JP2007257459A - Posture determining device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a posture determining device capable of easily and quickly determining the posture of a walker from an image around a road. <P>SOLUTION: The posture determining device has a binarization processing means 5 for extracting a binarized area from an image acquired through an imaging means 2 for imaging surroundings of the road, a run-length data preparing means 6 for preparing run-length data of the extracted binarized areas, a target object extracting means 7 for extracting target objects existing around the road on the basis of the prepared run-length data, a walker extracting means 8 for extracting walkers among the extracted target objects, and a walker posture determining means 9 for determining the postures of the extracted walkers on the basis of the prepared run-length data. The walker posture determining means 9 is provided with a means for calculating dot sequence data composed of coordinates of pixels of a middle point of each line of the prepared run-length data, a means for calculating an approximate straight lines that approximate the calculated dot sequence data, and a means for determining the postures of the walkers on the basis of the calculated approximate straight lines. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a posture determination device that determines the posture of a human such as a pedestrian from an image.

  In recent years, humans existing in the vicinity are detected from indoor images captured by an imaging means such as a CCD camera, the detected human posture is determined, and, for example, an air conditioner or the like is detected according to the determined result. Techniques for performing control and the like are known. According to this, since the human state is inferred from the posture, appropriate device control or the like according to the human state is performed. At this time, as an apparatus for determining a human posture, for example, an apparatus using shape determination of a human region extracted from an image has been proposed (see, for example, Patent Document 1).

  In the posture determination apparatus of Patent Document 1, a human region detection unit that detects a human region on infrared image data, a shape determination unit that determines the shape of a human region, and a human motion from time-series data of an image Motion detection means for performing the determination, and posture discrimination means for discriminating the human posture based on the discrimination result by the shape discrimination means and the detection result by the motion detection means. At this time, the shape discriminating means discriminates three types of shapes of portrait, landscape, and square according to the aspect ratio of the human region. The posture discriminating means is standing when the shape is vertically long and moving, standing in the case where the shape is vertically long and not moving and when the shape is horizontally long, and sitting when the shape is square, The human posture is classified into three types: standing, lying and sitting.

  On the other hand, a technique is known in which a pedestrian is detected from an image around a vehicle, a pedestrian with a high possibility of collision is determined, and information is presented to the driver (see Patent Document 2, for example). . In such a technique, for example, when a pedestrian suddenly jumps out to the roadway, in order to quickly determine the possibility of a collision and present information to the driver, the pedestrian must cross the road in advance. It is desirable to predict quickly. At this time, for example, it is considered that a pedestrian trying to cross a road and other pedestrians have different postures related to the body orientation, inclination, and the like at the time of being on the roadside. This is because when the pedestrian starts moving or jumps out onto the roadway, the trunk axis is tilted with respect to the traveling direction, while the pedestrian is stopped or the road This is because the inclination of the trunk axis is considered to be sufficiently small when the movement is continued along. Therefore, in order to predict the crossing of the road in advance, it is desirable to determine the posture of the roadside pedestrian. In particular, it is desirable to determine the posture of a pedestrian by a simple method in order to reduce the calculation process and reduce the time required for determining the possibility of a collision.

At this time, in order to predict the pedestrian's behavior such as crossing the road, the pedestrian must be in an upright position or a forward leaned position, not in a standing position, a standing position, or a sitting position. It is necessary to grasp the posture related to the inclination of the trunk axis (the direction and angle of the inclination of the trunk axis). However, in the shape determination using the aspect ratio of the region as in the posture determination device of Patent Document 1, it is difficult to determine the posture related to the inclination of the trunk axis such as the upright posture and the forward tilt posture. Furthermore, since the posture determination apparatus of Patent Document 1 detects a human motion from time-series data of an image and uses it to determine the posture, it takes time until the human posture is determined. Therefore, the posture determination device of Patent Document 1 cannot properly determine the posture of a roadside pedestrian or the like, for example, and cannot predict the behavior of a pedestrian crossing a road in advance. It was.
Japanese Patent Laid-Open No. 5-149725 JP 2001-6096 A

  The present invention provides an attitude determination device that can easily and quickly determine the pedestrian's posture (posture regarding the inclination of the trunk axis of the pedestrian) from images around the road, eliminating such inconvenience. Objective.

  In order to achieve such an object, the posture determination device of the present invention performs binarization processing on an image acquired via an imaging unit that images the periphery of a road, and the luminance value of a pixel in the image is a predetermined threshold value. The binarization processing means for extracting the above binarized area, the run length data creating means for creating run length data of the binarized area extracted by the binarization processing means, and the run length data creation Based on the run length data created by the means, object extraction means for extracting objects existing around the road, and walking for extracting pedestrians from the objects extracted by the object extraction means A pedestrian posture for discriminating the posture of the pedestrian extracted by the pedestrian extraction means based on the run length data created by the pedestrian extraction means and the run length data creation means And a different means.

  In the posture determination apparatus of the present invention, a binarized area is extracted by the binarization processing unit from an image acquired through the imaging unit, and the run length data of the binarized area is obtained by the run length data creating unit. An object existing around the road is extracted by the object extraction means based on the run length data, and a pedestrian is extracted from the objects by the pedestrian extraction means. And the said pedestrian attitude | position discrimination | determination means discriminate | determines the attitude | position of the extracted pedestrian based on the said run length data. At this time, the point sequence data of a predetermined point (for example, the middle point) of each line of the run length data is closely related to the inclination of the trunk axis of the pedestrian. Therefore, by using the run length data, it is possible to determine the posture related to the inclination of the trunk axis of the pedestrian. Based on the posture of the pedestrian determined as described above, the behavior of the pedestrian such as crossing the road can be predicted quickly in advance. At this time, since the run-length data can be calculated from only one frame image, the time required for the posture determination can be shorter than the determination based on the time-series data of the image, for example. The run-length data is calculated in the process of extracting a pedestrian from an image. By using this data for determination, the calculation load required for posture determination is reduced. Therefore, according to the pedestrian posture determination means, the posture of the pedestrian can be easily and quickly determined by a simple process.

  Further, in the posture determination device according to the present invention, the pedestrian posture determination means is composed of the coordinates of the pixel at the midpoint of each line of the run length data from the run length data created by the run length data creation means. Means for calculating the point sequence data, means for calculating an approximate line approximating the calculated point sequence data, and means for determining the posture of the pedestrian based on the calculated approximate line It is preferable to provide.

  According to this, the pedestrian posture discriminating means calculates point sequence data composed of the coordinates of the midpoint pixel of each line of the run length data, and calculates an approximate straight line that approximates the point sequence data. . At this time, the calculated approximate straight line corresponds to the trunk axis of the pedestrian. Therefore, by determining the pedestrian posture based on the approximate straight line by the pedestrian posture determination means, it is possible to easily and quickly determine the pedestrian posture (the posture related to the inclination of the trunk axis of the pedestrian). Can do.

  Specifically, in the posture determination device of the present invention, the pedestrian posture determination means can determine the posture of the pedestrian based on an angle formed by the approximate straight line and the vertical axis.

  That is, since the approximate line corresponds to the trunk axis of the pedestrian, the direction and angle of inclination of the trunk axis of the pedestrian can be determined from the angle formed by the approximate line and the vertical axis. Then, based on the pedestrian posture determined in this manner (the posture related to the inclination of the trunk axis of the pedestrian), the behavior of the pedestrian such as crossing the road can be predicted quickly in advance.

  An embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a functional block diagram of the posture determination device according to the present embodiment, and FIG. 2 is a flowchart showing the overall operation (posture determination processing) in the image processing unit of the posture determination device of FIG. 3 is an exemplary diagram of a processing image in the posture determination processing of FIG. 2, and FIG. 4 is an explanatory diagram regarding processing for creating run-length data and extracting an object in the posture determination processing of FIG. FIG. 5 is an explanatory diagram regarding processing for determining the posture of a pedestrian in the posture determination processing of FIG.

  With reference to FIG. 1, the attitude | position discrimination | determination apparatus of this embodiment has the image processing unit 1 which is an electronic unit provided with CPU (central processing unit). Connected to the image processing unit 1 are an infrared camera 2 mounted on the vehicle 10, a speaker 3 for issuing a warning by sound, and a display device 4 for displaying an image acquired via the infrared camera 2. Has been. The display device 4 includes, for example, a HUD (head-up display) that displays information on the driver by providing a screen at a position that does not obstruct the driver's forward view of the front window of the vehicle 10.

  The infrared camera 2 is a camera that can detect far-infrared rays, and has a characteristic that the higher the temperature of the object, the higher the output signal level (the luminance increases). The infrared camera 2 is arranged at the front of the vehicle 10 so that the optical axis of the infrared camera 2 faces the front of the vehicle 10. The infrared camera 2 corresponds to the imaging unit of the present invention.

  Although not shown in detail, the image processing unit 1 is composed of an electronic circuit including an A / D conversion circuit, a CPU, a RAM, a ROM, an image memory, and the like, and an output (analog signal) of the infrared camera 2 is A / D. It is converted into a digital signal and input through a conversion circuit. Then, the image processing unit 1 determines, based on the input data, processing for detecting an object such as a pedestrian or whether a predetermined requirement is satisfied for the detected object, and the requirement is satisfied. In such a case, processing for issuing a warning to the driver of the vehicle 10 through the speaker 3 and the display device 4 is executed.

  More specifically, the image processing unit 1 functions as the binarization processing means 5 for extracting the binarized area from the acquired image, and the run length for creating run length data of the extracted binarized area. Data creation means 6, object extraction means 7 for extracting objects existing around the vehicle 10 based on the created run length data, and pedestrian extraction for extracting pedestrians from the extracted objects Means 8 and pedestrian posture determination means 9 for determining the pedestrian posture based on the run length data are provided.

  The binarization processing unit 5 performs binarization processing on the image acquired via the infrared camera 2 and extracts a binarized region in which the luminance value of the pixel in the image is equal to or greater than a predetermined threshold. Specifically, the binarization processing means 5 has a region in which the luminance value of the pixel is brighter than a predetermined threshold with respect to a grayscale image obtained by A / D converting the output signal (infrared image) of the infrared camera 2. A process of setting “1” (white) and a dark area to “0” (black) is performed.

  The run length data creation means 6 creates run length data of the binarized area extracted by the binarization processing means 5. Here, the generated run-length data represents a binarized area as a set of lines that are one-dimensional connected pixels, and each line constituting the binarized area is represented by the coordinates of the start point and the start point to the end point. This is indicated by the length (number of pixels).

  The object extracting means 7 extracts objects existing around the road based on the run length data created by the run length data creating means 6. As a method for extracting the object, for example, a labeling process or the like is used. At this time, the extracted objects (binarized objects) include, for example, other vehicles, artificial structures such as electric poles and vending machines, in addition to pedestrians around the road.

  The pedestrian extraction means 8 extracts a pedestrian from the objects extracted by the object extraction means 7. As a method for extracting a pedestrian, for example, whether or not the object is a pedestrian based on the aspect ratio of the object or the ratio of the area of the object to the area of the circumscribed rectangle of the object is determined. A determination method is used.

  The pedestrian posture determination means 9 is based on the run length data created by the run length data creation means 6 and the pedestrian posture extracted by the pedestrian extraction means 8 (posture regarding the inclination of the trunk axis of the pedestrian). Is determined. At this time, the pedestrian posture discriminating means 9 calculates point sequence data composed of the coordinates of the midpoint pixel of each line of the run length data from the run length data, and approximates the calculated point sequence data. An approximate straight line is calculated. The pedestrian posture determination means 9 determines the posture (the direction and angle of the trunk axis inclination) related to the inclination of the trunk axis of the pedestrian based on the angle formed between the calculated approximate straight line and the vertical axis. .

  Next, the overall operation (posture determination processing) of the posture determination device of this embodiment will be described with reference to the flowchart shown in FIG. In the following description, a case where a pedestrian exists on the left side in front of the vehicle 10 and another vehicle exists in front of the vehicle 10 and the pedestrian is going to cross the road will be described as an example.

  Referring to FIG. 2, first, image processing unit 1 acquires an infrared image that is an output signal of infrared camera 2 (STEP001), performs A / D conversion (STEP002), and stores a grayscale image in an image memory. (STEP003). Next, the image processing unit 1 binarizes the image signal for the grayscale image (STEP 004). That is, a process is performed in which a region where the luminance value of the pixel of the grayscale image is brighter than the threshold value Ith is set to “1” (white) and a dark region is set to “0” (black). The threshold value Ith is a value that is experimentally determined in advance.

  FIG. 3A illustrates an image obtained by binarizing an image obtained by the infrared camera 2. In FIG. 3A, the hatched area is black, and the area surrounded by the thick solid line is white. A region surrounded by a thick solid line is a region of an object that has a high luminance level (at a high temperature) and is displayed as white on the screen in the image obtained from the infrared camera 2.

  Next, the image processing unit 1 creates run-length data from an area (binarized area) that has become “white” in the binarization process (STEP 005). Here, with reference to FIG. 4A, in FIG. 4A, the binarized region is represented by lines L1 to L8 which are one-dimensional connected pixels. Each of the lines L1 to L8 has a width of one pixel in the y direction (vertical direction) and is actually arranged without a gap in the y direction, but is shown separated for the sake of explanation. . The lines L1 to L8 have lengths of 2 pixels, 2 pixels, 3 pixels, 8 pixels, 7 pixels, 8 pixels, 8 pixels, and 8 pixels, respectively, in the x direction (horizontal direction). The run-length data shows the lines L1 to L8 with the coordinates of the start point of each line (the leftmost point of each line) and the length (number of pixels) from the start point to the end point (the rightmost point of each line). It is. For example, since the line L3 is composed of three pixels (x3, y5), (x4, y5), and (x5, y5), the run length data is (x3, y5, 3).

  Next, the image processing unit 1 extracts the target object by labeling the target object (STEP 006) based on the generated run length data (STEP 007). That is, among the lines represented by the run-length data, a line having an overlapping portion in the y direction is regarded as one object and a label (identifier) is attached to extract a connected region in the image as the object. . Here, with reference to FIG. 4B, among the lines L1 to L8 shown in FIG. 4A, the lines L1 to L3 having portions overlapping in the y direction are regarded as one object 1, The lines L4 to L8 are regarded as one object 2, and the object labels 1 and 2 are added to the run length data. Thereby, the binarized areas are extracted as the objects 1 and 2, respectively.

  Through the processing of STEP 005 to 007 described above, the binarized areas shown in FIG. 3A are extracted as objects (binarized objects) T1 to T6 as illustrated in FIG. 3B. At this time, for example, the object of the label T1 is indicated by n pieces of run-length data L1 to Ln. In the example of FIG. 3B, the object T1 is a binarized object corresponding to a pedestrian existing on the left side in front of the vehicle 10, and the objects T2 to T6 are in front of the vehicle 10. It is a binarized object corresponding to another vehicle that exists.

  Next, the image processing unit 1 calculates the area SA of the extracted object, the area SB of the circumscribed rectangle of the object, and the aspect ratio ASPECT of the object (STEP008). Specifically, the area SA of the target T1 is obtained by integrating the length of the line indicated by each run-length data Li (i = 1,..., N) with respect to the n pieces of run-length data of the target T1. To calculate. Further, the aspect ratio ASPECT of the object T1 is calculated as a ratio Dy / Dx between the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object T1.

  Next, the image processing unit 1 extracts a pedestrian from the extracted objects (STEP 009). At this time, when the aspect ratio ASPECT of the object and the ratio SA / SB of the area of the object and the circumscribed rectangle of the object are within a predetermined range, the object is determined to be a pedestrian. The object is extracted as a pedestrian. The predetermined range is a range that is determined in advance as a range of values that can be taken when the object is the upper body or the whole body of a pedestrian. Thereby, as illustrated in FIG. 3B, the object T1 surrounded by the frame R1 is extracted as a pedestrian from the objects T1 to T6.

  Next, in STEPs 010 to 013, processing for determining the posture of the extracted pedestrian T1 is performed. Below, it demonstrates with reference to Fig.5 (a) (b). FIGS. 5A and 5B each show a region (including a pedestrian T1) surrounded by the frame R1 illustrated in FIG. 3B.

  First, in STEP 010, the image processing unit 1 uses the extracted run-length data L1 to Ln of the pedestrian T1 to generate point sequence data P_T1 composed of the coordinates of the midpoint pixel of the line indicated by each run-length data Li. Is calculated. With reference to Fig.5 (a), the binarization area | region equivalent to the target object extracted as the pedestrian T1 is represented by n pieces of run-length data L1-Ln as shown in the figure. FIG. 5B shows run-length data L1 to Ln, and the midpoints M1 to Mn are as shown. The point sequence data P_T1 includes coordinate data {(X1, Y1), (X2, Y2),. . . , (Xn, Yn)}. In FIG. 5B, the coordinate axes X and Y are set as shown. At this time, the X-axis is an axis parallel to the left-right direction of the vehicle 10, and the Y-axis is an up-down direction axis.

Next, in STEP011, the image processing unit 1 calculates an approximate line S1 that approximates the calculated point sequence data P_T1. At this time, the least square method is used as an approximation method. Thereby, as shown in FIG.5 (b), approximate straight line S1 = aX + b is calculated. The approximate straight line S1 corresponds to the trunk axis of the pedestrian T1. Next, in STEP 012, as shown in FIG. 5B, an angle θ = tan ⁻¹ a formed by the approximate straight line S 1 and the vertical axis Y is calculated.

  Next, in STEP013, the image processing unit 1 determines the posture (the direction and angle of the inclination of the trunk axis) related to the inclination of the trunk axis of the pedestrian T1 based on the angle θ. Specifically, in this embodiment, it is determined whether or not the posture of the pedestrian T1 is a posture in which the trunk axis is tilted forward toward the center of the lane in which the vehicle 10 travels. In this embodiment, the angle θ is 0 degree in the vertical axis Y direction, and the direction from the vertical axis Y toward the center of the lane in which the vehicle 10 travels is positive (example shown in FIG. 5B). Is set to be positive). At this time, when the angle θ is equal to or larger than the threshold θ1 and equal to or smaller than the threshold θ2 (0 <θ1 <θ2), it is determined that the vehicle is in a forward leaning posture toward the lane in which the vehicle 10 travels. The threshold values θ1 and θ2 are preliminarily set as values indicating the range of angles that the trunk axis can generally take with respect to the traveling direction when the pedestrian starts moving or jumps out onto the roadway. It is a defined value. In the example of FIG. 5B, θ1 ≦ θ ≦ θ2, and it is determined that the pedestrian T1 is in a forward leaning posture toward the lane in which the vehicle 10 travels. Thereby, it is predicted that pedestrian T1 is going to cross the road.

  The above is the posture determination processing in the posture determination device of the present embodiment. Thereby, from the infrared image around the vehicle 10, the posture of the pedestrian T1 (the posture related to the inclination of the trunk axis of the pedestrian T1) can be easily and quickly determined by simple processing. Therefore, based on the determination result of the posture of the pedestrian T1, the behavior of the pedestrian T1 such as crossing the road is quickly predicted in advance.

  In the present embodiment, information is presented to the driver based on the determination result of the posture of the pedestrian T1. At this time, for example, when the pedestrian T1 is leaning forward toward the lane in which the vehicle 10 travels, it is predicted that the pedestrian T1 is about to cross the road. A sound alarm is issued through the speaker 3 or an image obtained by the infrared camera 2 is output to the display device 4 to emphasize the pedestrian T1 extracted on the image and display it to the driver of the vehicle 10. .

  In the present embodiment, as described above, the image processing unit 1 functions as the binarization processing means 5, run length data creation means 6, object extraction means 7, pedestrian, and the like. Extraction means 8 and pedestrian posture determination means 9 are included. More specifically, STEP 004 in FIG. 2 corresponds to the binarization processing means 5, STEP 005 in FIG. 2 corresponds to the run length data creation means 6, and STEPs 006 to 007 in FIG. 2 correspond to the object extraction means 7. 2 corresponds to the pedestrian extraction means 8, and STEPs 010 to 013 in FIG. 2 correspond to the pedestrian posture determination means 9.

  In this embodiment, the number of infrared cameras mounted on the vehicle 10 is one. However, as another embodiment, a plurality of infrared cameras mounted on the vehicle 10 are used, and any one of the plurality of infrared cameras is used. The posture determination process may be performed using an image obtained from one. For example, the vehicle 10 is equipped with two infrared cameras (which constitute a so-called stereo camera), and posture determination is performed using an image (for example, a reference image) obtained from one of the two infrared cameras. Processing is performed.

  In this embodiment, the infrared camera 2 is mounted on the vehicle 10 and images the road ahead of the vehicle 10. However, as another embodiment, for example, the infrared camera 2 is attached to a structure around the road to The image may be taken.

  In this embodiment, an infrared camera is used as the imaging means. However, as another embodiment, for example, a CCD camera that can detect only normal visible light may be used. However, by using an infrared camera, extraction processing of pedestrians, other vehicles, and the like can be simplified, and it can be realized even when the calculation capability of the calculation device is relatively low.

The functional block diagram of the attitude | position discrimination | determination apparatus of this invention. 3 is a flowchart showing posture determination processing in the image processing unit of the posture determination device in FIG. 1. FIG. 3 is an exemplary diagram of a processed image in the posture determination process of FIG. 2. Explanatory drawing regarding the process which produces the run length data in the attitude | position discrimination | determination process of FIG. 2, and extracts a target object. Explanatory drawing regarding the process which discriminate | determines the attitude | position of a pedestrian in the attitude | position discrimination | determination process of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 2 ... Infrared camera (imaging means), 5 ... Binarization processing means, 6 ... Run length data creation means, 7 ... Object extraction means, 8 ... Pedestrian extraction means, 9 ... Pedestrian posture discrimination means.

Claims (3)

Binarization processing means for performing binarization processing on an image acquired via an imaging means for imaging the periphery of a road, and extracting a binarized area in which the luminance value of a pixel in the image is equal to or greater than a predetermined threshold; ,
Run length data creating means for creating run length data of the binarized area extracted by the binarization processing means;
Based on the run-length data created by the run-length data creating means, an object extracting means for extracting objects existing around the road;
Pedestrian extraction means for extracting pedestrians from the objects extracted by the object extraction means;
A posture discrimination device comprising: a pedestrian posture discrimination means for discriminating a posture of a pedestrian extracted by the pedestrian extraction means based on the run length data created by the run length data creation means.   The pedestrian posture determining means calculates means for calculating point sequence data composed of the coordinates of pixels at the midpoint of each line of the run length data from the run length data created by the run length data creating means; The means for calculating an approximate straight line that approximates the calculated point sequence data, and means for determining the posture of the pedestrian based on the calculated approximate straight line. Attitude discrimination device.   The posture determination apparatus according to claim 2, wherein the pedestrian posture determination means determines the posture of the pedestrian based on an angle formed by the approximate straight line and a vertical axis.

Images