JP2016062356A - Solid object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a solid object detection device capable of detecting even a solid object low in height as a solid object with high accuracy, while maintaining a state in which the solid object and a road surface can be satisfactorily distinguished.SOLUTION: A solid object detection device increases a weight to voting by determining that the continuity of an approximate straight line on the predetermined two-dimensional plane is high and that the closer the inclination is to vertical the more likely the inclination appears to be a solid object, and a parallax calculation point is voted, by reflecting the weight, to each block of a plurality of parallax voting maps disposed with a plurality of predetermined blocks set so as to be long in side according as the parallax value increases, and in a parallax voting map, the number of votes of the parallax calculation point gives an attribute of the solid body exceeding a predetermined threshold value, and a position in a three-dimensional space of the solid body is obtained from information of the block given with the attribute of the solid body, thereby detecting the solid body.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a three-dimensional object detection device.

  Conventionally, an object determination device for a vehicle that determines whether an object around the vehicle is a three-dimensional object or a road surface has been reported (Patent Document 1, etc.).

  For example, in the technique described in Patent Document 1, left and right images having parallax are obtained by photographing objects around a vehicle from each of different viewpoints with two cameras, and points corresponding to each of the left and right images are obtained. Calculate the parallax. One side corresponds to the parallax, the other side corresponds to the horizontal position of the image, and the side corresponding to the parallax and the side corresponding to the horizontal position become longer as the parallax increases. A parallax voting map in which a plurality of blocks are arranged is set, and information on the parallax calculation point where the parallax is calculated is voted for each block of the parallax voting map. Then, in the parallax voting map, a block having a parallax calculation point voting number equal to or greater than a threshold value, the distribution of the voted parallax calculation points being small in the parallax direction and having a large distribution in the vertical direction of the image Gives the attribute of a three-dimensional object, and assigns the attribute of the road surface to the second block having a large distribution in the parallax direction of the voted parallax calculation points and a small distribution in the vertical direction of the image. Judging.

  Here, in the case of a three-dimensional object, the distance in the image depth direction does not change, so that the parallax value is substantially constant. In the case of a road surface, the distance in the image depth direction changes, and thus the parallax value changes. For this reason, the parallax calculation points of the three-dimensional object are distributed with a small spread in the distance direction and a large spread in the height direction in the real space coordinates, that is, a small distribution in the parallax direction and a vertical distribution on the image. There is a characteristic that it is large. On the other hand, the parallax calculation points on the road surface have a small spread in the height direction and a large spread in the distance direction in real space coordinates, that is, a small distribution in the vertical direction and a large distribution in the parallax direction on the image. There is a characteristic.

  Focusing on such characteristics, in the conventional object determination device described in Patent Document 1, in the parallax voting map, the number of votes of the parallax calculation points is a block having a threshold value or more, and A solid object is detected by assigning a solid object attribute to the first block having a small distribution in the parallax direction and a large distribution in the vertical direction of the image. On the other hand, the road surface is detected by assigning the attribute of the road surface to the second block having a large distribution in the parallax direction of the voted parallax calculation points and a small distribution in the vertical direction of the image. In the parallax calculation map, when the number of votes of the parallax calculation points is equal to or greater than the threshold, it is possible to determine whether the object is a three-dimensional object or a road surface because there are a large number of parallax calculation points corresponding to the left and right images. When the number of votes of calculation points is less than the threshold value, there is a high possibility that it is difficult to determine whether the object is a three-dimensional object or a road surface because the number of parallax calculation points that correspond between the left and right images is small.

JP 2006-236104 A

  However, in the prior art (Patent Document 1 and the like), although it is possible to detect the solid object and the road surface among the objects, the three-dimensional object has a low height such as a road shoulder or a fallen object on the road. It had the improvement that detection accuracy became low.

  Here, in order to detect a three-dimensional object with a low height, in the conventional object determination device described in Patent Document 1, a measure for relaxing the condition for detecting a three-dimensional object can be considered. For example, in the parallax calculation map, a threshold for determining that a predetermined number or more of parallax calculation points are voted blocks is set low, or a threshold for determining a first block to which a solid object attribute is assigned. It is possible to take measures such as adjusting so as to be low. However, if such a procedure is performed, a three-dimensional object with a low height can be detected, but at the same time, it becomes easy to pick up noise, and the road surface gradient itself is detected as a three-dimensional object. Will increase. Therefore, in order to detect a three-dimensional object with a low height, if a procedure for adjusting various thresholds that define the detection condition of the three-dimensional object is performed, the detection accuracy of the three-dimensional object is increased by increasing false detection of the three-dimensional object. Will be lowered. Moreover, it is not desirable for safety that the object determination result including the erroneous detection result of the three-dimensional object is used for driving support. As described above, in the conventional object determination device, a technique capable of distinguishing a three-dimensional object from a road surface and capable of detecting a three-dimensional object having a low height has been demanded.

  The present invention has been made in view of the above circumstances, and can detect a three-dimensional object having a low height as a three-dimensional object with high accuracy while maintaining a state in which the three-dimensional object and the road surface can be distinguished well. An object of the present invention is to provide a three-dimensional object detection device.

  The three-dimensional object detection device of the present invention includes a stereo camera imaging unit, and a parallax value calculation unit that calculates a parallax value for each predetermined pixel region that constitutes the image, based on an image acquired by the stereo camera imaging unit. , Extracting a predetermined vertical region obtained by dividing the image so as to include the pixel region in the vertical direction, and for each vertical region, the vertical axis included in the pixel position information of each pixel region of the image A two-dimensional plane that is projected and the parallax values corresponding to the respective ordinates are projected on the horizontal axis is set, and linear approximation is performed on the two-dimensional plane based on the projected ordinates and the parallax values. By performing, an approximate straight line generating unit that generates an approximate straight line composed of the projected ordinate and the parallax value, and the approximate straight line generated on the two-dimensional plane, the inclination and length of the approximate straight line A voting weight calculation unit that calculates weights that contribute to solid object determination by voting, and a parallax voting map in which a plurality of blocks defined by the parallax values and abscissas included in pixel position information of each pixel area of the image are arranged A parallax calculation point voting unit that reflects the weight calculated by the voting weight calculation unit for each block of the parallax voting map, and the parallax voting map, And a three-dimensional object detection unit that detects a three-dimensional object based on information of a block whose number of votes of the parallax calculation point is equal to or greater than a predetermined threshold.

  In the above three-dimensional object detection device, the voting weight calculation unit increases the amount of change in the ordinate relative to the amount of change in parallax, which represents the slope of the approximate line when the length of the approximate line is equal to or greater than a predetermined threshold. It is preferable to increase the weight.

  According to the three-dimensional object detection apparatus according to the present invention, it is possible to easily detect a three-dimensional object having a low height as a three-dimensional object while maintaining a state in which the three-dimensional object and the road surface can be distinguished well. . Thereby, the detection accuracy of a three-dimensional object with a low height can be improved, and as a result, there is an effect that a three-dimensional object can be detected more suitably.

FIG. 1 is a diagram illustrating an example of a configuration of a three-dimensional object detection device according to an embodiment. FIG. 2 is a diagram illustrating an example of an approximate line generated on the two-dimensional plane according to the embodiment. FIG. 3 is a diagram illustrating an example of a parallax voting map according to the embodiment. FIG. 4 is a flowchart illustrating an example of basic processing of the three-dimensional object detection device according to the embodiment.

  Hereinafter, embodiments of a three-dimensional object detection device according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment. In addition, constituent elements in the following embodiments include those that can be easily assumed by those skilled in the art or those that are substantially the same.

[Embodiment]
With reference to FIGS. 1-3, the structure of the solid-object detection apparatus which concerns on embodiment is demonstrated. Here, FIG. 1 is a diagram illustrating an example of a configuration of the three-dimensional object detection device according to the embodiment. FIG. 2 is a diagram illustrating an example of an approximate line generated on the two-dimensional plane according to the embodiment. FIG. 3 is a diagram illustrating an example of a parallax voting map according to the embodiment.

  The three-dimensional object detection device in the present embodiment is mounted on a vehicle (host vehicle), and typically includes a stereo camera imaging unit 1, a vehicle momentum detection device 2, an ECU 3, and an actuator 4.

  The stereo camera imaging unit 1 images the surrounding environment of the vehicle. In the present embodiment, an example in which the stereo camera imaging unit 1 is composed of two cameras will be described. The stereo camera imaging unit 1 includes a right camera 1a and a left camera 1b that can be imaged. The right camera 1a is installed on the right side in front of the vehicle, and the left camera 1b is installed on the left side in front of the vehicle. The right camera 1a and the left camera 1b are, for example, stereo cameras. The right camera 1a outputs a luminance image R that is an image obtained by capturing the traveling direction of the vehicle to the ECU 3. The left camera 1b outputs a luminance image L, which is an image obtained by capturing the traveling direction of the vehicle, to the ECU 3. Note that the stereo camera imaging unit 1 is not limited to the example including two cameras, and may include two or more cameras as long as the parallax can be calculated by processing of the ECU 3 described later.

  The vehicle momentum detection device 2 detects various information (vehicle speed, yaw rate, acceleration, etc.) indicating the vehicle momentum. The vehicle momentum detection device 2 includes at least a vehicle speed sensor 2a, a yaw rate sensor 2b, and an acceleration sensor 2c. The vehicle speed sensor 2a is a wheel speed detection device that is provided for each wheel and detects each wheel speed. Each vehicle speed sensor 2a detects a wheel speed that is a rotational speed of each wheel. Each vehicle speed sensor 2a outputs a wheel speed signal indicating the detected wheel speed of each wheel to the ECU 3. The ECU 3 calculates the vehicle speed, which is the traveling speed of the vehicle, based on the wheel speed of each wheel input from each vehicle speed sensor 2a. The ECU 3 may calculate the vehicle speed based on the wheel speed input from at least one of the vehicle speed sensors 2a. The ECU 3 acquires the calculated vehicle speed as vehicle motion information. The yaw rate sensor 2b is a yaw rate detection device that detects the yaw rate of the vehicle. The yaw rate sensor 2b outputs a yaw rate signal indicating the detected yaw rate to the ECU 3. The ECU 3 acquires the input yaw rate signal as vehicle motion information. The acceleration sensor 2c is an acceleration detection device that detects acceleration applied to the vehicle body. The acceleration sensor 2c outputs an acceleration signal indicating the detected acceleration to the ECU 3. The ECU 3 acquires the input acceleration signal as vehicle motion information.

  The ECU 3 controls the driving of each part of the vehicle, and is an electronic control unit mainly composed of a known microcomputer including a CPU, a ROM, a RAM, and an interface. The ECU 3 is electrically connected to the stereo camera imaging unit 1 and the vehicle momentum detection device 2 and receives an electric signal corresponding to the detection result. The ECU 3 performs various arithmetic processes according to the electrical signal corresponding to the detection result, and outputs a control command corresponding to the calculation result, thereby controlling the operation of the actuator 4 electrically connected to the ECU 3. For example, the ECU 3 outputs a control signal based on the calculation processing result to the actuator 4 and performs driving support control for controlling the behavior of the vehicle by operating the actuator 4.

  Details of various processing units of the ECU 3 will be described. The ECU 3 includes a luminance image acquisition unit 3a, a parallax value calculation unit 3b, an approximate straight line generation unit 3c, a voting weight calculation unit 3d, a parallax calculation point voting unit 3e, a three-dimensional object detection unit 3f, and a vehicle control unit 3i. And at least. Here, the three-dimensional object detection unit 3f further includes an attribute assignment unit 3g and a three-dimensional position calculation unit 3h.

  In the ECU 3, the luminance image acquisition unit 3a acquires the luminance image R and the luminance image L output from the right camera 1a and the left camera 1b of the stereo camera imaging unit 1, respectively. The luminance image acquisition unit 3a further corrects the luminance image R and the luminance image L so as to eliminate lens distortion of the right camera 1a and the left camera 1b as image distortion correction processing, and the optical axes of the right camera 1a and the left camera 1b. Also has a function of performing a process of correcting the luminance image R and the luminance image L so that they are parallel to each other. The luminance image R and the luminance image L acquired by the luminance image acquisition unit 3a and corrected for distortion are used for processing of the parallax value calculation unit 3b.

  The parallax value calculation unit 3b captures a surrounding environment of the vehicle and based on a plurality of luminance images (for example, the luminance image R and the luminance image L) obtained from the stereo camera imaging unit 1, a predetermined luminance image is configured. A parallax value is calculated for each pixel region.

  In the present embodiment, the predetermined pixel region may be a single pixel or a group of a plurality of pixels. The predetermined pixel area is set to an area having a common size among a plurality of luminance images when calculating the parallax. For example, when the pixel area of the luminance image L is a single pixel, the pixel area of the luminance image R is also a single pixel. When the pixel area of the luminance image L is a group of a plurality of pixels, the pixel area of the luminance image R is also a group of a plurality of images having the same size as the luminance image L. For example, when the pixel area of the luminance image L is a group of four pixels of 2 × 2 (two single pixels in the x direction and two single pixels in the y direction), the pixel area of the luminance image R Is a group of four 2 × 2 pixels. In addition, the pixel region may be, for example, a group of 9 × 3 × 3 pixels, a group of 16 × 4 × 4 pixels, or 5 × 5 × 25. It may be a group of a plurality of pixels.

  In the ECU 3, the parallax value calculation unit 3b calculates a parallax value by using a parallax calculation method known in the technical field (for example, a block matching method along an epipolar line, a Semi-Global Matching method, or the like). In this embodiment, since the continuity of the approximate straight line is determined by the continuity / tilt determination unit 3d described later, the Semi-Global Matching method in which smooth parallax is calculated in a relatively wide area of the image is used. preferable.

  Here, details of the parallax calculation processing by the parallax value calculation unit 3b will be described. The parallax value calculation unit 3b calculates the parallax based on the luminance image R and the luminance image L acquired by the luminance image acquisition unit 3a and subjected to distortion correction. In the present embodiment, the parallax value calculation unit 3b calculates the parallax based on only one of the luminance image L and the luminance image R. In other words, the parallax value calculation unit 3b calculates the parallax based on the luminance image L based on both the luminance image L and the luminance image R, or the luminance image based on both the luminance image L and the luminance image R. Parallax is calculated based on R.

  Specifically, the parallax value calculation unit 3b searches for a corresponding pixel area between the luminance image L and the luminance image R, and for each corresponding pixel area searched, the corresponding pixel area is the luminance image L and the luminance image. The amount of deviation indicating how much is deviated between R is calculated as the parallax value D (x, y). For example, when calculating the parallax based on the luminance image L, the parallax value calculation unit 3b calculates the amount of deviation from the luminance image L to the luminance image R as the parallax value D (x, y) between the corresponding pixel regions. To do. When calculating the parallax based on the luminance image R, the amount of deviation from the luminance image R to the luminance image L between the corresponding pixel regions may be calculated as the parallax value D (x, y). Further, the parallax value calculation unit 3b uses the calculated parallax value D (x, y) of the pixel area in one of the luminance image L and the luminance image R (that is, the luminance image based on the parallax calculation). A process of associating each pixel position information (x, y) (that is, the parallax calculation point (x, y) where the parallax value D (x, y) is calculated) is performed.

  More specifically, the parallax value calculation unit 3b calculates a parallax value for each partial pixel area constituting the image based on the luminance value information of the image (luminance image) acquired by the stereo camera imaging unit 1. The following processing is performed in the calculation process. The parallax value calculation unit 3b provisionally calculates the parallax value in the partial pixel region. Then, the parallax value calculation unit 3b includes a provisional parallax value, which is the provisionally calculated parallax value, and a pixel area at least in the vicinity of the pixel area located within the predetermined image range from the partial pixel area. The parallax value in the partial pixel region is calculated by correcting the temporary parallax value based on the comparison with the peripheral parallax value that has already been calculated by the parallax value calculation unit 3b. The parallax value calculated in this way is used when an approximate line is generated by an approximate line generation unit 3c described later.

  As shown in the upper diagram of FIG. 2, the approximate line generation unit 3 c of the ECU 3 includes a pixel region in the vertical direction of the image (that is, the luminance image L or the luminance image R based on the parallax calculation). In this way, a predetermined vertical region (a black line region drawn in the vertical direction of the image in FIG. 2) is extracted. Here, it is assumed that a plurality of regions corresponding to columns in an image having a predetermined width are set at regular intervals in the image. The number of column regions extracted in the image is determined according to the predetermined width of the column region and the width of the image.

  Then, as shown in the lower diagram of FIG. 2, the approximate straight line generation unit 3 c uses the ordinate (y coordinate in FIG. 2) included in the pixel position information of each pixel region of the image as the vertical axis for each column region. A two-dimensional plane is set on which the parallax values corresponding to the ordinates are projected on the horizontal axis. Here, the two-dimensional plane is set for each column area in the order from one end (for example, the left end) of the image to the other end (for example, the right end). The lower diagram in FIG. 2 shows, as an example, a plurality of column regions extracted from the image so as to straddle the road surface portion, the three-dimensional object portion (the opponent vehicle traveling in the opposite lane in FIG. 2), and the background portion. The two-dimensional plane regarding the 23rd column area counted from the left end of the set image is shown. The approximate line generation unit 3c extracts the ordinates included in the pixel position information of the pixel area in the column area and the parallax values corresponding to the ordinates in the order from bottom to top along the column area. However, these ordinates and parallax values are plotted on a two-dimensional plane.

  Then, as shown in the lower diagram of FIG. 2, the approximate straight line generation unit 3c projects on the two-dimensional plane using an approximate straight line method known in the technical field (for example, a Split-and-Merge method). An approximate straight line composed of the projected ordinate and parallax value is generated by performing linear approximation based on the ordinate and parallax value. The approximate line generation unit 3c generates an approximate line from points corresponding to the plotted ordinates and parallax values in each two-dimensional plane set for each column region of the image.

  Among the ECUs 3, the voting weight calculation unit 3 d is an approximate straight line that is more than a predetermined distance from the vertical axis toward the horizontal axis among the approximate straight lines generated on the two-dimensional plane, as shown in the lower diagram of FIG. 2. , Based on the inclination and length, the parallax points included in the approximate line determine the weight that contributes to the three-dimensional object determination by voting described later. That is, the voting weight calculation unit 3d calculates a weight that contributes to the three-dimensional object determination by voting from the inclination and the length of the approximate line for the approximate line generated on the two-dimensional plane.

  Here, it is assumed that the predetermined distance is set to an appropriate value capable of distinguishing between the approximate line corresponding to the background and the approximate line corresponding to the three-dimensional object and the road surface on the two-dimensional plane. On the two-dimensional plane, if the object is a three-dimensional object or background, the distance in the image depth direction does not change, so the parallax value is substantially constant. For example, as shown in the lower diagram of FIG. 2, for the background and the three-dimensional object in which the parallax value is almost constant, the change amount of the parallax value is small with respect to the change amount of the ordinate, so the inclination of the approximate straight line with respect to the vertical axis Are both smaller. Therefore, in order to distinguish between the background and the three-dimensional object, the determination is based on the difference in the parallax value. In general, the background exists on the far side from the vehicle in the traveling direction on the real space coordinates, while the three-dimensional object exists on the near side closer to the vehicle than the background on the real space coordinates. For this reason, the parallax value calculated in the pixel area corresponding to the background tends to be small, and the parallax value calculated in the pixel area corresponding to the three-dimensional object tends to be larger than that of the background. Therefore, in the present embodiment, among the approximate lines generated on the two-dimensional plane, the approximate lines that are less than a predetermined distance from the vertical axis toward the horizontal axis are determined to correspond to the background, and the approximate lines are continuous. It shall be excluded from the determination target of sex and inclination.

  As shown in the lower diagram of FIG. 2, the approximate straight line of the region corresponding to the three-dimensional object tends to have a large inclination (close to vertical) with respect to the horizontal axis. On the other hand, the above-mentioned inclination of the traveling road surface tends to be somewhat reduced. For this reason, it is possible to hit a solid object or a road surface to some extent from the inclination of the approximate straight line.

  Further, in a region where the influence of the parallax error is large, stable straight line approximation cannot be performed, and the straight line tends to be chopped.

  In view of the above-mentioned tendency, an approximate straight line having a length equal to or greater than a predetermined length threshold is determined. The weight of the parallax points included in the approximate straight line that contributes to the three-dimensional object determination by voting is increased. In this embodiment, for example, the weight is doubled. In addition, when the length of the approximate line is equal to or greater than a predetermined inclination threshold value, it is preferable to increase the weight as the change amount of the ordinate relative to the change amount of parallax, which represents the inclination of the approximate line, is larger.

  Here, considering the general property that farther three-dimensional objects appear smaller on the image and are more susceptible to parallax errors, the length threshold value and inclination threshold value change depending on the distance. Is preferred.

  Among the ECUs 3, the parallax calculation point voting unit 3e is a block defined by the parallax value and the abscissa included in the pixel position information of each pixel area of the image, and the vertical side of the block is the parallax value, A parallax voting map in which a plurality of blocks in which the horizontal sides of the block correspond to the abscissa and the vertical sides and the horizontal sides become longer as the parallax value increases are arranged. Set.

  Here, with reference to FIG. 3, the details of the parallax voting map used when the parallax calculation point voting unit 3e votes the parallax calculation points will be described. As shown in FIG. 3, in the parallax voting map, the side in the horizontal direction corresponds to the position in the horizontal direction (x coordinate axis direction) of the image, and the side in the vertical direction has the size of the parallax value D (x, y). A plurality of rectangular blocks determined to correspond to each other are arranged. Each block increases the horizontal length of the image as the parallax value increases (that is, the length of the side corresponding to the abscissa (x coordinate in FIG. 3) increases) and the parallax value increases. By increasing the resolution of the parallax value as it becomes (that is, by increasing the length of the side corresponding to the parallax value), the block area is determined to increase as the parallax value increases.

  In the present embodiment, the parallax voting map is divided into a plurality of areas of the planes 1 to 5, and each of the plurality of areas is determined to correspond to a predetermined parallax width. In the embodiment, when the parallax value is represented by 8 bits, the plane 1 has a parallax value 0 to 15, the plane 2 has a parallax value 16 to 31, the plane 3 has a parallax value 32 to 63, the plane 4 has a parallax value 64 to 127, The plane 5 is determined so as to correspond to the parallax 128 to 255.

  The size of each block on the plane 1 is N pixels in the horizontal direction and the vertical direction (parallax direction) is 1 pixel, and the size of each block on the plane 2 is 2N pixels in the horizontal direction and the vertical direction (parallax direction). ) Is 2 pixels, the size of each block on the plane 3 is 4N pixels in the horizontal direction, the vertical direction (parallax direction) is 4 pixels, and the size of each block on the plane 4 is 8N pixels in the horizontal direction. The vertical direction (parallax direction) is 8 pixels in size, and the size of each block on the plane 5 is determined so that the horizontal direction is 16N pixels and the vertical direction (parallax direction) is 16 pixels. The blocks are arranged in a grid pattern. N is an arbitrary integer. N may be set to 1, for example, or may be set to 4.

  Then, the parallax calculation point voting unit 3e votes the parallax calculation points for each block of the parallax voting map as illustrated in FIG. 3 by reflecting the weight calculated by the voting weight calculation unit 3d. Specifically, the parallax calculation point voting unit 3e is calculated by the voting weight calculation unit 3d for the voting that is normally counted as “1” for the parallax values on the approximate straight line included on the two-dimensional plane. Multiply the weights to vote for the corresponding block on the parallax voting map. Since the two-dimensional plane is set for each column region of the image, the parallax calculation point voting unit 3e reflects the weight calculated by the voting weight calculation unit 3 for all the parallax points on the two-dimensional plane. And vote for the corresponding block on the parallax voting map.

  Of the ECU 3, the three-dimensional object detection unit 3 f detects a three-dimensional object based on information of blocks in which the number of votes of parallax calculation points is equal to or greater than a predetermined threshold in the parallax voting map. Here, the three-dimensional object detection unit 3f includes an attribute assignment unit 3g and a three-dimensional position calculation unit 3h.

  In the parallax voting map, the attribute assigning unit 3g gives the attribute of the three-dimensional object to a block in which the number of votes at the parallax calculation point is equal to or greater than a predetermined threshold. Specifically, the attribute assigning unit 3g determines whether or not the number of votes of the parallax calculation points is equal to or greater than a predetermined threshold set for each plane, so that a solid object is added to a block equal to or greater than the predetermined threshold. Gives the attribute of a solid object. Moreover, the attribute provision part 3g provides the attribute of a road surface about the block below a predetermined threshold value. Here, as the distance is shorter, the correspondence between the left and right images can be taken, and the number of parallax calculation points increases. Therefore, the predetermined threshold is set to gradually increase from the plane 1 toward the plane 5. . In addition, the predetermined threshold value is set to an appropriate value for each of the planes 1 to 5 that has been confirmed in advance by experiments or the like so that the road surface and the three-dimensional object can be distinguished.

  The three-dimensional position calculation unit 3h calculates the three-dimensional actual position where the vehicle exists from the abscissa of the block to which the attribute of the three-dimensional object is assigned, the average value of the parallax being voted, and the vertical variation in the image of the parallax points to be voted. It is calculated how high a three-dimensional object exists at which position in the space.

  Of the ECU 3, the vehicle control unit 3i performs driving support control for controlling the vehicle behavior so that the vehicle avoids the three-dimensional object detected by the three-dimensional object detection unit 3f. The vehicle control unit 3i, for example, exercise information (vehicle speed, yaw rate, acceleration) acquired from the vehicle momentum detection device 2, various information indicating a region in which the vehicle can travel, the position of a three-dimensional object to be avoided, etc. Based on the above, a travel locus, a travel speed, and the like that the vehicle can avoid the three-dimensional object are calculated. And the vehicle control part 3i outputs the control signal based on this calculation process result to the actuator 4, and performs avoidance control by operating the actuator 4. As the avoidance control, the vehicle control unit 3i performs steering assistance so that the vehicle avoids the three-dimensional object by controlling the steering angle of the steering wheel of the vehicle via the actuator 4 such as EPS, for example. The vehicle control unit 3i may execute the steering support in combination with the brake support as the avoidance control so that the three-dimensional object can be avoided more reliably. In this manner, the vehicle control unit 3i functions as an avoidance control unit that avoids the movement of the vehicle to the position of the three-dimensional object.

  Next, a three-dimensional object detection method executed by the processing of the three-dimensional object detection device configured as described above will be described with reference to FIG. Here, FIG. 4 is a flowchart illustrating an example of basic processing of the three-dimensional object detection device according to the embodiment.

  As illustrated in FIG. 4, the luminance image acquisition unit 3a acquires the luminance image R and the luminance image L output from the right camera 1a and the left camera 1b of the stereo camera imaging unit 1, respectively (step S10).

  And the parallax value calculation part 3b is based on the luminance value information of the several luminance image (For example, the luminance image R and the luminance image L) acquired by the process of the luminance image acquisition part 3a in step S10, The said luminance image A parallax value is calculated for each predetermined pixel area that constitutes (step S20).

  Then, as shown in FIG. 2 described above, the approximate straight line generation unit 3c is an image (that is, the luminance image R and the luminance image L acquired by the processing of the luminance image acquisition unit 3a in step S10) on the two-dimensional plane. In step S20, a predetermined column area obtained by dividing the image in the vertical direction so as to include a pixel area in the vertical direction by the process of the parallax value calculation unit 3b in step S20 is extracted. Then, the approximate straight line generation unit 3c projects the ordinate included in the pixel position information of each pixel region of the image on the vertical axis for each column region, and the processing of the parallax value calculation unit 3b in step S20. A two-dimensional plane in which the parallax values corresponding to the calculated ordinates are projected on the horizontal axis is set. Then, the approximate line generator 3c generates an approximate line composed of the projected ordinate and parallax value by performing linear approximation on the two-dimensional plane based on the projected ordinate and parallax value (step S30). ).

  Then, as shown in FIG. 2 described above, the voting weight calculation unit 3d moves from the vertical axis to the horizontal axis among the approximate lines generated on the two-dimensional plane by the process of the approximate line generation unit 3c in step S30. Based on the length and inclination of the approximate straight line that is more than a predetermined distance toward the front, the parallax points included in the approximate straight line calculate the weight that contributes to the solid object determination by voting described later (step S40).

  The parallax calculation point voting unit 3e is a block defined by the parallax value and the abscissa included in the pixel position information of each pixel area of the image, as shown in FIG. A block defined such that the side in the direction corresponds to the parallax value, the side in the horizontal direction of the block corresponds to the abscissa, and the side in the vertical direction and the side in the horizontal direction become longer as the parallax value increases, A parallax voting map in which a plurality of are arranged is set. Then, the parallax calculation point voting unit 3e votes the parallax calculation points for the target block of the parallax voting map by reflecting the weight calculated by the voting weight calculation unit 3d in step S40 (step S50).

  Then, in the parallax voting map in which the parallax calculation points are voted by the process of the parallax calculation point voting unit 3e in step S50, the attribute assigning unit 3g determines whether the number of votes of the parallax calculation points of the target block is equal to or greater than a predetermined threshold value. Is determined (step S60).

  Here, in step S60, when the attribute assigning unit 3g determines that the number of votes of the parallax calculation point of the target block is equal to or greater than a predetermined threshold (step S60: Yes), the three-dimensional object with respect to the target block Is added (step S70). Thereafter, the process proceeds to step S90.

  On the other hand, in step S60, when the attribute assigning unit 3g determines that the number of votes of the parallax calculation point of the target block is less than a predetermined threshold (step S60: No), the attribute of the road surface with respect to the target block. (Step S80). Thereafter, the process proceeds to step S90.

  After the processing of step S70 and step S80, the attribute assigning unit 3g finishes the attribute providing processing for all blocks of the parallax voting map in which the parallax calculation points are voted by the processing of the parallax calculation point voting unit 3e in step S50. It is determined whether or not (step S90).

  Here, in step S90, when the attribute assigning unit 3g determines that the attribute assigning process has not been completed for all blocks of the parallax voting map (step S90: No), the process returns to the process of step S60, and the attribute assigning is performed. The block next to the processed block is set as a target block, and the process of step S70 or the process of step S80 is performed again based on the determination result of step S60.

  On the other hand, when the attribute assigning unit 3g determines in step S90 that the attribute assigning process has been completed for all blocks of the parallax voting map (step S90: Yes), the process proceeds to the next step S100.

  Then, the three-dimensional position calculation unit 3h, in step S70, the abscissa of the block to which the attribute of the three-dimensional object has been assigned by the processing of the attribute assigning unit 3g, the average value of the parallax that has been voted, and the image of the parallax point to be voted From the vertical variation, it is calculated at which position and how high a three-dimensional object exists in the three-dimensional real space where the vehicle exists (step S100). Thereafter, this process is terminated.

  In the present embodiment, after the process of step S100, the ECU 3 detects the road surface based on the pixel region of the image corresponding to the block to which the road surface attribute is given by the process of the attribute assigning unit 3g in step S80. May be. Specifically, the ECU 3 totals the pixel position information of the pixel area of the image indicated by the parallax calculation points voted for the block to which the road surface attribute is given by the process of the attribute giving unit 3g, and the totaled pixel position For the pixel area corresponding to the information, the road surface is detected by determining that the road surface exists in the image.

  In the present embodiment, after the process of step S100, the vehicle control unit 3i performs driving support control for controlling the vehicle behavior so that the vehicle avoids the three-dimensional object detected by the process of the three-dimensional object detection unit 3f. May be. In this case, the vehicle control unit 3i, for example, exercise information (vehicle speed, yaw rate, acceleration) acquired from the vehicle momentum detection device 2, various information indicating the region in which the vehicle can travel, and a three-dimensional object to be avoided. Based on the position of the vehicle, a travel locus, a travel speed, and the like that the vehicle can avoid the three-dimensional object are calculated. And the vehicle control part 3i outputs the control signal based on this calculation process result to the actuator 4, and performs avoidance control by operating the actuator 4. As the avoidance control, the vehicle control unit 3i performs steering assistance so that the vehicle avoids the three-dimensional object by controlling the steering angle of the steering wheel of the vehicle via the actuator 4 such as EPS, for example. The vehicle control unit 3i may execute the steering support in combination with the brake support as the avoidance control so that the three-dimensional object can be avoided more reliably.

  As described above, according to the three-dimensional object detection device of the present embodiment, a three-dimensional object with a low height is easily detected as a three-dimensional object while maintaining a state in which the three-dimensional object and the road surface can be distinguished well. Can be. Thereby, the detection accuracy of a three-dimensional object with a low height can be improved, and a three-dimensional object can be detected more suitably than the prior art.

DESCRIPTION OF SYMBOLS 1 Stereo camera imaging part 1a Right camera 1b Left camera 2 Vehicle momentum detection apparatus 2a Vehicle speed sensor 2b Yaw rate sensor 2c Acceleration sensor 3 ECU
3a Luminance image acquisition unit 3b Parallax value calculation unit 3c Approximate line generation unit 3d Voting weight calculation unit 3e Parallax calculation point voting unit 3f Solid object detection unit
3g attribute assignment part
3h 3D position calculation unit 3i Vehicle control unit 4 Actuator

Claims (1)

A stereo camera imaging unit;
A parallax value calculation unit that calculates a parallax value for each predetermined pixel region that constitutes the image, based on the image acquired by the stereo camera imaging unit;
A predetermined vertical region obtained by dividing the image into a plurality of vertical regions so as to include the pixel region is extracted, and for each vertical region, the vertical coordinate included in the pixel position information of each pixel region of the image is projected on the vertical axis. In addition, a two-dimensional plane in which the parallax values respectively corresponding to the ordinates are projected on the horizontal axis is set, and linear approximation is performed on the two-dimensional plane based on the projected ordinates and the parallax values. Thus, an approximate line generation unit that generates an approximate line composed of the projected ordinates and the parallax value;
A voting weight calculating unit that calculates a weight that contributes to the determination of a three-dimensional object by voting from the inclination and length of the approximate line, with respect to the approximate line generated on the two-dimensional plane;
A parallax voting map in which a plurality of blocks determined by the parallax value and the abscissa contained in the pixel position information of each pixel area of the image is arranged is set, and the parallax calculation is performed for each block of the parallax voting map A parallax calculation point voting unit for voting a point reflecting the weight calculated by the voting weight calculation unit;
In the parallax voting map, a three-dimensional object detection unit that detects a three-dimensional object based on information of blocks in which the number of votes of the parallax calculation points is equal to or greater than a predetermined threshold;
A three-dimensional object detection device comprising:

Images