JP4558758B2 - Obstacle recognition device for vehicles - Google Patents

Description

  The present invention relates to an obstacle recognition device for a vehicle that is mounted on a vehicle and recognizes an obstacle using a sensor that measures the relative position and relative speed of a surrounding object with respect to the vehicle and an image sensor that captures the periphery. is there.

In order to improve the performance of conventional systems that consist of a single sensor that measures the relative position and relative speed of surrounding objects relative to the host vehicle, such as an automatic travel control device and collision damage mitigation device mounted on a vehicle, an image sensor There is an increasing number of fusions.
In such a system, there are cases where the results output by the two sensors are different. Therefore, it is necessary to obtain the effect of fusion while maintaining the performance of the conventional system.
In particular, when a monocular camera that can be configured at low cost is used as an image sensor, it is difficult to identify a three-dimensional object.

For example, as disclosed in Patent Document 1, a vehicle candidate group based on a detection result of a laser radar, a camera, and the like relating to a fusion technique between a sensor for measuring the relative position and relative speed of a peripheral object with respect to the own vehicle and a monocular camera. The rectangular area corresponding to the preceding vehicle based on the preceding vehicle recognition result of the preceding vehicle is compared, and the portion where the vehicle candidate group based on the detection result of the laser radar overlaps the rectangular area corresponding to the preceding vehicle based on the preceding vehicle recognition result of the monocular camera is Some are judged to exist.
Or as shown in patent document 2, there exists a thing which selects the shorter one among the vehicle width detected by the laser radar, and the vehicle width detected by the preceding vehicle recognition of the camera.

Japanese Patent Laid-Open No. 08-329393 (pages 2 and 3, FIG. 1) JP-A-2006-240454 (pages 5-11, FIG. 1)

However, in Patent Documents 1 and 2, since a monocular camera cannot recognize a three-dimensional object, it is very difficult to detect when objects overlap each other or when a target is a stationary object, and erroneous detection is likely to occur. There is.
For this reason, even when taking into account turning points and monocular camera systems, control intervention may not be performed properly in collision damage mitigation devices, etc., compared to systems consisting of single sensors such as laser radar and millimeter wave radar. Unnecessary control intervention (malfunction) may occur.
Further, according to Patent Document 2, in order to select the shorter one of the vehicle width detected by the laser radar and the vehicle width detected by the preceding vehicle recognition of the camera, When the laser radar is correctly detecting (for example, when the camera selects a part of the vehicle such as a rear window or a license plate as an edge), the control intervention may be delayed.

  The present invention has been made in order to solve the above-described problems. Even when a sensor for measuring the relative position or relative speed of an object around the own vehicle and a monocular camera are used, an obstacle is removed. An object of the present invention is to obtain an obstacle recognition device for a vehicle that can be recognized.

In the vehicle obstacle recognition device according to the present invention, the vehicle obstacle recognition device that is mounted on a vehicle and recognizes an obstacle against the vehicle, a detection sensor that detects an object around the vehicle, and an output of the detection sensor. Based on object information calculating means for calculating the relative position and relative speed between the vehicle and the object, an image sensor for picking up an image around the vehicle, an image acquiring means for acquiring an image picked up by this image sensor, and an object information calculating means The image processing means for processing the image acquired by the image acquisition means, the vehicle information acquisition means for detecting the driving state of the vehicle, the object information calculation means, and the output of the vehicle information acquisition means determining obstacle determining means likely to be, and on the basis of the output of the object information calculating means, determining chromatic whether output is enabled or the image processing unit Comprising sex determination hand stage, the obstacle determining unit, in accordance with the output of the validity determining means, with using the output of the image processing means for determination of the obstacle, the validity determining means, the output of the object information calculating means And a traveling state estimation unit that estimates the traveling state of the object, and the traveling state estimation unit detects a part of the object further ahead of the preceding object of the vehicle as the traveling state of the object. If it is estimated, it is determined that it is not effective .

As described above, the present invention is a vehicle obstacle recognition device that is mounted on a vehicle and recognizes an obstacle against the vehicle, based on a detection sensor that detects an object around the vehicle, based on the output of the detection sensor, Object information calculation means for calculating the relative position and relative speed between the vehicle and the object, an image sensor for picking up an image around the vehicle, an image acquisition means for acquiring an image picked up by the image sensor, and an output of the object information calculation means Based on the output of the image processing means for processing the image acquired by the image acquisition means, the vehicle information acquisition means for detecting the driving state of the vehicle, the object information calculation means, and the output of the vehicle information acquisition means, the object may become an obstacle to the vehicle. determining obstacle determining means sex, and on the basis of the output of the object information calculating means, Bei validity determination means to determine whether the output of the image processing means is enabled , The obstacle determining unit, in accordance with the output of the validity determining means, with using the output of the image processing means for determination of the obstacle, the validity determining means based on the output of the object information calculating means, the object traveling state of When the travel state estimation means estimates that a part of the object further ahead of the preceding object of the vehicle is detected as the travel state of the object by the travel state estimation means since judged to be invalid, while maintaining detection performance by the detection sensor alone, to increase the detection performance when use of the image processing means is enabled, the output of the image processing means when the use of the image processing unit is not appropriate Malfunctions caused by the use of can be reduced.

Embodiment 1 FIG.
Hereinafter, the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings.
1 is a block diagram showing a schematic configuration of a vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.
In FIG. 1, the vehicle obstacle recognition device 100 includes a millimeter wave radar 110, a camera 120, a CPU 130, an alarm device 140, an automatic control device 150, a vehicle speed sensor 160, and a yaw rate sensor 170.
The millimeter wave radar 110 (detection sensor) receives reflected waves from surrounding objects of the millimeter wave radar and detects obstacles such as vehicles and guardrails existing in the forward direction of the host vehicle.
The camera 120 (video sensor) is a monocular camera that images the front of the vehicle.
The vehicle speed sensor 160 and the yaw rate sensor 170 detect the vehicle speed and yaw rate of the host vehicle.
Although only two sensors are described here, other sensors necessary for determining the behavior of the vehicle, such as a blinker and a steering wheel angle, may be used.
The alarm device 140 generates an alarm for notifying the driver that there is an obstacle that becomes an obstacle of the host vehicle, based on a command from the CPU 130 or the like. Here, although it was set as the alarm device, you may comprise with a display apparatus.
The automatic control device 150 outputs control signals such as brake control, throttle control, and AT shift control, for example, according to commands from the CPU 130.

The CPU 130 includes object information calculation means 131 that calculates object information from the millimeter wave radar 110, image acquisition means 132 that acquires an image captured by the camera 120, and vehicle information (vehicles) from the vehicle speed sensor 160 and the yaw rate sensor 170. Vehicle information acquisition means 133 that acquires information on the driving state of the vehicle), image processing means 134 that processes an image with information acquired by the object information calculation means 131, image acquisition means 132, and vehicle information acquisition means 133, and object Obstacle determination means 135 that determines an obstacle based on output information from the information calculation means 131, vehicle information acquisition means 133, and image processing means 134, alarm means 136 that outputs an alarm to the alarm device 140, and the determined obstacle Obstacle avoiding means 137 for avoiding the object and the validity judgment for judging the validity of the object information A cutting means 138 is provided.
Although all processing is performed by one CPU 130, the image acquisition unit 132 and the image processing unit 134 may be included in the camera 120 or provided with a dedicated device. Further, the object information calculation unit 131 may be included in the millimeter wave radar 110 or may be provided with a dedicated device.
Further, instead of the millimeter wave radar 110, a laser radar may be used.

FIG. 2 is an external view showing an example of mounting of the millimeter wave radar and camera on the vehicle by the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.
In FIG. 2, the millimeter wave radar 110 is attached to a front bumper or the like of the vehicle 200, and acquires a reflected wave from a peripheral object in front of the vehicle. The camera 120 is attached to the interior of the vehicle 200, in the vicinity of the rearview mirror, and photographs the situation around the front of the vehicle.
Here, for the sake of explanation, the millimeter wave radar 110 and the camera 120 are shown conspicuously, but they should be attached so as not to impair the appearance of the vehicle.

FIG. 3 is a flowchart showing processing of the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.
FIG. 4 is a flowchart showing the processing of the image processing means of the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.

FIG. 5 is a diagram showing an image example of the image processing means of the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.
In FIG. 5, in the image example 300, a coordinate 301 is an image coordinate P (i) indicating a relative position between the own vehicle and the moving body. A coordinate 302 is a coordinate P (j) indicating a relative position between the own vehicle and the stopped object. An area 303 is a moving object detection area R (i), and an area 304 is a stationary object detection area R (j).

FIG. 6 is a diagram for explaining edge candidate extraction of the image processing means of the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.
In FIG. 6, a histogram 310 is a result of taking a histogram in the vertical direction of the image of edge detection points scanned and detected in the horizontal direction of the image in the detection areas 303 and 304. The edge candidate 311 is a result of further extracting edge candidates that are peak points of a predetermined value or more from the result. The edges 312 and 313 are determined edges.

FIG. 7 is a view for explaining the wrap amount of the image processing means of the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.
In FIG. 7, the lap amount 401 is a size corresponding to the overlap in the width direction between the own vehicle 200 and the detection object 400.

FIG. 8 is a flowchart showing the processing of the obstacle determination means of the vehicle obstacle recognition apparatus according to Embodiment 1 of the present invention.
FIG. 9 is a diagram for explaining avoidance of the obstacle determination means and the alarm threshold value of the obstacle recognition apparatus for a vehicle according to Embodiment 1 of the present invention.
FIG. 9A is a diagram in which the avoidance threshold value THc increases in proportion to the lap amount L, and FIG. 9B is a diagram in which the alarm threshold value THw increases in proportion to the lap amount L.

FIG. 10 is a diagram for explaining the effectiveness determining means of the vehicle obstacle recognizing device according to the first embodiment of the present invention.
In FIG. 10, a guardrail 500 is shown.

Next, the operation of the vehicle obstacle recognition apparatus in the first embodiment will be described.
The millimeter wave radar 110 receives reflected waves from surrounding objects of the millimeter wave radar, and detects obstacles such as a vehicle and a guardrail existing in the forward direction of the host vehicle. The detected result is output to the CPU 130. The camera 120 captures the front of the vehicle, and the captured image is captured by the CPU 130 via the image acquisition unit 132. The vehicle speed sensor 160 and the yaw rate sensor 170 output the vehicle speed and yaw rate of the host vehicle to the CPU 130.

As shown in FIG. 3, the CPU 130 executes vehicle information acquisition means 133 and object information calculation means 131, and then executes image acquisition means 132 and image processing means 134. Further, the obstacle determination unit 135 is executed based on the effectiveness determination unit 138 and its output.
After the execution of the obstacle determination unit 135, the alarm unit 136 or the alarm unit 136 and the obstacle avoidance unit 137 are executed according to the result of the obstacle determination unit 135.
If there is no alarm target or avoidance target, the process ends without doing anything.

The processing of each means will be described below.
The vehicle information acquisition unit 133 acquires information such as the vehicle speed and yaw rate of the host vehicle from the vehicle speed sensor 160 and the yaw rate sensor 170.
The object information calculation unit 131 determines, based on the output of the millimeter wave radar 110, the relative distance to the detected object, the relative lateral position, the relative speed, and whether the detected object is a moving object or a stopped object.
Here, the output of the object information calculation means 131 is that the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp. , Relative speed Vr}. i and j are integers starting from 0 and given to each object.
The image acquisition unit 132 acquires an image of a predetermined area used for the image processing unit 134 from the camera 120. The predetermined area referred to here may be all image data of the camera, or may be set based on the output of the object information calculation means 131.
The image processing unit 134 performs predetermined image processing based on the output of the object information calculation unit 131.
In the first embodiment, the image processing unit 134 detects the edge of the detected object, and calculates the amount of lap with the own vehicle from the result.

Next, the processing of the image processing means will be described with reference to FIG.
In the image processing unit 134, first, based on the output of the object information calculation unit 131, a detection region for detecting the edge of each moving body or stationary object is set by the detection region setting unit S 01. For example, as shown in FIG. 5, the detection area includes image coordinates P (i) for the relative position between the vehicle and the moving object obtained by the object information calculation unit 131, and P ( j) to obtain regions R (i) and R (j) in which a predetermined margin is added to the height and width of one vehicle.

Subsequently, the edge detection means S02 detects vertical edge components in the detection region for each moving object or stationary object, and further extracts edge candidates from the detected result.
The histogram 310 in FIG. 6 is obtained by taking a histogram in the vertical direction of the image of edge detection points detected by scanning in the horizontal direction of the image in the detection areas 303 and 304. As a result, the edge candidate 311 further includes a predetermined value among the results. This is a result of extracting edge candidates that are the above peak points.

  In the edge determination unit S03, the noise position included in the detection result of the edge detection unit S02 is removed, and the edge position of each detected object is determined. For the determination of the edge, for example, it is conceivable to use the symmetry of the detected object or pattern matching in consideration of the position of the edge candidate point in the detection region. Thereby, for example, a pair of edge candidates is determined as the edges 312 and 313.

  In the width detection unit S04, the width of the detected object is detected from the result of the edge determination unit S03. The width detection results are: moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp, relative speed Vr. , Width W} is added as information. A width W = NULL is added to an object for which edge detection could not be performed.

The lap amount calculation means S05 considers the road curvature estimated from the yaw rate in the result of the edge determination means S03, and considers the result of determining the traveling direction of the own vehicle, and calculates the lap amount L between each detected object and the own vehicle. calculate.
The lap amount 401 has a size corresponding to the overlap in the width direction of the host vehicle 200 and the detection object 400 as shown in FIG. The lap amount 401 includes the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp. , Relative speed Vr, width W, lap amount L} are added. A lap L = NULL is added to an object for which edge detection could not be performed.
Here, the processing is performed for all the results of the object information calculation unit 131. However, the object information calculation unit 131 narrows down objects that may be an obstacle of the host vehicle in advance, and then the image processing unit. 134 may be implemented.

Next, the function of the validity determination unit 138 will be described.
First, the case where the effectiveness determination unit 138 is not provided, that is, the case where the obstacle determination unit 135 is implemented following the image processing unit 134 will be described.
In the obstacle determination unit 135 of FIG. 8, first, in the obstacle candidate selection unit S11, an object whose edge cannot be detected from the information of the moving body Mv (i) and the stop St (j), that is, the width W = NULL. If the possibility of being an object on the road is low, such as when the width W is too wide, the obstacle determination flag Fo = NULL is set as a moving object. Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle determination flag Fo} and stop St (j) = {relative distance Dis, relative lateral position Ltp, The information is added as information on the relative speed Vr, the width W, the lap amount L, and the obstacle determination flag Fo}. For other objects, an obstacle determination flag Fo = 1 is added.

Subsequently, the collision prediction time calculation means S12 is performed. The predicted collision time TTC can be calculated by Equation 1 from the relative distance DIS and the relative speed Vr of the moving object Mv (i) and the stopped object St (j).
TTC = Dis / Vr Equation 1
The predicted collision time TTC is the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle determination flag Fo, predicted collision time TTC} and stationary object St ( j) = {information as relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle determination flag Fo, collision prediction time TTC}.

The avoidance target determination unit S13 determines that the target is an avoidance target when the predicted collision time TTC is less than a predetermined avoidance threshold value THc. Here, the avoidance threshold value THc is variable according to the lap amount L in each detection target, and the avoidance threshold value THc is also increased in proportion to the lap amount L as shown in FIG.
If there is an avoidance target, the warning means 136 is executed to notify the driver that there is an obstacle that is an obstacle to the own vehicle, and the obstacle avoiding means 137 is executed to avoid the obstacle. Control.
If there is no avoidance target, the process proceeds to alarm target determination means S14.

The alarm target determination means S14 determines that the target is an alarm when the predicted collision time TTC is below a predetermined alarm threshold THw. Here, the alarm threshold value THw is variable according to the lap amount L in each detection target. For example, as shown in FIG. 9B, the alarm threshold value THw is also increased in proportion to the lap amount L.
If there is an alarm target, the alarm means 136 is executed to notify the driver that there is an obstacle that is an obstacle to the vehicle.
If there is no alarm target, the obstacle determination means 135 is terminated.
As shown in FIG. 9, by increasing the avoidance threshold value and the alarm threshold value in proportion to the lap amount, the alarm / control intervention is advanced for an object with a large lap amount and a high risk of collision.

Here, the warning means 136 uses the warning device 140 to notify the driver that there is an obstacle with a sound. Although not described in FIG. 1, an image or a schematic diagram of the camera 120 may be provided using a display device.
The obstacle avoiding means 137 performs brake, AT shift, throttle control, etc. via the automatic control device 150, and controls the host vehicle in a direction to avoid the obstacle. Further, along with this, control such as seat belt or airbag deployment for protecting the driver or the occupant may be performed.

Next, a case where the validity determination unit 138 is provided will be described.
A sensor capable of detecting depth such as a stereo camera can distinguish between a vehicle and a background object such as a guard rail. However, when performing image processing using a monocular camera as in the first embodiment, Have difficulty.
In particular, moving objects on the road are likely to be vehicles or two-wheeled vehicles and can be detected with high accuracy even with a monocular camera, but there are various objects such as backgrounds, stopped vehicles, guardrails, etc., which can be erroneously detected. The nature is very high. For example, the guard rail 500 as shown in FIG. 10 cannot be distinguished from being a guard rail, and there is a possibility that a part of the guard rail is determined as the edge of the stop object.
In the case where the validity determination means 138 is not provided, as described above, the obstacle candidate selection means S11 of the obstacle determination means 135 cannot detect the edge or the road W is too wide. If it is determined that the possibility of being an upper object is low, it is excluded because the possibility of being an obstacle is low, but in the case of FIG. 10, there is a high possibility that it cannot be excluded.
Therefore, in such a case, it is determined that the target is an alarm target or an avoidance target, an unnecessary alarm or braking is applied, and the performance of the millimeter wave radar single system cannot be maintained. In particular, when the alarm / control intervention is advanced according to the lap amount as in the first embodiment, the influence of malfunction due to erroneous detection becomes very large.

Therefore, the validity determination unit 138 determines whether it is valid or invalid to consider the output of the image processing unit 134 in the obstacle determination unit 135 for each detected object based on the output of the object information calculation unit 131. judge.
In the first embodiment, as described above, in the image processing by the monocular camera, it is determined to be invalid if it is a stationary object that is highly likely to be erroneously detected, and it is determined to be valid if it is a moving object.
For an object determined to be invalid, the validity object determination flag Fv = NULL is set to the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle. Determination flag Fo, effectiveness determination flag Fv} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle determination flag Fo, effectiveness determination flag Fv } Is added as information. The validity determination flag Fv = 1 is set for an object determined to be valid.

In addition, in the case of having the validity determination means 138,
The avoidance target determination unit S13 and the alarm determination unit S14 are changed as follows.
In the avoidance determination means S13, it is determined as an avoidance target when the predicted collision time TTC is less than a predetermined avoidance threshold THc. Here, when the effectiveness determination flag Fv = 1, the avoidance threshold value THc is variable according to the lap amount L in each detection target, and is proportional to the lap amount L as shown in FIG. The avoidance threshold THc is also increased. However, when the validity determination flag Fv = NULL, the avoidance threshold THc is determined using a predetermined fixed threshold C.
If there is an avoidance target, the warning means 136 is executed to notify the driver that there is an obstacle that is an obstacle to the own vehicle, and the obstacle avoiding means 137 is executed to avoid the obstacle. Control.
If there is no avoidance target, the process proceeds to alarm target determination means S14.

The alarm target determination means S14 determines that the target is an alarm when the predicted collision time TTC is below a predetermined alarm threshold THw. Here, when the validity determination flag Fv = 1, the alarm threshold value THw is variable according to the lap amount L in each detection target, and is proportional to the lap amount L, for example, as shown in FIG. The alarm threshold value THw is also increased. However, when the validity determination flag Fv = NULL, the warning threshold THw is determined using a predetermined fixed threshold W.
If there is an alarm target, the alarm means 136 is executed to notify the driver that there is an obstacle that is an obstacle to the vehicle.
If there is no alarm target, the obstacle determination means 135 is terminated.
As shown in FIG. 9, by increasing the threshold value in proportion to the lap amount, warning / control intervention is accelerated for an object with a large lap amount and a high risk of collision.

  According to the first embodiment, for a stationary object that is highly likely to be erroneously detected in the image processing by the monocular camera, while maintaining the performance of the conventional millimeter wave radar single system, an alarm / Control intervention can be accelerated, and the effect of fusion with the camera can be obtained.

Embodiment 2. FIG.
The second embodiment of the present invention will be described below with reference to the drawings.
FIG. 11 is a block diagram showing a schematic configuration of a vehicle obstacle recognition apparatus according to Embodiment 2 of the present invention.
In FIG. 11, reference numerals 100, 110, 120, 130 to 138, 140, and 150 are the same as those in FIG. In FIG. 11, the CPU 130 is provided with a reliability determination unit 139 subsequent to the image processing unit 134, and the output of the reliability determination unit 139 is input to the obstacle determination unit 135.
The example of mounting the millimeter wave radar and the camera to the vehicle is the same as that in FIG.

FIG. 12 is a flowchart showing processing of the vehicle obstacle recognition apparatus according to Embodiment 2 of the present invention.
FIG. 13 is a flowchart showing the processing of the reliability determination means of the vehicle obstacle recognition apparatus according to Embodiment 2 of the present invention.

Next, the operation of the vehicle obstacle recognition apparatus according to the second embodiment having the reliability determination unit 139 will be described. However, the description of the same parts as those in Embodiment 1 is omitted.
In FIG. 12, the CPU 130 executes vehicle information acquisition means 133 and object information calculation means 131, and then executes image acquisition means 132 and image processing means 134. Further, the obstacle determination unit 135 is executed based on the effectiveness determination unit 138, the reliability determination unit 139, and the output thereof.
After execution of the obstacle determination means 135, if the obstacle determination means 135 determines that there is an obstacle to be alarmed, the alarm means 136 further determines that there is an obstacle to be avoided. In this case, the obstacle avoiding means 137 is executed.
If there is no alarm target or avoidance target, the process ends without doing anything.

Next, the reliability determination unit 139 will be described with reference to FIG.
The reliability determination means 139 determines the reliability based on the time-series data of each moving object and the stationary object.
In FIG. 13, first, the detection stability confirmation unit S21 stores the information on the moving body and the stationary object detected by the millimeter wave radar 110 within the past predetermined time and the information calculated this time. Compare.
As a result of comparison, if it is determined that the object is the same as the stored information, the stability determination counter of each information is incremented.
If it is determined in the holding information that there is no same object in the new information, the stability determination counter is decremented. Here, when the counter reaches 0, the information is deleted.
If the new information determines that there is no same object in the stored information, the new information is stored.

In the stability determination means S22, the counter incremented in the detection stability confirmation means S21 is compared with a predetermined threshold, and if it is equal to or greater than the threshold, it is determined that the detection is stable and it is determined that the reliability is high.
If the predetermined threshold value is not reached, it is determined that the reliability is low because stable detection is not possible.
For an object determined to have low reliability, the reliability determination flag Fr = NULL is set to the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, Obstacle determination flag Fo, validity determination flag Fv, reliability determination flag Fr} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle determination The information is added as information of the flag Fo, the validity determination flag Fv, and the reliability determination flag Fr}. For an object determined to have high reliability, the reliability determination flag Fv = 1 is set.

The flowchart of the obstacle determination means 135 in the second embodiment is the same as that in FIG.
In the second embodiment, the following changes are made to the first embodiment in the avoidance target determination unit S13 and the alarm target determination unit S14.

In the avoidance determination means S13, it is determined as an avoidance target when the predicted collision time TTC is less than a predetermined avoidance threshold THc. Here, when the validity determination flag Fv = 1 and the reliability determination flag Fr = 1, the avoidance threshold THc is variable according to the lap amount L of each detection target, for example, as shown in FIG. In addition, the avoidance threshold value THc is also increased in proportion to the lap amount L. However, if the validity determination flag Fv = 1 or NULL, the reliability determination flag Fr = NULL, the validity determination flag Fv = NULL and the reliability determination flag Fr = 1, the avoidance threshold THc is a predetermined fixed threshold C Determine using.
If there is an avoidance target, the warning means 136 is executed to notify the driver that there is an obstacle that is an obstacle to the own vehicle, and the obstacle avoiding means 137 is executed to avoid the obstacle. Control.
If there is no avoidance target, the process proceeds to alarm target determination means S14.

The alarm target determination means S14 determines that the target is an alarm when the predicted collision time TTC is below a predetermined alarm threshold THw. Here, when the validity determination flag Fv = 1 or NULL and the reliability determination flag Fr = 1, the alarm threshold value THw is variable according to the lap amount L of each detection target, for example, as shown in FIG. Thus, the alarm threshold value THw is also increased in proportion to the lap amount L. However, when the validity determination flag Fv = 1 or NULL and the reliability determination flag Fr = NULL, the alarm threshold THw is determined using a predetermined fixed threshold W.
If there is an alarm target, the alarm means 136 is executed to notify the driver that there is an obstacle that is an obstacle to the vehicle.
If there is no alarm target, the obstacle determination means 135 is terminated.
As shown in FIG. 9, by increasing the threshold value in proportion to the lap amount, warning / control intervention is accelerated for an object with a large lap amount and a high risk of collision.

According to the second embodiment, for a stationary object that is highly likely to be erroneously detected in the image processing by the monocular camera, while maintaining the performance of the conventional millimeter wave radar single system, an alarm / Control intervention can be accelerated, and the effect of fusion with the camera can be obtained.
Furthermore, even if there is a high possibility of a stationary object that is likely to be erroneously detected by image processing with a single-lens camera, it is possible to provide information to the driver early by an alarm if it is highly reliable. The effect by can be acquired.

Embodiment 3 FIG.
The third embodiment of the present invention will be described below with reference to the drawings.
FIG. 14 is a block diagram showing a schematic configuration of a vehicle obstacle recognition apparatus according to Embodiment 3 of the present invention.
In FIG. 14, reference numerals 100, 110, 120, 130 to 138, 140, 150 are the same as those in FIG. In FIG. 14, the validity determination means 138 is provided with a traveling state estimation means 138a for estimating that the traveling state is such that the preceding vehicle and the preceding vehicle partially overlap each other.

FIG. 15 is a diagram for explaining the erroneous detection due to the overlap with the interrupting vehicle of the validity judging means of the vehicle obstacle recognizing device according to Embodiment 3 of the present invention.
In FIG. 15, by performing edge detection on the detection area 320 for the interrupting vehicle 601, there is a possibility that an edge is erroneously detected due to the overlap 321 with the preceding vehicle 600.

FIG. 16 is a diagram for explaining erroneous detection of a two-wheeled vehicle by the effectiveness determining means of the vehicle obstacle recognition apparatus according to Embodiment 3 of the present invention.
In FIG. 16, by performing edge detection on the detection region 320 for the two-wheeled vehicle 701, there is a possibility that the edge of the first-coming vehicle instead of the two-wheeled vehicle is erroneously detected due to the overlap 321 with the first-first vehicle 700.

An external view showing an example of mounting the millimeter wave radar of the third embodiment to a vehicle is the same as that of the first embodiment.
Hereinafter, description of the same parts as those in the first embodiment will be omitted, and the validity determination unit 138 in the third embodiment will be described.

If it is a sensor that can detect the depth of a stereo camera or the like, in addition to the background object such as the vehicle and the guardrail, it is possible to detect the preceding vehicle even when the preceding vehicle and a part of the preceding vehicle overlap, It is very difficult to perform image processing using a monocular camera as in Form 3.
In particular, when there is no preceding vehicle, or when the preceding vehicle and the preceding vehicle are completely overlapped and only the preceding vehicle is visible, it can be detected with a single camera, but if the preceding vehicle and the preceding vehicle partially overlap, a false detection is detected. Very likely to do.
For example, the following four examples can be given as examples where the possibility of erroneous detection is high.
(A) When there is a preceding vehicle, when an interrupting vehicle occurs (B) When there is a preceding vehicle, when a lane change of the preceding vehicle occurs (C) When there is a preceding vehicle, the preceding vehicle (D) When a preceding vehicle is present, the preceding vehicle is offset in the width direction with respect to the preceding vehicle (when there is a preceding vehicle, the lateral position of the preceding vehicle in the width direction) Is the state that is deviated from the vehicle ahead)

(A) (B) (D)
As shown in FIG. 15, the image processing unit 134 may detect an edge erroneously due to the overlap 321 with the preceding vehicle 600 by performing edge detection on the detection region 320 for the interrupting vehicle 601.
In the case of (C),
As shown in FIG. 16, the image processing means 134 can detect the edge of the preceding vehicle instead of the two-wheeled vehicle by detecting the edge of the detection region 320 for the two-wheeled vehicle 701 by overlapping 321 with the preceding vehicle 700. There is sex.

When the obstacle candidate selection unit S11 of the obstacle determination unit 135 of FIG. 8 determines that the possibility of being an object on the road is low, such as an object for which edge detection could not be performed, or when the width W is too wide. Are excluded because they are unlikely to be obstacles, but in the case of FIGS. 15 and 16, there is a high possibility that they cannot be excluded.
Therefore, in such a case, it is determined that the target is an alarm target or an avoidance target, an unnecessary alarm or braking is applied, and the performance of the millimeter wave radar single system cannot be maintained. In particular, when the alarm / control intervention is advanced according to the lap amount as in the third embodiment, the influence of malfunction due to erroneous detection becomes very large.

For this reason, in the validity determination unit 138, it is effective or invalid to add the output of the image processing unit 134 to the processing in the obstacle determination unit 135 for each detected object based on the output of the object information calculation unit 131. Determine whether.
In the third embodiment, as described above, it is estimated that the validity determination unit 138 is in a traveling state in which a preceding vehicle and a preceding vehicle partially overlap each other, which are likely to be erroneously detected by image processing with a monocular camera. The travel state estimating means 138a is provided, and if it is estimated that the travel state is such that the preceding vehicle and the preceding vehicle partially overlap each other, it is determined to be invalid, and otherwise it is determined to be valid.
For an object determined to be invalid, the validity object determination flag Fv = NULL is set to the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle. Determination flag Fo, effectiveness determination flag Fv} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle determination flag Fo, effectiveness determination flag Fv } Is added as information. The validity determination flag Fv = 1 is set for an object determined to be valid.

According to the third embodiment, as shown in FIG. 15, even when the traveling vehicle and the preceding vehicle partially overlap each other, while maintaining the performance of the conventional millimeter wave radar single system, Alarm / control intervention can be accelerated with respect to the driving state of the vehicle, and the effect of fusion with the camera can be obtained.

Embodiment 4 FIG.
Embodiment 4 of the present invention will be described below with reference to the drawings.
FIG. 17 is a diagram for explaining the relative position at which it is determined that the validity determination means of the obstacle recognition device for a vehicle according to Embodiment 4 of the present invention is invalid.
In FIG. 17, an erroneous detection area 351 for the detection target 350 is shown.

FIG. 18 is a diagram for explaining the erroneous detection of the effectiveness determination means of the vehicle obstacle recognition apparatus according to Embodiment 4 of the present invention.
In FIG. 18, the actual detection point of the vehicle 340 should be the center position of the vehicle, but there may be an end portion of the vehicle like the detection point 341. In this case, a detection area is set like the detection area 342 in the image processing means 134.
The configuration of the fourth embodiment and the example of mounting the millimeter wave radar to the vehicle are the same as those of the first embodiment.

Next, the effectiveness determination means 138 in Embodiment 4 will be described.
Since the millimeter wave radar 110 is mechanically or electrically scanned to detect a detection target, the lateral position detection accuracy generally deteriorates according to the relative distance.
Therefore, for example, as shown in FIG. 18, the actual detection point of the vehicle 340 may be the end of the vehicle 340 like the detection point 341. In this case, the image processing unit 134 sets a detection area like the detection area 342.
In such a case, the background image or other vehicle is determined as a detection target by camera image processing, and there is a high possibility of further adverse effects.

Therefore, the validity determination unit 138 determines, based on the output of the object information calculation unit 131, whether it is valid or invalid to add the output of the image processing unit 134 by the obstacle determination unit 135 for each detected object. To do.
In the fourth embodiment, as described above, if a detected object is present at a relative position where the millimeter wave radar 110 is likely to be erroneously detected, it is determined to be invalid, and otherwise, it is determined to be valid.
The relative position determined to be invalid is, for example, an erroneous detection region 351 with respect to the detection target 350 as shown in FIG.
For an object determined to be invalid, the validity object determination flag Fv = NULL is set to the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle. Determination flag Fo, effectiveness determination flag Fv} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, width W, lap amount L, obstacle determination flag Fo, effectiveness determination flag Fv } Is added as information. The validity determination flag Fv = 1 is set for an object determined to be valid.

  According to the fourth embodiment, as shown in FIG. 17, even when a detection object exists at a relative position where the millimeter wave radar is likely to be erroneously detected and there is a concern about further adverse effects due to camera image processing, While maintaining the performance of the single millimeter wave radar system, it is possible to speed up alarm and control intervention for other driving conditions, and to obtain the effect of being fused with the camera.

Embodiment 5 FIG.
Embodiment 5 of the present invention will be described below with reference to the drawings.
FIG. 19 is a flowchart showing processing of the image processing means of the vehicle obstacle recognition apparatus according to Embodiment 5 of the present invention.
The configuration of the fifth embodiment and the example of mounting the millimeter wave radar to the vehicle are the same as those of the first embodiment.
In the fifth embodiment, the image processing means and the obstacle determination means in the fourth embodiment are changed as follows.

The image processing unit 134 performs predetermined image processing based on the output of the object information calculation unit 131.
In the fifth embodiment, the image processing unit 134 identifies the detected object as a vehicle, a two-wheeled vehicle, a pedestrian, or the like.

Next, the processing of the image processing unit 134 will be described with reference to FIG.
First, based on the result of the object information calculation means 131, the detection area setting means S01 sets a detection area for performing image processing on each moving body or stationary object. For example, the detection area converts the relative position between the vehicle and the moving body obtained by the object information calculation unit 131 into image coordinates P (i), and the relative position between the stationary object and P (j) obtained by coordinate conversion. Regions R (i) and R (j) are obtained by adding a predetermined margin to the height and width of one of the largest vehicles among the detection targets.

Subsequently, the detection target identification unit S52 identifies that each moving body or stop is a vehicle, a two-wheeled vehicle, a pedestrian, or the like using, for example, pattern matching.
The identification result of the detection target is that the moving body Mv (i) = {relative distance Dis, relative lateral position Ltp, relative speed Vr, detection target ID} and stationary object St (j) = {relative distance Dis, relative lateral position Ltp, It is added as information of relative speed Vr, detection target ID}. A detection target ID = NULL is added to an object whose detection target cannot be identified.

The flowchart of the obstacle determination means 135 in the fifth embodiment is the same as that in FIG.
However, in the fifth embodiment, the following changes are made to the fourth embodiment in the avoidance target determination means S13 and the alarm target determination means S14 of FIG.

The avoidance target determination unit S13 determines that the target is an avoidance target when the predicted collision time TTC is less than a predetermined avoidance threshold value THc. Here, when the effectiveness determination flag Fv = 1, the avoidance threshold THc is determined using a predetermined fixed threshold C (ID) set for each detection target. However, when the validity determination flag Fv = NULL, the avoidance threshold THc is determined using a predetermined fixed threshold C (ID) when the detection target is a vehicle.
If there is an avoidance target, the warning means 136 is executed to inform the driver that there is an obstacle that is an obstacle to the vehicle, and the obstacle avoidance means 137 is executed to avoid the obstacle. Control the car.
If there is no avoidance target, the process proceeds to alarm target determination means S14.

The alarm target determination unit S14 determines that the target is an alarm when the predicted collision time TTC is below a predetermined alarm threshold THw. Here, when the validity determination flag Fv = 1, the alarm threshold THw is determined using a predetermined fixed threshold W (ID) set for each detection target. However, when the validity determination flag Fv = NULL, the alarm threshold THw is determined using a predetermined fixed threshold W (ID) when the detection target is a vehicle.
If there is an alarm target, the alarm means 136 is executed to notify the driver that there is an obstacle that is an obstacle to the vehicle.
If there is no alarm target, the obstacle determination means 135 is terminated.
Here, it is assumed that the fixed thresholds C (ID) and W (ID) increase in the order of pedestrian> two-wheeled vehicle> vehicle or other. As a result, warning / control intervention can be expedited for an object such as a pedestrian or a motorcycle that wants to increase the damage reduction effect at the time of collision.

  According to the fifth embodiment, in a system in which a detection target is identified and the timing of alarm / control intervention is variable for each detection target, even if a detection target is erroneously detected, the conventional millimeter wave radar single system While maintaining the performance of the camera, the effect of the fusion with the camera can be obtained.

It is a block diagram which shows schematic structure of the obstruction recognition apparatus for vehicles by Embodiment 1 of this invention. It is an external view which shows the example of attachment to the vehicle of the millimeter wave radar and camera of the obstruction recognition apparatus for vehicles by Embodiment 1 of this invention. It is a flowchart which shows the process of the obstacle recognition apparatus for vehicles by Embodiment 1 of this invention. It is a flowchart which shows the process of the image processing means of the obstacle recognition apparatus for vehicles by Embodiment 1 of this invention. It is a figure which shows the example of an image of the image processing means of the obstacle recognition apparatus for vehicles by Embodiment 1 of this invention. It is a figure explaining edge candidate extraction of the image processing means of the obstacle recognition device for vehicles by Embodiment 1 of this invention. It is a figure explaining the amount of laps of the image processing means of the obstacle recognition device for vehicles by Embodiment 1 of this invention. It is a flowchart which shows the process of the obstacle determination means of the obstacle recognition apparatus for vehicles by Embodiment 1 of this invention. It is a figure explaining the avoidance of the obstacle determination means of the obstacle recognition apparatus for vehicles by Embodiment 1 of this invention, and an alarm threshold value. It is a figure explaining the effectiveness determination means of the obstacle recognition apparatus for vehicles by Embodiment 1 of this invention. It is a block diagram which shows schematic structure of the obstruction recognition apparatus for vehicles by Embodiment 2 of this invention. It is a flowchart which shows the process of the obstacle recognition apparatus for vehicles by Embodiment 2 of this invention. It is a flowchart which shows the process of the reliability determination means of the obstruction recognition apparatus for vehicles by Embodiment 2 of this invention. It is a block diagram which shows schematic structure of the obstruction recognition apparatus for vehicles by Embodiment 3 of this invention. It is a figure explaining the misdetection by the overlap with the interruption vehicle of the effectiveness determination means of the obstacle recognition apparatus for vehicles by Embodiment 3 of this invention. It is a figure explaining the erroneous detection of the two-wheeled vehicle of the effectiveness determination means of the obstacle recognition apparatus for vehicles by Embodiment 3 of this invention. It is a figure explaining the relative position judged to be invalid of the effectiveness determination means of the obstacle recognition apparatus for vehicles by Embodiment 4 of this invention. It is a figure explaining the erroneous detection of the effectiveness determination means of the obstruction recognition apparatus for vehicles by Embodiment 4 of this invention. It is a flowchart which shows the process of the image process means of the obstruction recognition apparatus for vehicles by Embodiment 5 of this invention.

Explanation of symbols

100 vehicle obstacle recognition device 110 millimeter wave radar 120 camera 130 CPU
140 alarm device 150 automatic control device 160 vehicle speed sensor 170 yaw rate sensor 131 object detection means 132 image acquisition means 133 image processing means 134 vehicle information acquisition means 135 obstacle determination means 136 alarm means 137 obstacle avoidance means 138 effectiveness determination means 138a travel State estimation means 139 Reliability determination means

Claims (8)

In a vehicle obstacle recognition device that is mounted on a vehicle and recognizes an obstacle to the vehicle,
A detection sensor for detecting objects around the vehicle,
Object information calculation means for calculating the relative position and relative speed between the vehicle and the object based on the output of the detection sensor;
An image sensor that captures an image around the vehicle,
Image acquisition means for acquiring an image captured by the image sensor;
Image processing means for processing the image acquired by the image acquisition means based on the output of the object information calculation means;
Vehicle information acquisition means for detecting the driving state of the vehicle;
Obstacle determining means for determining the possibility that the object becomes an obstacle to the vehicle by the outputs of the object information calculating means and the vehicle information acquiring means,
And validity determination means for determining whether the output of the image processing means is valid based on the output of the object information calculation means,
The obstacle determining means uses the output of the image processing means for determining an obstacle according to the output of the effectiveness determining means,
The effectiveness determination means includes travel state estimation means for estimating a travel state of the object based on an output of the object information calculation means, and the travel state estimation means determines the travel state of the object as the travel state of the object. An obstacle recognition apparatus for a vehicle, characterized in that when it is estimated that a part of an object further ahead of a preceding object is detected, the vehicle is not valid. The running state of the running condition the object estimated by the estimating means, when the preceding vehicle is present, vehicular obstacle recognition system according to claim 1, characterized in that it contains a state in which the interrupt vehicle is generated . The running state of the running condition the object estimated by the estimating means, when there is wherever line vehicle, the preceding vehicle vehicular obstacle recognition according to claim 1, characterized in that it includes a state changing lane apparatus. The running condition of the object estimated by the running state estimating means, when there is wherever line vehicle, the preceding vehicle vehicular obstacle recognition according to claim 1, wherein the contained state is motorcycle apparatus. The traveling state of the object estimated by the traveling state estimating means includes a state in which the lateral position in the width direction of the preceding vehicle is deviated from the preceding vehicle when the preceding vehicle exists. The obstacle recognition apparatus for a vehicle according to claim 1 . Based on the time-series data of the output of the object information calculation means, comprising reliability determination means for determining the reliability of the output of the image processing means,
The obstacle determining means, in accordance with the output of the output and the reliability judging means of the validity determining unit, according to claim 1 claims, characterized in that used for the determination of the obstacle the output of said image processing means The obstacle recognition apparatus for a vehicle according to any one of 5 . The vehicle obstacle recognition according to any one of claims 1 to 6 , wherein the image processing means detects a lap amount which is a size of an overlap between the vehicle and the object. apparatus. The vehicle obstacle recognition apparatus according to any one of claims 1 to 6 , wherein the image processing means identifies the object into a plurality of types including a vehicle and a pedestrian.