JP4069916B2 - Object detection apparatus and method - Google Patents

Description

  The present invention relates to an object detection apparatus and method for detecting a moving object existing in front of a vehicle.

  The following peripheral monitoring device is known from Patent Document 1. Based on the imaging time interval between the image 1 and the image 2 ahead of the host vehicle and the speed of the host vehicle, the host vehicle captures the image 1 until the image 2 is captured. Calculate the distance traveled. Then, the estimated image obtained by projective transformation of the road surface image of image 1 by horizontal movement based on the calculated moving distance is compared with image 2, and objects that do not require detection such as paint and dirt on the road surface are deleted. Thus, these are not erroneously detected as other vehicles.

JP 2000-74645 A

  However, in the conventional apparatus, the detection unnecessary object is deleted in consideration of only the image capturing time interval and the speed of the host vehicle. There has been a problem that objects that do not need to be detected cannot be correctly deleted and these may be erroneously detected as other vehicles.

  The present invention performs image processing on a captured image in front of the host vehicle, calculates lateral speed information of an object present in the image, and based on the lateral speed information of the object present in the calculated image. Set an area containing an object with a speed direction different from the background on the image, and extracted the area set for the diagonal line on the image from the set area as a false detection area. An erroneous detection area extracted from the area is deleted, and an object included in the remaining area is determined to be a moving object existing ahead of the host vehicle.

  According to the present invention, when vertical vibration is generated in the vehicle at the time of image capture, the lateral movement speed is detected from the oblique line of the stationary object present on the image, The region set for the diagonal line is extracted as a false detection region and deleted. This makes it possible to accurately determine erroneous detection when vertical vibrations occur in the vehicle during image capture, and to delete objects that do not require detection.

  FIG. 1 is a block diagram illustrating a configuration example of an embodiment of an object detection device according to the present embodiment. The object detection device 100 is mounted on a vehicle and performs image processing on a camera 101 that captures an image in front of the vehicle, an image memory 102 that stores an image captured by the camera 101, and an image captured by the camera 101. A control device 103 that detects an existing object and a counter memory 104 that stores a count value of a pixel counter to be described later are provided.

  The camera 101 has an image sensor such as a CCD or a CMOS, for example, and is installed in front of the upper part of the interior of the vehicle so that its optical axis faces the front front direction of the vehicle. Then, the vehicle front is continuously photographed at regular time intervals Δt, and the captured image is output to the image memory 102 and stored. The control device 103 performs processing described later with reference to FIG. 2, performs image processing on the captured image in front of the host vehicle stored in the image memory 102, and detects a moving object existing in front of the host vehicle.

  FIG. 2 is a flowchart showing the operation of the object detection apparatus 100 of the present embodiment. The processing shown in FIG. 2 is executed by the control device 103 as a program that starts when an ignition switch (not shown) is turned on. In step S <b> 1, reading of a captured image captured by the camera 101 at a fixed time Δt interval and stored in the image memory 102 is started. That is, images taken at regular time intervals Δt are acquired from the image memory 102 one by one until the end of processing.

  Thereafter, the process proceeds to step S2, the lateral speed detection process described later with reference to FIG. 3 is executed to detect the lateral speed of each pixel in the captured image, and the process proceeds to step S3. In step S3, as will be described later with reference to FIGS. 4 and 5, "a screen area different from the background" is detected, and an area including an object popping toward the own vehicle ahead of the own vehicle is set. . Then, it progresses to step S4. In step S4, as will be described later with reference to FIG. 6 and FIG. 7, the vibration (pitching) of the host vehicle in the vertical direction is detected among the detected “screen area different from the background”, although it is actually stationary. The area including the object whose lateral velocity has been detected by is detected as an erroneous detection area, and the erroneous detection area is deleted.

  Thereafter, the process proceeds to step S5, and it is determined that the “screen area different from the background” remaining as a result of deleting the erroneous detection area is an area including a moving object existing ahead of the host vehicle. Thereafter, the process proceeds to step S6. In step S6, it is determined whether or not the ignition switch is turned off. If it is determined that the ignition switch is not turned off, the process returns to step S2 to repeat the above-described processing. On the other hand, if it is determined that the ignition switch is turned off, the process is terminated.

  FIG. 3 is a flowchart showing the flow of the speed detection process in the lateral direction in step S20 described above. In step S <b> 11, a lateral differential image is obtained based on the image acquired from the image memory 102. Thereafter, the process proceeds to step S12, and the edge image obtained by extracting the edge in the horizontal direction is obtained by binarizing the horizontal differential image using a predetermined threshold. After that, the process proceeds to step S13, where the edge image from which the edge in the horizontal direction is extracted is thinned to accurately determine the center of the edge, thereby accurately grasping the movement of the edge. Then, it progresses to step S14 and the thinned edge is expanded. Thereby, an edge image having a uniform width can be obtained. Thereafter, the process proceeds to step S15.

  In step S15, the pixel counter stored in the counter memory 104 is updated. The pixel counter is a counter corresponding to each pixel of the edge image, and 1 is added to the count value of the pixel counter corresponding to the pixel where the edge exists for each frame, and the pixel counter corresponds to the pixel where the edge does not exist. The count value of the pixel counter is initialized with zero. Accordingly, a pixel having a long edge existence time has a large count value, and a pixel having a short edge existence time has a small count value. After updating the pixel counter in step S15, the process proceeds to step S16.

  In step S16, it is determined whether or not the above-described processing has been executed for a predetermined number of frame images set in advance. If it is determined that the processing for the predetermined number of frame images set in advance has not been completed, the process returns to step S11 and the processing is repeated. On the other hand, if it is determined that the processing for a predetermined number of frame images set in advance has been completed, the process proceeds to step S17.

  In step S17, the time required for the edge to move by one pixel is calculated by calculating the difference in the edge existence time in each pixel by taking the difference in the count value of the pixel counter corresponding to each pixel adjacent in the horizontal direction. Get. Then, by obtaining the reciprocal of this value, the lateral speed in the image space at each pixel can be calculated. The horizontal speed in the image space at each pixel corresponds to the horizontal speed of the edge included in each pixel. Thereafter, the process returns to the process shown in FIG.

  Next, with reference to FIGS. 4 and 5 which are images obtained by capturing the front of the host vehicle, the “screen region different from the background” detection process in step S3 described above will be described. When the host vehicle is traveling straight, the background in the captured image has a speed direction radially extending from the FOE (Focus of Expansion) 4a in the image existing in the traveling direction toward the outside of the image as shown in FIG. That is, when the host vehicle is traveling straight, the white lines 6c and 7a on the road surface of the background 4b move so as to spring out from the center of the screen toward the outside of the screen. On the other hand, a pop-out object that tries to pass in front of the host vehicle, such as the other vehicle 4c, has a speed direction from the outside of the screen toward the center of the screen.

  Therefore, in the edge image for which the speed in the horizontal direction is calculated, it is determined that the pixel having the speed direction toward the center of the screen includes the edge of the protruding object. Then, as shown in FIG. 5, the edge image is divided into vertically long strips, and each strip region has a pixel having a speed direction different from the background, that is, a pixel having a speed direction from the outside of the screen toward the center of the screen. 5a to 5c are set as “screen areas different from the background”.

  Next, the erroneous detection area deletion processing in step S4 described above with reference to FIGS. 6 and 7 will be described. When the camera 101 vibrates up and down due to pitching of the host vehicle, a stationary object on the image captured by the camera 101 vibrates up and down accordingly.

  In this case, as shown in FIG. 6, the horizontal line on the image moves in the vertical direction from a position 6e on the image at ΔT seconds before the current time T to a position 6f on the image at the current time T. The horizontal movement on the image, that is, the moving speed in the horizontal direction is not detected, the stationary object is erroneously detected due to the vertical vibration of the camera 101, and the region including the stationary object is the above-described “screen different from the background”. It is not mistakenly set as “area”. Similarly, since the vertical line on the image also moves in the vertical direction from the position 6a on the image at ΔT seconds before the current time T to the position 6b on the image at the current time T, the movement speed in the horizontal direction is A region including a stationary object is not erroneously set as a “screen region different from the background” in accordance with the vertical vibration of the camera 101 without being detected.

  On the other hand, a diagonal line existing on the image, for example, a white line 6c on the road surface, is present at the current time T from the position 6c on the image at ΔT seconds before the current time T due to the vertical vibration of the camera 101. It moves like position 6d on the image. As a result, in the image, the horizontal movement is detected together with the vertical movement, and the moving speed in the horizontal direction is detected. As a result, despite the fact that the object is actually stopped, the lateral speed toward the center of the screen is erroneously detected, and the area containing the white line on the stationary road surface is the `` screen area different from the background '' It may be set incorrectly.

  In such a case, it is necessary to delete the “screen area different from the background” set based on the erroneously detected stationary object. For example, as shown in FIG. 7, it is assumed that the moving speed in the horizontal direction is detected from the other vehicle 4c on the image and the white line 7a on the road surface, and each area indicated by a bold line is set as a “screen area different from the background”. In FIG. 7, the “screen area different from the background” set based on the white line 7a on the road surface is an area erroneously set by erroneously detecting the lateral speed based on the stationary object. Therefore, these ranges are deleted as follows.

  First, an arbitrary “screen area different from the background” set on the image is selected. Next, the presence or absence of a “screen area different from the background” is confirmed in the adjacent strip-shaped area. Here, when there are “screen areas different from the background” in the adjacent strip-shaped areas, and they are continuously present (distributed) diagonally, they are present on the image as white lines 7a on the road surface. Since there is a high possibility that the horizontal speed is erroneously detected from the oblique line of the stationary object, these “screen areas different from the background” are extracted and deleted.

  On the other hand, if there is no “screen area different from the background” in the adjacent strip-shaped area, or if there are “screen areas different from the background” side by side as in the areas 5a to 5c, an error occurs. Do not delete because it is not considered to be detected. By performing this procedure for all “screen areas different from the background”, it is possible to delete “screen areas different from the background” that are erroneously set based on the diagonal lines of the stationary object existing on the image.

  In addition, as shown in FIG. 8, even when “screen areas different from the background” existing in adjacent strip-shaped areas are not obliquely continuous, these areas are also diagonal lines of still objects existing on the image. There is a possibility that the “screen area different from the background” is set erroneously based on the above. Therefore, as described below, it is assumed that each area shown in FIG. 8 is a false detection due to diagonal lines, and each “screen area different from the background” is erroneously set by examining the validity of the assumption. It is judged whether it is a thing.

First, an arbitrary “screen area different from the background” 8a set on the image is selected. A straight line y = α (x−X _V ) + Y _V passing through the lower end (X _V , Y _V ) is defined for the selected region 8a as indicated by reference numeral 9a in FIG. Α is a parameter. Here, a temporary value is set and estimated later. Next, the lower end of another “screen area different from the background” existing in the vicinity of the “screen area different from the background” 8a is used as sample data, and the data set (xi, yi) is obtained. Then, the parameter α is estimated so that the defined straight line 9a passes as close as possible to the data set (xi, yi). In this embodiment, the parameter α is estimated by the least square method using the data set (xi, yi), and the straight line 9b based on the estimated α is estimated.

The average value of the difference between the y coordinate values of the straight line 9b estimated at this time and each point in the data set (xi, yi) is calculated as an error E. If the error E is equal to or less than a predetermined threshold value _ETH , an arbitrary value is obtained. The “screen area different from the background” 8a and other “screen areas different from the background” in the vicinity thereof are areas that are erroneously set based on a stationary diagonal line approximated by the diagonal line 9b existing on the image. Therefore, these “screen areas different from the background” are extracted and deleted. In contrast, if the error E is greater than a predetermined threshold E _TH, these areas are not removed because not considered to have been set incorrectly based on the shaded stationary approximated by hatching 9b. The predetermined threshold value E _TH is set to a small value when the image resolution is high and the speed detection accuracy on the image is high, and is set to a large value when the accuracy cannot be expected so much.

According to the present embodiment described above, the following operational effects can be obtained.
(1) When the lateral movement speed of an actually stationary object is erroneously detected due to the vertical vibration of the vehicle, the “screen area different from the background” including the object is deleted and An area including an object having a high possibility of colliding is determined. As a result, it is possible to detect a moving object (a protruding object) that is present in front of the host vehicle stably regardless of the behavior of the vehicle.
(2) In deleting the “screen area different from the background” set based on the erroneously detected stationary object, the area is deleted with respect to the oblique line of the stationary object existing on the image. As described above, since only the oblique lines on the image that are likely to be erroneously detected are targeted, it is possible to efficiently delete the “screen area different from the background” that is set in error.

(3) If “screen areas different from the background” exist diagonally and continuously on the image, these “screen areas different from the background” are areas that are erroneously set based on stationary diagonal lines. Judging that there was, it was decided to delete the area. As a result, it is possible to delete an erroneously set area based on a diagonal line that is accurately stationary from the top of the image.
(4) Even if the “screen area different from the background” does not exist diagonally continuously on the image, the straight line 9b is set based on the data set at the lower end of the “screen area different from the background” Based on the error represented by the average value of the difference of the y-coordinate values between the straight line 9b and each point in the data set, these “screen areas different from the background” were erroneously set due to the erroneous detection of the diagonal lines. It was judged whether it was a thing. If it is set by mistake, the area is deleted. As a result, even a false detection area that is spatially discontinuous and has a large variation can be deleted accurately.

-Modification-
In addition, it can also deform | transform as follows.
(1) In the embodiment described above, the lateral speed of the captured image is detected by the process shown in FIG. However, the present invention is not limited to this. For example, an optical flow using a gradient method or block matching may be calculated to detect the lateral velocity of the captured image.

(2) In the embodiment described above, when detecting the “screen area different from the background”, the edge image is divided into vertically long strips, and in each strip area, a pixel having a speed direction from the outside of the screen toward the center of the screen is obtained. The existing range is set as “screen area different from background”. However, an area in which pixels having a speed direction from the outside of the screen toward the center of the screen exist at a predetermined density or more may be set as a “screen area different from the background”. Alternatively, a plurality of screen areas different from the background that have been set may be combined to set the combined area as a “screen area different from the background”.

(3) In the embodiment described above, in estimating the immediately preceding 9b, the lower end of another “screen area different from the background” existing in the vicinity of the “screen area different from the background” 8a is used as sample data, and the data set ( xi, yi). However, an arbitrary point in another “screen area different from the background” existing in the vicinity of the “screen area different from the background” 8a may be used as sample data to obtain the data set.

(4) In the embodiment described above, the parameter α is estimated by the least square method using the data set (xi, yi) in order to estimate the immediately preceding 9b. However, the present invention is not limited to this. You may estimate using an approximation method.

(5) In the above-described embodiment, when deleting the “screen area different from the background”, the average value of the difference between the y-coordinate values between the estimated straight line 9b and each point in the data set (xi, yi) is determined as an error. Although the calculation is performed as E, the error E may be calculated by other methods.

  The correspondence between the constituent elements of the claims and the embodiment will be described. The camera 101 corresponds to an imaging unit, and the control device 103 corresponds to a speed information calculation unit, a region setting unit, an erroneous detection region extraction unit, and a determination unit. Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired.

It is a block diagram which shows the structural example of one Embodiment at the time of mounting an object detection apparatus in a vehicle. FIG. 5 is a flowchart showing the operation of the object detection apparatus 100. It is a flowchart figure which shows the flow of the speed detection process of a horizontal direction. FIG. 11 is a first diagram illustrating a specific example of detection processing of “screen area different from background”. FIG. 11 is a second diagram illustrating a specific example of the detection process of “screen area different from background”. It is the 1st figure which showed the specific example of the deletion process of an erroneous detection area | region. It is the 2nd figure which showed the specific example of the deletion process of an erroneous detection area | region. It is the 3rd figure which showed the specific example of the deletion process of an erroneous detection area | region. It is the 4th figure showing the example of deletion processing of a false detection field.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Object detection apparatus 101 Camera 102 Image memory 103 Control apparatus 104 Counter memory

Claims (4)

Imaging means for capturing an image in front of the host vehicle;
Speed information calculating means for performing image processing on an image captured by the imaging means and calculating lateral speed information of an object present in the image;
Area setting means for setting an area including an object having a speed direction different from the background on the image based on the speed information in the horizontal direction of the object existing in the image calculated by the speed information calculating means;
A false detection region extraction means for extracting, as a false detection region, a region set for the oblique line on the image from the regions set by the region setting means;
The erroneous detection area extracted by the erroneous detection area extraction means is deleted from the areas set by the area setting means, and the objects included in the remaining areas are determined to be moving objects existing in front of the host vehicle. An object detecting device. The object detection apparatus according to claim 1,
The misdetection area extracting means, when a plurality of areas set by the area setting means are continuously present in an oblique direction on the image, these areas are set as areas set with respect to the oblique lines on the image. An object detection apparatus characterized by extracting. In the object detection device according to claim 1 or 2,
The erroneous detection area extraction means calculates a straight line that approximates a set of arbitrary points in each area when a plurality of areas set by the area setting means exist discontinuously on the image, If an error calculated based on the straight line and a set of arbitrary points in each area is equal to or less than a predetermined value, these areas are extracted as areas set with respect to diagonal lines on the image, Object detection device. Image processing is performed on the captured image in front of the vehicle, and lateral speed information of an object existing in the image is calculated.
Based on the calculated lateral velocity information of the object present in the image, a region including an object having a velocity direction different from the background on the image is set,
Extract the area set for the diagonal line on the image from the set area as a false detection area,
An object detection method comprising: deleting the erroneous detection area extracted from the set area, and determining that an object included in the remaining area is a moving object existing in front of the host vehicle.

Images