JP4575315B2 - Object detection apparatus and method - Google Patents

Description

  The present invention relates to an object detection apparatus and method for detecting an approaching object from an image captured by an image capturing apparatus attached to a moving body such as an automobile in a rearward direction.

  Conventionally, a method using one video camera has been proposed as a method for detecting an obstacle around a moving object such as an automobile. The method using one video camera is advantageous in that it can be easily installed in an automobile or the like and the production cost can be reduced as compared with a method using a plurality of video cameras.

  As a method using a single video camera, a method of detecting a forward vehicle by determining a horizontal edge in an image taken by a camera installed in an automobile as a ground line of the forward vehicle has been proposed (patent). Reference 1).

  In addition, the optical flow is detected from the time-series images taken by the video camera, and the height from the road surface of each flow is estimated using the own vehicle speed information from the vehicle speed sensor. There has been proposed a method for detecting only the object it has (see Patent Document 2).

Furthermore, a method has been proposed in which a tracking area that does not include a road surface is set and an obstacle such as a three-dimensional object is detected only within the tracking area (see Patent Document 3).
JP 7-280517 A JP 2000-123183 A JP-A-8-147599

  However, in the method of detecting a horizontal edge in Patent Document 1, there is a possibility that a sign on the road surface, an asphalt joint, or the like is determined as a ground contact position of a preceding vehicle and erroneous object detection is performed.

  Further, in the method of detecting an optical flow in an image of Patent Document 2, since it is assumed that all the flows in the road surface are stationary, the shadow of another vehicle passing through the tracking area is used as an obstacle. There is a problem of false detection.

  The problems of Patent Documents 1 and 2 are caused by the fact that the tracking area of the object includes a road surface. In this case, it is necessary to distinguish whether the detected horizontal line or optical flow is a road surface or a three-dimensional object.

  On the other hand, in the method of setting a tracking region that does not include a road surface in Patent Document 3, a three-dimensional object is detected from motion information only in the vicinity of a preset vanishing line (infinity line). In the vicinity of the vanishing line, since feature quantities located below the camera position such as road patterns and shadows are not detected, it is possible to suppress erroneous detection caused by a method of setting a tracking area including a road surface. However, in this method, the left and right edges of the object are estimated from only the horizontal motion vector near the vanishing line, and it is determined whether the object is approaching or moving away from only the motion vectors at both ends. It is difficult to stably detect an obstacle in a scene with a complicated background where both ends of the image cannot be cut out well.

  The present invention has been made in view of the above, and proposes an object detection apparatus and method capable of more reliably detecting an object even if the image is captured by a single image capturing apparatus. Objective.

The present invention relates to an image input means for inputting a time-series image acquired by an imaging device attached backward to a moving body moving on a surface, and a specific direction within a tracking region along the vanishing line in the image. Object candidate tracking means for extracting a line segment having a component, setting an object candidate by searching around the line segment, and specifying the position of the object candidate for each frame of the time-series image; and the object An enlargement ratio calculation means for calculating an enlargement ratio of the object candidate in at least one frame before and after using an enlargement ratio calculation area that includes a candidate and is perpendicular to the erasure line, and that exceeds a first threshold An object detection apparatus comprising: an object detection unit configured to detect the object candidate having an enlargement ratio as an object approaching the moving body.
The present invention also provides an image input means for inputting a time-series image acquired by a photographing device attached backward to a moving body that moves on a surface, and an object in a tracking region along the vanishing line in the image. A candidate is set, and object candidate tracking means for specifying the position of the object candidate for each frame of the time-series image is selected, and only an object candidate having a movement vector in a direction emanating from a vanishing point in the image is selected, An enlargement factor calculating means for calculating an enlargement factor of the object candidate in at least one frame before and after using an enlargement factor calculation region that includes the selected object candidate and is perpendicular to the vanishing line; An object detection apparatus comprising: an object detection unit configured to detect the object candidate having the enlargement ratio exceeding a threshold value as an object approaching the moving body.

  According to the present invention, it is possible to determine whether or not an object is approaching by examining the enlargement ratio of the object candidate.

  Hereinafter, the object detection apparatus 10 of the embodiment of the present invention will be described.

  In the present embodiment, rear side monitoring, that is, detection of an overtaking vehicle traveling in an adjacent lane of the host vehicle 1 will be described as an example. As shown in FIG. 2 and FIG. 3, the TV camera 2 is mounted with the road surface facing downward in the image, and the optical axis is mounted in the vicinity of the side mirror or in the side mirror with a certain angle in the road surface and the traveling direction. And The same method can be applied to other installation methods. When the TV camera 2 rotates in the roll direction, the same method can be applied by correcting the infinity line (disappearance line) on the road surface to be horizontal in the image. In the following, it is assumed that the vertical direction in the image coincides with the vertical direction of the real space, and the infinity line on the road surface in the image is simply referred to as “disappearance line”.

  FIG. 1 shows a configuration example of the object detection apparatus 10 of the present embodiment.

  The object detection apparatus 10 includes an image input unit 12, an object candidate tracking unit 14, an enlargement ratio calculation unit 16, and an object detection unit 18. Each function of each part 12-18 is realized by a program stored in a computer.

(1) Image input unit 12
First, a time series image is acquired by the TV camera 2 in the image input unit 12. In this case, in the present embodiment, an image as shown in FIG. 4 is obtained.

(2) Object candidate tracking unit 14
The object candidate tracking unit 14 sets a tracking region of an object candidate having a predetermined height near the vanishing line as shown in FIG. As can be seen from FIG. 4, an object having a height higher than the installation position of the TV camera 2 always moves in the horizontal direction within this tracking area, and road signs such as white lines and shadows of other vehicles are in this area. There is no movement, and the movement on the road surface and the movement of the three-dimensional object can be easily separated.

  The object candidate tracking unit 14 tracks a vertical line segment in the tracking area. Since the movement in the tracking area is dominant in the horizontal direction, the vertical line segment is detected and tracked in order to accurately track these movements.

(2-1) Detection of Vertical Lines Various methods are applicable as a method for detecting vertical line segments. The method of this embodiment will be described.

  First, a vertical Sobel filter is applied to the tracking region to obtain a vertical edge dv (x, y) at a point (x, y) in the image.

  Next, a vertical edge is added in the y direction in the tracking region using the following formula to obtain an addition edge sv (x).

sv (x) = Σdv (x, y)

The vertical range to which dv (x, y) is added is the height of the object candidate tracking area, and the horizontal range is about ± 1 pixel of the target x coordinate. Thereby, it is possible to give a certain degree of freedom in the direction of the detected vertical line segment.

  Next, the peak of the addition edge sv (x) is searched, and a vertical line segment is set to the x coordinate where the peak exists.

  This vertical line segment is obtained in the object candidate tracking area for each frame, and will be used as a search area when object candidates are to be matched between frames in the future.

  FIG. 5 shows the detected vertical line segments. 6 represents a vertical line segment in which a vertical line in the object candidate tracking area in FIG. 5 is detected. In addition, since the own vehicle is reflected depending on the installation direction of the TV camera 2 like the vehicle side at the right end in the figure, it is assumed that the entire processing of the own vehicle portion is not performed within the object candidate tracking area.

(2-2) Setting of rectangular area A rectangular area having a predetermined width and height is set around the vertical line segment obtained as described above, and this is set as an object candidate.

  A suitable value should be set as appropriate for the size of the object candidate. In the present embodiment, the width is set to about half of the object candidate tracking area based on the height.

(2-3) Object Candidate Tracking Next, object candidates are tracked in time series (tracking processing) to obtain horizontal motion of the object candidates.

  Although various methods exist for tracking, in the present embodiment, tracking is performed on dv (x, y) obtained for the detected object candidate. In the tracking, the rectangular region of the object candidate one frame before is used as a template, and template matching is performed only around the x coordinate of the horizontal line segment detected in the current frame. In this embodiment, the SAD (Sum of Absolute Difference) in the frame before and after the vertical edge dv (x, y) is calculated, and the x coordinate with the smallest SAD is updated to the x coordinate of the object candidate in the current frame. FIG. 6 shows object candidates obtained as described above.

(2-4) Calculation of Movement Vector The object candidate tracking unit 14 holds the position of the object candidate for each frame for a certain period of time so that the motion vector can be calculated as appropriate. As can be seen from the movement vector in FIG. 6, the background has a movement that converges to the vanishing point, and the overtaking vehicle has a movement that springs from the vanishing point. That is, it is highly possible that an object candidate having a movement that springs from the vanishing point is an overtaking vehicle.

(2-5) In the case of curve driving However, when driving on a curve as shown in FIG. 7, the background has a movement that springs from the vanishing point in the same way as an overtaking vehicle. Since it is difficult to separate the background, the enlargement factor calculation unit 16 calculates the vertical enlargement factor for each object candidate, and the object detection unit 18 determines whether the object candidate is approaching. Judgment is made.

(2-6) Modified Example The extraction of the vertical line segment and the tracking method in the object candidate tracking unit 14 are not limited to the present embodiment.

  For example, the Okada, Onoguchi, `` Monocular image processing system for low-speed inter-vehicle control, '' PRMU2002-140, pp.-69-74, 2002, a method for extracting and tracking a horizontal line segment is proposed. By using the vertical line segment extraction and tracking in this embodiment, it is possible to extract and track the vertical line segment stably and at high speed.

(3) Enlargement rate calculation unit 16
The enlargement factor calculation unit 16 calculates the enlargement factor in the vertical direction for each object candidate being tracked by the object candidate tracking unit 14. When the object candidate expands in the vertical direction, it can be determined that the vehicle is an overtaking vehicle. When the object candidate is reduced, it can be determined that the object candidate is a background or a vehicle moving away from the host vehicle.

(3-1) Calculation method of enlargement ratio A method of calculating the enlargement ratio of the present embodiment will be described.

  First, in the present embodiment, an enlargement ratio calculation region as shown in FIG. 8 including an object candidate and extending from the lower end to the upper end of the image is set.

  Next, as shown in FIG. 8, the horizontal edge is calculated within the enlargement factor calculation area at time t, the peak of the horizontal edge is calculated in the same manner as the object candidate tracking unit 14, and the horizontal line segment is extracted.

  Next, tracking is performed in the vicinity of the y coordinate of the horizontal line segment obtained at time t in the enlargement ratio calculation region at time t + 1, and the corresponding point in the vertical direction between time t and time t + 1 is obtained. Tracking is performed on a horizontal edge image. A plurality of horizontal lines in the enlargement ratio calculation area in FIG. 8 represent the horizontal lines extracted by the enlargement ratio calculation section 16, and a plurality of arrows between the enlargement ratio calculation areas at time t and time t + 1 indicate the extracted horizontal line segments. It represents the correspondence between the two.

  Next, two horizontal line segments are selected from the corresponding horizontal line segments, the vertical distance between time t and time t + 1 is calculated, and the distance dt + 1 at time t + 1 is divided by the distance dt at time t. To calculate the enlargement ratio.

  Note that the y-coordinate at each time is held for the corresponding horizontal line segment as in the object candidate tracking so that the history can be referred to as appropriate.

  Thereby, even when the distance between the horizontal lines does not change in one frame, the enlargement ratio can be calculated retrospectively to any past time. When a plurality of horizontal line segments are detected and tracked as shown in FIG. 8, a plurality of enlargement rates of two horizontal lines are also calculated depending on how to select them. In this embodiment, when a plurality of enlargement rates are calculated for one object candidate, the average value is calculated and determined as the enlargement rate of the object candidate.

  Without calculating the enlargement ratio, it is possible to determine that the object candidate belongs to the overtaking vehicle when the horizontal line segment in the object candidate has a downward movement in the image. Due to the vibration, all horizontal line segments of the object candidate have a downward movement, and there is a possibility of erroneous determination that they are approaching despite being in a parallel running state. Since the enlargement ratio does not change with respect to the vibration component in the pitching direction, no such erroneous determination is made.

(3-2) Modification 1
A single enlargement ratio calculation region may include a plurality of horizontal line segments due to three-dimensional objects such as passing vehicles and horizontal line segments due to road surface signs. At this time, if the average of all enlargement rates is taken, the correct enlargement rate cannot be calculated.

  Therefore, in the enlargement ratio calculation area, only horizontal line segments that have moving components in the same direction are grouped, the enlargement ratio is calculated only from the horizontal line segments that belong to the same group, and the average is taken to obtain a more accurate enlargement ratio. Can be calculated.

  In this case, there are a plurality of groups (one belonging to a three-dimensional object, one belonging to a road surface), and the enlargement ratio of the group located at the upper part of the image is determined as the enlargement ratio of the object candidate. This is because the group belonging to the moving object is observed at the upper part of the image than the group belonging to the road surface.

(3-3) Modification 2
In this embodiment, the enlargement ratio calculation area image is a rectangular area extending from the lower end to the upper end. However, for example, the enlargement ratio calculation area can be limited to a predetermined length from the disappearance line.

  As a result, it is possible to reduce the number of horizontal lines of multiple objects that enter the enlargement ratio calculation area, calculate the enlargement ratio more stably, and reduce the calculation cost of the enlargement ratio calculation. is there.

(3-4) Modification 3
In addition to the method of tracking a plurality of horizontal segments as described above, various methods can be applied to calculate the enlargement ratio.

  For example, using the enlargement factor calculation area at time t in the figure as a template, an expandable image stretched in the vertical direction at a plurality of enlargement factors around the disappearance line is generated.

  Next, the degree of similarity between the generated stretched image and the enlargement ratio calculation area at time t + 1 is calculated, and the enlargement ratio of the stretchable image having the highest similarity may be set as the enlargement ratio of the object candidate currently focused on. Is possible. For example, SAD between two regions can be used for calculating the similarity.

(3-5) Modification 4
It is not necessary to calculate the enlargement ratio for all object candidates. For example, if only an overtaking vehicle is desired to be detected, higher-speed processing is possible by calculating the enlargement rate only for object candidates that have a movement that springs from the vanishing point.

(4) Object detection unit 18
The object detection unit 18 extracts only an object approaching the host vehicle from the object candidates based on the motion of the object candidate detected by the object candidate tracking unit 14 and the enlargement rate of the object candidate calculated by the enlargement rate calculation unit 16. .

(4-1) Object Detection Method FIG. 9 shows the flow of object detection.

  First, the object candidate tracking unit 14 extracts only object candidates having a movement that springs from the vanishing point from the object candidates being tracked. As described above, it cannot be distinguished whether the vehicle is an approaching vehicle or the background at the time of a curve only by the horizontal motion information of the object candidate.

  Next, when the enlargement ratio of the extracted object candidate is larger than 1, it is determined that the object is expanding, and the vehicle is determined to be an overtaking vehicle.

  Next, when it is smaller than 1, it is determined that the vehicle is reduced, and it is determined that the background is far away from the host vehicle or the vehicle is stopped.

  If neither of the above applies, it is determined that the extracted object candidate is a parallel running vehicle that moves at a constant speed in the traveling direction of the host vehicle.

(4-2) Modification Example Although the method for calculating the enlargement ratio for each frame and detecting the approaching vehicle from the object candidates has been described above, the object detection unit 18 is not limited to this.

  For example, the enlargement ratio calculated for each frame can be added as a cumulative evaluation value in time series, and it can be determined as an overtaking vehicle only in a frame where the cumulative evaluation value exceeds a threshold value.

  Thereby, even when the tracking of the horizontal line segment is unstable, it is possible to stably extract only the overtaking vehicle from the object candidates.

  The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist thereof.

1 is a block diagram showing a configuration of an object detection apparatus according to a first embodiment of the present invention. The figure for demonstrating the attachment position of the imaging device to the motor vehicle in which the said object detection apparatus is installed. It is a figure for demonstrating the imaging | photography area | region by the said imaging device. It is a figure which shows an example of the image image | photographed by the said imaging device and input into the said object detection apparatus. It is a figure which shows the vertical line segment set in the object candidate tracking area | region by the said object candidate tracking part. It is a figure which shows the object candidate detected by the said object candidate tracking part, and its motion vector. It is a figure which shows the mode at the time of the curve driving | running | working of the motion vector of the object candidate detected by the said object candidate tracking part. It is a figure which shows the expansion rate calculation area | region in the said expansion rate calculation part. It is a block diagram which shows the determination method of the object in the said object detection part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object detection apparatus 12 Image input part 14 Object candidate tracking part 16 Magnification ratio calculation part 18 Object detection part

Claims (13)

An image input means for inputting a time-series image acquired by a photographing device attached rearward to a moving body moving on the surface;
A line segment having a specific direction component is extracted in a tracking region along the disappearance line in the image, and an object candidate is set by searching around the line segment, for each frame of the time-series image. Object candidate tracking means for specifying the position of the object candidate;
An enlargement ratio calculation means for calculating an enlargement ratio of the object candidate in at least one frame before and after using an enlargement ratio calculation area that includes the object candidate and is perpendicular to the disappearance line;
Object detection means for detecting the object candidate having the enlargement ratio exceeding a first threshold as an object approaching the moving body;
An object detection apparatus comprising: An image input means for inputting a time-series image acquired by a photographing device attached rearward to a moving body moving on the surface;
Object candidate tracking means for setting an object candidate in a tracking region along the disappearance line in the image and specifying the position of the object candidate for each frame of the time-series image;
Select only object candidates having a movement vector in the direction of boiling from the vanishing point in the image, using the enlargement ratio calculation region that includes the selected object candidate and is perpendicular to the vanishing line, An enlargement ratio calculating means for calculating an enlargement ratio of the object candidate in at least one frame before and after;
Object detection means for detecting the object candidate having the enlargement ratio exceeding a first threshold as an object approaching the moving body;
An object detection apparatus comprising: The enlargement ratio calculating means includes
A rectangular area that includes the object candidate and extends from the top to the bottom of the image is set as the enlargement ratio calculation area.
The object detection apparatus according to claim 1, wherein an enlargement ratio in the vertical direction of the object candidate is calculated using the enlargement ratio calculation area. 3. The object detection unit according to claim 1, wherein the object detection unit determines that the object candidate having the enlargement ratio smaller than a second threshold is a background or an object having a movement away from the moving body. Object detection device. The object detection means determines that the object candidate having the enlargement ratio between the first threshold value and the second threshold value is an object running in parallel with the moving object. The object detection apparatus according to claim 1 or 2. On the computer,
An image input function for inputting a time-series image acquired by a photographing device attached backward to a moving body that moves on the surface;
A line segment having a specific direction component is extracted in a tracking region along the disappearance line in the image, and an object candidate is set by searching around the line segment, for each frame of the time-series image. An object candidate tracking function for identifying the position of the object candidate;
An enlargement ratio calculation function for calculating the enlargement ratio of the object candidate in at least one frame before and after using the enlargement ratio calculation area that includes the object candidate and is perpendicular to the vanishing line;
An object detection function for detecting the object candidate having the enlargement ratio exceeding a first threshold as an object approaching the moving body;
Object detection program for realizing On the computer,
An image input function for inputting a time-series image acquired by a photographing device attached backward to a moving body that moves on the surface;
An object candidate tracking function that sets an object candidate in a tracking region along the vanishing line in the image and identifies the position of the object candidate for each frame of the time-series image;
Select only object candidates having a movement vector in the direction of boiling from the vanishing point in the image, using the enlargement ratio calculation region that includes the selected object candidate and is perpendicular to the vanishing line, An enlargement ratio calculation function for calculating the enlargement ratio of the object candidate in at least one frame before and after,
An object detection function for detecting the object candidate having the enlargement ratio exceeding a first threshold as an object approaching the moving body;
Object detection program for realizing The enlargement ratio calculation function is
A rectangular area that includes the object candidate and extends from the top to the bottom of the image is set as the enlargement ratio calculation area.
The object detection program according to claim 6, wherein an enlargement ratio in the vertical direction of the object candidate is calculated using the enlargement ratio calculation area. The object detection function determines that the object candidate is a background or an object having a movement away from the moving object when the enlargement ratio is smaller than a second threshold value. Object detection program. The object detection function determines that the object candidate is an object running in parallel with the moving object when the enlargement ratio is between the first threshold value and the second threshold value. The object detection program according to claim 6 or 7. A computer inputs a time-series image acquired by a photographing device attached backward to a moving body that moves on the surface,
A line segment having a specific direction component is extracted in a tracking region along the disappearance line in the image, and an object candidate is set by searching around the line segment, for each frame of the time-series image. The position of the candidate object is identified by the computer;
Using the enlargement factor calculation area that includes the object candidate and is perpendicular to the vanishing line, the enlargement factor of the object candidate is calculated by the computer in at least one frame before and after,
The object detection method, wherein the computer determines that the object candidate is an object approaching the moving body when the enlargement ratio is equal to or greater than a first threshold. A computer inputs a time-series image acquired by a photographing device attached backward to a moving body that moves on the surface,
Setting an object candidate in a tracking region along the vanishing line in the image, and specifying the position of the object candidate for each frame of the time-series image by the computer;
Select only object candidates having a movement vector in the direction of boiling from the vanishing point in the image, using the enlargement ratio calculation region that includes the selected object candidate and is perpendicular to the vanishing line, Calculating the enlargement ratio of the object candidate by at least one frame before and after by the computer;
The object detection method, wherein the computer determines that the object candidate is an object approaching the moving body when the enlargement ratio is equal to or greater than a first threshold. When calculating the enlargement ratio by the computer,
A rectangular area that includes the object candidate and extends from the top to the bottom of the image is set as the enlargement ratio calculation area.
The object detection method according to claim 11, wherein an enlargement ratio in the vertical direction of the object candidate is calculated using the enlargement ratio calculation area.

Images