JP3820995B2 - Obstacle detection device and obstacle detection method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that detects an obstacle on a road ahead from an image captured from one camera, and in particular, an obstacle for identifying whether a captured object is a three-dimensional object or a planar object. The present invention relates to an object detection device and an obstacle detection method.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a technique has been developed in which a camera is mounted on a vehicle to capture the vehicle traveling direction and detect an obstacle on the road. In order to determine whether the imaged object is an obstacle, it is important to identify whether the object is a three-dimensional object or a planar object. Many methods have been researched to identify whether a captured object is a three-dimensional object or a planar object, but image processing to identify with only one camera is more costly and processing-intensive than using multiple cameras. There are many advantages.
[0003]
Japanese Patent Laid-Open No. 10-222679 discloses a technique for identifying whether an object imaged using one camera is a three-dimensional object or a planar object. This is effective as a method for detecting a moving object on the road. However, since pattern matching (for all images) of images taken at times T ₀ and T ₁ is performed, an object that has a large amount of processing and becomes an obstacle is detected. to the problem that it takes time to extract the time _"T 0", the _{"T 1"} at captured image projected on the planar image, the time _{"T 0"} planar image and time _"T 0 -T ₁ of By calculating the predicted plane image at time “T ₁ ” from the vehicle travel distance between “the” and comparing the predicted plane image with the plane image at time “T ₁ ”, a solid object or a plane object, that is, an obstacle is obtained. It is possible to quickly identify whether or not it can be.
[0004]
That is, in the two planar images taken from different viewpoints, the three-dimensional object has a different shape and the planar object has the same shape, and the amount of movement of the host vehicle is used and the planar image at time “T ₀ ” and the time “T The feature of enabling comparison in pixel units by matching the position of the ₁ ″ planar image is used to quickly identify whether the object is a three-dimensional object or a planar object.
[0005]
[Problems to be solved by the invention]
However, in such a conventional technique, the predicted plane image at the time “T ₁ ” is calculated from the plane image at the time “T ₀ ” using only the movement amount of the own vehicle, so that the object to be determined moves. , time and predicted planar images "T _1", since the resulting different position of the object in the plane image at the time "T _1", can not be compared in pixel units.
[0006]
Therefore, it is necessary to align the object, and in order to do this, the amount of movement of the object to be determined between time “T ₀ -T ₁ ” is calculated, and the position of the object on the predicted plane image at time “T ₁ ” is calculated. A separate process such as correction is required. In other words, when detecting a moving object, it is necessary to add a process for calculating the amount of movement of the object and a process for correcting the object position on the planar image, and the processing load increases accordingly. was there.
[0007]
The present invention has been made in view of the above-described problems of the prior art, and even when an object is moving, whether the object is a three-dimensional object or a planar object, that is, whether it can be an obstacle is quickly identified. An object of the present invention is to provide an obstacle detection device and an obstacle detection method.
[0008]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, in the obstacle detection device according to claim 1, an imaging unit that is mounted on a vehicle and images the front of the host vehicle, a vehicle behavior detection unit that detects a traveling state of the host vehicle, Object extraction means for extracting an object on a road from an image obtained by the imaging means, image position of the object extracted by the object extraction means, and vehicle behavior obtained by the vehicle behavior detection means Based on the object distance calculation means for calculating the distance to the object, object image size calculation means for calculating the vertical image size of the object extracted by the object extraction means, and the object extraction Object identifying means for identifying the object extracted by the means between image frames, and a vertical image size with respect to the distance for each object extracted by the object extracting means. The object information holding means that holds the image and the change in the vertical image size with respect to the distance of all the objects held by the object information holding means is close to either the three-dimensional object model curve or the planar object model curve And a three-dimensional object plane object identifying means for identifying whether the object is a three-dimensional object or a plane object.
[0009]
Further, in claim 2, in the obstacle detection device according to claim 1, the object distance calculation means includes a vehicle pitching angle detected by the vehicle behavior detection means and an object extracted by the object extraction means. The distance from the vertical image position of the object to the object on the road is calculated.
[0010]
In the obstacle detection device according to any one of claims 1 and 2, the three-dimensional object plane object identification unit may detect a three-dimensional object when the object is determined to be a three-dimensional object. The height of the three-dimensional object is calculated from a three-dimensional object model curve obtained by modeling a change in the image size with respect to the distance.
[0011]
According to a fourth aspect of the present invention, the object on the road is extracted from the first step of detecting the vehicle behavior of the host vehicle based on the running state of the host vehicle and the image obtained by the imaging means mounted on the vehicle. The distance from the own vehicle to the object is calculated based on the second step, the image position of the object extracted in the second step, and the vehicle behavior obtained in the first step. A third step; a fourth step for calculating a vertical image size of the object extracted in the second step; and the object extracted in the second step is identified between image frames. A fifth step, a sixth step that holds a change in the vertical image size with respect to the distance for each object extracted in the second step, and all that are held by the sixth step. Against the distance of the object And determining whether the change in the vertical image size is close to a three-dimensional object model curve or a planar object model curve, and identifying whether the object is a three-dimensional object or a planar object, It is characterized by that.
[0012]
【The invention's effect】
According to the first aspect of the present invention, the change in the vertical image size with respect to the distance of the object is observed, and this data is a three-dimensional object that models the change in the vertical image size with respect to the distance of the three-dimensional object. Since the model and the plane object model that modeled the change in the vertical image size with respect to the distance of the plane object, it is possible to identify whether the object is a solid object or a plane object. Even when the object moves, it can be identified whether it is a three-dimensional object or a planar object, that is, whether it can be an obstacle.
[0013]
Further, according to the invention of claim 2, since the camera visual axis direction at the time of imaging is known from the detected vehicle behavior and the direction of the visual axis at the time of camera installation, the distance to the object on the road can be determined even when the vehicle is shaken. It can be measured.
[0014]
According to the invention of claim 3, since the three-dimensional object model is obtained by modeling the change in the vertical image size with respect to the distance according to the height of the three-dimensional object, the distance between the object identified as the three-dimensional object The height of the three-dimensional object can be calculated from the data of the change in the vertical image size with respect to.
[0015]
According to the invention of claim 4, the change in the vertical image size with respect to the distance of the object is observed, and this data is a three-dimensional object model in which the change in the vertical image size with respect to the distance of the three-dimensional object is modeled. Whether the target object is a three-dimensional object or a planar object is identified based on whether the object is closer to a planar object model that models a change in the vertical image size with respect to the distance of the planar object. Therefore, even when the object moves, it is possible to identify whether the object is a three-dimensional object or a planar object, that is, whether it can be an obstacle.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
<< Distance of 3D object, plane object-Explanation of size calculation formula >>
First, prior to the detailed description of the obstacle detection apparatus according to the present invention, a three-dimensional object distance-size calculation formula, a plane object distance-size calculation formula, and a target object image used in the obstacle detection apparatus according to the present invention are three-dimensional objects. A method for determining whether the image is a flat image or a flat object will be described.
[0017]
<Distance of three-dimensional object-Size calculation formula>
When a three-dimensional object having a distance “L” and a height “h _t ” is picked up from a camera installed in a place where the visual axis direction is “θ _m ” and the height is “h _s ” as shown in FIG. If the look angle with respect to the upper part of the three-dimensional object is “θ _tu ” and the look angle with respect to the lower part of the three-dimensional object is “θ _tl ”, the following equation is established.
[0018]
h _t = L · tan θ _tl −L · tan θ _tu (1)
When the instantaneous viewing angle (angle per pixel) of the sensor is “θ _s ”, the look-out angles “θ _tu ” and “θ _tl ” and the pixel coordinates “Y _tl ” and “Y _tu ” In the meantime, the following equation holds.
[0019]
θ _tl = θ _s · Y _tl (2)
θ _tu = θ _s · Y _tu (3)
In the case of an in-vehicle camera, since the camera installation height is low and the imaging target is far away, “L >> h _s between the distance“ L ”from the camera and the height“ h _t ”of the three-dimensional object. Therefore, the approximation shown in the following equation can be performed.
[0020]
tanθ = θ (4)
Here, substituting Equations (2) and (3) into Equation (1), eliminating “θ”, and performing the approximation shown in Equation (4),
h _t = L · θ _s (Y _tl −Y _tu ) (5)
When this equation (5) is rearranged, the following equation is obtained.
[0021]
Y _tl −Y _tu = h _t / (L · θ _s ) (6)
<Plane object distance-size calculation formula>
Further, when a plane object having a distance “L” and a depth “d _t ” is imaged from a camera installed in a place where the visual axis direction is “θ _m ” and the height is “h _s ” as shown in FIG. If the look angle with respect to the _{vicinity of} the flat object is “θ _tn ” and the look angle with respect to the lower part of the flat object is “θ _tf ”, the following equation is established.
[0022]
tanθ _tn = h _s / L (7)
tanθ _tf = h _s / (L + d _t ) (8)
If the instantaneous viewing angle (angle per pixel) of the sensor is “θ _s ”, the look-out angles “θ _tn ” and “θ _tf ” and the pixel coordinates “Y _tn ” and “Y _tf ” In the meantime, the following equation holds.
[0023]
θ _tn = θ _s · Y _tn (9)
θ _tf = θ _s · Y _tf (10)
In the case of an in-vehicle camera, since the camera installation height is low and the imaging target is far away, “L >> h _s between the distance“ L ”from the camera and the height“ h _t ”of the three-dimensional object. "Holds, and the approximation shown in the following equation can be performed.
[0024]
tan θ = θ (11)
Here, after substituting the equation (9) into the equation (7) and substituting the equation (10) into the equation (8), the approximation shown in the equation (11) is performed, and both sides are subtracted.
(Y _tn −Y _tf ) · θ _s = (h _s · d _t ) / {L · (L + d _t )} (12)
When this equation (12) is rearranged, the following equation is obtained.
[0025]
(Y _tn −Y _tf ) = (h _s · d _t ) / {θ _s · L · (L + d _t )} (13)
<Method for Determining Whether the Object Image is a Solid Object Image or a Plane Object Image>
Then, in the obstacle detection device according to the present invention, the image of the same object is specified from each image frame obtained by the camera, and the object image of each image frame is expressed by the equation (6). Applying a distance-stereoscopic size calculation model when the object image is a three-dimensional object image and a distance-stereoscopic size calculation model when the object image shown in the equation (13) is a flat object image, It is checked which distance-size calculation model holds through each image frame, and based on the check result, it is determined whether the object image is a solid object image or a planar object image.
[0026]
<Description of form>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0027]
FIG. 1 shows the configuration blocks of this embodiment. As shown in FIG. 1, the obstacle detection device 18 according to the present invention includes a camera 1, a sensor 2, a microcomputer 3, a memory 4, a vehicle control device 5, and an alarm device 6.
[0028]
The camera 1 is provided in the front part of the vehicle and images the front of the vehicle. The sensor 2 is composed of a gyro and the like, and detects the running state (pitch angle, etc.) of the vehicle. The microcomputer 3 processes the image captured by the camera 1 and the vehicle running state data detected by the sensor 2, and stores the distance and image size for each captured object in the memory 4. When the object comes close, all the sampling data of the object is read from the memory 4 and it is output whether it is a three-dimensional object or a flat object, that is, whether an obstacle is close. The vehicle control device 5 performs a brake operation or the like based on the output result, and the alarm device 6 issues an alarm such as a vehicle collision based on the output result.
[0029]
FIG. 2 shows a processing block. This is a functional representation of the main part of the building block. When compared with the building block, the camera 1 is the imaging means 10, the sensor 2 is the vehicle behavior detecting means 11, the microcomputer 3 is the object detecting means 12, and the object. The object image size calculating unit 13, the object distance calculating unit 14, the object identifying unit 15, the countermeasure object information holding unit 16, and the three-dimensional object plane object identifying unit 17 are equivalent to each other.
[0030]
FIG. 3 shows an example of the imaging operation of the camera 1. First, FIG. 3 shows a case where the distance to the vehicle ahead is measured by one camera 1, and the camera 1 is determined from the camera installation line-of-sight direction angle with respect to the vehicle at the time of installation and the vehicle pitch angle at the time of imaging input from the sensor 2. The direction direction angle with respect to the road surface is calculated, and the object azimuth angle for viewing the object is calculated from the vertical image position of the object input from the microcomputer 3, the angle of view of the camera 1, the image size, and the visual axis direction angle. The The distance of the object on the road can be calculated from this object azimuth and the height from the road to the camera 1.
[0031]
For example, when a three-dimensional object is imaged as shown in FIG. 6 used in the above description, an object in front is imaged from the in-vehicle camera 1, so that “tan θ = θ” from the condition when the object azimuth is small. From the equation (6), a three-dimensional object distance-vertical image size model can be expressed by the following equation.
[0032]
Y _b = (h _t · Φ) / (L · Y _max ) (14)
However, Y _b : Vertical image size h _t of the target object: Height of the three-dimensional object Φ: Angle of view L of the camera: Distance Y _max : Total vertical image size When a plane object is imaged as shown in FIG. From the equation (13), a model of the distance between the planar object and the vertical image size can be expressed by the following equation.
[0033]
Y _f = (h _s · d _t · Φ) / {L · (L + d _t ) · Y _max } (15)
However, _Yf : Vertical image size hs of the object _hs : Camera installation height _dt : Depth of flat object Φ: Camera angle of view L: Distance _Ymax : All vertical image size FIG. A processing flow performed by the obstacle detection apparatus 1 according to the basic principle is shown.
[0034]
S1 to S4 are processing flows for extracting a target object for one image input by the camera 1 and calculating a position, a vertical image size, and a distance for each target object. In S1, it is a process for determining whether or not an object that has not been extracted remains in the image. If it remains, the remaining extraction process after S2 is performed, and if it does not remain, the identification process in S5 is performed. . In S2, image edge processing is performed and an image position of an object having the same shape as the object is output. In S3, the vertical image size is output by calculating the upper and lower edge intervals based on the position information output in S2. In S4, the distance to the object is calculated using the position information output in S2 and the own vehicle pitching angle input from the sensor 2.
[0035]
S5 to S8 are identifications for associating a plurality of objects extracted from the current frame image with a plurality of objects extracted from all frame images and outputting link information between objects determined to be the same It is a processing flow. In S5, it is a process for determining whether or not there is an object that has not been subjected to the identification process extracted in the previous frame. If it remains, the remaining identification process after S6 is performed, and if it does not remain, the process of S9 is performed. Perform the retention process. In S6, similarly, the loop processing of the object extracted in the current frame is performed. In S7, an identification determination is performed focusing on the positions of the previous frame object and the current frame object. If the position difference is less than or equal to the allowable value, the identification process S8 is performed as the same object. If it is not, S6 is performed. In step S8, a process of linking the same object of the identified previous frame and current frame on the internal data is performed.
[0036]
S9 and S10 are data holding process flows for holding the vertical image size and distance information for each object based on the link data of the object output from the identification process. In S9, in the loop process for holding the data of the object extracted from the current frame image, if there is an unprocessed object, the information holding process of S10 is performed, and if there is no unprocessed object, the determination process of S11 To implement. In S10, information is held based on the link information output in S8. If there is no object to be identified, a new object data holding area is created because it is a new object, and the distance and the vertical image size are written. If there is an object to be identified, the distance and the vertical image size are added to the data holding area of the object.
[0037]
S11 and S12 are determination flows for determining whether the object is a three-dimensional object or a planar object based on the change in the vertical image size data with respect to the distance for each object held by the memory 4. In S11, it is determined whether or not the object is close to within "40m". If it is within "40m", whether the object is a three-dimensional object or a planar object is determined in S12, and "40m" If it is further away, no judgment is made.
[0038]
In S12, a process of determining whether the object is a three-dimensional object or a planar object is performed using a change in vertical image size data with respect to a distance during which the object is close to “40 m”. First, by using the vertical image size data for the nearest distance to calculate the unknown “three-dimensional object height“ h _t ”” of the equation (14), the three-dimensional object when the object is a three-dimensional object is calculated. Estimate a vertical image size model for object distance. Similarly, by calculating the “planar object depth“ d _t ”” which is an unknown quantity in the equation (15), a distance-vertical image size model of the planar object when this object is a planar object is estimated. . Next, as shown in FIG. 5, by calculating the total sampling data of the vertical image size with respect to the distance of the object and the variance value of the three-dimensional object model and the planar object model, it is determined which model is closer, Whether the object is a solid object or a planar object is identified.
[0039]
As described above, in this embodiment, the front side is imaged by the camera 1, the traveling state (pitch angle, etc.) of the host vehicle is detected by the sensor, and each detection result and imaging result are supplied to the microcomputer 3. Since the object extraction process, the object identification process, the information holding process, and the determination process as to whether the object is a three-dimensional object or a planar object are performed, an object in front of the host vehicle is moving. Even in this case, it is possible to identify whether the object is a three-dimensional object or a flat object, that is, whether it can be an obstacle (effect of claim 1).
[0040]
In this embodiment, the camera 1 captures the front and the sensor detects the traveling state (pitch angle, etc.) of the host vehicle, and causes the microcomputer 3 to process the detection results and the imaging results. From the camera installation line-of-sight direction angle with respect to the vehicle at the time and the vehicle pitch angle at the time of imaging input from the sensor 2, the indication direction angle with respect to the road surface of the camera 1 is calculated, and the vertical image position of the object and the image of the camera 1 Calculates the object azimuth angle when viewing the object from the angle, image size, and visual axis direction angle, and calculates the distance of the object on the road from the object azimuth angle and the height from the road to the camera 1 Therefore, even when the host vehicle is shaken, the distance to the object on the road can be accurately measured (effect of claim 2).
[0041]
According to the invention of claim 3, as shown in the equation ( 14 ), a three-dimensional object model that models a change in the vertical image size with respect to a distance corresponding to the height of the three-dimensional object is used. Since the height is calculated, the height of the three-dimensional object in front of the host vehicle can be calculated in real time.
[0042]
The form described above is described in order to facilitate understanding of the present invention, and is not described in order to limit the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.
[Brief description of the drawings]
FIG. 1 is a configuration block diagram showing an embodiment of an obstacle detection apparatus of the present invention.
FIG. 2 is a processing block diagram of the obstacle detection apparatus of the present invention.
FIG. 3 is a basic principle diagram of distance calculation processing according to the present invention.
FIG. 4 is a flowchart illustrating an embodiment of the obstacle detection device of the present invention.
FIG. 5 is a three-dimensional object / planar object distance-vertical image size model according to the three-dimensional object plane object identifying means of the present invention;
FIG. 6 is a basic principle diagram of a three-dimensional object model according to the present invention.
FIG. 7 is a basic principle diagram of a planar object model according to the present invention.
[Explanation of symbols]
1 Camera (imaging means)
2 Sensor (Mechanical vehicle behavior detection means)
3 microcomputer (object extraction means, object image size calculation means, object distance calculation means, object identification means, and solid object flat object identification means)
4 Memory (object information holding means)
DESCRIPTION OF SYMBOLS 5 Vehicle control apparatus 6 Alarm apparatus 10 Imaging means 11 Vehicle behavior detection means 12 Object extraction means 13 Object image size calculation means 14 Object distance calculation means 15 Object identification means 16 Object information holding means 17 Three-dimensional object plane object identification means

Claims (4)

An imaging means mounted on the vehicle for imaging the front of the host vehicle;
Vehicle behavior detecting means for detecting the traveling state of the host vehicle;
An object extraction means for extracting an object on the road from the image obtained by the imaging means;
An object distance calculating means for calculating a distance to the object based on the image position of the object extracted by the object extracting means and the vehicle behavior obtained by the vehicle behavior detecting means;
An object image size calculating means for calculating a vertical image size of the object extracted by the object extracting means;
Object identifying means for identifying the object extracted by the object extracting means between image frames;
Object information holding means for holding a change in vertical image size with respect to the distance for each object extracted by the object extracting means;
It is determined whether the change in the vertical image size with respect to the distance of all the objects held by the object information holding means is closer to the three-dimensional object model curve or the planar object model curve, and the object is a three-dimensional object Three-dimensional object plane object identifying means for identifying whether the object is a plane object,
An obstacle detection device comprising: The obstacle detection device according to claim 1,
The object distance calculating means calculates a distance to the object on the road from a vehicle pitching angle detected by the vehicle behavior detecting means and a vertical image position of the object extracted by the object extracting means. ,
An obstacle detection device characterized by that. In the obstacle detection device according to any one of claims 1 and 2,
The three-dimensional object plane object identifying means, when it is determined that the object is a three-dimensional object, from the three-dimensional object model curve modeling the change in the vertical image size with respect to the distance of the three-dimensional object. Calculate the height,
An obstacle detection device characterized by that. A first step of detecting a vehicle behavior of the host vehicle based on a running state of the host vehicle;
A second step of extracting an object on the road from an image obtained by an imaging means mounted on the vehicle;
A third step of calculating a distance from the host vehicle to the object based on the image position of the object extracted in the second step and the vehicle behavior obtained in the first step;
A fourth step of calculating a vertical image size of the object extracted in the second step;
A fifth step of identifying the object extracted in the second step between image frames;
A sixth step of maintaining a change in vertical image size with respect to the distance for each object extracted in the second step;
It is determined whether the change in the vertical image size with respect to the distance of all objects held by the sixth step is closer to the three-dimensional object model curve or the planar object model curve, and whether the object is a three-dimensional object A seventh step of identifying whether the object is a plane object;
An obstacle detection method comprising:

Images