JP3503543B2 - Road structure recognition method, distance measurement method and distance measurement device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distance measuring technique for calculating a distance to an object existing in an image.

[0002]

2. Description of the Related Art Conventionally, as a technique for measuring a distance, there are a technique using a laser radar and a technique using a stereo method using two visible cameras.

As a system using only one visible camera, there is a system disclosed in Japanese Patent Application Laid-Open No. 11-44533, "Preceding Vehicle Detection Device".

FIG. 10 shows an explanatory view of the conventional method. Figure 10
In the figure, reference numeral 91 represents an image pickup surface, and 92 represents a lens. The prime number of the y coordinate from the center of the image of the preceding vehicle estimated point is dy2, the focal length of the lens 92 of the CCD camera of the image input unit is f,
When the number of pixels of the image pickup surface 91 is IY and the screen size is DH, the angle dθ2 formed by the optical axis and the straight line connecting the estimated point of the preceding vehicle on the image pickup surface 91 and the center point of the lens 92 is expressed by the following equation. .

Dθ2 = atan (DH / IY · dy2 / f) At this time, the distance in real space to the preceding vehicle estimation point is L2.
When the angle between the optical axis of the lens 92 and the horizontal line in the vertical direction is θ1, and the mounting height of the CCD camera is H, the distance L2 is expressed by the following equation.

L2 = H / tan (θ1-dθ2) The distance L2 obtained by the above equation is regarded as the inter-vehicle distance.

[0007]

In such a conventional technique, a laser radar and two visible cameras are required to measure the distance. Further, the distance measurement from one visible camera is limited to the case where the camera is fixed or the road from the camera to the target object is a horizontal plane even when the camera is not fixed.

In the present invention, when the camera itself is moving,
And even when the road surface is not horizontal, the visible camera 1
The purpose is to measure the distance accurately using only the table.

[0009]

In order to solve this problem, the present invention is to detect an object existing on a road from an image input from a camera, and at the same time recognize the structure of the road surface. And a distance calculation means for calculating the distance in the real space from the camera to the object using the position of the detected object in the image and the road surface structure, and are configured to measure the distance. It is a thing.

As a result, even when the camera itself is moving and the road surface is not horizontal, it is possible to obtain a distance measuring device capable of accurately calculating the distance from the image of only one visible camera.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION The invention according to claim 1 of the present invention is to provide a road at a position sufficiently close to a camera installed in a traveling vehicle.
It is assumed that the road is horizontal, and whether the road edge position is sufficiently close to the camera of the input image at the current time based on this assumption .
Obtain the position of the road edge Minoru Luo space, further, the calculated road construction at the present time from the position of the road edge in the real space, the amount of movement of the real space of the traveling vehicle from the previous time to the current time Using the road structure obtained by the previous time and the road structure at the current time, the road structure recognition method for calculating the road structure up to the current time is used, and the camera itself moves. Even when the road surface is not horizontal, the road structure can be recognized from the image of only one visible camera.

The invention according to claim 2 is the real space of a traveling vehicle.
The amount of movement between the current time and the coordinate axis of the input image at the previous time
The amount of translation and the rotation angle of the input image to the coordinate axes.
The road structure recognition method according to claim 1 , wherein even when the camera itself is moving and the road surface is not horizontal, the road structure at the current time and the road structure up to the previous time are
It has the effect that it can be connected accurately .

The third aspect of the present invention is a road road in a real space.
The position of the edge is the position of the road edge included in the partial image,
Camera focal length, camera lens center height and turtle
The road according to claim 1 or 2, which is calculated from the inclination of the optical axis.
The structure recognition method is used to calculate the position of the road edge in real space from the image of only one visible camera even when the camera itself is moving and the road surface is not horizontal.
It has the effect of being able to.

According to a fourth aspect of the present invention, the position of the object existing on the road in the input image is detected using the input image at the current time taken by the camera installed in the traveling vehicle. Step and position close enough to the camera
Let's assume that the road is horizontal and
Serial obtain the position of the road edge in the real space from the position of the road edge in close enough range of the input image at the present time cameras, further, it calculates the road structure of the current time from the position of the road edge of the real space , Using the movement amount of the traveling vehicle in the real space from the previous time to the current time, connecting the road structure obtained by the previous time and the road structure at the current time,
Using the second step of calculating the road structure up to the current time, the position of the detected object in the input image and the road structure up to the current time, the distance in real space from the camera to the object is calculated. And a third step of calculating the distance measurement method, wherein the camera itself is moving,
And even when the road surface is not horizontal, the visible camera 1
It has an effect that the distance can be measured from the image of only the table.

According to a fifth aspect of the present invention, the traveling vehicle is in the real space.
The amount of movement between the current time and the coordinate axis of the input image at the previous time
The amount of translation and the rotation angle of the input image to the coordinate axes.
The distance measuring method according to claim 4 is used, and even when the camera itself is moving and the road surface is not horizontal, the road structure at the current time and the road structure up to the previous time can be accurately measured.
It has the effect that it can be connected to .

The invention according to claim 6 is a road space in a real space.
The position of the edge is the position of the road edge included in the partial image,
Camera focal length, camera lens center height, camera
The distance measurement according to claim 4 or 5, which is calculated from the inclination of the optical axis.
This is a fixed method, and even if the camera itself is moving and the road surface is not horizontal, the position of the road edge in real space can be calculated from the image of only one visible camera .
It has the effect of being able to

The invention according to claim 7 is the first step.
Holds the characteristics of multiple detection target objects in advance, and
Extract the linear component from the differential binary image of the object included in the force image.
Of the object represented by the length of the straight line component
The object that most closely resembles the characteristics of the object to be detected.
Claim to extract the body and detect the position of the extracted object
The distance measuring method according to any one of items 4 to 6, which has an effect that the distance can be measured from the image of only one visible camera even when the camera itself is moving and the road surface is not horizontal. Have.

The invention according to claim 8 is the first step.
Is a plurality of detection target objects that are different in distance from the camera in advance.
The learning image, which is the image of
Collate the learning images to find the most similar area.
The position of the object is detected by and.
The distance measuring method described as the shift has an effect that the distance can be measured from the image of only one visible camera even when the camera itself is moving and the road surface is not horizontal.

According to a ninth aspect of the present invention , the road is horizontal at a position sufficiently close to the camera installed in the traveling vehicle.
And the camera of the input image at the current time is based on this assumption.
A partial image extraction means for extracting a partial image of sufficiently close range, and the real space road edge detection means from the position of the road edge Ru determined Me a position of the road edge in the real space contained in the partial image, the real space By the current time road structure calculation means for calculating the road structure at the current time from the position of the road edge and the movement amount of the traveling vehicle in the real space from the previous time to the current time. A road structure recognition device having a road structure connecting means for connecting the road structure and the road structure at the current time and calculating the road structure up to the current time, wherein the camera itself is moving, and Even if the road surface is not horizontal, the road structure can be recognized from the image of only one visible camera.

The invention according to claim 10 is an embodiment of a traveling vehicle.
The amount of space movement is the current time from the coordinate axis of the input image at the previous time.
The translation amount and rotation angle of the input image of the input image to the coordinate axes.
The road structure recognition device according to claim 9, wherein the road structure at the current time and the road structure up to the previous time are calculated even when the camera itself is moving and the road surface is not horizontal.
It has the effect that it can be connected accurately .

The invention described in claim 11 is a road in a real space.
The position of the edge is the position of the road edge included in the partial image.
Position, camera focal length, camera lens center height,
The calculation according to claim 9 or 10, which is calculated from the tilt of the optical axis of the melody
This is a road structure recognition device, and even if the camera itself is moving and the road surface is not horizontal, the position of the road edge in real space can be determined from the image of only one visible camera.
It has the effect that it can be calculated .

According to a twelfth aspect of the present invention, an object detection for detecting a position of an object existing on a road in the input image using an input image at the current time taken by a camera installed in a traveling vehicle. Means and position close enough to the camera
Let's assume that the road is horizontal and
Serial obtain the position of the road edge in the real space from the position of the road edge in close enough range of the input image at the present time cameras, further, it calculates the road structure of the current time from the position of the road edge of the real space , Using the movement amount of the traveling vehicle in the real space from the previous time to the current time, connecting the road structure obtained by the previous time and the road structure at the current time,
Using the road structure recognition means for calculating the road structure up to the current time, the position in the input image of the detected object and the road structure up to the current time, the distance in real space from the camera to the object is calculated. It is a distance measuring device having a distance calculating means for calculating, and it is said that the road structure can be recognized from the image of only one visible camera even when the camera itself is moving and the road surface is not horizontal. Have an effect.

FIG. 1 shows an embodiment of the present invention.
9 to 9 will be described.

(Embodiment 1) FIG. 1 shows a block diagram of a distance measuring apparatus according to Embodiment 1 of the present invention. In FIG. 1, 11 is an image input unit for inputting an image of an object, 12 is an area of the object extracted from the image data 15 input by the image input unit 11, and position information 1
The object detecting means for detecting 6 and the road structure information 17 for the image data 15 input by the image input unit 11.
Is a road structure recognizing means, and 14 is a distance calculating means for calculating the distance from the camera to the object using the position information 16 and the road structure information 17 and outputting the measurement result 18.

The operation of the distance measuring device configured as described above will be described below. Here, FIG. 2 shows an example of the image data 15, and the detection target object 21 is present on the road 23, and the detection target object 21 and the road 23 are in contact with each other at the position 22. White lines 24 and 25 are drawn on the road 23. FIG. 3 shows an example of the differential binary image, and 31 represents the detected model of the preceding vehicle.
FIG. 4 shows an example of a road structure recognition result, where 41 represents a road shape on a horizontal plane when viewed from above, 42 represents a road shape on a vertical cross section when viewed from the side, and both ends of a line segment 43. The left and right road edges of are in a one-to-one correspondence. Also, 44
Represents the position in real space. FIG. 5 shows the road edges detected from the image data 15 and their correspondence.
Reference numeral 52 represents a detected road edge, and 53 represents a line segment representing the correspondence between the left and right road edges.

The image input unit 11 inputs an image of a road to be recognized and an object on the road to be detected by one camera. In the present invention, if the camera is moving more camera itself to be mounted on a vehicle, and the road surface is also available when not horizontal like slope as shown in FIG.

The object detecting means 12 includes the image input section 11
2 is input, the position 22 in the image of the detection target object 21 is detected, and this is held as position information 16.

Here, as an example of the method of detecting the position of the object from the image, the Information Research Institute CV37-4 (1985)
Year) "Identification and tracking of preceding vehicle on highway". The above-mentioned method is based on the assumption that a vehicle traveling ahead is detected as the detection target object, and its contents will be briefly described below. First, the input image obtained by photographing the rear surface of the preceding vehicle is subjected to the secondary differential processing and binarization processing to obtain a differential binary image as shown in FIG. Next, the target horizontal edge component is extracted from the obtained image, and this set is used as the model 31 of the preceding vehicle.

The road structure recognition means 13 receives the image data 15 input by the image input section 11 as shown in FIG.
To recognize the structure of the road 23 in the real space,
This is held as road structure information 17. Here, as an example of the method of recognizing the structure of the road surface in the real space from the image, there is a method shown in "Reconstruction of Road Shape by Local Plane Approximation" in Information Research Institute CV62-3 (1989). . The above method obtains the road structure as shown in FIG. 4 from the road edge in the image based on the knowledge about the road shape called the road model. The contents will be briefly described below. The road model is defined as a road edge which is a locus of end points when line segments having a constant length move horizontally while intersecting vertically at a midpoint on a smooth curve.

First, as shown in FIG. 5, the road edges 51 and 52 are detected with respect to the input image, and the one-to-one correspondence between the left and right road edges is set at both ends of the line segment having the constant length. Ask for. Next, with respect to the corresponding points of the left and right road edges, a three-dimensional position is restored based on the road model and a smooth curve is fitted to the three-dimensional road shapes 41, 4
2 is set.

The distance calculating means 14 is the object detecting means 1
2 calculates the distance from the camera by obtaining the position of the detection target object 21 in the real space from the position information 16 detected by 2 and the road structure information 17 recognized by the road structure recognizing means 13, and measures this. The result 18 is output. For example, with respect to the image data as shown in FIG. 2, the target object 21 is detected by the object detecting means 12 and the position 22 in the image is obtained.

On the other hand, the road structure recognition means 13 recognizes a road structure as shown in FIG. At this time, as shown in FIG. 5, the white lines in the image are detected as road edges 51 and 52, and the correspondence between the left and right road edges is obtained. Therefore, the line segment 53 in the image corresponding to the detected position 22 is obtained. .
The line segment 53 is the line segment 43 in the road structure recognition means 13.
Since it has been restored to, the position 44 in the real space is obtained correspondingly from the position 22 in the image. From this position 44, the distance from the camera to the object is calculated.

(Second Embodiment) FIG. 6 shows a block diagram of a distance measuring device according to a second embodiment of the present invention. The difference from the first embodiment lies in the object detecting means 12 and the object learning image database 19. Yes, only the different parts will be described below.

The operation of the object detecting means 12 in the second embodiment will be described below. Here, FIG. 7 shows an example of the learning image.

When the object detecting means 12 detects the position of the object, a plurality of learning images as shown in FIG. 7 are registered in the object learning image database 19 in advance.
The learning images registered in the object learning image database 19 are
For each object to be detected, an image of a size corresponding to the appearance at each distance is registered for each distance from the camera set at an appropriate interval.

The image data 15 as shown in FIG. 2 input by the image input unit 11 is read and collated with the learning image registered in each object learning image database 19 to find the most similar learning image and input image. And the area of. The position in the input image at this time is held as the position information 16.

At the same time, a rough distance from the camera to the target object can be obtained from a learning image having a size corresponding to the appearance at the collated distance.

(Third Embodiment) Next, a distance measuring device according to a third embodiment of the present invention will be described. However, the processing of the road structure recognizing means 13 is different from that of the first embodiment. It described below means 13.

The operation of the road structure recognition means in the third embodiment will be described below. Here, FIG. 8 shows the relationship between the image and the real space, where 71 is the lens center and 7
2 is the coordinate axis of the image, 73 is the coordinate axis of the real space, 74 is the position in the image, and 75 is the position of the real space. However, in both cases, the y-axis is the direction perpendicular to the paper surface. 9A and 9B show the road structure in the real space. FIG. 9A shows the road structure at a position sufficiently close to the camera at the current time, FIG. 9B shows the road structure recognition result at the previous time, and FIG. ) Represents the road structure recognition result at the current time, and 81 represents the coordinate axes at the previous time.

Consider the case where the camera is installed in the rear of the traveling vehicle and continuously captures the direction opposite to the traveling direction in the road structure recognition means 13, and it is assumed that the road is horizontal at a position sufficiently close to the camera. . The image input section 11
2 is input and the position (ix, iy) of the white line in the image is detected at a position sufficiently close to the camera. As shown in FIG. 8, assuming that the focal length is f, the height of the lens center is h, and the angle from the horizontal direction to the camera optical axis is θ, the position of the white line (px, py, pz) is as in (Equation 1).

[0041]

[Equation 1] The position of the white line in the real space is calculated from (Equation 1) from all positions of the white line in the detected image that are sufficiently close to the camera, and if a curve is applied, the position is sufficiently close to the camera as shown in FIG. 9 (a). The road structure in real space is required.

Here, it is assumed that the road structure in the real space as shown in FIG. 9B has been output at the previous time on the coordinate axes at the previous time. Further, the amount of movement of the vehicle itself from the previous time to the current time, that is, the amount of parallel movement from the coordinate axis at the previous time to the coordinate axis at the current time and the rotation angle are separately obtained from the vehicle side. Therefore, by connecting the road structure shown in FIG. 9B at the previous time to the road structure at a position sufficiently close to the camera shown in FIG. The road structure at the current time as shown in (c) can be obtained, and this is held as road structure information 17.

[0043]

As described above, according to the present invention, it is advantageous that the distance can be accurately measured from the image of only one visible camera even when the camera itself is moving and the road surface is not horizontal. The effect is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a distance measuring device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an example of image data according to the first embodiment of the present invention.

FIG. 3 is a diagram showing an example of a differential binary image according to the first embodiment of the present invention.

FIG. 4 is a diagram showing an example of a recognition result of a road structure according to the first embodiment of the present invention.

FIG. 5 is a diagram showing an example of road edge detection and its correspondence according to the first embodiment of the present invention.

FIG. 6 is a block diagram showing a distance measuring device according to a second embodiment of the present invention.

FIG. 7 is a diagram showing an example of a learning image according to the second embodiment of the present invention.

FIG. 8 is a diagram showing a relationship between an image and a real space according to the third embodiment of the present invention.

FIG. 9 is a diagram showing a road structure in a real space according to the third embodiment of the present invention.

FIG. 10 is an explanatory diagram of a conventional distance measuring method.

[Explanation of symbols]

11 Image input section 12 Object detection means 13 Road structure recognition means 14 Distance calculation means 15 image data 16 Location information 17 Road structure information 18 measurement results 19 Object learning image database 21 Object to be detected 22 Object position 23 road 24, 25 white line 31 Leading vehicle model 41 Road shape on horizontal plane 42 Road shape on vertical section 43 Line segment representing the correspondence of road edges 44 Position in real space 51,52 Road edge 53 Line segment that represents the correspondence of road edges 71 Center of lens 72 Image coordinate axes 73 Coordinate axes in real space 74 Position in image 75 Position in real space 81 Coordinate axes at the previous time

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-9-48299 (JP, A) JP-A-11-259658 (JP, A) JP-A-5-46730 (JP, A) JP-A-11- 232426 (JP, A) JP-A-7-120258 (JP, A) Masayuki Nakajima Two others, identification and tracking of a preceding vehicle on a highway, Information Processing Society of Japan Research Report, Information Processing Society of Japan, July 18, 1985 Sun, Vol. 85, No. 26, CV 37-4, p. 1-8 (58) Fields investigated (Int.Cl. ⁷ , DB name) G01C 3/06 G01B 11/00 G06T 7/60 180

Claims (12)

(57) [Claims] 1. Close enough to a camera installed in a traveling vehicle
The position assumes that the road is horizontal and the basis of this assumption is
To obtain the position of the road edge in the real space from the position of the road edge in the range sufficiently close to the camera of the input image at the current time, further, calculate the road structure at the current time from the position of the road edge in the real space, Using the amount of movement of the traveling vehicle in the real space from the previous time to the current time, the road structure obtained by the previous time and the road structure at the current time are connected to each other, and a road structure up to the current time is obtained. Road structure recognition method to calculate. 2. The road structure recognition method according to claim 1, wherein the moving amount of the traveling vehicle in the real space is a parallel moving amount and a rotation angle from the coordinate axis of the input image at the previous time to the coordinate axis of the input image at the current time. 3. The position of the road edge in the real space is calculated from the position of the road edge included in the partial image, the focal length of the camera, the height of the lens center of the camera, and the tilt of the camera optical axis. 2. The road structure recognition method described in 2. 4. A first step of detecting a position in the input image of an object existing on a road using an input image of a current time taken by a camera installed in a traveling vehicle, and the camera Roads should be level when close enough
And the input image at the current time based on this assumption
The position of the road edge in the real space is obtained from the position of the road edge sufficiently close to the camera, and the road structure at the current time is calculated from the position of the road edge in the real space, from the previous time to the current time. Second step of connecting the road structure obtained up to the previous time and the road structure at the current time using the movement amount of the traveling vehicle in the real space, and calculating the road structure up to the current time And using the detected position of the object in the input image and the road structure up to the current time,
Third, calculating the distance in real space from the camera to the object
And a distance measuring method including the steps of. 5. The distance measuring method according to claim 4, wherein the moving amount of the traveling vehicle in the real space is a parallel moving amount and a rotation angle from the coordinate axis of the input image at the previous time to the coordinate axis of the input image at the current time. 6. The position of the road edge in the real space is calculated from the position of the road edge included in the partial image, the focal length of the camera, the height of the lens center of the camera, and the tilt of the optical axis of the camera. 5. The distance measuring method described in 5. 7. A first step is to store the characteristics of a plurality of detection target objects in advance, extract a linear component from a differential binary image of the object included in the input image, and calculate the length of the linear component. 7. The distance measuring method according to claim 4, wherein the feature of the expressed object is obtained, the object most similar to the feature of the detection target object is extracted, and the position of the extracted object is detected. 8. The first step is to register a learning image, which is an image of a plurality of detection target objects having different distances from a camera, in advance, collate the learning image with an input image, and obtain the most similar image. 7. The distance measuring method according to claim 4, wherein the position of the object is detected by obtaining the area. 9. Close enough to a camera installed in a traveling vehicle
The position assumes that the road is horizontal and the basis of this assumption is
A partial image extraction means for extracting a partial image of sufficiently close range of the input image at the present time cameras, the road edge real space road location Ru determined Me a position from the real space road edge included in the partial image Using the edge detection means, the current time road structure calculation means for calculating the road structure at the current time from the position of the road edge in the real space, and the movement amount of the traveling vehicle in the real space from the previous time to the current time. A road structure recognizing device having a road structure connecting means for connecting the road structure obtained up to the previous time and the road structure at the current time and calculating the road structure up to the current time. 10. The road structure recognition device according to claim 9, wherein the movement amount of the traveling vehicle in the real space is a translation amount and a rotation angle from the coordinate axis of the input image at the previous time to the coordinate axis of the input image at the current time. 11. The position of the road edge in the real space is calculated from the position of the road edge included in the partial image, the focal length of the camera, the height of the lens center of the camera, and the tilt of the camera optical axis. 10. The road structure recognition device according to item 10. 12. An object detecting means for detecting a position in the input image of an object existing on a road by using an input image at a current time taken by a camera installed in a traveling vehicle, and the camera being sufficient. The road should be horizontal in close proximity.
And the input image at the current time based on this assumption
The position of the road edge in the real space is obtained from the position of the road edge sufficiently close to the camera, and the road structure at the current time is calculated from the position of the road edge in the real space, from the previous time to the current time. A road structure recognizing means for connecting the road structure obtained up to the previous time and the road structure at the current time using the movement amount of the traveling vehicle in the real space to calculate the road structure up to the current time. And a distance calculation device having a distance calculation means for calculating the distance in real space from the camera to the object using the detected position of the object in the input image and the road structure up to the current time.