JPH08278126A - Distance measuring equipment - Google Patents

Abstract

PURPOSE: To provide distance measuring equipment capable of establishing the corresponding relationships of object features more accurately and lessening a calculation burden by extracting horizontal edges from the photographed images of a pair of cameras, and preparing the histograms of the edges projected to the vertical axes of the images. CONSTITUTION: A pair of cameras 12 and 14 are laid at different height and kept faced to the same direction for photographing the image of an object 10. Also, a horizontal edge extraction means 16 extracts horizontal edges from the photographed images of the cameras 12 and 14, and histograms with the extracted horizontal edges projected to the vertical axes of the images are prepared for both cameras 12 and 14, using a histogram preparation means 18. In addition, corresponding peaks are searched in order from both histograms, starting from the lower sections of the vertical axes, and a parallax calculation means 20 computes the positional difference of the corresponding peaks on the vertical axes as the parallaxes of the cameras 12 and 14. Then, a distance calculation means 22 calculates a distance up to the object 10 on the basis of the principle of the trigonometrical survey.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for photographing the same object with two upper and lower cameras and measuring the distance to the object.

[0002] In factories and the like, unmanned vehicles are introduced in order to reduce the number of personnel, but it is desirable to avoid collision with pedestrians on the premises and surrounding equipment. Further, when the driver of the vehicle becomes tired while traveling on the highway, it is also required to prevent a rear-end collision with a vehicle ahead. Therefore, it has been proposed to mount an apparatus of this kind on an unmanned vehicle or a general vehicle to automatically control a braking mechanism of the vehicle or to issue an alarm to a vehicle driver.

[0003]

2. Description of the Related Art Referring to FIG.
Is following, and there are two cameras 20 in the following vehicle 202.
4,206 are installed.

The cameras 204 and 206 are used for the vehicle 20.
They are attached to the front part of the vehicle 2 at different heights and point toward the front of the vehicle.

In FIG. 3, a method of "tracking by template" is described. The front vehicle 200 is photographed by both cameras 204 and 206, and the characteristics of the vehicle 200 (for example, a bright portion: a license plate in FIG. 3) are shown. Is included in the captured images of both cameras 204 and 206.

Here, since a vertical line (epipolar line) passing through the vehicle feature in the image captured by the upper camera 204 penetrates the same vehicle feature in the image captured by the lower camera 206, The same vehicle feature is searched along the epipolar line in the upper and lower captured images.

Then, the height position of the searched vehicle feature is obtained in pixel units for both captured images, and the difference between the two positions is taken as the parallax of both cameras 204, 206.
The distance to the vehicle 200 ahead of 206 is calculated from this parallax according to the principle of triangulation.

The process of searching for the same vehicle feature along the epipolar line and calculating the distance to the vehicle 200 ahead is performed every time the images taken by both cameras 204 and 206 are obtained, and the vehicle 200 traveling ahead is searched. License plates are tracked continuously in chronological order.

In addition to the above "tracking by template", a feature-non-tracking system is employed in which the same two-dimensional vehicle feature is independently searched for each time an upper and lower photographed image is obtained. In this system, Only the vehicle features for distance calculation are narrowed down from those existing in the captured image.

[0010]

When the camera shakes laterally due to vibrations while the vehicle is running, the vehicle features often deviate from the epipolar line, making it impossible to perform the feature search and distance calculation processing.

This is the case where the "tracking by template" method is applied, and another vehicle 201 interrupts the front vehicle 200 as shown in FIG. 4 to conceal the characteristics (license plate) of the vehicle 200. The same applies when you do this.

When the feature non-tracking method is applied, a large number of vehicle features exist in each image frame as shown in FIG. 5, so that the vehicle features for distance calculation are narrowed down for each frame cycle. Extremely difficult in terms of speed.

The present invention has been made in view of the above circumstances, and can more reliably associate object features, and
The purpose is to provide a distance measuring device that can significantly reduce the calculation load.

[0014]

[Means for Solving the Problems]

Claim 1: In Fig. 1, a pair of cameras 12 and 14 which are provided at different heights and are oriented in the same direction to capture an image of the object 10, respectively, and a horizontal edge from the captured images of the cameras 12 and 14 are displayed. A horizontal edge extracting means 16 for extracting,
Histogram creating means 18 for creating a histogram in which the extracted horizontal edges are projected on the vertical axis of the captured image for both cameras 12 and 14, and the corresponding peaks are searched in order from the bottom of the vertical axis from both created histograms. The difference between the positions of the corresponding peaks on the vertical axis.
It has a parallax calculating means 20 for calculating the parallax of 2 and 14 and a distance calculating means 22 for calculating the distance to the object 10 from the calculated parallax based on the principle of triangulation.

According to a second aspect of the present invention, the horizontal edge extracting means detects the intensity of the horizontal edge in the horizontally long rectangular pixel area and transfers it to the histogram creating means, and both of the cameras. Means for repeating horizontal scanning of the edge operator from the lower side to the upper side in the captured image.

According to a third aspect of the present invention, the histogram creating means counts the number of pixels in which the detected horizontal edge intensity exceeds a predetermined value, and the counted value of the photographed image. Means for projecting onto a vertical axis.

According to a fourth aspect of the present invention, in the apparatus according to the first aspect, the camera has an apex at an intersection of two lines indicating the left and right ends of a strip-shaped plane area where the object and both cameras are present and a horizontal line including an infinite point. A region cutout unit is provided that cuts out a triangular region whose bottom is the lower side of the shooting region and delivers it to the horizontal edge extraction unit from the shot images of both cameras.

In a fifth aspect of the present invention, the histogram creating means is provided with means for adjusting the peak height in inverse proportion to the width between the hypotenuses in the triangular region.

Claim 6: In the apparatus of claim 4, the histogram creating means is provided with means for adjusting the height of the peak in proportion to the position on the vertical axis.

Claim 7: In the apparatus of claim 1, the parallax calculating means selects a peak having a variation amount exceeding a set value from both histograms and a corresponding peak in order from the lower side of the vertical axis. Means to search from the peaks
including.

Claim 8: In the apparatus of claim 1, the parallax calculating means includes means for excluding beforehand a histogram portion of an image area where the visual fields of both cameras do not overlap from the corresponding peak search range.

Claim 9: In the apparatus of claim 1, the parallax calculating means stores means corresponding to each position on the vertical axis and the distance from both cameras to each point on the plane where the object exists. , Means for searching corresponding peaks from both histograms in order from the lower side of the vertical axis, and means for searching the stored contents for a distance corresponding to the positions on the vertical axis of the searched both peaks, and comparing both distances, Means for calculating the difference in position on the vertical axis of both peaks for which the comparison result of distance disagreement has been obtained, as parallax of both cameras.

[0023]

[Action]

Claim 1: The cameras 12 and 14 are provided at positions having different heights and are directed in the same direction so that the images of the object 10 are captured.

Then, horizontal edges are extracted from the images taken by the cameras 12 and 14, and the histogram obtained by projecting the extracted horizontal edges on the vertical axis of the images is taken by the cameras 12 and 14.
Create for 14.

Furthermore, the corresponding peaks are searched in order from the lower side of the vertical axis from both histograms (since it is required to detect the object at the closest position at a higher level in order to avoid collisions, etc., Is performed sequentially from the lower side of the vertical axis), and the difference between the positions of the corresponding peaks on the vertical axis is calculated as the parallax between the cameras 12 and 14, and the distance from the parallax to the object 10 is calculated by the principle of triangulation. Ask.

As described above, according to the present invention, a histogram in which the horizontal edge of the object 10 is projected on the vertical line is created, and the distance to the object 10 is one-dimensionally calculated from the lower side of the histogram.

According to a second aspect of the present invention, the intensity of the horizontal edge is detected in the horizontally long rectangular pixel area, and the horizontal scanning of the pixel area is performed from the lower side to the upper side in the images captured by both cameras. Repeatedly, the intensity of the horizontal edge is projected onto the vertical axis of the captured image to create histograms for both cameras.

In a third aspect of the present invention, a histogram is created by projecting the number of pixels whose horizontal edge strength exceeds a predetermined value onto the vertical axis of the captured image.

A fourth aspect of the present invention is the apparatus according to the first aspect, in which a strip-shaped plane area in which an object and both cameras are present (specifically,
Two lines (specifically, straight lines or dashed white lines or yellow lines drawn on both sides of the driving lane) and infinity indicating the left and right edges of the lane on which the vehicle equipped with this device is running. A triangular area having an intersection with a horizontal line (horizon) that includes a point as a vertex and a lower side of a camera shooting area as a base is cut out from a captured image of both cameras, and a horizontal edge extraction range is limited to the triangular area.

In order to measure the distance to the preceding vehicle, the histogram of only the horizontal edge of the preceding vehicle is sufficient, so only the traveling lane portion including the preceding vehicle is taken out and the surrounding scenery portion is excluded.

Claim 5: The undulation of the histogram becomes small at a position away from the camera (vertex side of the triangular region). Therefore, in the device of claim 4, the height of the peak is adjusted in inverse proportion to the width between the hypotenuses in the triangular region.

The sixth aspect of the present invention adjusts the height of the peak in proportion to the position on the vertical axis in consideration of the fact that the undulation of the histogram becomes small at a position away from the camera.

(7) In the apparatus of (1), a peak whose fluctuation amount exceeds a set value is selected from both histograms, and a corresponding peak is searched from the peaks selected in order from the lower side of the vertical axis. Attention is paid to only sharp horizontal edges as appropriate to show the features of the object.

Claim 8: In the apparatus of claim 1, the histogram part of the image area where the visual fields of both cameras do not overlap is excluded from the peak search range in advance, and only the image area where the visual fields of both cameras overlap is noted.

Claim 9: In the apparatus of claim 1, each position on the vertical axis and the distance from each camera to each point on the plane where the object is present are stored in advance in correspondence with each other and both histograms are used. The peaks are searched in order from the lower side of the vertical axis, the distances corresponding to the positions of the searched peaks on the vertical axis are searched from the stored contents, both distances are compared, and a comparison result of distance disagreement is obtained. The difference between the positions of the peaks on the vertical axis is calculated as the parallax of both cameras.

Marks, shadows, reflections in rainy weather, etc. on the road are recognized as planar objects on the road surface. Therefore, only objects having a certain height from the road surface are targets for distance measurement.

[0037]

【Example】

First Embodiment: In FIG. 6, the images taken by the cameras 204, 204 are given to the image input units 600, 602, respectively (also in the present embodiment, the cameras 204, 204 are mounted in front of the vehicle 202 as shown in FIG. 7). Are installed at different positions and point toward the front of the vehicle), the image input unit 600,
Reference numeral 602 denotes A / D conversion of the images taken by the cameras 204, 204 into digital images (steps 802, 80 in FIG. 8).
4).

These digital images are transferred to the vehicle lane extraction units 604 and 606, and the vehicle lane extraction units 604 and 604 are transmitted.
606 detects the white line of the lane in which the vehicle 202 is traveling from the transferred captured image (steps 806 and 8 in FIG. 8).
08), the traveling lane in the triangular area where the infinite line (horizon line) and the white line, which are prepared in advance, intersect is cut out as shown in FIG. 9 (steps 810 and 812 in FIG. 8) and transferred to the edge histogram creation units 608 and 610, respectively.

At that time, the processing of FIG. 11 is performed by using two edge operators (1 pixel vertically / 3 pixels horizontally) of FIG. 10 (for the edge operator, “Image Analysis Handbook” Mikio Takagi, Yoshihisa Shimoda). Supervision "University of Tokyo Press pp.
550 to 575), the height positions Y1 and Y2 are first set (step 1100).

Then, set both edge operators to the height position Y.
Scanning is performed from the horizontal center of the image at 1 and Y2 toward the left end, and the maximum value of the pixel and its horizontal position X are recorded (steps 1102, 1104, 1106, 1 in FIG. 8).
108, 1110, 1112, 1114, 1116, 1
117, 1118).

Next, when it is confirmed that both maximum values are within the threshold value (steps 1120 and 1122), a straight line connecting the lateral positions X recorded for the height positions Y1 and Y2 is calculated (step 1124). )

Further, the intersection of the straight line and the line at infinity is calculated (step 1126), and the white lines (yellow lines) drawn on both sides of the road from the road width are estimated (step 1128), and the estimated white line (yellow) is calculated. A running lane portion of a triangular area having a diagonal line as a line and a bottom frame as an image bottom is cut out (step 113).
0).

The edge histogram creation unit 608 of FIG.
610 detects a horizontal edge from the transferred driving lane portion (lane area) (FIG. 8_steps 814 and 816),
A histogram is created by projecting onto the vertical axis as in FIG. 12 (steps 818 and 820 in FIG. 8).

When a horizontal edge is detected, the edge operator (3 vertical pixels / 9 horizontal pixels) shown in FIG. 13 is scanned, and in an actual apparatus, the edge strength (edge operator value) is extracted.

Then, the correspondence unit 612 in FIG. 6 searches for corresponding peaks on both histograms in order from the bottom (FIG. 8_steps 822, 824, 826, 828), and the peak pairs are sent to the distance calculation unit 614 for each search. Send out.

The distance calculation unit 614 obtains the distance to the object feature from the distance table of FIG. 14 by the Y coordinate on both histograms, and when the difference between these distances is within the threshold value, the object feature is a mark on the road surface. Occasionally, the transmission of the next peak pair is requested (Fig. 8_step 830).

In FIG. 14, a distance table is prepared in advance for both cameras 204 and 206 (step 800 in FIG. 8), and the contents of the position range in the photographing screen in the vertical direction and the distance range from the cameras 204 and 206 correspond to each other. Has been done.

Further, when the difference between the two distances is within the threshold value, the object feature is a planar mark, shadow, or reflection in the rain on the road surface, and therefore the distance is not calculated. If it is equal to or more than the threshold value, since it is a car or a building, the distance from the Y coordinate on both histograms to the feature of the preceding vehicle (license plate, etc.) and the height of the feature are calculated according to the principle of feature triangulation (Fig. 7). Then, after outputting this to the outside, the operation of the corresponding unit 612 is ended (FIG. 8_step 832).

Therefore, the device output is obtained when the closest object existing in front of the vehicle is recognized,
Therefore, when another vehicle cuts in with the preceding vehicle (see FIG. 4), the driver can be immediately warned of a rear-end collision with the interrupting vehicle when the distance to the interrupting vehicle is unacceptable.

In this embodiment, the feature of the vehicle ahead is not recognized two-dimensionally, the distance to the feature is calculated by one-dimensional processing, and the range to be subjected to the calculation processing is the extraction of the road portion or the distance table. Because it is limited by the use of, the collision warning can be issued almost instantly.

In order to measure the distance to the preceding vehicle, the histogram of only the horizontal edge of the preceding vehicle is sufficient. Therefore, only the traveling lane portion including the preceding vehicle is taken out and the surrounding scenery portion is excluded. Therefore, the disturbance due to the scenery portion can be avoided and the malfunction of the device can be more reliably prevented.

The number of pixels whose horizontal edge strength exceeds a predetermined value may be projected on the vertical axis of the captured image to create a histogram. Further, considering that the undulation of the histogram becomes small at the positions distant from the cameras 204 and 206, the peak height is adjusted in inverse proportion to the width between the hypotenuses in the triangular region as shown in FIG. It is also preferable to adjust in proportion to the position on the axis.

Further, by selecting a peak whose fluctuation amount exceeds the set value from both histograms and searching for a corresponding peak from the peaks selected in order from the lower side of the vertical axis,
It is preferable to focus only on sharp horizontal edges as the appropriate ones to characterize the object.

It is preferable that the histogram portion of the image area in which the fields of view of both cameras do not overlap is excluded from the peak search range in advance, and only the image area in which the fields of view of both cameras overlap is focused to reduce the processing load.

Second Embodiment: FIG. 16 shows the configuration of the second embodiment. The image input unit 603, the vehicle lane extraction unit 607, and the edge histogram creation unit 611 are shown in FIG.
Unlike the embodiment, it is unified.

According to this embodiment, the scale of the device (including software) can be greatly reduced and the manufacturing cost of the device can be significantly reduced as compared with the first embodiment. Note that image input, vehicle lane extraction, and edge histogram creation processing are performed in series rather than in parallel, so it takes time to perform these processing, but one-dimensionalization of the processing, By reducing the load by, for example, limiting the area targeted for edge operator scanning and processing to the traveling lane of the host vehicle, it is possible to secure a processing speed that is practically sufficient.

[0057]

As described above, according to the present invention, a histogram in which horizontal edges of an object existing in front of the camera are projected on a vertical line is created, and the distance to the object is one-dimensional from the lower side of the histogram. Therefore, the processing load can be significantly reduced without using the epipolar line.

Therefore, even if a person or vehicle cuts in front of the camera, the distance to the object in front of the camera can be measured without error, and this distance measurement can be completed with sufficient margin within the cycle of the image frame. It is also possible.

Moreover, since the distance to the front object is calculated by one-dimensional processing of the horizontal edge histogram,
Even if the camera shakes laterally due to the vibration of the vehicle while traveling, a stable distance calculation result can always be obtained.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the principle of the first invention.

FIG. 2 is an explanatory view of a mounting position of a camera pair.

FIG. 3 is an explanatory diagram of tracking using a template.

FIG. 4 is an explanatory view of a characteristic concealing action when a vehicle interrupts.

FIG. 5 is an explanatory diagram of intra-frame search candidates.

FIG. 6 is a diagram illustrating the configuration of the first embodiment.

FIG. 7 is an explanatory diagram of a distance calculation formula.

FIG. 8 is a flowchart illustrating an outline of processing according to the first embodiment.

FIG. 9 is an explanatory view of the operation of cutting out the own vehicle lane.

FIG. 10 is an explanatory diagram of an edge operator for cutting out the own lane.

FIG. 11 is a flowchart illustrating a traveling lane cutout process.

FIG. 12 is an explanatory view of the operation of the first embodiment.

FIG. 13 is an explanatory diagram of a histogram edge operator.

FIG. 14 is an explanatory diagram of contents of a distance table.

FIG. 15 is an explanatory diagram of a weighting operation of a histogram.

FIG. 16 is an explanatory diagram of the configuration of the second embodiment.

[Explanation of symbols]

 200, 201, 202 vehicle 204, 206 camera 600, 602, 603 image input unit 604, 606, 607 own vehicle lane extraction unit 608, 610, 611 edge histogram creation unit 612 corresponding unit 614 distance calculation unit

Claims (9)

[Claims] 1. A pair of cameras, which are provided at different heights and are oriented in the same direction to capture an image of an object, and a horizontal edge extraction means for extracting a horizontal edge from the captured images of both cameras. Histogram creating means for creating a histogram in which the horizontal edges are projected on the vertical axis of the captured image for both cameras, and the corresponding peaks are searched in order from the bottom of the vertical axis from the created histograms and the corresponding peak A distance measuring device comprising: a parallax calculating unit that calculates a position difference on an axis as a parallax of both cameras; and a distance calculating unit that calculates a distance to an object from the calculated parallax. 2. The horizontal edge extraction means detects an edge strength of a horizontal edge in a horizontally long rectangular pixel area and transfers it to the histogram creation means, and an edge operator from the lower side to the upper side in the images captured by both cameras. The distance measuring device according to claim 1, further comprising: a unit that repeats horizontal scanning of an edge operator. 3. The histogram creating means includes means for counting the number of pixels in which the detected horizontal edge intensity exceeds a predetermined value, and means for projecting the counted value on the vertical axis of the captured image. The distance measuring device according to claim 2, further comprising: 4. A triangle having an apex at the intersection of two lines displaying the left and right ends of a strip-shaped plane area where an object and both cameras are present and a horizontal line including the point at infinity, and having a lower side of the camera photographing area as a base. 2. The distance measuring device according to claim 1, further comprising area cutout means for cutting out the area from the images taken by both cameras and delivering it to the horizontal edge extraction means. 5. The distance measuring device according to claim 4, wherein the histogram creating means is provided with means for adjusting the height of the peak in inverse proportion to the width between the hypotenuses in the triangular region. 6. The distance measuring device according to claim 4, wherein the histogram creating means is provided with means for adjusting the height of the peak in proportion to the position on the vertical axis. 7. The parallax calculating means, means for selecting a peak whose variation amount exceeds a set value from both histograms, and means for searching for a corresponding peak from the peaks selected sequentially from the lower side of the vertical axis, 1. The method according to claim 1, wherein
The described distance measuring device. 8. The distance measuring device according to claim 1, wherein the parallax calculating means includes means for excluding beforehand a histogram portion of an image region where the fields of view of both cameras do not overlap from a corresponding peak search range. . 9. The parallax calculating means stores, in correspondence with each position on the vertical axis and the distance from both cameras to each point on the plane where the object is present, and the corresponding peaks from both histograms are stored vertically. A means for searching sequentially from the bottom of the axis, a means for comparing the two distances by searching the stored contents for the distances corresponding to the positions of both searched peaks on the vertical axis, and a comparison result of distance disagreement were obtained. The distance measuring device according to claim 1, further comprising: a unit that calculates a difference between positions of both peaks on a vertical axis as a parallax of both cameras.