JPH09145337A - Method for determining angle of depression in image pickup unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine the angle of depression which fluctuates due to pitching of a pair of image pickup units for image detector calculating the distance from a pair of image to an object accurately through a simple method. SOLUTION: The pavement parallax of image is previously calculated optogeometrically for a plurality of angles of depression within a predetermined range. On the other hand, a correlation value is calculated from the difference between a translated image and other image. Using the movement of translation for the minimum correlation value as a parallax, a correlation value obtained during the process for calculating the distance to object, where the pavement parallax calculated for each angle of depression is equal to the movement of translation of one image at the time of calculating a correlation value, is multivalued (Step 103). Variation is then calculated for the angle of depression of multivalued correlation value and the angle of depression for maximum variation is determined as the angle of depression of image pickup unit (Step 104).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for determining a depression angle of an image pickup device which is provided in an object detection device for detecting an object in front and is mounted on a vehicle and used.

[0002]

2. Description of the Related Art Development of a system for mounting an image pickup device such as a CCD camera on a vehicle or the like to obtain a front image, detecting an object in front, and enabling automatic traveling such as obstacle avoidance However, in this case, the image may be vertically shaken due to pitching of the vehicle or the like, and accurate front recognition may not be possible. Therefore, a countermeasure such as providing a pitching sensor or detecting the vertical displacement of the vanishing points of the parallel lane marks on the traveling road surface on the image to know the pitching amount of the vehicle and make a correction can be considered.

[0003]

However, since the absolute amount of depression angle cannot be detected by the pitching sensor, it is necessary to readjust each time the posture of the vehicle changes due to luggage loading or the like. Further, the detection by the lane mark is also lacking in generality because the condition that the lane mark exists on the traveling road surface is required.

As a method for detecting the attitude angle of a vehicle, an adaptive digital filter is applied to image data (Journal of the Institute of Electronics, Information and Communication Engineers, March 1992, Vol.
J75-D-II No. 3 P. 490 to 499), etc., but since complicated calculations are required, the calculation load of the microcomputer mounted on the vehicle becomes large.

The present invention solves such a problem, and provides a depression angle determining method for an image pickup apparatus which can easily and accurately calculate the pitching absolute amount of a vehicle and the like without being influenced by the running condition. To aim.

[0006]

According to the present invention, a plane image is obtained by a pair of image pickup devices which are arranged at regular intervals on the left and right sides, and an analysis area in each plane image is moved by a predetermined amount in the horizontal direction. The correlation value is calculated from the difference between the one plane image and the other plane image that are translated in parallel, and the parallax of the object captured in the analysis region is obtained from the change in the correlation value with respect to the movement amount. In the depression angle determination method of the imaging device of the object detection device for detecting the object by calculating the distance of, the road surface parallax in the analysis region, is calculated in advance for a plurality of depression angles in a predetermined range, the correlation value Of these, the amount of movement that the plane image is translated in parallel when calculating the correlation value,
The correlation value having the same road surface parallax calculated for each depression angle is multivalued, and then the change amount of the multivalued correlation value with respect to the depression angle is calculated, and the depression angle when the change amount becomes maximum is imaged. The depression angle of the device is determined (Claim 1).

The depression angle of the image pickup device can be obtained only by using the correlation value obtained in the process of calculating the distance from the object, and no image processing is required.

According to the method of the present invention, the analysis area is divided into a plurality of sub-analysis areas at least in the vertical direction, and the correlation values multivalued for each sub-analysis area as shown in FIG. Then, the depression angle at which the amount of change with respect to the depression angle is maximum is determined as the depression angle of the image pickup device (step 104) (claim 2).

Since the correlation value is locally obtained for each small analysis area, the depression angle of the image pickup device can be obtained with good accuracy even for a complicated planar image.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIGS. 2 and 3 are block diagrams of an object detecting apparatus for carrying out the depression angle determining method of the present invention. The stereo image pickup device 1 which is an image pickup device is provided with a pair of CCD cameras at the left and right positions of the vehicle V (FIG. 3) at the same height h (mm) parallel to the horizontal road surface toward the front of the vehicle. The plane image in front of the vehicle is input from each CCD camera to the left camera image storage memory 21 and the right camera image storage memory 22, respectively, and the input plane image is stored in the left camera image storage memory 21 and the right camera image storage. It is temporarily stored in the memory 22 and is updated every control cycle. A distance calculation circuit 3 is provided with the data of the plane images stored in the left camera image storage memory 21 and the right camera image storage memory 22 as an input, and the parallax of the object is obtained based on the correlation value of the two plane images. Is calculated, and the distance to the object in the plane image captured by the camera is calculated. The distance data calculated by the distance calculation circuit 3 is input to the microcomputer 6 via the distance data storage memory 4 which temporarily stores the data.

On the other hand, the correlation value calculated by the distance calculating circuit 3 is input to the microcomputer 6 via the correlation value data storage memory 5 which temporarily stores the correlation value.

The method of the present invention will be described together with the operation of the above object detecting apparatus.

The left-side plane image and the right-side plane image obtained by the stereo image pickup device 1 are the left camera image storage memory 2.
1, input to the right camera image storage memory 22 and temporarily stored. The stored left and right plane images (hereinafter, simply referred to as images) A and B data are read into the distance calculation circuit 3. The images A and B are shown in (1) and (2) of FIG. For convenience of explanation, as shown in the figure, the x coordinate is horizontal (the number of pixels (dots) from the left edge of the screen), and y is vertical.
Set the coordinates (the number of pixels (dots) from the top edge of the screen). The image is the area (0, 0) to (512, 24) of the entire screen.
0), regions (50, 100) to (450, 24)
0) is a width calculation target area of 400 dots,
The analysis area has a vertical width of 140 dots. The analysis area is divided into a small analysis area (not shown) having a horizontal width of 4 dots and a vertical width of 4 dots in a grid pattern. Addresses are attached to the respective small analysis areas for convenience. In the figure, O is a three-dimensional object, and the horizontal road surface G has a vanishing point M on the horizontal line H. Due to parallax, the image A on the left side is slightly closer to the right side than the image B on the right side.

In the distance calculation circuit 3, the read left camera image data is translated in the left-right direction by a predetermined movement amount z (dots) to be converted into another image data. Travel amount z
Is a variable dispersed at a constant interval within a predetermined range in which the parallax of the object can be taken, and the parallel translation of the image is executed for each z.

Image data L (x + z, y) on the left side translated in accordance with the equation in each sub-analysis region and image data R on the right side read from the image storage memory 22 for the right camera.
The absolute value of the difference of (x, y) is calculated for the pixels in each small analysis area, the sum of the absolute values of the differences is calculated, and the correlation value S (z) is obtained for each small analysis area.

[0016]

(Equation 1)

FIG. 5 shows the image A on the left side and the image B on the right side superposed on each other. The two images are laterally displaced by the parallax of the object. Therefore, the closer the movement amount z of the parallel movement of the image A is to the parallax of the object captured in the small analysis region,
The image data L (x + z, y) on the left side and the image data R (x, y) on the right side, which are translated, are in good agreement, and | L (x +
Since z, y) -R (x, y) | approaches 0, the relationship between the calculated correlation value S (z) and the movement amount z is concave as shown in FIG. 6, and the movement amount z of the parallel movement is as described above. If it matches the parallax of the object, the correlation value S (z) becomes the minimum. Then the correlation value S
Let z that minimizes (z) be the parallax w of the object.

FIG. 7 shows an optical path diagram of the stereo image pickup device 1. In the stereo image pickup device 1, a pair of left and right lenses 11L, 11 are provided at a constant interval toward the front of the vehicle.
R is provided, and CCD elements 12L and 12R are provided behind the focal length f (mm). In the distance calculation circuit 3, the parallax w obtained by calculating the correlation value S (z) from the distance d to the object geometrically (hereinafter referred to as the inter-object distance) is calculated.
From the CCD of the projected image on the CCD element 12L of the object O
Deviation a (dot) from the center of the element 12L, deviation b (dot) from the center of the CCD element 12R of the projected image of the object O on the CCD element 12R, baseline length c (mm) of the lenses 11L and 11R, coefficient It is calculated by the formula as k (mm / dot). d = k × f × (a + b) / c = kfw / c ...

The inter-object distance d is obtained for each small analysis area, input to the distance data storage memory 4, and stored as an inter-object distance map in the analysis area. The correlation value S
(Z) is input to the correlation value data storage memory 5, and is stored using the address and movement amount z added to the small analysis area as an index.

When the correlation values S (z) are stored in the correlation value data storage memory 5 for all the small analysis areas of the analysis area, the microcomputer 6 follows the flow chart shown in FIG. Start the calculation of. Computer 6 to start calculation
Feature amount V (θi) set on the program of
Reset (i = 1 to 31).

The microcomputer 6 executes an equation with M as the y coordinate of the vanishing point on the image for the range of Ws to We, which is the range of angles that the depression angle can take in advance from a vehicle pitching measurement test or the like. Parallax r (θi, yj
) Is calculated. The y coordinate M of the vanishing point is calculated by the following equation with the y coordinate of the center position of the screen as y0. Note that Ws and We are, for example, −3 ° and + 3 °, respectively.
The angle θi that this depression angle can take is in the above-mentioned predetermined range Ws to We.
Among them, a plurality of them are selected at regular intervals. For example, -3 °
When the interval is 0.2 ° in the range of + 3 °, i = 1 to 31. Further, y is selected in the range of 100 to 240 dots at the same 4-dot intervals as the vertical width of the small analysis region, and j = 1 to 35. r ([theta] i, yj) = c / h * (yj-M) ... M = y0-k * f * tan [theta] i.

As is known from the equation, the road surface parallax r
(Θi, yj) changes depending on the depression angle θi and also changes depending on the vertical position yj on the image. That is, y
It becomes zero when j = M, and linearly increases with increasing yj (that is, below the image).

Each y coordinate yj is the same as the vertical width of the small analysis area, 4
Since the dot intervals are selected, the small analysis regions are arranged in the x direction for each y coordinate yj. Since the width of the analysis area in the x direction is 400 dots, the number of analysis small areas arranged in the x direction is 100. r (θi at the y coordinate yj
, Yj) are read out, and for 100 small analysis areas arranged in the x direction, the index is sequentially stored from the correlation value data storage memory 5 in which the correlation value S (z) is stored and z = r. The correlation value S (z) that is (θi, yj) is read and binarized by a predetermined threshold value, and V
It is added to (θi) (step 103).

Correlation value S for the above 100 small analysis areas
When all (z) have been added to V (θi), the binarized correlation value S of the small analysis regions arranged in the x direction with respect to the y coordinate yj + 1 is similarly obtained for the next y coordinate yj + 1. (Z)
Further addition to V (θi), all y for j = 1 to 35
Correlation value S binarized into V (θi) for coordinate yj
An operation for adding (z) is executed. Then the feature value V
The sum of the binarized correlation values S (z) of all the small analysis areas in the analysis area is obtained as (θi) (step 102).

Then, the feature quantity V (θi + 1
) Is similarly calculated, and the feature value V (θi) is obtained for θi of i = 1 to 31 (step 101).

Since the small analysis area has a vertical width of 4 dots,
Road parallax differs depending on the position in the small analysis area. However, in the range of about 4 dots, each road surface parallax is almost the same, and r calculated by one of the formulas is r.
It can be regarded as (θi, yj). The road surface parallax at the position within the small analysis region is r (θi, where yj is replaced by y in the equation).
Therefore, S (z) is equal to v (θi) approximately represented by the equation.

[0027]

(Equation 5)

As is clear from the equation, v (θi) is the left side image A (FIG. 4 (1)) corresponding to the road surface parallax r (θi, y) when the depression angle is θi at each position in the small analysis region. Data of the translated image and right image B (Fig. 4 (2))
It is the sum within the small analysis area of the difference of the data.

FIG. 8 is an image C obtained by performing image processing for translating the left image A at each position by the road surface parallax r (θi, y), and FIG. 9 is a difference image D between the image C and the right image B. . If θi is equal to the true depression angle θr, the road surface images in the image C and the right image B match, the road surface image in the difference image D disappears as shown in FIG. Only the differences appear. On the contrary, if r (θi, y) as the amount of parallel movement is equal to the three-dimensional object parallax, the difference image D
The three-dimensional object disappears and only the difference of the road surface image remains. This can be expressed in terms of v (θi) and the sum of binarized and ternarized values with a certain threshold value over the entire analysis region (hereinafter referred to as image comparison variable). Become. The image comparison variable is the sum of the road surface image component and the three-dimensional object image component captured in the analysis area. The road surface image component takes a small value at the true depression angle θr, and has a large value before and after that. On the other hand, the three-dimensional object component takes a small value before the true depression angle θr.

With respect to such an image comparison variable, the inventors have found that the change amount with respect to θi is maximum at the true depression angle θr for all components, and therefore the change amount with respect to θi of the image comparison variable, which is the sum of both components, is the true depression angle. It has been confirmed that it has the maximum property at θr. Then, after the binarization of the correlation value S (z) that approximately matches v (θi), the feature amount V (θ
i) also has the same property as the image comparison variable, and its variation amount with respect to θi is maximum at the true depression angle θr.

At step 104, the feature amount V (θi)
For i (i = 1 to 35), the feature value V (θ i
), V (θi + 1) difference is calculated, and the difference is used as the change amount to compare the magnitudes. The true depression angle θr is set to θi that maximizes the difference (step 104). Then, in the next control cycle, the same routine is executed again for the correlation value S (z) newly stored in the correlation value data storage memory 5.

FIG. 10A shows an example of the calculation result of the feature amount V (θi) and the angle θi that the depression angle can take, and (2) shows the calculated amount of the change of the feature amount V (θi) and the angle θi that the depression angle can take. And the true depression angle θr is the maximum feature amount V (θi).
From the change amount of, it can be obtained as 0.4 °. As described above, according to the method of the present invention, the distance calculation circuit 3 can be used without performing image processing.
The depression angle can be determined only by using the correlation value calculated in the process of calculating the distance.

When the depression angle is determined, the computer 6 reads the inter-object distance map from the distance data storage memory 4, corrects it according to the depression angle, and knows the correct situation ahead even if there is pitching or the like. be able to.

The calculation of the characteristic amount V (θi) is executed after all the correlation value data is transferred from the distance calculation circuit 3, but the road surface parallax r (θi, yj) is calculated in advance. In the distance calculation circuit 3 and z = r.
At the time when the correlation value S (z) of each small analysis region is calculated for the movement amount z that is (θi, yj), the correlation value S
(Z) may be transferred to the microcomputer 6. In this case, the calculation result of the true depression angle θr can be obtained almost at the same time when the calculation of the correlation value S (z) is completed for all the small analysis regions.

Further, the method of dividing the analysis area into the small analysis areas is not limited to the above-mentioned one, and is arbitrary unless it goes against the gist of the present invention, and the analysis area is not divided into a plurality of small analysis areas. Can also be carried out.

Although S (z) is binarized in calculating the feature amount V (θi), it may be binarized or more.

Before calculating the change amount of the feature quantity, V
It is also possible to remove the noise component by performing a moving average of (θi).

Further, the range of the analysis region, the range of possible depression angles, and the like are arbitrary as long as they do not violate the gist of the present invention.

[Brief description of the drawings]

FIG. 1 is a flowchart illustrating the method of the present invention.

FIG. 2 is a block diagram of an apparatus to which the method of the present invention is applied.

FIG. 3 is a side view of a vehicle equipped with the above device that travels on a road.

FIG. 4 (1) is a front view of a first image illustrating the method of the present invention, and (2) is a front view of a second image illustrating the method of the present invention.

FIG. 5 is a front view of a third image illustrating the method of the present invention.

FIG. 6 is a first graph illustrating the method of the present invention.

FIG. 7 is a schematic view of a main part of the apparatus.

FIG. 8 is a front view of a fourth image illustrating the method of the present invention.

FIG. 9 is a front view of a fifth image illustrating the method of the present invention.

FIG. 10 (1) is a second graph illustrating the method of the present invention, and (2) is a third graph illustrating the method of the present invention.

[Explanation of symbols]

 1 Stereo image pickup device (image pickup device)

Claims (2)

[Claims] 1. A plane image is obtained by a pair of image pickup devices provided at regular intervals on the left and right sides, and an area at the same position in each plane image is defined as an analysis area. The amounts of movement in the left and right directions are changed to make parallel movements, and the correlation value of the images in the two analysis areas is calculated from the difference between the parallel movement analysis area of the one plane image and the parallel analysis area of the other plane image. The correlation between the two images is determined, and the movement amount when the correlation between the two images is the most is determined as the parallax of the target captured in the analysis region. In the depression angle determination method of the image pickup device of the object detection device that detects the object in front of the image pickup device by calculating the distance to the object, the road surface parallax in the analysis region, for a plurality of depression angles within a predetermined range Calculate in advance and From the correlation values, the amount of movement of the plane image moved in parallel when calculating the correlation value and the road surface parallax calculated for each depression angle are the same as the multi-valued correlation value, and the multi-valued correlation value A depression angle determination method for an imaging device, which calculates a variation amount with respect to a depression angle, and determines the depression angle when the variation amount is maximum as the depression angle of the imaging device. 2. The depression angle determination method for an image pickup apparatus according to claim 1, wherein the analysis region is partitioned at least in a vertical direction into a plurality of small analysis regions, and the entire analysis region of the multivalued correlation value is calculated for each small region. Is calculated, and the depression angle when the amount of change with respect to the depression angle of the sum is maximum is determined as the depression angle of the imaging device.