JPH02216413A - Range finding method - Google Patents

Abstract

PURPOSE:To find a range extending from a photographing position to the surface of an object with high accuracy by determining the parts to be allowed to correspond by detecting a feature point with the position unit of the size of a picture element, and thereafter, obtaining a parallax of this part with finer accuracy than the picture element by calculating a local correlation value. CONSTITUTION:The principle of a stereoscopic vision measurement is utilized, and the left and the right images to become stereo-images are inputted to input parts 1, 1', and stored in store parts 2, 2', respectively. Feature point detecting parts 3, 3' detect a feature point, respectively and obtain the attribute of the feature point and a position coordinate on an image. A local correlation arithmetic part 5 derives the local correlation value of the left and the right images from position coordinates on the left and the right images allowed to correspond by the feature point, and a parallax arithmetic part 6 calculates the parallax of a pair of original images by using an interpolation, based on the obtained correlation. A distance arithmetic part 7 calculates a distance to a photographing object by using this parallax information.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a distance measuring method, and particularly to a distance measuring method that applies the principle of stereoscopic viewing to measure the distance of an object in an indoor or outdoor scene.

For mechanical systems, it is desired to efficiently and automatically input three-dimensional shape data of objects.

[Conventional technology]

To obtain three-dimensional distance information from a pair of stereo images taken of a target object from the viewpoints of both the left and right eyes, first find the corresponding parts between the left and right images and calculate the difference (parallax) between the positions on the images. 8th This is done by applying this parallax to a triangulation formula.

FIG. 1 shows the principle of stereoscopic measurement as a top view, and L3. Lg is a lens of an imaging camera having the same focal length and whose optical axes are arranged parallel to each other; E
i and Hz are the imaging surfaces of lenses L, Lt, and C+, respectively.
, C* are the points where the optical axes of each lens intersect on the imaging plane E+Ez.

P is the measurement target point, and Ql and Qo are the imaging plane El.

The projection point of point P projected onto E8, 2 is the distance between the depths of point P as seen from the lens, f is the distance between the lens and the first layer image plane, and u is the point P from the center of lens L1 The displacement xl is measured from the midpoint M between the center of the lens L and the center of the lens L along the straight line C,ex in the direction from C1 to Ct.

X is a point Q on the X-axis, which is set in the direction from C2 to CI along the straight line C5C8 with C1 and Ct as origins, respectively.
, are the coordinates of Qt. Here, from the figure, Xi /f = (u+d/2)/z (
1)) Cg /f= (u-d/2) /2
(2), and transforming this gives Z ”'d-(/ (Xi Xs ) (
It will be 31. Therefore, by detecting a set of coordinates (x+rX,) representing corresponding points from the left and right images, the distance in the depth direction of that point can be determined. Here, the coordinates X, and x
The difference ΔX=X, -X@ is called parallax.

[Problem to be solved by the invention]

Therefore, the problem is how to obtain disparity information with high accuracy. Conventional methods of this type include (1) providing a moving window for calculation over a pair of stereo images and calculating the local correlation; There are two methods: (2) extracting feature points (edges, etc.) from a pair of stereo images and finding the correspondence between them; 8 for being informed
For example, in the vicinity of the edge of an object that blocks the background, there is a problem in that the portion that extends toward the object and the portion that protrudes toward the background when viewed from the edge have the same parallax. Furthermore, in the latter method, the positions of feature points are given by two-dimensional quantum specific gravity (pixels) of the image, so it has been impossible to detect parallax with a finer precision than this unit.

(Means for Solving Problem ii) The present invention aims to solve the problems of the above two methods and take advantage of the strengths of both methods. This method is characterized by determining the size in units of position, and then calculating the local correlation value to obtain the parallax of this part with an accuracy finer than that of pixels, and will be explained below with reference to one drawing.

FIG. 2 shows an embodiment of the present invention, in which 11' is an image input section, 2.2' is an image storage section, 3.3' is a feature point detection section, 4 is a corresponding feature point detection section, 5 is a local correlation calculation unit, 6
7 is a parallax calculation unit, and 7 is a distance calculation unit. To explain this operation, first, left and right images to be a stereo image are respectively inputted by an image input unit 1.1' consisting of an image input device such as a television camera, and the input images are stored in an image storage unit consisting of a frame memory etc. 2.2' respectively.

The feature point detection unit 3.3' performs feature point detection on each of the stored images and obtains the attributes of the feature points and the position coordinates on the image. Here, specific feature points include, for example, edges, which are discontinuous parts of the image's shading and color, and its attributes include the inclination angle and the shading gradient within the image plane. As for the method for finding this edge, the known technique described in Section 12.2 of "Image Processing Handbook" (edited by Morio Onoe et al., published by Shokodo Co., Ltd.) is used. The detected feature points are detected by the corresponding feature point detection unit 4 as a pair corresponding to each other in the left and right images. As a specific method for detecting corresponding pairs, 9, for example, the relation =
A known technique disclosed in Stereo Matching Using Shape Information of One Segment (IEICE technical report PRU-84, 1988) can be used. The local correlation calculation unit 5 fixes the correlation center for the left image at two of the position coordinates on the left and right images that are correlated by the feature points, for example, at the feature point position of the left image, and fixes the correlation center for the left image at the feature point position of the left image, and fixes the correlation center at the corresponding feature point position of the right image. In the vicinity of
The cross-correlation value is determined while moving in the direction parallel to +Cx. The parallax calculation unit 6 calculates parallax using an interpolation method based on the obtained correlation value. The distance calculation unit 7 applies the parallax information to equation (3) to calculate the distance to the measurement target point.

FIG. 3 shows an example of the calculation principle in the parallax calculation section 6 of FIG. 2. Now, quadratic neighboring X coordinates p, q, and r are found around the X coordinate X of a point on the right image that corresponds to the point having the X coordinate X on the left image.

3p-max (Sp, Sq, 5r) Sp-3 (p, xt) SQ song S (q, X,) 3r-3 (r, X+) -q-1 r=q+1 In the above formula, S (m, n) indicates the cross-correlation value when a correlation window is provided at the position of the X coordinate value m of the left image and the position of the X coordinate value n of the right image, ideally qw z, but edge detection Due to problems such as positional accuracy in
p, Sp).

Find a parabola that passes through (q, Sq), (r, Sr) and find the X coordinate value X□° when the correlation value is maximum on the parabola
Using , 8X-X, -Xg' is the parallax to be found. As a result, parallax can be obtained with a precision finer than that of one pixel. Here is X'.

Using p, q, r, Sp, Sq, Sr.

It says.

In the explanation of the embodiment shown in FIG. 2 above, we have explained the method of using two stereo images to find the correspondence between feature points such as edges, but in order to further increase the credibility of the detection of corresponding feature points, , 3N stereo system using three or more stereo images (for example, Kaneko: A study on three-dimensional shape input using one-point reference image,” Television Society Technical Report, I
It is also possible to calculate the parallax as shown in Figure 3 after finding the corresponding feature point position in pixel units using a method such as C388-30, 1988. Two appropriate images, such as the leftmost and rightmost images, are selected to perform local correlation.) Also in the above explanation.

The distance is measured only for the parts that appear as feature points on the object, but the parts that do not appear as feature points
For areas (tS light, where there is no change in color, and where the area is flat), the distance can be determined by performing interpolation processing using information obtained from the feature point area.

〔Effect of the invention〕

As described above, according to the present invention, the distance from the photographing position to the surface of the object can be measured with high accuracy and reliability.

[Brief explanation of the drawing]

Figure 2 is a diagram explaining the principle of distance measurement using stereoscopic vision.
FIG. 2 shows the configuration of an embodiment of the present invention, and FIG. 3 is an explanatory diagram illustrating the operation of the parallax calculating section in the present invention. In the figure, 1.1' is an image input section, 2.2' is an image storage section,
3.3' is a feature point detection section, 4 is a corresponding feature point detection section, 5 is a local correlation calculation section, 6 is a disparity calculation section, and 7 is a distance calculation section. Patent applicant Nippon Telegraph and Telephone Corporation

Claims (1)

[Claims] A process of detecting feature points of a pair of images forming a stereoscopic image, a process of detecting a corresponding feature point forming a pair between each image from a set of feature points on each detected image, and a position of the corresponding feature point. A process of calculating local correlation between a pair of original images, a process of obtaining disparity information of the pair of original images by applying an interpolation method to the local correlation value, and a process of calculating the distance to the photographed object using the disparity information. 1. A distance measuring method comprising: calculating a distance to a photographing target with high precision using parallax information with a precision finer than that of a pixel, which is a two-dimensional quantization unit of an image.