JPH03115916A - Distance image generation system - Google Patents

Abstract

PURPOSE:To reduce errors included in distance information on a section connecting two points on an object by converting parallax to the same point on the object which is obtained from two pieces of image information, and then interpolating the distance information and calculating distance information on the surface of the object. CONSTITUTION:The object is observed in different directions to obtain two pieces of image information 111 and 113, to which two picture elements which are estimated to correspond to the same point on the object are made to correspond in order. A parallax calculating means 117 calculates the parallax from said two picture elements and a distance calculating means 119 calculates the distance to the point on the object corresponding to the picture elements by using the parallax. A distance interpolating means 121 calculates distance information on a noncorresponding picture element by interpolation according to the distance information on the corresponding point on the object. A three- dimensional coordinate calculating means 123 calculates the three-dimensional coordinate values of the corresponding point by using the positions and distances of the respective picture elements.

Description

[Detailed Description of the Invention] [Summary] This invention relates to a distance image generation method that performs stereoscopic recognition of an object based on three-dimensional information generated by binocularly viewing the object. The purpose is to reduce the error included in the distance information of the section connecting the two pixels that are estimated to correspond to the same point on the object from two image information obtained by observing the object from different directions. a disparity calculation means that calculates the parallax from two associated pixels, and a distance calculation means that calculates the distance to a point on the object corresponding to each pixel using the parallax. , distance interpolation means for interpolating and calculating distance information of uncorresponding pixels forming the surface of the object from each image information based on distance information of a point on the object corresponding to each pixel; and three-dimensional coordinate calculation means for calculating the three-dimensional coordinate value of the corresponding point using the position and distance of the point.

[Industrial application field]

The present invention relates to a distance image generation method that performs stereoscopic recognition of an object based on three-dimensional information generated by binocularly viewing the object.

In particular, the present invention calculates the distance to the same point on the object using the principle of triangulation from both image information obtained by observing the object with two cameras, and determines the three-dimensional coordinates of the object. This relates to a distance image generation method.

[Conventional technology]

FIG. 7 is a block diagram illustrating a conventional distance image generation method.

In the figure, the object is observed by two cameras, and two image information, a left image 211 and a right image 213, are extracted. The edge extraction unit 215 extracts a left image 211, a right image 21,
Extract the edges of the object from 3. The mapping unit 217 sequentially correlates the edges of the left image 211 and the right image 213 that are estimated to correspond. The parallax calculation unit 519 calculates the parallax from the associated edges, and also interpolates the parallax between the edges in order to calculate all the distances to the object surface. The distance calculation unit 521 calculates parallax and camera parameters (
Based on the distance between the two cameras used for observation, focal length, etc.), distance information to the object is calculated using the principle of triangulation. The three-dimensional coordinate calculation unit 225 calculates three-dimensional coordinate values of the object surface based on the distance information.

The operation of the association section 217 will be described below with reference to FIG.

In the figure, the left image has three edges a on the scan line y0.
, b, c and both ends of the image. Also, the right image is the scanning line y.

3 edges d, e, f on top and a total of 5 on both sides of the image
It has dotted edges. Note that the scanning line y0 indicates scanning performed in the horizontal direction of each image.

The mapping unit 217 correlates edges on the scanning line y0 of each image using dynamic programming.

Dynamic programming uses a combination of edges in the left and right images to create a search plane (left image edge points n, right image edge points n
This is a method of configuring a network (having R nodes) and processing it as an optimization problem of route selection from node O to node M corresponding to both ends of the object (References, Ota Tomo-1 Katsuo Ikeda Author, T. Iffl Transactions of the IEICE, '8
5/4 Vol. J-68D No. 4, "Segmental correspondence method for stereo images using dynamic programming").

The operation of the parallax calculation unit 219 will be described below with reference to FIG. 9.

In the left image, an edge of the object is recognized at the coordinates aL and bL on the scanning line y0 (Fig. 9 (a)), and in the right image, an edge is recognized on the scanning line y0 at the coordinates allsb11 (Fig. 9). (B)).

Parallax is the difference in the coordinates of corresponding edges in the left and right images when they are displayed in each image, and is an amount equivalent to the lateral shift in the position of the object recognized in each image.

The parallax calculation unit 219 calculates coordinates aL, b based on the left image.
Parallax (at-a, l), (b+
, bR) (Fig. 9 (c)) and the point (at, ,
The interpolation of the parallax between at-aR) and (bt, bt bR) is performed for each scanning line (FIG. 9 (C) ◯).

The operation of the distance calculation section 221 will be described below with reference to FIG. 1O.

From the parallax (at a*), the distance R between the camera and the object
1 is calculated, and the distance R2 between the camera and the object is calculated from the parallax (bL-bR) (FIG. 10). In addition, the parallax calculation means 219 calculates the parallax (at att), (
The respective distances are calculated from the parallaxes interpolated between bL b++) (FIG. 10, 0).

[Problems to be Solved by the Invention] By the way, in the conventional distance image generation method described above, the parallax is interpolated, and this is converted into a distance to calculate the section (object surface) connecting points (edges) on the object. I was getting 3D information.

However, since the interpolation performed by the parallax calculation unit is performed for each image that has already been quantized, it contains an error in the quantization level, and the distance information calculated from the parallax that includes this error is changed from the parallax to the distance information. An even larger error will be included in the conversion process. That is, for example, the distance of a surface, which should originally change smoothly, changes stepwise in accordance with the constraint of the parallax quantization level, and unnatural three-dimensional information is generated (○ in FIG. 10).

The present invention is intended to solve these problems,
It is an object of the present invention to provide a distance image generation method that can reduce errors included in distance information of a section connecting two points on a target object.

[Means to solve the problem]

FIG. 1 is a bronc diagram of the principle of the present invention.

In the figure, the means 115 for associating two image information 11L1], 3 obtained by observing the object from different directions.
Then, two pixels estimated to correspond to the same point on the object are sequentially associated.

The parallax calculation means 117 calculates the parallax from two correlated pixels.

The distance calculation means 119 uses parallax to calculate the distance to a point on the object corresponding to each pixel.

The distance interpolation means 121 interpolates and calculates the distance information of uncorresponding pixels forming the surface of the object from each image information based on the distance information of the point on the object corresponding to each pixel.

The three-dimensional coordinate calculation means 123 uses the position and distance of each pixel to calculate the three-dimensional coordinate value of the corresponding point.

[For production]

The object is observed from different directions and has two image information 1
11.113. Two image information 11
1.113, two pixels estimated to correspond to the same point on the object are sequentially associated. The parallax is extracted from each associated pixel, and the distance to the point on the object is calculated.

Distance information of uncorresponding pixels is calculated by interpolating distance information obtained from matched pixels. Three-dimensional coordinate values are calculated from this distance and used for three-dimensional recognition of the object.

In other words, since the distance information of the pixels that make up the object surface is calculated by interpolating the distance information obtained from the points on the object, it is possible to reduce errors included in the distance information of the section connecting two points. .

〔Example〕

Hereinafter, embodiments of the present invention will be described in detail based on the drawings.

FIG. 2 is a block diagram showing the configuration of an embodiment of the present invention.

In the figure, the object observed by two cameras is output to the edge extraction unit 215 as a left image 211 and a right image 213. The output of the edge extraction section 215 is mapped to section 2.
17. The output of the association section 217 is output to the parallax calculation section 219. The output of the parallax calculation unit 219 is
It is output to the distance calculation section 221. The output of the distance calculation unit 221 is sent to the distance interpolation unit 223 and the three-dimensional coordinate calculation unit 225.
is output to.

Here, the correspondence between FIG. 1 and FIG. 2 will be explained.

Image information 111.113 includes left image 211 and right image 21.
Corresponds to 3.

The mapping means 115 corresponds to the mapping section 217.

The parallax calculation means 117 corresponds to the parallax calculation section 219.

The distance calculation means 119 corresponds to the distance calculation half section 221.

The distance interpolation means 121 corresponds to the distance interpolation section 223.

The three-dimensional coordinate calculation unit 123 includes a three-dimensional coordinate calculation unit 225
corresponds to

In the left image 211 and the right image 213, after edges are extracted in the edge extraction unit 215, edges that are estimated to correspond to the same point on the object are associated in the association unit 217, and then the edges are associated with the disparity calculation unit 0 219. is output to. The parallax calculation unit 219 calculates the parallax of the edge and outputs it to the distance calculation unit 221.

The distance calculation unit 221 calculates the distance from this parallax.

The principle of distance calculation will be explained below with reference to FIGS. 3 and 4.

FIG. 3 is a diagram explaining the observation system.

In the figure, the left image 211 displays the observed object in the coordinate system XL't, and the right image 213 displays the observed object in the coordinate system XR3'R.

In addition, in the entire observation system, the straight line connecting the origin of each coordinate system of the left image 211 and the right image 213 is the Z axis, and the left image 211,
The midpoint of the straight line connecting the origins of each coordinate system of the right image 213 is the origin O, the straight line connecting the origin O and the object is the Y axis, the Z axis,
A three-dimensional coordinate system is constructed in which the Z-axis is perpendicular to the Y-axis.

Note that the positions of the left image 211 and the right image 213 in the three-dimensional space are the positions where two cameras that capture both images are installed.

FIG. 4 is a diagram explaining the principle of distance calculation.

■■ The figure is a diagram of the above-mentioned three-dimensional coordinate system viewed from the Z-axis direction.

In the figure, the origin A of the coordinate system of the left image 211, the right image 2
The origin B of the 13 coordinate systems is (-
a, 0.0), (a, 0.0), the distance between the two cameras (baseline length) is 2a, and the convergence angle of both optical axes (convergence angle)
is in the relationship 2α (0≦α<π/2).

The case of observing a dot on an object will be described below. Note that the dot on the object has coordinates (η
1. ηy), and in the right image 213, the point F has coordinates (ξ, ξy). Also, let the distance of line segment DE be R14, the distance of line segment DF be RR, the intersection of the straight line obtained by extending line segment DB and the Z axis be point G, and the intersection of line segment DF and the Z axis be point I]. .

Since ΔABC and ΔGHD are similar, the following relationship holds true, and the length of the line segment DG is as follows.

Here, the relationship in which point E (η, ηy) and point F (ξ8.ξy) are projected onto the Z axis of the three-dimensional coordinate system parallel to the optical axis is (η, ηy) → (-a −t− 7/x/cosα+
O+ 77y) (3) (ξ8.ξy) Mourning (a0ξx /cosα, Q, ξy
)(4) becomes.

Therefore, the length of the line segment GH in the three-dimensional coordinate system is becomes.

As shown in equation (5), the line segment GH has the term (η9-ξ,) and is a function of the difference in coordinate values of the object observed in the left image 211 and the right image 213, that is, the parallax. It turns out that there is something.

3 Using equation (5), the line segment DC shown in equation (2) can be calculated as (6
) is transformed into.

On the other hand, point G and χ, y1. of the left image 211. If the distance to the plane is Lo, and the distance between point H and the XRy* plane of the right image 213 is Ro, then the distance is. , Ro and the coordinate values η8 and ξ8 in the left and right images of the object are as follows: 1- o = + 77 X l tan rx
−−−, −, , −(7) Ro=lξ, l ta
n α---(8).

Using equation (7), the distance R3° between the left image 211 and the dot is RL-DG-1 when η8×0. ---(9)η
When 8≧0, R, -DG+L. −θ■.

Also, using equation (8), the distance MR,l from the right image 213 to the point is RR-D C, 1RO--CII when 4 ξ8<0
) When ξ8≧0, RR=DG−R,−021.

In this way, the coordinates of the points (edges) on the object observed by the left image 211 and the right image 213, the parallax, and each image (
The distance between the left image, right image (camera) and the object can be calculated from the distance between the left and right images (camera) and the angle between the optical axis and the Y axis.

The distance calculation unit 221 calculates an image 21 to the left of the parallax corresponding to the two edges on the object output from the parallax calculation unit 219.
The distances R1 and R2 corresponding to the distance RL between No. 1 and the object are determined, and the distances R, , R, and the coordinate values of the edges are output to the distance interpolation unit 223. The distance interpolation unit 223 calculates D D A (d
digital differential analysis
er) Interpolate distance information between edges by a circuit.

Hereinafter, with reference to FIGS. 5 and 6, the distance interpolation unit 22
The operation of the DDA circuit No. 3 will be explained.

The distance interpolation unit 223 calculates the coordinate values aL, bc of the two edges in the left image 211 output from the distance calculation unit 221 (here, it is assumed that aL<bL).
Then, the distances R,,R2 (here, R,>R2) to the points on the object corresponding to the left image 211 and the two edges are input, and the point P(al,,R1) and the point P(al,,R1) are input. Q(
bL, R2) (Fig. 5, between) is interpolated.

■ fat -bLt is assigned to the variable LENGTH.

■ Next, L E N G T II and lR + ' R
The magnitude of zl is compared. LENGTII is IR, -
If it is smaller than R21, IJ?
I-RZ I is assigned.

That is, the number of pixels in the χ-axis direction or the Y-axis direction is set as the number of repetitions of the interpolation process.

■ Incremental values X4, Y i in the X and Y axis directions used for interpolation
n c is X,, lc= (a,, -su,, ) / LEN
GTI(-03)Y+, 1c-(R+ -R2)/
LENGTH---04).

■ The starting points of interpolation (χ, Y) are respectively X = a
L + 0.5 −−1−−−−−−−
05) y = R, +0.5 ---'-
- Set to -00, the interpolation process iteration number parameter I
is set to "1".

Here, 0.5 is a value used in the process of converting X and Y into integers.

■ The number of iterations parameter I is L E N G T II
It is determined whether or not it has exceeded.

If the number of iterations parameter I does not exceed LENGTH, the next interpolation point is X = X + X, nc−−−−−−−−θη
Y = Y 10 Y i , 1cm--1・-1--08
), and "1" is added to the repetition number parameter I.

If the number of iterations parameter (2) exceeds LENGTI-1, the interpolation process ends.

In this way, point (at, R+), point (at).

R2), points (bL, Rz), and points (bt, R,), on a grid that takes integer values, generate the sequence of points closest to the line segment PQ (○ in Figure 5), and interpolate. It is done.

This interpolation result is calculated by the distance calculation unit 221 from the three-dimensional It is output to the coordinate calculation section 225.

When the interval connecting the two edges is interpolated and distance information between the camera and the object (distance information on the object surface) is generated,
By converting the coordinate systems of the left image 21i and the right image 213 into a three-dimensional coordinate system, the three-dimensional coordinate values of the object surface can be calculated.

Coordinate transformation of the coordinate system of the left image 211 to the three-dimensional coordinate system is performed by the following, and coordinate transformation of the coordinate system of the right image 213 to the three-dimensional coordinate system is performed by the following.

The three-dimensional coordinate calculation unit 225 calculates the formula 09), C! Three-dimensional information can be obtained by using either one of the e-formulas.

〔Effect of the invention〕

According to the present invention, after converting the parallax with respect to the same point on the object obtained from two image information into distance information, this distance information is interpolated to calculate the distance information with respect to the object surface. It is possible to reduce the error included in the distance information of the section connecting the above two points, and it is possible to reproduce natural image information even for a three-dimensional object.

[Brief explanation of drawings]

Fig. 1 is a block diagram of the principle of the present invention; Fig. 2 is a block diagram showing the configuration of an embodiment of the invention; Fig. 3 is a diagram illustrating the observation system; Fig. 4 is a principle diagram illustrating distance calculation; 9 Figure 5 is a diagram explaining interpolation processing, Figure 6 is a diagram explaining the operation of the DDA circuit, Figure 7 is a block diagram showing the configuration of a conventional example, and Figure 8 is a diagram explaining dynamic programming. FIG. 9 is a diagram for explaining the operation of the parallax calculation section, and FIG. 10 is a diagram for explaining the operation of the distance calculation section. In the figure, 111 and 113 are image information, 115 is a correspondence means, 117 is a parallax calculation means, 119 is a distance calculation means, 121 is a distance interpolation means, 123 is a three-dimensional coordinate calculation means, 211 is a left image, 213 is the right image, 215 is the edge extraction section, 2]7 is the correspondence section, 219.519 is the disparity calculation section, 0 221.521 is the distance calculation section, 223 is the distance interpolation section, 225 is the three-dimensional coordinate calculation section It is. 1 Principle block diagram of the present invention Fig. 1 Block diagram showing the configuration of an embodiment Fig. 2 Fig. illustrating the observation system Fig. Principle diagram explaining distance calculation Fig. 1 Block diagram showing the configuration of the conventional example figure

Claims (1)

[Claims] (1) An association means (115) that sequentially associates two pixels estimated to correspond to the same point on the object from two image information (111, 113) obtained by observing the object from different directions. a parallax calculation means (117) that calculates the parallax from the two correlated pixels; and a distance calculation means (119) that calculates the distance to a point on the object corresponding to each pixel using the parallax. ), and distance interpolation that calculates by interpolating distance information of uncorresponding pixels forming the surface of the object from each image information based on distance information of a point on the object corresponding to each pixel. Means (1
21) and three-dimensional coordinate calculation means (
123).