JP2813211B2 - Range image generation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Summary] The present invention relates to a range image generation method for performing stereoscopic recognition of an object based on three-dimensional information generated by stereoscopically viewing the object. It is an object to reduce an error included in distance information of a section connecting two pixels, and two pixels estimated to correspond to the same point on the object from two pieces of image information obtained by observing the object from different directions. An associating means for sequentially associating, a disparity calculating means for calculating the disparity from the associated two pixels, a distance calculating means for calculating a distance to a point on an object corresponding to each pixel using the disparity, Based on distance information of a point on the object corresponding to each pixel, distance interpolation means for calculating by interpolating distance information of uncorresponding pixels constituting the surface of the object from each image information, Using position and distance,
Three-dimensional coordinate calculation means for calculating three-dimensional coordinate values of corresponding points.

[Industrial applications]

The present invention relates to a range image generation method for performing stereoscopic recognition of an object based on three-dimensional information generated by stereoscopically viewing the object.

In particular, the present invention calculates the distance to the same point on the object based on the principle of triangulation from both image information obtained by observing the object with two cameras, and obtains three-dimensional coordinates of the object. The present invention relates to a range image generation method.

[Conventional technology]

FIG. 7 is a block diagram illustrating a conventional range image generation method.

In the figure, an object is observed by two cameras, and two pieces of image information of a left image 211 and a right image 213 are extracted.
The edge extracting unit 215 extracts an edge of the object from the left image 211 and the right image 213. The associating unit 217 sequentially associates edges of the left image 211 and the right image 213 that are estimated to correspond. The parallax calculation unit 519 calculates the parallax from the associated edges and interpolates the parallax between the edges in order to obtain all the distances to the object surface. Distance calculator 521
Calculates distance information to an object based on parallax and camera parameters (distance between two cameras used for observation, focal length, etc.) based on the principle of triangulation. The three-dimensional coordinate calculation unit 225 calculates three-dimensional coordinate values of the object surface based on the distance information.

Hereinafter, the operation of the association unit 217 will be described with reference to FIG.

In the figure, the left image, the scanning lines y ₀ on the three edges a, b, and has both ends total 5 points edge of c and image. The right image, the scanning lines y ₀ on the three edges d, e, and has both ends total 5 points edge of f and image, is to be noted that the scan line y ₀ is the horizontal direction of the image 2 shows the scanning performed.

Association unit 217 associates the dynamic programming an edge on the scan line y ₀ for each image.

In the dynamic programming, a search plane (left image edge points n _L × right image edge points n _{R) is obtained by} combining the edges of the left image and the right image.
This is a method of processing a route as an optimization problem from a node O to a node M corresponding to, for example, both ends of an object (reference literature, Yuichi Ota, Katsuo Ikeda) , IEICE Transactions, '85 / 4 Vol.J68D
No4, "Segment correspondence method for stereo images by dynamic programming").

Hereinafter, the operation of the parallax calculation unit 219 will be described with reference to FIG.

In the left image, the edges are recognized at the coordinates a _L and b _L at the scanning line y ₀ in the left image (FIG. 9 (A)), and in the right image, the scanning line y ₀
Then, an edge is recognized at the coordinates a _R and b _R (FIG. 9 (b)).

The parallax is a difference in coordinates when corresponding edges of the left image and the right image are displayed in each image, and is an amount corresponding to a lateral displacement of a position of an object recognized in each image.

Parallax calculating unit 219, based on the left image, coordinates a _L, a parallax corresponding to each edge of the _{b L (a L -a R)} , determine the (b _L -b _R) (FIG. 9 (c) ●), points (a _L , a _L −a _R ), (b _L , b _L −
Interpolation of parallax between b _R ) is performed for each scanning line (Fig. 9 (c)-).

Hereinafter, the operation of the distance calculation unit 221 will be described with reference to FIG.

Parallax than (a _L -a _R), the calculated distance R ₁ between the camera and the object, the parallax distance between the camera and the object from (b _L -b _{R) R 2}
Is calculated (Fig. 10 ●). Further, the parallax calculating unit 219 calculates distances from the parallaxes interpolated between the parallaxes (a _L −a _R ) and (b _L −b _R ) (FIG. 10A).

[Problems to be solved by the invention]

By the way, in the above-described conventional range image generation method, the parallax is interpolated and converted into a distance to obtain three-dimensional information of a section (object surface) connecting points (edges) on the object. However, the interpolation performed by the disparity calculation unit includes a quantization level error because the interpolation is performed in units of images that have already been quantized, and distance information calculated from disparity including this error is converted from disparity to distance information. In the conversion process, a larger error is included. That is, for example, the distance of the surface that should originally change smoothly changes stepwise according to the restriction on the quantization level of parallax, and unnatural three-dimensional information is generated (○ in FIG. 10).

An object of the present invention is to solve such a point, and it is an object of the present invention to provide a distance image generation method capable of reducing an error included in distance information of a section connecting two points on an object.

[Means for solving the problem]

 FIG. 1 is a block diagram showing the principle of the present invention.

In the figure, the associating means 115 obtains two pieces of image information 111 and 113 obtained by observing an object from different directions.
Two pixels estimated to correspond to the same point on the object are sequentially associated.

The parallax calculating means 117 calculates the parallax from the two associated pixels.

The distance calculation unit 119 calculates a distance to a point on the object corresponding to each pixel using the parallax.

The distance interpolation means 121 calculates the distance information of the non-corresponding pixels constituting the surface of the object from each image information based on the distance information of the point on the object corresponding to each pixel.

The three-dimensional coordinate calculation unit 123 calculates the three-dimensional coordinate value of the corresponding point using the position and distance of each pixel.

(Operation)

The object is observed from different directions and the two image information
Extracted as 111 and 113. Two pieces of image information 111 and 113
Is sequentially associated with two pixels that are estimated to correspond to the same point on the object. The parallax is extracted from each associated pixel, and a distance calculation for a point on the object is performed.

The distance information of the uncorresponding pixel is calculated by interpolating the distance information obtained from the associated pixel. A three-dimensional coordinate value is calculated from this distance and used for three-dimensional recognition of the object.

That is, since the distance information of the pixels constituting the object surface is calculated by interpolating the distance information obtained from the points on the object, errors included in the distance information of the section connecting the two points can be reduced. .

〔Example〕

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 2 is a block diagram showing the configuration of the embodiment of the present invention.

In the figure, the objects observed by the two cameras are output to the edge extraction unit 215 as a left image 211 and a right image 213. The output of the edge extraction unit 215 is output to the association unit 217. The output of the associating unit 217 is output to the parallax calculating unit 219. The output of the parallax calculation unit 219 is output to the distance calculation unit 221. The output of the distance calculation unit 221 is output to the distance interpolation unit 223 and the three-dimensional coordinate calculation unit 225.

 Here, the correspondence between FIG. 1 and FIG. 2 will be described.

The image information 111 and 113 correspond to the left image 211 and the right image 213.

 The associating unit 115 corresponds to the associating unit 217.

 The parallax calculating unit 117 corresponds to the parallax calculating unit 219.

 The distance calculating means 119 corresponds to the distance calculating unit 221.

 Distance interpolation means 121 corresponds to distance interpolation section 223.

The three-dimensional coordinate calculation unit 123 corresponds to the three-dimensional coordinate calculation unit 225.

After the edges are extracted by the edge extraction unit 215, the left image 211 and the right image 213 are output to the disparity calculation unit 219 in association with the edges presumed to correspond to the same point on the object in the association unit 217. Is done. The parallax calculation unit 219
Edge parallax is obtained and output to distance calculation section 221.

 The distance calculation unit 221 calculates a distance from the parallax.

Hereinafter, the principle of the distance calculation will be described with reference to FIGS. 3 and 4.

 FIG. 3 is a diagram illustrating an observation system.

In the figure, the left image 211 represents the observed object in the coordinate system x
Displayed in _L− y _L , the right image 213 shows the observed object in the coordinate system.
Expressed as x _R −y _R.

In the observation system as a whole, a straight line connecting the origins of the respective coordinate systems of the left image 211 and the right image 213 is defined as the X axis, and the left image 211 and the right image 2
A three-dimensional coordinate system is constructed in which the middle point of the straight line connecting the origins of the 13 coordinate systems is defined as the origin O, the straight line connecting the origin O and the object is defined as the Y axis, and the X axis and the Y axis are perpendicular to the Z axis. I do.

Note that the positions of the left image 211 and the right image 213 in the three-dimensional space are positions where two cameras for capturing both images are installed.

FIG. 4 is a diagram for explaining the principle of distance calculation. The figure is a diagram in which the above-described three-dimensional coordinate system is viewed from the Z-axis direction.

In the figure, the origin A of the coordinate system of the left image 211, the right image 213
The origin B of the coordinate system is (−a, 0,
0), (a, 0,0), the distance (base line length) between the two cameras is 2a, and the angle (convergence angle) between both optical axes is 2α (0 ≦ α).
<Π / 2).

Hereinafter, a case where the point D on the target object is observed will be described. Note that a point D on the object is a point E at the coordinates (η _x , η _y ) in the left image 211 and a coordinate (ξ _x ,
点_y ). Also, the distance of the line segment DE
Let R _{L be} the distance between line segment DF and R _R, and extend straight line DE and X
The intersection of the axes is point G, and the intersection of the line segment DF and the X axis is point H.

ΔABC and ΔGHD are similar, so Holds, the length of the line segment DG is Becomes

Here, the relationship of projecting the point E (η _x , η _y ) and the point F (ξ _x , ξ _y ) onto the X-axis of the three-dimensional coordinate system in parallel with the optical axis is (η _x , η _y ) (− a−η _x / cos α, 0, η _y ) (3) (ξ _x , ξ _y ) (a−ξ _x / cos α, 0,, ξ _y ) (4)

Therefore, the length of the line segment GH in the three-dimensional coordinate system is Becomes

As shown in the equation (5), the line segment GH has a term of (η _x , _{ｘ x} ), and a difference between coordinate values of an object observed in the left image 211 and the right image 213, that is, a function of disparity. It turns out that it is.

Using equation (5), the line segment DG shown in equation (2) is Is transformed into

On the other hand, the distance between x _L y _L plane of the point G and the left image 211 and L _0, Taking the distance between the x _R y _R plane of the point H and the right image 213 R _0, the distance L _0, R ₀ and the relationship between the coordinate values η _x, ξ _x of the left and right images of the _{object, L 0 = | η x |} tanα ...... (7) R 0 = | a _{tanα ...... (8) | ξ x} .

(7) using equation, the distance R _L of the left image 211 and the point D, eta _x <0, when _{R L = DG-L 0 ......} (9) η x ≧ 0, R L = DG + L ₀ (10)

Further, (8) using the formula, the distance R _R to the right image 213 and the point D, when xi] _x <0, when _{R R = DG + R 0 ......} (11) ξ x ≧ 0, R R = DG−R ₀ (12)

In this way, the coordinates of the point (edge) on the object observed by the left image 211 and the right image 213, the parallax, the distance between the images (cameras), and the angle between the optical axis and the Y axis, The distance between the left image and the right image (camera) and the object can be calculated.

The distance calculation unit 221 obtains distances R ₁ and R ₂ corresponding to the distance _RL between the left image 211 and the object from the parallax corresponding to the two edges on the object output from the parallax calculation unit 219, The distances R ₁ and R ₂ and the coordinate values of the edge are output to the interpolation unit 223. The distance interpolation unit 223 is a digital differential analyzer (DDA)
A circuit interpolates distance information between edges.

Hereinafter, with reference to FIG. 5 and FIG.
The operation of the DDA circuit 3 will be described.

Distance interpolation section 223 outputs coordinate values a _L and b _{L of the} two edges output from distance calculation section 221 in left image 211 (here, a _L <b _L is assumed) and left images 211 and 2. Distances R ₁ and R ₂ (here, it is assumed that R ₁ > R ₂ ) to points on the object corresponding to the two edges are input, and a point P (a _L , R ₁ ) is input.
And the distance between the points Q (b _L , R ₂ ) (FIG. 5, ●) is interpolated.

| A _L −b _L | is assigned to the variable LENGTH.

Next, the magnitudes of LENGTH and | R ₁ −R ₂ | are compared. LENG
When TH is smaller than | R ₁ −R ₂ |, | R ₁ −R ₂ | is substituted for LENGTH.

That is, the larger number of pixels in the X-axis and Y-axis directions is set as the number of repetitions of the interpolation process.

X and Y axis increments X _inc and Y used for interpolation
_inc is set to X _inc = (a _L −b _L ) / LENGTH (13) Y _inc = (R ₁ −R ₂ ) / LENGTH (14)

The starting point (X, Y) of the interpolation is set as follows: X = a _L +0.5 (15) Y = R ₁ +0.5 (16) "1"
Is set to

Here, 0.5 is a value used in the process of converting X and Y into integers.

It is determined whether or not the repetition number parameter I has exceeded LENGTH.

If the repetition number parameter I does not exceed LENGTH, the next interpolation point is obtained by X = X + X _inc (17) Y = Y + Y _inc (18) Is added.

When the iteration number parameter I exceeds LENGTH, the interpolation processing ends.

In this way, on a grid that takes an integer value between points (a _L , R ₁ ), points (a _L , R ₂ ), points (b _L , R ₂ ), and points (b _L , R ₁ ) Generates a point sequence closest to the line segment PQ (FIG. 5A), and performs interpolation.

This interpolation result is output from the distance calculation unit 221 to the three-dimensional coordinate calculation unit 225.

When the section connecting the two edges is interpolated and the distance information between the camera and the object (distance information on the surface of the object) is generated, the distance from the coordinate system of the left image 211 and the right image 213 to the three-dimensional coordinate system is calculated. By the coordinate conversion, a three-dimensional coordinate value of the object surface can be calculated.

The coordinate transformation of the coordinate system of the left image 211 into a three-dimensional coordinate system is as follows: The coordinate transformation of the coordinate system of the right image 213 into a three-dimensional coordinate system is performed as follows. It is done in.

The three-dimensional coordinate calculation unit 225 can obtain three-dimensional information by using one of the equations (19) and (20).

〔The invention's effect〕

According to the present invention, after converting the parallax for the same point on the object obtained from the two pieces of image information into distance information, the distance information is calculated by interpolating the distance information. The error included in the distance information of the section connecting the above two points can be reduced, and natural image information can be reproduced even for a three-dimensional object.

[Brief description of the drawings]

 FIG. 1 is a block diagram showing the principle of the present invention, FIG. 2 is a block diagram showing the configuration of an embodiment of the present invention, FIG. 3 is a diagram illustrating an observation system, FIG. FIG. 5 is a diagram illustrating an interpolation process, FIG. 6 is a diagram illustrating the operation of a DDA circuit, FIG. 7 is a block diagram illustrating a configuration of a conventional example, and FIG. 8 is a diagram illustrating a dynamic programming method. FIG. 9 is a diagram for explaining the operation of the parallax calculating unit, and FIG. 10 is a diagram for explaining the operation of the distance calculating unit. In the figure, 111 and 113 are image information, 115 is associating means, 117 is disparity calculating means, 119 is distance calculating means, 121 is processing interpolation means, 123 is three-dimensional coordinate calculating means, 211 is a left image, and 213 is The right image, 215 is an edge extraction unit, 217 is an association unit, 219 and 519 are disparity calculation units, 221 and 521 are distance calculation units, 223 is a distance interpolation unit, and 225 is a three-dimensional coordinate calculation unit.

Continuation of front page (58) Fields investigated (Int.Cl. ⁶ , DB name) G01B 11/00-11/30 G01C 11/00-11/34 H04N 13/00-13/04

Claims (1)

(57) [Claims] An associating means for sequentially associating two pixels estimated to correspond to the same point on the object from two pieces of image information (111, 113) obtained by observing the object from different directions. 115); disparity calculating means (117) for calculating the disparity from the two pixels associated with each other; and distance calculating means for calculating a distance to a point on an object corresponding to each pixel using the disparity. (119) and a distance calculated by interpolating distance information of an uncorresponding pixel constituting the surface of the object from each of the image information based on distance information of a point on the object corresponding to each of the pixels. A distance image generation unit comprising: an interpolation unit (121); and a three-dimensional coordinate calculation unit (123) that calculates a three-dimensional coordinate value of a corresponding point using the position and distance of each pixel. method.