JPS62249286A - Position measuring system using correspondence relation between images - Google Patents

Abstract

PURPOSE:To execute accurately the corresponding between the points of a pair of images by counting the force of gravity corresponding to the characteristic quantity (brightness, and changing quantity of the brightness of a near point) of the pair of images. CONSTITUTION:The image information photographed by a TV camera 11 is stored into an image memory part 12. In case of a binocular stereopsia, two images photographed simultaneously from a different position are stored. A pretreatment part 13 reads image information from the image memory part 12, counts the average brightness and the changing quantity of the brightness of a near point for a small area of suitable dimensions and sends the result to a corresponding point determining part 14. The corresponding point determining part 14 counts the corresponding relation between respective small areas with the received data and sends the instruction to the pretreatment part 13 based upon the result. The pretreatment part 13 executes the changing of the size of an area and the setting of an initial value. By repeating such a processing, when the calculating result of the corresponding relation is sufficiently converged, the counting result is sent to a three-dimensional distance determining part 15 and here, the position is counted.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a position measuring device, and in particular to various devices ( For example, robots with vision, automatic control devices for vehicles that respond to the external environment.

It is indispensable for process control equipment (such as process control equipment that works with ITV).

[Conventional technology]

In position measuring devices of the type mentioned above, it is necessary to determine by automatic processing which point in one image corresponds to which point in the other image. One of the conventional methods for this purpose is "science."

Volume 194, 15. October 1976 ("5cienc
e” Vol.

194, 150ctober, 1976) No. 28
Cooperative calculation of P-steric disequilibrium by D. Marr and T. Poggio” published on pages 3-287 (D, N
arr.

T, Poggio Cooperativa Com
putation of 5tare.

Dfsparity'')].

[Problem that the invention seeks to solve]

This method is suitable for high-speed processing because parallel calculations are possible. However, this method only works for images consisting of pixels that are quantized to two levels.

Since the purpose is to find the correspondence between pixels, the correspondence is rough; that is, it is not possible to correspond a certain pixel in one image to a position intermediate between pixels in the other image, and as a result, the calculated The accuracy of the position is insufficient. Further, since only images quantized to two levels can be handled, incorrect correspondences are made, and the calculated positions are often greatly incorrect.

An object of the present invention is to refine the correspondence between points in a pair of images, and to improve the accuracy of position measurement.

[Means for solving problems]

The feature of the present invention is to calculate the gravitational force corresponding to the feature amount of one image and the feature amount (brightness, amount of change in brightness of neighboring points, etc.) of the other image.

Usually, each point has a plurality of corresponding points where the gravitational force becomes large. One of these corresponding points is the correct corresponding point. In order to find this correct corresponding point, a first-order property is used. In other words, to take advantage of the fact that variations in depth on the surface of an object are generally smooth, and discontinuous changes occur only at the edges of the object.

Consider a network that connects each point in one image with each point in the other image.

At each node (connection point) on this network, the attractive force acting between corresponding points on both images is calculated, and corresponding points are determined from the evaluation value distribution of this attractive force. In order to find the distribution of such attractive forces, we introduce a coefficient to evaluate the attractive force, and evaluate the degree of correspondence by multiplying this coefficient by the attractive force.6 The value of the coefficient at each node on the network is calculated from that node. A steady-state solution is found by adjusting the correspondence evaluation values of nodes in the vicinity of . At this time, the evaluation value of the attractive force possessed by each node represents the degree of correspondence between the points on the image corresponding to those nodes.

[Effect]

Normally, the deviation between corresponding points between two images to be processed is very small, so the number of points (number of nodes on the network) connected to a point on each image is not very large. Automatic processing using a network like this becomes practical.

〔Example〕

FIG. 1 is a block diagram of an embodiment of a position measuring device to which the present invention is applied. The information representing the image captured by the TV camera 11 is temporarily stored in the image storage unit 13. In the case of monocular stereoscopic vision, two images captured simultaneously from different positions are stored. If there is a relative movement between the measurement subject and the measurement target, the calculation unit 10 may store two images taken at different times.

In this example, it consists of a preprocessing section 13, a corresponding point determination section]4, and a distance calculation section 15. The preprocessing section 13 is.

The image information is read from the image storage section 12, and the average brightness and the amount of change in brightness of neighboring points are calculated for each small region of appropriate size, and the results are sent to the corresponding point determination section 14. The corresponding point determining unit 14 calculates the correspondence between each small region using the received data, and sends an instruction to the preprocessing unit 13 based on the result. Performs changes in size, preprocessing, and setting initial values for calculations.

When such processing is repeated an appropriate number of times and the calculation result of the correspondence relationship is sufficiently converged, the calculation result is sent to the three-dimensional distance calculation section 15, where the position calculation is performed.

In the case of single binocular stereoscopic viewing, two TV cameras 11 corresponding to the left and right eyes are provided, and their positions are known. It is. In this case, the search for corresponding points is 5pi
It can be localized on polar 1ine. e
pipolar 1ine is, as shown in Figure 2,
Right camera lens center 22. Left camera lens center 23
and the intersection line 21 between the plane determined by the three points of interest point P and the images 24 and 25 of each camera ("Information Processing J
Vol, 24. Nn, 12°p, 1449 (see “Image Understanding Research in the United States”).

In the image storage unit 12, one image information is converted and stored so that epipolar 1ine matches the corresponding scanning line of both images. By storing the data in this format, it is only necessary to check the correspondence between points on corresponding scanning lines of one image.

Next, the processing in the preprocessing section 13 will be explained.

In FIG. 3, 31 is a left eye image, and 32 is a right eye image. 33 is a scanning line for which corresponding points are determined; e
It matches pipolar 1ine. Pre-processing section 1
3 is a small area 34i of nXn pixels including this scanning line 33.
Take (i=1.2...tm).

The average brightness of the small area is calculated using the following formula.

Here, n is the number of pixels on one side of the small area, RI (x.

y) e LI (x+ y) is the position (x, y) respectively
Q is the number of the scanning line for determining the corresponding point and finding the three-dimensional distance, and m is the number of small areas.

Next, using the brightness of each small area obtained in this way, the amount of change (differential value) in brightness is calculated using the following equation.

Here, d is a constant representing the width of the neighborhood where differentiation is performed, and σ1 is a parameter that performs smoothing at the same time as differentiation.

The above is the content of the processing in the preprocessing section 13.

Next, the corresponding point determination unit 14 will be explained. The corresponding point determination unit 14 assumes a network as shown in FIG. 4 and calculates the attractive force of each node on the network. In FIG. 4, each differential value BR(i) for the right eye image is taken as a point 46 in the vertical axis direction, and the differential value BL (j) for the left limit image is taken as a point 47 in the horizontal axis direction, and each point in one image is This is a network constructed by providing and connecting nodes from one point to another point in the image. Note that the points on the opposing image connected to each point i can actually be limited to an appropriate number of opposing points and their vicinity, and only nine points are illustrated in FIG.

The corresponding point determining unit 14 performs calculations (n) to (III) of the first-order calculation procedure several times for this network, and determines corresponding points when a nearly stationary solution is reached.

(1) Set the initial value of the coefficient of attraction R(x, j) for each node.

(If) The degree of correspondence X(Le j) for each node is calculated using the following equation.

Here, σ2 is a parameter that determines the strength of attraction.

(III) The number of attractions R(ie j) for each node
is updated using the following formula.

R' (1tj) =R(i tj)+β・ΔR(le
j) ...(4) However.

, where ω is a weighting coefficient for the evaluation value X of the attractive force, and is the value of the function shown in FIG. Further, β is an appropriate correction coefficient. Q is a constant indicating the weighting width, and in the example of FIG. 5, 0=4.

When the steady solution is obtained as described above, the corresponding points can be found as shown in the following equation.

Here, pI: number of the small region of the left eye image corresponding to the i-th small region of the right eye image; a: a constant indicating iJ for finding the corresponding point. Equations (8) and (9) are calculated by selecting the node with the largest evaluation value of attractive force from the nodes connected to the i-th small region of the right eye image, and calculating ±a with this node as the center.
This indicates that the evaluation values of the attractive forces of nodes within the width are weighted and averaged using the distance between nodes as weights. Note that when pI includes a decimal part, the decimal part is
It is interpreted as indicating the point between one subregion and the next subregion.

Next, the calculation procedure will be explained in detail using the flowchart shown in FIG. First, in step 1, the average brightness of each small area is calculated using equation (1), and R(
xy j).

Ru. R (i, j) at the start of calculation should be set to an appropriate value of about 5.0. In step 2 of the 0th order procedure, using the average brightness of this small area, equation (2) Calculate the differential quantity by. Using this differential amount, in steps 3 and 4, calculation of X(it j) by equation (3) and calculation of R(t, j) by equations (4) to (7) are repeated several times. return. However, by repeating this calculation,
3) After repeating steps 3 and 4 several times, gradually decreasing σ2 in the equation and refining the correspondence.

Return to step 2, reduce σ1, calculate equation (2), refine the differential value, and repeat the calculations of steps 3 and 4. After completing these calculations, step 5
Then, calculate the corresponding points and return to step 1. In step 1, the size n of the area for which the average brightness is calculated is reduced and calculation is performed using equation (1), and in step 5
Based on the calculation results of the corresponding points, R(zej
) to set the initial value.

R (i * j) = 10.0 - (j - (i + p'
1))" - (10) However, if -R (i, j) < 0.0, set R (x v j) = 0.0. Also, p1'
As a result of step 5, is the shift amount of the point i of the left eye image to which point i of the right eye image corresponds on the new network.

Steps 2 to 5 using the initial value of equation (lO)
Repeat the calculation.

The final corresponding points can be obtained by repeating the calculations described above while successively decreasing the size n of the small area for which the average brightness is to be obtained.

The above is the processing procedure in the corresponding point determining section 14. Here, Equations (4) to (7) have the following meanings. Generally, in an environment where a moving body moves, the surface of the object is considered to change smoothly at most places (this is called continuity assumption), and the change in distance between adjacent pixels is small.

As shown in Equations (4) 2 to (7), if there is a node with a large evaluation value of attraction, the neighboring nodes of that node will increase the coefficient of attraction R(i, j), and the farther away they are, the more R
R(
The continuity assum
ption can be satisfied.

Next, the distance calculation section 15 will be explained using FIG. As shown in Fig. 7, VL and VR are the center coordinates of the optical systems of the left and right cameras, respectively, and PL(xLy ym)tP*
(xR@yR) is one point P (x
, yRz) are the positions of images formed on the left and right camera imaging surfaces, and f is the focal length of the lens. At this time, the position of one point P is given by XmuXR, as is well known. In the distance calculation unit 15, (XR*yR)t(X
The three-dimensional distance of each point on the object corresponding to each point on the image is calculated using Equation (11).

Examples of processing results according to this embodiment are shown in FIGS. 8 and 9. FIGS. 8(a) and 8(b) are right eye images and left eye images on a pair of corresponding scanning lines stored in the 1-image storage unit 12, respectively, where the horizontal axis is the pixel number and the vertical axis is the brightness. For this image data, which shows

The preprocessing unit 13 calculates the average brightness and its differential value for each small area of 2×2 pixels, and the corresponding point determination unit 14 performs calculations to determine the corresponding points. Next, the corresponding point determination unit 14 determines the initial value of the resistance using this result, and performs one round calculation using the average brightness of each small area of 1×1 pixel.
The final results obtained are shown in FIG. 9(a).
The horizontal axis is the pixel number, and the vertical axis is the number of pixels in which the left eye color is shifted to the right with respect to the right eye image. Here, only the calculation results for pixels with large absolute values of differential values are displayed as they are, and for other points, the above-mentioned points are interpolated. FIG. 9(b) shows the three-dimensional distance calculation unit 15 based on FIG.
This shows the distance in the y direction (depth) calculated in .

The present invention, which has been described above with respect to embodiments in binocular images, is applicable not only to cases where corresponding points need to be found only on corresponding scanning lines, such as in binocular images, but also to objects that are relatively displaced in arbitrary directions. This method can also be applied to cases where it is necessary to find corresponding points in a two-dimensional range, such as images taken at different times.

〔Effect of the invention〕

According to the present invention, when determining corresponding points, the shading information of an image can be treated as is (without being quantized into binary values),
In addition, it is not limited to mapping between pixels, but it is now possible to map pixels in one image to positions between pixels in the other image, which in turn improves the accuracy of position or distance measurement. do.

[Brief explanation of drawings]

FIG. 1 is a block diagram of an apparatus for implementing the present invention, and FIG. 2 is an epipolar 1 diagram for binocular vision.
FIG. 3 is a schematic diagram of an image for explaining preprocessing in the case of binocular vision, FIG. 4 is a diagram of a network for determining corresponding points on the corresponding scanning line, and FIG.
1 is a graph of weighting coefficients used to update the coefficient of gravity,
FIG. 6 is a flowchart for explaining the processing procedure according to the present invention, FIG. 7 is a schematic diagram showing the relationship between the positions of corresponding points on an image and the positions of measured points, and FIGS. 8 and 9 are 3 is a graph showing an example of processing results according to the present invention. 1st/U 5th Figure W″ 2nd P z %3 口 → Z → χ40th closed Figure M7

Claims (1)

[Claims] 1. Means for generating image information representing an image of the external world, processing this image information to determine pairs of mutually corresponding points in a pair of images, and determining the positions of two points forming each pair. In a position measuring device equipped with an arithmetic unit that calculates the position of a point in the external world that corresponds to both points based on the relationship, each pair of corresponding points is calculated based on the feature amount of each point in one image and the position of the other image. A position measurement method that uses a correspondence relationship between images, which is determined based on the distribution of gravitational force generated between each point and the feature quantity. 2. A position measurement method using the correspondence between images as described in item 1, characterized in that the feature amount of each point in the image is the brightness of each point. 3. The position measurement method using the correspondence between images as described in item 1, wherein the feature amount of each point of the image is the amount of change in brightness of a point in the vicinity of each point.