JPS6166108A - Method and apparatus for measuring position and shape of object - Google Patents

Abstract

PURPOSE:To judge the correctness or error of correspondence, by obtaining the intersection of projected straight lines of visual lines on the third image with respect to possible corresponding points, which are obtained from two images, and studying the evaluated value obtained by comparing the vicinity of the intersection and the possible corresponding points. CONSTITUTION:A point (c) agrees with the intersection of two projected straight lines L1 and L2. Meanwhile, the corresponding point of a point (a) in an image 17 on an image 18 is located on a projected straight line L3 of a visual line OAP on the image 18. When a possible corresponding point (b) is selected on said projected straight line L3, the point on an image 19, which simultaneously corresponds to both (a) and (b) much become the intersection (c) of the projected straight line L1 and the projected straight line L2. When an erroneous corresponding point, e.g., b' is selected, a projected straight line of a visual line OBb' on the image 19 becomes L4, and the point, which simultaneously corresponds to both (a) and b', becomes c'. Therefore, similarity of (a) and c' and b' and c' is strikingly lowered. Thus the fact that b' is erroneous correspondence can be readily judged.

Description

[Detailed Description of the Invention] [Industrial Field of Application] The present invention is directed to a plurality of 2-sheet images obtained from at least three locations.
This relates to an object position/shape measurement method that three-dimensionally measures the position/shape of an object from a dimensional image by stereoscopic corresponding point processing, and in particular, measures the position/shape of an object by identifying visible and hidden parts due to camera position. The present invention relates to a method for measuring the position and shape of an object, and an apparatus for implementing this method.

[Prior art]

Conventionally, as a method for measuring the position and shape of an object, a three-dimensional measurement method using a television camera image as a two-dimensional image has been common, and the methods can be roughly classified into the following three types.

Namely, 1) a projection type that illuminates an object with spot light, slit light, or a grating image, and then observes and processes the projected image with a television camera; 2) automatic focusing that adjusts the lens position to make the image clear; Formula (also known as sharpness detection formula), 3) 2
Observe with a TV camera on a stand, and process the corresponding points of both images to find the parallax between the images. Based on the principle of triangulation,
There is a binocular stereoscopic method that converts into dimensional position and shape.

Among these, the above 1) floodlight type and 2) automatic focus type require mechanical scanning such as light scanning or lens scanning, so they have problems with high-speed performance and durability. There is an essential problem that resolution is difficult to obtain.

On the other hand, the binocular stereoscopic viewing method described in 3) above does not require mechanical scanning, is capable of high-resolution measurement, and has fewer restrictions on the object to be measured than methods 1) and 2) above. , it is also an excellent general-purpose method.

However, in this binocular stereoscopic viewing method, there are many problems in the method of processing corresponding points between both images.

Below, the principle of binocular stereoscopic viewing and problems related to processing of visible and hidden parts depending on the camera position will be specifically explained with reference to the drawings.

An example of the configuration of a conventionally proposed binocular stereoscopic viewing method is shown in Part 1.
To illustrate and explain, in the figure, 1 and 2 are each X
The television cameras are placed on the axis (in the X direction) and equidistant from each other from the coordinate origin 0, and the orientations of the television cameras 1 and 2 are approximately parallel to each other.

3 is a target object (hereinafter abbreviated as object), and this object 3
is placed on the Y axis at a distance that is sufficiently larger than the distance between the television cameras 1 and 2 from the coordinate origin O. Note that 2 is the z-axis.

FIG. 11 is an explanatory diagram showing an example of the distribution of evaluation values obtained by the local correlation method as a typical method for determining corresponding points. In these images, 4 and 5 are images taken by television camera 1 and television camera 2, respectively, shown in FIG. 10.

Then, any feature point a of the image 4 shown in FIG. 11(a)
The corresponding points in image 5 shown in FIG.

The equation of this straight line is that the two cameras are OY (see Figure 10).
37.12 ('
73) (hereinafter referred to as Document 1), in P1105, formula (4, 17).

Then, while sweeping a window (for example, a 5 x 5 matrix) on the projection straight line, the pattern in the window and the pattern in the same size window centered on an arbitrary feature point a of image 4 are calculated. Compare and find evaluation values such as pattern composition, brightness, and power. Generally, a cross-correlation function is often used as this evaluation value.

This cross-correlation function is generally well known in the field of image matching technology, and a typical example is the above-mentioned document 10 PII.
It is described in formula (5,2) in IO. and,
When the aperture conditions of the camera lens are almost the same when acquiring two images, the formula (5
, 4) can be used.

Hereinafter, the value of this error evaluation function will be referred to as evaluation value ρ. This evaluation value ρ becomes smaller as the comparison parts correspond,
A complete match has a value of 0. Conversely, if the comparison parts do not correspond, they have a bold value.

6 in FIG. 11(c) shows the change in the evaluation value ρ when the window is swept using a projection straight line. t is a threshold value of the evaluation value, and a value larger than the ρ value at the corresponding point is given in advance. Then, since the evaluation value should be the minimum at the corresponding point and smaller than the threshold value t, the point where the evaluation value is smaller than the set threshold value and is the minimum is determined as the corresponding point. Once these corresponding points are determined, the inter-image parallax can be calculated and converted into three-dimensional position coordinates of the object 3 (see FIG. 10).

[Problem to be solved by the invention 3 As mentioned above, in binocular stereopsis, performing corresponding point processing between two images is an important procedure, but there are the following problems. .

That is, in the conventional binocular stereoscopic viewing method, it is difficult to handle visible and hidden parts depending on the camera position. If it is a very simple object and the shape of the object and the scene in the background are known in advance, it is not necessarily impossible to handle visible and hidden parts, but in this case, advanced image recognition processing is required in advance. It also requires knowledge about the object and the background. How this processing and knowledge is provided remains unresolved and remains a challenge for the future.

For this reason, conventionally, it is often assumed that there are no visible or hidden parts. In order to satisfy this premise, it is necessary to narrow the distance between the cameras. However, if the distance between the cameras is narrowed, the measurement error increases in inverse proportion to the distance, so it is difficult to make the distance too narrow.

On the other hand, if visible and hidden parts are allowed, corresponding points cannot be detected even if the edges of the visible and hidden surfaces are visible in both images. The reason for this will be explained with reference to FIG.

This FIG. 12 is a diagram for explaining the handling of visible and hidden parts in binocular stereoscopic viewing, in which (a) shows the left image, and (1)) shows the right image.

7 and 8 are images of the rectangular parallelepiped observed by the left camera and the right camera, respectively. a and a' are the feature points of the image 7, and b and b' are the desired correct corresponding points on the image 8 corresponding to the feature point a and the feature point a' of the image 7, respectively.

Then, the left surface 9 of the feature point a and the right surface 1 of the corresponding point b'
0 is the visible/hidden portion, and 11.12 is the ridge line that can be measured by binocular stereoscopic viewing.

Now, if we compare the surrounding area of the feature point a and the surrounding area of the corresponding point S, unless the left side of the feature point a has exactly the same brightness as the background, the brightness distribution will be significantly different, so the evaluation value ρ becomes extremely high, and in the conventional method, it is determined that the feature point a and the corresponding point S are not corresponding points. Similarly, it is determined that the feature point a' and the corresponding point b' are not corresponding points. Therefore, the vertical ridgeline portion in FIG. 12 does not have corresponding points. Edges in the horizontal direction cannot be correlated using binocular stereoscopic viewing, and correspondence cannot be obtained between the two edges 11 and 12 because one side of the edges is hidden and visible. Therefore, it is impossible to measure the position of all edges.

As seen in the example of FIG. 12, in the conventional binocular stereoscopic viewing method, it is difficult to handle visible and hidden parts depending on the camera position.

In view of the above points, the present invention was made in order to solve the force problem and eliminate such drawbacks, and the purpose is to measure objects that have many visible and hidden parts depending on the camera position, which was difficult in the past. It is possible to measure edges where one side is visible and hidden, and to identify visible and hidden areas, and the processing algorithm is extremely simple and reliable because it uses corresponding point processing between multiple images. Since it is possible to improve object positioning and camera spacing, it is possible to increase the distance between cameras.
It is an object of the present invention to provide a shape measurement method and an apparatus for carrying out the method.

[Means for solving problems]

The present invention acquires two-dimensional images from at least three locations on non-colinear lines, and finds corresponding point candidates by associating two of the two-dimensional images, and identifies the feature points of the reference image and its corresponding point candidates. Find respective projection straight lines on other images from the corresponding point candidates for the feature points, predict the existing positions of the corresponding points on the other images from the intersections of the projected straight lines,
The degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate is determined independently for one side and the other side of the edge, and by identifying the presence of visible and hidden parts on one side of the edge, the corresponding point can be determined. This is a method that can measure the position and shape of an object by determining whether it is correct or incorrect and selecting corresponding points.It uses an image input section and the output of this image input section as input, and a parameter list of the image input section. and a preprocessing calculation unit that extracts feature points, and the output of this preprocessing calculation unit is used as input to perform correspondence between two of the two-dimensional images obtained from at least three locations on the non-uniform orthogonal helix. Then, from the feature points of the reference image and the corresponding point candidates for the feature points, project straight lines onto other images, and from the intersections of the projected straight lines, find the corresponding points on the other images. The existence position of the edge is predicted, and the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate is determined independently for one side of the edge and the other side, and the existence of a hidden part on one side of the edge is determined. There is a corresponding point detection calculation section that executes a corresponding point sorting function that determines whether the corresponding points are correct or incorrect by identifying them, and selects the corresponding points. The apparatus includes a coordinate calculation unit that converts the obtained feature points into real space coordinates and calculates the coordinates.

Furthermore, the present invention acquires two-dimensional images from at least three locations on non-colinear lines, and independently examines the degree of correspondence between one side and the other side of an edge from two of the two-dimensional images. Find corresponding point candidates independently by Predict the location of the corresponding point, check the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate, identify the existence of visible and hidden parts, determine whether the corresponding point candidate is correct, and select the corresponding point. This method is capable of measuring the position and shape of an object by doing so, and includes an image input section and a preprocessing calculation section that receives the output of this image input section and extracts the parameter list and feature points of the image input section. Then, using the output of this preprocessing calculation unit as input, one side and the other side of the edge are independently processed from two of the two-dimensional images obtained from at least three locations on the non-isogonal orthogonal helix. Corresponding point candidates are determined independently by examining the degree of the reference image, and respective projection straight lines are determined from the edge of the reference image and the corresponding point candidates for that edge onto other images, and from the intersection of the projected straight lines, the corresponding point candidates are determined from the other image. Predicting the existing position of the corresponding point above, checking the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate, and determining whether the corresponding point candidate is correct or incorrect by identifying the presence of visible or hidden parts, A corresponding point detection calculation unit that performs a corresponding point selection function to select corresponding points, and the output of this corresponding point detection calculation unit is input, and the feature points for which correspondence has been obtained are converted into real space coordinates by the calculation unit. This device includes a coordinate calculation unit that calculates coordinates using the coordinate calculation unit.

Furthermore, the present invention obtains two-dimensional images from at least three locations on non-colinear lines, and determines corresponding point candidates by associating two of the two-dimensional images, and determines the characteristics of the reference image. Find the respective projection straight lines onto other images from points and their corresponding point candidates for the feature points, predict the existing positions of the corresponding points on the other images from the intersections of the projected straight lines, and calculate the positions and the above criteria. A first corresponding point selection function that determines whether a corresponding point candidate is correct or incorrect by checking the degree of correspondence with the feature points of the image or the corresponding point candidate and selects corresponding points; For feature points for which no corresponding point was obtained, the degree of correspondence between the predicted location of the corresponding point and the feature point of the reference image or the corresponding point candidate is determined independently for the −・ side of the edge and the other side. A second corresponding point selection function that determines whether the corresponding points are correct or incorrect by identifying the presence of visible and hidden parts on one side of the edge, and selects the corresponding points from two of the two-dimensional images obtained above. Corresponding point candidates are obtained independently by examining the degree of correspondence on one side and the other side of the edge, and a projection straight line is drawn from the edge of the reference image and the corresponding point candidate for that edge onto another image. , predict the existing position of the corresponding point on the other image from the intersection of the projection straight line, and check the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate to determine the existence of visible and hidden parts. This is a method in which the position and shape of an object can be measured by determining the correctness of corresponding point candidates by identifying them, and obtaining corresponding points by a third corresponding point selection function that selects corresponding points. an input section, a preprocessing operation section that takes the output of this image input section as input and extracts the parameter list and feature points of the image input section; A corresponding point detection calculation unit that performs a third corresponding point selection function, and the output of this corresponding point detection calculation unit is input, and the feature points for which correspondences are obtained by the calculation unit are converted into real space coordinates and the coordinates are This device includes a coordinate calculation unit that calculates

[Effect]

In the present invention, a plurality of two-dimensional images are acquired from at least three locations on the same straight line, and stereoscopic corresponding point processing is performed between the images to create a brightness distribution not only around the edge but also on one side of the edge. By comparing the values and checking the degree of correspondence, we can identify the visible and hidden parts around the edge due to the camera position, and determine whether the correspondence is correct or not.
It measures the shape in three dimensions.

〔Example〕

Hereinafter, embodiments of the present invention will be described in detail based on the drawings.

FIG. 1 is a block diagram for explaining an embodiment of the object position/shape measuring method according to the present invention, and shows an embodiment of the camera arrangement according to the present invention when a television camera is used as a means for acquiring two-dimensional images. It is something that

In the figure, x, y, and z are spatial coordinates, 13 is a television camera, 14 is a television camera placed in the X direction of this television camera 13, and 15 is a television camera placed in two directions diagonally to the upper right of the television camera 13. It is. Also, 16 is Y
A target object (object) placed on the axis, in this example a cube.

FIG. 2 will be explained by taking as an example a case where there is no visible or hidden part due to the camera position for telepicture in the embodiment shown in FIG.

In the figure, 1T is the image of the television camera 13, -20=18 is the image of the television camera 14, and 19 is the image of the television camera 1.
This is the image of No. 5. Here, each television camera 13.14.
15 lens centers oA and o, respectively.

and Oc. The image (point) a + b of the point P of the child on the object 16 and the point C are the line of sight OAP.
Since point C is on OBP and OcP, point C is a projection type a of line of sight OAP and line of sight OBP onto image 19.
lL+ r Exists on L2. That is, point C coincides with the intersection of these two projection straight lines Ll + L2. On the other hand, the corresponding point on the image 18 to the point a in the image 17 is on the projection straight line L3 of the line of sight OAP onto the image 18.

When corresponding point candidates are selected on this projection straight line L3, the image on the image 19 corresponding to both a and b at the same time must be at the intersection C of the projection straight line L1 and the projection straight line L2. However, if you select an incorrect corresponding point candidate, for example b' in the figure, the line of sight 0Bb
The projection straight line of l onto image 19 is L4, a and b.
The point that simultaneously corresponds to both the projection straight line L4 and the projection straight line L4 becomes the intersection C', which deviates greatly from the correct intersection C. Therefore, the similarity (corresponding to the reciprocal of the above-mentioned evaluation value) in the vicinity of a and C' and b' and C' both decrease significantly,
It can be easily determined that b' is an incorrect correspondence.

As described above, for the corresponding point candidates obtained from the two images, the intersection points of the projection straight lines of each line of sight onto the third image (hereinafter referred to as predicted corresponding points) are found, and the vicinity and corresponding point candidates are calculated. It is possible to determine whether the correspondence is correct or incorrect by examining the evaluation values compared.

Next, how to obtain a projection straight line will be explained with reference to Document 1 and FIG.

First, Document 1 imposes strict constraints on the positions and directions of two cameras, and describes how to obtain a projection straight line under those conditions. That is, x, y, z shown in Figure 10
In each coordinate system, we are dealing with a case where the positions and directions of two television cameras have a symmetrical relationship about two axes. However, in the three-lens image, as is clear from FIG. 1, at least the television camera 13 and the television camera 15, and the television camera 14 and the television camera 15 are not symmetrical about the Z axis. For this reason, the projection straight line [111L2
+L3 cannot be obtained by the method shown in Document 1 or its modification. Here, since the method of finding a projected straight line between images from two television cameras placed at arbitrary coordinates and in any direction has been completely unknown, the method will be described below as a case of finding the linear equation of the projected straight line L3. A specific explanation will be given as an example.

Now, as camera parameters of the television camera 13, the lens focal length is fA, and the lens center OA is the coordinates (aA, b
A, hA), and the optical axis makes an angle of αA with the y-z plane and fA with the x-y plane. Also, TV camera 21
As a camera parameter of 4, the lens focal length is fn+
The lens center OB is at the coordinates (XB.

Y B + Z a ), and the optical axis is in the Y-Z plane.
B I X It is assumed that it forms an angle of ψB with the Y plane.

In addition, as other camera parameters, the intersection of the camera imaging plane and the optical axis (optical axis point) is (hoA+VoA) in the camera 2 imaging plane coordinates for the television camera 13, and (hge) in the camera imaging plane coordinates for the television camera 14. ,voB
), and the conversion ratio of the size of the image and the real image is (M
XA, Myn) + (MXB, Myn)
shall be.

Then, any point (hA
, VA) is expressed in (x, y, z) coordinates as follows.

However, ... (2) ξXA" MXA (hOA hA)
@@@*@(3) ξyA=My(vAvoA)
e @ @ @ * (4). On the other hand, if the (x, y, z) coordinates of the corresponding points (hB, VB) are (Xn + YB r ZB), then (XA I
YA rZA ) has the following relationship.

When this simultaneous equation is solved, the result is A(XB-in) + B(Yn-bn) + C(Zu-ha
)=O" (7) However, A-qz(bB-bA)-(hB-hA)...e...(
8) n=qz(bn hA) Qz(JLBaA)
"""(9)C"" QX(bB bA) +
bBbA*5essQO)qz = (XA-aA)/
(YA-bA) ・11111111aυqz
” '(ZA hA)/(YAbA) a
@ @ 1111 (121It can be proven that equation (7) also holds true when the denominator is -0 in the above equation.

On the other hand, (Xn + Ys + ZB) and (hB + vn
) is given by rewriting A −+ B in equations (1) to (4).

Then, the following equation is obtained by this rewritten equation and the above equation (7).

ulξzn+u2ξ)'b+u3-=0”
””Q31 However, by replacing A with B in the above equations (3) and (4) and substituting it into equation OJ, the following equation can be introduced as the projection straight line (I, J).

vI I V2J V3=Oeas*eQ5 However, we have explained how to find the linear equation of L3 above, but Ll +
A projection straight line between images from a camera having arbitrary camera parameters such as L2 * L4 can also be found in exactly the same way.

Next, a distance calculation method when corresponding points are detected will be explained. In the following, a case will be described taking as an example a case where a point in the image 18 is selected as the corresponding point to the point a in the image 17 in FIG. 2.

Now, a is the coordinates of the imaging plane of the television camera 13 (hB r
vl), b is the coordinates of the imaging plane of the television camera 14, and (h
b, vb), then the coordinates (Xa) of a and b in the (x, y, z) coordinate system according to the above equations (1) to (4).

Ya, Za) and (Xb, Yb, Zb
) can be found. Then, the coordinates (x, y
, z) is the intersection of straight lines OAa and OB b, so
The following simultaneous equations hold true.

From this simultaneous equation, (qx(Yb bn) (Xb-am))Ya (
bAqz-(aA-aB))(Yl)-bn)
-b B (Xb-1n)...0 Y is determined.

Ta, QX=(Xa aA)/(Ya bA)
...Fist・(A) Then, X is obtained by substituting Y obtained by equation α1 into equation (Iη). 2 is obtained by substituting it into the following equation.

In this way, the coordinates of a point on the object can be calculated from the result of the corresponding point processing.

FIG. 3 is a flowchart showing an example of the flow of stereoscopic corresponding point processing for trinocular images, and shows the first step of the entire corresponding point processing. Note that for visible and hidden parts of the object, a result of no correspondence is obtained here, and corresponding points are processed again in the visible and hidden part processing (second step), which will be described later.

In FIG. 3, 20 and 21 are preprocessing that is commonly performed in the field of image processing, and 20 is an edge detection process that detects the edge strength and direction of three images, and various methods have been proposed in the past. has been done. Here Robin
Allows the operator to detect edges.

This edge detection processing 20 is performed by the cited "image processing subroutine package J 5PIDERUSER'.
This can be easily executed by using the image processing subroutine package 5PIDER described in S. MANTJAL, 1982, Kyodo System Development Co., Ltd. (hereinafter referred to as Document 2). Further, processing 21 is a thinning process for obtaining feature points by thinning the lines. Then, the change in edge strength in the direction perpendicular to the edge direction (the direction of the edge line in the case of a three-dimensional edge part) at all points where the edge strength is somewhat large is measured.The point where the edge strength is maximum is defined as a feature point (edge).

Furthermore, a thinning processing subroutine is also described in 5PIDER of this document 2.

22 is the equation of the projection straight line L 1 + Lll for each feature point (edge) of the image 1T of the television camera 13, and the projection straight line L2 and L5 for each feature point (edge) of the images 18 and 19 of the television camera 14 and 15, respectively.
This is the process of calculating the equation.

Here, the method of calculating this has already been shown in the above-mentioned equations (1) to α0. Then, in process 22', for each feature point (edge), the edge is approximately parallel to the projection straight line L3 (
In the embodiment, when the direction angle difference is within 25 degrees, it is treated as parallel), the process is assigned to process 26, otherwise, the process is assigned to process 23. 23 is a process for performing an initial correspondence between the image 17 of the television camera 13 and the image 18 of the television camera 14, in which candidates for corresponding points on the image 18 for each feature point (edge) of the image 1T are selected with priority. Then, the evaluation value is determined by a local correlation method that compares the neighboring density distributions of the feature points (edges) of the image 17 and the feature points on the projection straight line of the image 18. Specifically, this local correlation method creates a 5×5 window near the feature points (edges), and
Window image IP1 from window density analysis in 7 and image 18,
A method was used in which rp was created and compared using the following equation.

Here, the window image IP2 corresponds to a template image in image matching.

・1''・(2) In addition, in the experiment, a value of 100 was used as the threshold value, but since there was no particular problem with 200, 10
A value between 0 and 200 seems appropriate. Then, it is determined that feature points (edges) on the image 18 whose edge directions do not match within 45 degrees or whose ρ is larger than a threshold value are not corresponding point candidates.

Here, for feature points (edges) for which there is no corresponding point candidate, processing is suspended as there is no corresponding point in step 23'.

24 is a process of calculating the intersection of two projection straight lines on the image 19, that is, a predicted corresponding point for each corresponding point candidate obtained in process 23. And two projective straight lines a
r I b p J cl-0 and &2I-b2J (
! 2”0(al +bl +e1 +a21b!+02
is a constant) is the intersection of ((elbg czbt)/(a
tbz azbtL (az c+ at 02)
/(ILI bz a2bl)).

Here, if an edge (feature point) exists in a range within five times the pixel interval around this predicted corresponding point, the nearest edge point is corrected as the correct predicted corresponding point. This is because the predicted corresponding point that should be on the edge may shift due to camera position/pan meter errors, image distortion, etc. Also, shifts may occur due to accumulation of rounding errors due to integer calculations. It is. According to the experimental results of this example, a deviation of four times the maximum pixel interval was observed locally.

25 is a process of obtaining evaluation values for the feature points (edges) of the image 17 and the predicted corresponding points of the image 19 by the local correlation method. Here, the calculation of this evaluation value is performed by the above-mentioned formula (22+. Then, in step 29, among the corresponding point candidates, the one whose evaluation value is less than or equal to the threshold and the smallest is selected, and is determined as the corresponding point.

On the other hand, 26 is a process of performing initial association with image 19 instead of image 18. Then, the setting value is processed in step 23.
As in the case of , the above-mentioned local correlation method was used to compare and find the density distributions near the feature points (edges) of image 17 and the feature points (edges) on the projection line of image 19. This evaluation formula is the above-mentioned formula (221).As a result, the image 19
Select the corresponding point candidate above. Here, for feature points (edges) for which no corresponding point candidates were found, process 2
At 6', processing is suspended as there is no corresponding point.

21 is the image 19 obtained by processing 26 in the same manner as processing 24.
For each corresponding point candidate above, two projection lines on image 18
This is a process of calculating the intersection of 1iL3 and L5, that is, the predicted corresponding point. However, L5 is image 1 of the line of sight OcP.
This is the projection straight line to 8. 28 is image 1 as in process 25.
This is a process of comparing the neighborhood distribution of the feature points (edges) of 7 and the predicted corresponding points of image 18 using the local correlation method to obtain an evaluation value.

Then, in process 29, among the corresponding point candidates, the one whose evaluation value is less than or equal to the threshold value and has the smallest value is selected, and this is set as the corresponding point. Here, if a corresponding point is not found, the process 29' enters into a pending state and waits for the second step process.

30, each feature point (
This is a process of finding real space coordinates for edges), and is calculated using equations 07) to 01) described above. Then, process 31
Now, the above processing results are displayed in three-dimensional form. Specifically, it is converted into x-y, y-z, x-ZO perspective views.

As explained above, when visible and hidden parts are hidden, each point on the object captured by at least three television cameras can be converted into real space coordinates (x, y, z) and recorded. Therefore, excluding visible and hidden parts, it is possible to find the distance and direction from a certain reference point to an object, and also recognize the shape of the object from the positional relationship between each point of the object and the object, and identify the type of object. be able to.

Next, the principle of handling visible and hidden parts depending on the camera position, which is difficult in the binocular stereoscopic viewing method, will be explained using FIG. 4.

The processing in FIG. 4, which is an explanatory diagram for explaining the method of handling visible and hidden parts in the present invention, is based on the flow shown in FIG. This is performed as a second step for feature points that have not yet been processed.

In this FIG. 4, 32 shown in (C) and 33 shown in (d) are the left image and right image respectively taken by the first camera and the second camera as described above, and 34° and 35 are the visible and hidden parts ( side). 36 shown in (a) and 36' shown in (b) are examples of images taken at different positions of the third camera, respectively.

In the image 36 shown in (a), the side surface 34 is visible and the side surface 35 (see (b)) is hidden.
In the image 36' shown in , the side surface 35 is visible, and the side surface 34 ((a
)0ne) is visible.

First, the third camera position where the third image 36 shown in (a) is obtained will be explained.

The corresponding point on the right image 33 shown in (d) for the point a on the left image 32 shown in (c) cannot be found using the conventional local correlation method explained in the first step in the flow chart of FIG. A corresponding point candidate a" is found on the image 36 shown in (a).

If the predicted corresponding point on the image 33 between point a and corresponding point candidate a'' is a', then when a'' is the correct corresponding point, a' is on the edge and only around one side of the edge. (hereinafter referred to as the one-sided correlation method, and the comparison on the right side is especially called the right-side correlation method, and the comparison on the left side is especially called the left-side correlation method. ), the evaluation value ρ (both evaluation numbers will be described later) of the right-side correlation method becomes smaller, indicating that &r&' and aII are in a corresponding relationship. Ru. If a'' is an incorrect corresponding point, a' will usually deviate from the edge by a fairly large distance, so the error in correspondence can be easily identified.

On the other hand, when thinking about dosing, if we switch the left and right sides,
Correspondence can be obtained using exactly the same procedure, and visible and hidden parts can be confirmed. On the other hand, for feature point C in image 32, a corresponding point candidate C'' is found only in image 36.Here, even if a corresponding point candidate is found in image 33, the predicted corresponding point on image 36 is usually Since the predicted corresponding points on the image 33 from C and c'' are on the hidden edge 83, the evaluation value is high and the error in correspondence can be clearly determined. , image 33
It is not on the visible edge. Therefore, C” is C
Is it possible to confirm the corresponding point? In such a case, the process is temporarily put on hold, and when the trinocular corresponding point process has been completed for all feature points, the third step described below is performed. Then, in the third step, the points whose correspondence is determined in the second step are removed from the corresponding point candidates, and the corresponding point candidates and their regions are narrowed down to find corresponding points.

Next, the case of an image 36' in which the position of the third camera is different will be explained.

In the case of the aforementioned image 36, the left image 32 is used as the reference image, but in the case of the image 36', the same processing as in the case of the image 36 can be performed by using the right image 33 as the reference image. However, if possible, it would be better to unify the reference image to maintain consistency in processing. here,
A method of processing the reference image as the left image 32 even in the case of the image 36' will be described. When a corresponding point candidate for a is searched for in the image 33 using the one-sided correlation method, a point a' with a small right side correlation evaluation value ρ' is found. Then, the predicted corresponding point a'' on the image 36' for a and a' exists on the edge if a/ is a correct corresponding point, and the evaluation value ρ is It can be seen that the neighborhood of a' and a'' correspond well to each other. Therefore, it is determined that a, a' and a'' have a correspondence relationship with each other.

Next, in contrast to the case of a, the evaluation value ρ' of b becomes smaller by the left side correlation method with b' of image 33, and b/ is selected as a corresponding point candidate. Then, when we calculate the predicted corresponding point b" on the image 36' from b and b', it lies on the edge, and the evaluation value ρ due to the local correlation around b' and b" is small, so b , b' and b" have a correspondence relationship. If there is a corresponding point candidate other than b', the predicted corresponding point does not lie on the edge in image 36', so the correspondence It can be determined that the point candidate is an incorrect corresponding point.

Here, the one-sided correlation method will be specifically explained.

First, cut out a micro region (window) of nXm (for example, 5X5 matrix) on the right side of the edge and very close to the edge.
If the evaluation value ρ' is calculated using the above formula (2) for that part, the degree of correspondence on the right side of the edge can be obtained.The same applies to the comparison on the left side of the edge. The degree of correspondence can be determined by cutting out a minute area (window) very close to the edge perpendicular direction and calculating the evaluation value ρ' using the above formula (2) for that area. Since the change in brightness is not as large as that at the edges, in the example described below, the above method was not used, but instead the following method was used: In other words, a perpendicular line was drawn at the edge, and the pixel spacing was 1.4 times or 2.8 times the pixel spacing. After cutting out a micro area of 3 x 3 or 5 x 5 pixels centered on a position twice as far away, remove the pixels at the apex position of the square area from the area.Then, remove the pixels at the apex position of the square area from the area. Calculate the average brightness and the variance representing the brightness variation. These two physical quantities are taken as the feature values on one side of the edge. The evaluation value by comparing the two edges E1 r B2 on one side is as follows. However, m r n is selected under the condition of m+n=1, but usually m=n==0.5.Furthermore, a corresponding threshold value of 50 to 100 was used.

Although the case of the image 36 and the image 36' has been explained separately above, in reality, it is not possible to distinguish between the cases of the image 36 and the image 36' in advance and then process them. Therefore, there is a need for a procedure to automatically perform case classification in the second step process.

An embodiment of the second step including this procedure is shown in FIG.

In FIG. 5 showing a flowchart of the corresponding point processing according to the present invention, the entire procedure is shown in steps 1 to 3 shown in FIG. 5(a).
It is classified into steps.

This first step is the process 20 to 20 disclosed in FIG. 3 above.
It is 31. The second step is processing for visible and hidden parts, and the third step is processing for parts for which correspondence cannot be obtained even in step M2.

Here, this second step processing deals with only the feature points that were unresolved in the first step.

The reference image will be referred to as image 32, which will be referred to as image A, and image 33 will be referred to as image B2, image 36-j, or image 36' will be referred to as image C.

In FIG. 5(b) showing the second step, in process 37, the edge direction of the feature point of image A and the direction of the projection straight line L1 are compared, and if they are parallel, the process moves to process 43, and if they are not parallel, the process moves to process 38. Move. In this process 38, initial corresponding point processing between image A and image C is performed in the same way as process 237 in the above-mentioned diagram M3. Then, if a corresponding point candidate exists, in step 39, the predicted corresponding point in image B is determined as the intersection of the two projection straight lines obtained by the above equation (1) to α0, and if the predicted corresponding point is located at or near the Find out if it is. If it is, in step 40, the evaluation value ρ' for both sides of the edge is determined independently using the above formula (231) using the one-sided correlation method for that point and image A or image C, and one side of the edge is set to the threshold value. If it has a smaller evaluation value, the correspondence is determined. Here, processing 37°38', 4
If the result is G or 42, the process 13Jl moves to process 43. Then, in processes 43 to 47, similar processes are performed in processes 38 to 42, except that the images (IJB and image C are replaced).
7, the process further moves to process 48, where the evaluation value ρ'(
Equation (2) is determined, and corresponding point candidates having evaluation values smaller than the threshold are selected in order of priority. If there are corresponding point candidates, in step 50, the predicted corresponding points in image C are calculated for each corresponding point candidate in the first diagram shown in FIG.
As in the case of the step, it is determined as the intersection of two projection straight lines Lt and L2 (corresponding to the projection straight line Ll + L2 in Fig. tJ2) from image A and image B to image C.

Then, in process 51, if the predicted corresponding point exists on the edge, the evaluation value ρ (formula (2)) is calculated using the local correlation method for image B and image C. The value is smaller than the threshold value and the minimum value is searched, and if it exists, the correspondence is confirmed.In addition, even if the predicted corresponding point is not on the edge in process 40.45 and process 51, it is found near the edge. If so, it is determined that it is on the edge.

Process 54 is a process for determining the correspondence. In this example, if the distance between the predicted corresponding point and the nearest edge is within 5 times the pixel distance, it is assumed that the edge point is on the edge, and the edge point closest to the predicted corresponding point is corrected as the correct predicted corresponding point. . Further, although the process 51 is not necessarily necessary, in the normal case where correspondence of high edges is obtained, this process 51
Processing efficiency may be higher if this judgment is included.

Even in the second step shown in FIG. 5(6), unresolved feature points (processing pending state in (2)) are processed in the third step shown in FIG. 5(c). .

In this Figure 5(c), 55.56...-62
indicate each process.

In this third step, if the feature points determined up to the second step are included in the initial corresponding point candidates of M2 or the first step in process 55 or process 60, respectively, they are removed and the corresponding point candidates are created. Squeeze. As a result, when there is only one candidate, or when only one of the remaining candidates has a significantly high degree of correspondence (evaluation value is small), that candidate is determined to be a corresponding point. In addition, if it is acceptable to impose a constraint condition that image A and image B or image A and Ifii image C have the same corresponding order of the projective lineman, then the feature points determined up to the second step can be used. , it is possible to limit the area where the corresponding points to be found exist, and the reliability of the processing can be further improved.

As described above, visible and hidden parts can be processed even when the visible and hidden parts depending on the camera position are not known in advance.

In this way, the object position/shape measurement method of the present invention acquires two-dimensional images from at least three locations on non-collinear lines,
The degree of correspondence is independently examined on the right and left sides of the edge in the image, and the presence of hidden parts on the right or left side of the edge is identified, thereby performing a process of determining whether the corresponding point candidate is correct or incorrect.

FIG. 6 is a block diagram showing an example of an apparatus implementing the above-mentioned object position/shape measurement method, and shows an example of the overall configuration of an object position/shape measurement apparatus that utilizes corresponding point processing between trinocular images. It is something.

In this Figure 6, 13, 14, and 15 are the three television cameras shown in Figure 1 above, 63a, 63b+8.
Is each 3c a TV camera? An analog-to-digital converter (hereinafter abbreviated as A/D converter) that corresponds to 14.15.16 and converts the video signal from each television camera 13 to 15 into a digital signal, and these constitute an image input section. There is. 64 is an A/D converter 63a to 6
Image memory 65 that receives and temporarily stores each output of 3c.
numeral 66 is a preprocessing arithmetic circuit that receives the output of the image memory 64 as an input, and a feature table 66 records the positions and feature amounts of feature points obtained by the preprocessing arithmetic circuit 65. These are the features of the image input section. It constitutes a preprocessing calculation section 6T that receives the output as input and extracts the parameter list and feature points of the image input section. 68 is a camera parameter table.

69 is an arithmetic circuit that receives the output from the feature table 66 in the preprocessing arithmetic unit 61 and calculates a projection straight line parameter based on the camera parameter table 68;
71 is a projection straight line parameter table, and 71 is a corresponding point table for recording the corresponding point processing result which is the calculation result obtained by the calculation circuit 69, and these constitute the trinocular corresponding point detection module T2.

Then, this trinocular corresponding point detection module T2 obtains corresponding point candidates by performing correspondence between two of the two-dimensional images obtained from at least three locations on non-colinear lines, and Obtain respective projection straight lines onto other images from the feature points of the image and their corresponding point candidates for the feature points, predict the existing positions of the corresponding points on the other images from the intersections of the projected straight lines, and calculate the positions. The degree of correspondence between the feature point of the reference image or the corresponding point candidate is determined independently for one side of the edge and the other side, and the correctness of the corresponding point is determined by identifying the presence of visible and hidden parts on one side of the edge. The first corresponding point selection function determines and selects corresponding points, and for feature points for which correct corresponding points are not obtained by this first corresponding point selection function, the predicted existing positions of the corresponding points and the reference image are used. The degree of correspondence with the feature point or the corresponding point candidate is determined independently for one side of the edge and the other side, and the correctness of the corresponding point is determined by identifying the existence of a visible or hidden part on one side of the edge. A second corresponding point selection function that selects the corresponding points and independently examines the degree of correspondence on one side and the other side of the edge from two of the two-dimensional images obtained above to obtain corresponding point candidates; Then, obtain a projection straight line onto another image from the edge of the reference image and the corresponding point candidate for the edge, predict the existing position of the corresponding point on the other image from the intersection of the projected straight line, and calculate the position and A third corresponding point selection function is executed to determine whether the corresponding point candidates are correct or incorrect by checking the degree of correspondence with the feature points of the reference image or the corresponding point candidates and identifying the existence of visible and hidden parts, and to select the corresponding points. It constitutes a corresponding point detection calculation unit.

73 is the output of the corresponding point table 71 in this 3rd sale corresponding point detection module 72 and the camera parameter table 6
This coordinate calculation module γ3 is a coordinate calculation module that receives the output of 8 as input, and this coordinate calculation module γ3 has a coordinate calculation unit that converts the feature points whose correspondence has been obtained by the corresponding point detection calculation unit into real space coordinates and calculates the coordinates. It consists of 74 is a coordinate table that records the conversion results of this coordinate detection module 73;
Reference numeral 75 denotes a graphic module that three-dimensionally displays the conversion result obtained by the coordinate calculation module 73, 76 a graphic display device that receives the output of this graphic module 75 as input, and 76 various external devices (not shown) based on the results. This is an external input/output interface provided to control the

Next, the operation of the embodiment shown in FIG. M6 will be explained with reference to FIG.

First, the video signals obtained by the three television cameras 14, 15, and 16 are sent to A/D converters 63a r 63b * 63
c is converted into a digital signal and temporarily stored in an image memory 27 consisting of three or more images. Then, the preprocessing calculation circuit 65 sequentially performs edge detection processing 20 and thinning processing 21 shown in FIG. The operations up to this point are the operations of the preprocessing calculation section 60, which is generally often referred to.

Next, the projection straight line harameter for each of the obtained feature points is calculated by the calculation circuit 69 based on the camera parameter table 68, and the result is recorded in the projection straight line table T1. In addition, for the projected multi-edge parameters, all necessary parameters are obtained in advance and created in the projected straight line parameter table 7.
Instead of recording it as 0, it may be calculated every time it becomes necessary in the next stage. Subsequently, the initial correspondence shown in FIG. 3 is performed by the arithmetic circuit 69 from processing 23 to corresponding point selection determination processing 29. In this process, the image memory 64 and the feature table 66 are referred to as necessary, and intermediate results are recorded or read from the corresponding point table γ1. After the processing is completed, by checking the corresponding point table 71, the history up to the determination of the corresponding points can be seen at a glance. Next, the feature points for which corresponding points have been obtained are converted into real space coordinates by the coordinate calculation module 73, and the results are recorded in the coordinate table 74.

The results converted into real space coordinates by the coordinate calculation module 73 are converted into a perspective view or an overview view of the object by the graphics module 75, and colored edges,
The graphic display device 76 displays the image three-dimensionally. Further, the object coordinate data obtained by the coordinate calculation module 73 is used for external device control through the external input/output interface 7T.

Next, an embodiment of the apparatus of the present invention will be described with reference to FIGS. 7 and 8, targeting an object that has many visible and hidden parts depending on the camera position.

This figure 7 shows an example of an image of a building block observed with three television cameras.The building block has a hexagonal pyramid with a height of 30 days mounted on a hexagonal pillar with a ridge length of 30 days. The image is observed with a television camera, the contents of the image memory are displayed on a graphic display device 76 shown in FIG. 6, and a photograph is taken with a video printer to show the edges of the image. A, B,
C each indicates an input image.

The position coordinates of the camera are expressed as (x, y, z), and (
-200 garden, 0, 100m+x), (200 temple, 0°100m), (-100m, 0, 40
0m+), the depression angles are 0.1, 0.1 and 0, respectively.
4, and the Ym marks at the intersection of the optical axis and the Y-2 plane are both 90
0mm, the focal length of the lens is 55 mm.

Figure 8 shows the result of converting the image in Figure 7 into object real space coordinates using the corresponding point processing of the present invention. After recording in the coordinate table 74 shown in FIG.

The result is shown after being converted into an X-Y, Y-Z coordinate perspective view, displayed on a graphic display device γ6, and photographed using a video printer. and,
From the left, the results are 1 for the first step, result II for the second step, and result I for the third step. Also, (a)
is a perspective view of X-2 seen from the front (can)
(c) shows a perspective view X-Y seen from directly above, and (c) shows a perspective view y-z seen from directly side.

In Fig. 8, the broken lines are originally the parts where the correspondence cannot be confirmed in the processing up to the second step', but under the condition that the correspondence order is preserved between images except for visible and hidden parts. The correspondence was obtained in the second step, which was the stage of determining visible and hidden parts. As a result of corresponding point processing through experiments, the number of feature points is 1164, and the uncorresponding points are 121.
．． Wrong response O, response confirmation rate approximately 90%, non-response rate 10
%, it was Person O who responded incorrectly. Also, the maximum measurement error is X-Z
The axial direction is within ±0.3 m + Y-axis direction ±4ttaa, which almost matches the quantization error due to the image sampling interval. In addition, in the conventional binocular stereoscopic viewing method using the local correlation method, the correspondence accuracy rate was only 17%.

As described above, when the object position/shape measuring device according to the present invention is used, an object is observed with at least three cameras, the real space coordinates of each point on the object including visible and hidden parts are determined, and the object position/shape is recorded. It can be displayed in 3D.

FIG. 9 is a configuration diagram showing an example of application of the present invention to an autonomous robot.

In FIG. 9, 13, 14, and 15 are the aforementioned television cameras, and 16 is a target object.

78 is the object position/shape measuring device illustrated in FIG. 6, T7 is the aforementioned external input/output interface included in this object position/shape measuring device 7B, 19 is a system main controller, and 80 is this system main controller. A robot controller 81 is controlled by the device T9, and a robot body 82 is controlled by the robot controller 80.
is a robot arm.

Next, the operation of the application example shown in FIG. 9 will be explained.

First, an image signal of the object 16 is obtained by the television cameras 13, 14, 15, and the image signal is input to the object position/shape measuring device γB, where the coordinates of each point on the object 16 are obtained and a three-dimensional display is performed. be done. And the system main controller 7
At step 9, the amount of movement, direction, and trajectory for moving the robot arm 82 to the object position are calculated based on the object coordinate data transmitted from the object position/shape measuring device 78 via the external input/output interface T1. Furthermore, the direction of the fingers and the distance between the fingers when grasping the object 16 are calculated.

Next, the robot controller 80 calculates the rotation speed and speed of a drive motor (not shown) for each joint corresponding to the amount of movement of the robot main body 81 and the rotation speed and speed of the motor of the joint that drives the arm, and Generates a drive signal. On the other hand, the object position and shape measuring device γ8 constantly observes the position and shape of the object 16, and when the object 16 moves or deforms or an obstacle enters, the system main controller 79 issues a command to change the movement of the robot body 81. emits. Further, in the object position/shape measuring device 78, when the robot arm 82 approaches the object 16, the robot arm 82
Since the position and direction of object 2 can also be recognized, the positional relationship between object 16 and robot arm 82 can be determined. Based on this positional relationship, the system main controller 79 determines detailed movements of the robot arm 82, and the robot controller 80
can give commands to.

In the application example shown in FIG. 9, when an arbitrary object is placed at an arbitrary position, the position and shape of the object are visually recognized, and based on the results, the robot's arm 82 is moved to move the object. It can be made to perform independent movements such as picking it up and carrying it to a predetermined position. Furthermore, even if the object 16 is moving within the response speed of the system, it can be picked up by the robot arm 82 by following its movement.

Above, the present invention has been mainly explained using a vertical edge as a representative example. Edges in all directions can be treated in exactly the same way. That is, this is the above-mentioned FIG. 5(a).

(b) , (The flowchart shown in (()) is a process that can be applied to edges in all directions.

In addition, although the above-mentioned example was explained using a case where a three-eye image is used as an example, the present invention is not limited to this. The accuracy of the processing can be further improved by performing verification similar to the above method using images from the fourth eye onward.

In addition, in the above embodiment, after selecting in advance a plurality of corresponding point candidates with low evaluation values, that is, with a high degree of correspondence, through initial corresponding point processing, the correspondence with the third image is checked for each corresponding point candidate. Although the description has been given using an example of a case where processing to determine correctness or incorrectness is performed, the present invention is not limited to this, and since many similar patterns are included, it is possible to find a correct corresponding point among the top several corresponding point candidates. There may be cases where the In this case, or if there is a possibility of this happening, the procedure is to search for a point with an evaluation value below the threshold value along the projection straight line, and as soon as it is found, check the correspondence with the M3 image and determine whether it is correct or incorrect. By using this in combination, it is possible to improve the accuracy of correct/incorrect judgment and to detect correct corresponding points from among similar patterns.

Furthermore, in the above embodiment, the first step is carried out for the entire image, and the second step is started after the completion of the first step. , the first to second steps may be implemented.

Further, in the above embodiment, a method using images from a television camera was explained as an example, but the present invention is not limited to this, and it is also possible to use photographs obtained from a still camera, depending on the camera used. The type does not matter. As long as parameters such as observation position and direction at the time of image input are clear, no special conditions are given to the image input means.

〔Effect of the invention〕

As explained above, the present invention acquires two-dimensional images from at least three locations on non-collinear lines, independently examines the degree of correspondence on the right and left sides of edges in the images, and
Since the process is performed to determine whether a corresponding point candidate is correct or incorrect by identifying the presence of hidden parts on the right or left side of edges, it is also possible to measure objects with many visible and hidden parts depending on the camera position, which was previously difficult. It is possible to measure edges where one side is visible and hidden, and to identify visible and hidden areas.The processing algorithm is extremely simple and reliable because it uses corresponding point processing between multiple images. Since the properties are improved, the practical effect is extremely large.

Furthermore, in the past, in order to eliminate visible and hidden parts, the camera interval was narrowed and accuracy was sacrificed, but according to the present invention, it is now possible to process visible and hidden parts, so the camera interval can be freely increased. , it is possible to perform highly accurate measurements and increase the accuracy of processing, and camera parameters such as camera position/direction and focal length can be set arbitrarily for each camera. Not only is it easier to set up the
Instead of setting three or more cameras at the same time, one camera is moved one after another to obtain three or more images, and the stereoscopic viewing processing according to the present invention is performed between the images. It is extremely effective in that it is easy to reduce the amount of noise, and to freely change the distance between cameras to flexibly change the measurement area and required accuracy depending on the object.

As described above, the method of the present invention has a great effect compared to the conventional binocular stereoscopic viewing method, and it is possible to identify objects by stereoscopic corresponding point processing from a plurality of two-dimensional images acquired from at least three locations. When three-dimensionally measuring the position and shape of an object, this method is unique as an object position/shape measurement method that identifies visible and hidden parts depending on the camera position.

Further, the present invention is extremely effective in that a highly reliable and general-purpose object position/shape measuring device can be realized.

[Brief explanation of drawings]

Figure 1 is a block diagram for explaining an embodiment of the object position/shape measurement method according to the present invention, and Figure 2 is the principle of corresponding point processing between images from three television cameras in the embodiment of Figure 1. An explanatory diagram, FIG. 3 is a 70-chart of stereoscopic vision compatible processing of trinocular images to explain the operation of FIGS. 1 and 2, and FIG. 4 is an explanation to explain the method of handling visible and hidden parts in the present invention. 5 is a flowchart showing the flow of corresponding point processing in the present invention, FIG. 6 is a block diagram showing an embodiment of the object position/shape measuring device according to the present invention, and FIG. An explanatory diagram showing an image observed by a camera; FIG. 8 is an explanatory diagram showing the result of converting the image in FIG. 7 into object space coordinates using the corresponding point processing of the present invention and displaying the image;
FIG. 9 is a block diagram showing an example of application of the present invention to an autonomous robot; FIG. 10 is a block diagram showing an example of a conventional binocular stereoscopic viewing method;
FIG. 11 is an explanatory diagram showing an example of the distribution of evaluation values obtained by the local correlation method, and FIG. 12 is an explanatory diagram for explaining the handling of visible and hidden parts in the conventional binocular stereoscopic viewing method. 13-15...Television camera, 16...Target object, 11-19...Image, 32...Left image, 3
3... Right image, 34.350... Visible and hidden parts,
36.36'...fist/image, 63a ~63(! @
1111 @ A/D converter, 54 @ @ Image memory, 65... Pre-processing calculation circuit, 66... Feature amount table, 67... Pre-processing calculation unit, 68...
Turtle 2 parameter table, 69...fist calculation circuit,
γ0... Projection straight line parameter table, 71o...
- Corresponding point table, T2... Trinocular corresponding point detection module, 13... Coordinate calculation module, 74... Fist...
Coordinate table, 75...Graphic module 1
6.eII.Graphic display device.

Claims (6)

[Claims] (1) Obtain two-dimensional images from at least three locations on non-colinear lines, find corresponding point candidates by associating two of the two-dimensional images, and identify the feature points of the reference image and their Find each projection straight line from the corresponding point candidate for the feature point onto another image, predict the existing position of the corresponding point on the other image from the intersection of the projected straight line, and calculate the position and the characteristics of the reference image. The degree of correspondence with the point or the corresponding point candidate is determined independently on one side and the other side of the edge,
The object position is characterized in that the object position and shape can be measured by determining whether the corresponding points are correct or incorrect by identifying the existence of a visible or hidden part on one side of the edge, and by selecting the corresponding points.・Shape measurement method. (2) An image input unit, a preprocessing calculation unit that receives the output of this image input unit as input and extracts the parameter list and feature points of the image input unit, and a preprocessing calculation unit that receives the output of this preprocessing calculation unit and uses non-collinear lines as input. Corresponding point candidates are obtained by associating two of the two-dimensional images obtained from at least three locations, and the corresponding point candidates for the feature points of the reference image and the feature points are transferred to other images. Find each projection straight line, predict the existing position of the corresponding point on the other image from the intersection of the projected straight line, and calculate the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate of the edge. For detecting corresponding points, which calculates one side and the other side independently, determines whether the corresponding points are correct or incorrect by identifying the existence of visible and hidden parts on one side of the edge, and executes a corresponding point sorting mechanism that selects the corresponding points. comprising a calculation unit and a coordinate calculation unit that receives the output of the corresponding point detection calculation unit and calculates the coordinates by converting the feature points for which the correspondence has been obtained by the calculation unit into real space coordinates. An object position/shape measuring device featuring: (3) Obtain two-dimensional images from at least three locations on non-collinear lines, and independently examine the degree of correspondence between one side and the other side of the edge from two of the two-dimensional images. Find corresponding point candidates, find the edge of the reference image and each projection straight line from the corresponding point candidate for that edge onto another image, and determine the location of the corresponding point on the other image from the intersection of the projected straight lines. is predicted, and the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate is checked to identify the presence of a visible or hidden part, and the correctness of the corresponding point candidate is determined and the corresponding points are selected. Therefore, an object position/shape measuring method is characterized in that the object position/shape can be measured. (4) An image input section, a preprocessing operation section that takes the output of this image input section as input and extracts the parameter list and feature points of the image input section, and uses the output of this preprocessing operation section as input and Correspondence point candidates are independently determined by independently examining the degree of correspondence on one side and the other side of the edge from two of the two-dimensional images obtained from at least three locations on the reference image. Find each projection straight line from the corresponding point candidate for the edge onto another image, predict the existing position of the corresponding point on the other image from the intersection of the projected straight line, and calculate the position and the feature point of the reference image. or a corresponding point detection calculation unit that executes a corresponding point selection function that determines whether the corresponding point candidate is correct or incorrect by checking the degree of correspondence with the corresponding point candidate and identifying the existence of visible and hidden parts, and selects corresponding points; It is characterized by comprising a coordinate calculating section which receives the output of the corresponding point detection calculating section and calculates the coordinates by converting the feature points for which correspondence has been obtained by the calculating section into real space coordinates. Object position/shape measurement device. (5) Obtain two-dimensional images from at least three locations on non-colinear lines, find corresponding point candidates by associating two of the two-dimensional images, and identify the feature points of the reference image and their Find each projection straight line from the corresponding point candidate for the feature point onto another image, predict the existing position of the corresponding point on the other image from the intersection of the projected straight line, and calculate the position and the characteristics of the reference image. A first corresponding point selection function that determines whether a corresponding point candidate is correct or incorrect and selects corresponding points by checking the degree of correspondence with a point or the corresponding point candidate, and a correct response depending on the first corresponding point selection function. For the feature points for which no points were obtained, the degree of correspondence between the predicted existing position of the corresponding point and the feature point of the reference image or the corresponding point candidate is determined independently for one side and the other side of the edge, and the A second corresponding point selection function that determines whether the corresponding points are correct or incorrect by identifying the presence of a visible or hidden part on one side of the edge, and selects the corresponding points; independently find corresponding point candidates by independently examining the degree of correspondence on the side of the reference image and the other side, and find a projection straight line from the corresponding point candidate to the edge of the reference image and the edge onto another image, By predicting the existing position of the corresponding point on the other image from the intersection of the projection straight lines, and checking the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate, and identifying the presence of a visible or hidden part. An object position characterized in that the position and shape of an object can be measured by obtaining corresponding points by determining whether corresponding point candidates are correct or incorrect and selecting a third corresponding point selection function to select corresponding points.・Shape measurement method. (6) An image input section, a preprocessing operation section that takes the output of this image input section as input and extracts the parameter list and feature points of the image input section, and uses the output of this preprocessing calculation section as input and Corresponding point candidates are obtained by associating two of the two-dimensional images obtained from at least three locations, and the corresponding point candidates for the feature points of the reference image and the feature points are transferred to other images. , predicting the location of the corresponding point on the other image from the intersection of the projection straight line, and checking the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate. A first corresponding point selection function that determines the correctness of corresponding point candidates and selects corresponding points, and a feature point for which correct corresponding points were not obtained by the first corresponding point selection function, By determining the degree of correspondence between the existing position of the corresponding point and the feature point of the reference image or the corresponding point candidate independently for one side and the other side of the edge, and identifying the presence of a visible or hidden part on one side of the edge. A second corresponding point selection function that determines whether the corresponding points are correct or incorrect and selects the corresponding points; and independently examining the degree of correspondence on one side and the other side of the edge from two of the acquired two-dimensional images. , independently obtain corresponding point candidates, obtain a projection straight line from the edge of the reference image and the corresponding point candidate for that edge onto another image, and calculate the corresponding point on the other image from the intersection of the projected straight line. Predict the existing position, check the degree of correspondence between the position and the feature point of the reference image or the corresponding point candidate, identify the existence of visible and hidden parts, determine whether the corresponding point candidate is correct, and select the corresponding point. A corresponding point detection calculation unit that performs a third corresponding point selection function, and the output of this corresponding point detection calculation unit is input, and the feature points for which correspondence is obtained by the calculation unit are converted into real space coordinates. 1. An object position/shape measuring device comprising: a coordinate calculating section for calculating coordinates.