JPH04130587A - Three-dimensional picture evaluation device - Google Patents

Abstract

PURPOSE:To recognize a three-dimensional object accurately and rapidly by comparing the degree of match between picture information to be obtained when image-picking up a model in an estimated position and posture and the picture information outputted from an image pickup part. CONSTITUTION:A three-dimensional position/posture estimation part 15 estimates the position and the posture of the object based on the picture information and the three-dimensional data of the model. A predicting picture information generating part 18 predicts the picture information to be obtained when image- picking up the model in the position and the posture estimated by the three- dimensional position/posture estimation part 15, and a picture information comparing part 19 compares the picture information datarized from the image pickup part and the picture information outputted from the predicting picture information generating part 18 and finds the degree of the match. Thus, it is possible to recognize the three-dimensional object accurately and rapidly.

Description

Detailed Description of the Invention The present invention relates to a three-dimensional image used to recognize a three-dimensional object based on image information obtained from a television camera, an image scanner, etc. Regarding an evaluation device.

For example, "Image and Language Recognition Engineering" (by Makoto Nagao, published by Corona Publishing Co., Ltd. in 1989, p. 142-p. 1)
As shown in 49), there is a known method of object recognition by comparing image information obtained by imaging a two-dimensional object with image information of a pre-stored model. Image evaluation devices that apply this type of method are used.

Similarly, 3D image evaluation devices are also used that recognize 3D objects based on image information of a subject obtained from multiple imaging means and 3D data of a model stored in advance. .

In this type of 3D image evaluation device, for example, based on image information obtained from two imaging means, the position and posture of each part of the subject in 3D space is determined as 3D data, and this 3D data and The three-dimensional object is recognized by comparing the three-dimensional data of the model.

However, the 3D data of the subject can only be obtained when image information is obtained from both imaging means, and at least one of the imaging devices can detect lines hidden behind the subject. Three-dimensional data cannot be obtained for parts such as minutes and vertices. On the other hand, the three-dimensional data of a model includes information regarding all the parts that make up the model.

Therefore, even if the three-dimensional data of the subject and the model are compared, the degree of agreement between the data is low, and therefore, there has been a problem in that it is difficult to improve the recognition accuracy of three-dimensional objects.

In particular, when performing recognition by capturing images of multiple subjects, some parts may be hidden by other subjects and not captured.
layer, recognition accuracy tends to decrease.

Furthermore, as described above, calculations for obtaining three-dimensional data based on image information obtained from the imaging means are complex;
Another problem was that it was difficult to quickly obtain recognition results.

In view of the above points, the present invention aims to provide a three-dimensional image evaluation device that can recognize a three-dimensional object with high accuracy and quickly.

In order to achieve the above-mentioned object, the present invention provides an imaging unit including an imaging unit that images a subject and outputs image information, a model information storage unit that stores three-dimensional data on a model of the subject, 3D position/position estimation for estimating the position and orientation of the subject based on the image information and the 3D data of the model;
Posture estimation unit and the estimated position! and a predicted image information generation unit that predicts image information that would be obtained when imaging the model in the posture, image information output from the imaging unit, and image information output from the predicted image information generation unit. and an image information comparison section that compares the images to determine the degree of coincidence.

The predicted image information generation unit predicts image information that will be obtained when the model is imaged at the position and orientation estimated by the three-dimensional position/orientation estimation unit, and the image information comparison unit , the image information output from the imaging section and the image information output from the predicted image information generation section are compared to determine the degree of coincidence.

Figure 1 is a block diagram showing the configuration of a three-dimensional image evaluation apparatus in an embodiment of the present invention.

In FIG. 1, 11 to 13 are television cameras as imaging means, 14 is an image preprocessing unit that constitutes an imaging unit together with the television cameras 11 to 13, 15 is a three-dimensional position/orientation estimation unit, 16 is a data input device, 17 is a model information storage unit, 1
8 is a predicted image information generation section, 19 is an image information comparison section, 20
21 is an evaluation section, and 21 is an evaluation result output section.

The model information storage unit 17 is previously inputted with data indicating the positions in the three-dimensional space of each vertex and edge in the model M as shown in FIG. 2, for example, through the data input device 16. Data regarding vertices and edges connected to the vertices are stored in a data structure in which they are associated with each other.

As shown in FIG. 3, the television cameras 11 to 13 are arranged at predetermined positions so as to take images of the object H from different directions. These TV cameras 1
Images H1l to H13 taken at points 1 to 13 are projected onto imaging surfaces 11a to 13a, respectively. Here, the imaging surfaces 11a to 13a are actually located behind the imaging lens, but for convenience of explanation, they will be illustrated and described as being inverted on the optical axis and located in front of the lens.

More specifically, the images H11 to H13 are obtained as grayscale images, as shown in FIGS. 4(a) to 4(c), for example. In addition, in the figure, the shading is indicated by the type of hunting line.

The image preprocessing unit 14 extracts image information H1l'' to H13' of vertices and sides as shown in FIGS. 5(a) to (C) based on the grayscale images H1l to H13. Here, the above image information H1,1'' to H13'
More specifically, the extraction of
"Algorithm for P" (Tomita, Takahashi, IEICE Pattern Recognition Understanding Research Group PRU86-87 (19
By obtaining data indicating the position and direction of each vertex and edge using the method shown in 87)), Figures 5(a) to (C) show the reproduction of images based on the obtained data. An example is shown.

The image information H1l" to H13" can be obtained, for example, as data within a predetermined absolute coordinate system in a three-dimensional space, but the invention is not limited thereto.
Alternatively, the data may be obtained as data within a coordinate system specific to the imaging surfaces 11a to 13a.

The three-dimensional position/orientation estimation unit 15 estimates the position and orientation of the object H based on the image information H11′ to H13′ and the data of the model M output from the model information storage unit 17. ing.

More specifically, ``For example, first, a plurality of image information H1l
3 of any vertex in the object H based on °...
coordinates in dimensional space, and the 2 connected to that vertex
Find the angle between the sides. Next, two sides of the model M that form an angle equal to the above angle and a vertex to which the two sides are connected are searched. Then, the model M is arranged such that the vertex and two sides of the model M overlap with the vertex and two sides of the object H.
Find the translation and rotation matrices for moving and rotating .

Note that the position and orientation of the object H are estimated based on a plurality of pieces of image information H1l'' as described above, thereby reducing the number of incorrectly estimated movement matrices, etc.
Although it is possible to reduce the processing load in the predicted image information generation unit 18, etc., which will be described later, the present invention is not limited to this.
andom Sample Cons5ensus:A
Paradigm+ for Model Fitti
ngevith Ap-plications to
Image Analysts and Autom
atedCartography J (Marti
n A, Fischler and Robert C,
Written by Bolles, Graphcs and Imag
eProcessing.

Communications of the ACM
24-6 P, 381-395 (1981)), any three points or three line segments in the image information of the object H in a single screen, and any three points or three line segments in the model. The position and orientation of the object H in the three-dimensional space may be estimated on the assumption that the line segments correspond to each other.

The predicted image information generation unit 18 performs coordinate transformation of the coordinates indicating the positions of each vertex and edge in the model M, transformation according to the perspective law, hidden line processing, etc. based on the movement matrix and rotation matrix, and As shown in FIGS. (a) to (C), image information H11'' to H13'' of vertices and edges that will be obtained when images are captured by each television camera 11-13 is predicted.

Here, the prediction of the image information H11'' is -d,
Based on the data of model M, line drawing image pattern,
Furthermore, as shown in FIGS. 7(a) to (C), a grayscale image pattern H1l"a-H2S"a is generated by performing processing such as ray tracing in consideration of the illumination condition, light reflection, refraction, etc. However, this may be performed by performing the same processing as that of the image preprocessing section 14.

That is, if such processing is performed, even if there is a side that is difficult to detect depending on the characteristics of the image preprocessing section 14, similar image information H1l'' can be obtained, so that the evaluation section 20 described later can obtain the same image information H1l''. Evaluations are made more accurately.Also,
Similar processing is performed on shadows caused by lighting conditions, light reflection, refraction, etc., so image information H1l'' based on the above shapes etc. can also be subject to evaluation. On the other hand, edges for which it is difficult to obtain image information H11' because they are boundaries of regions with similar density can be excluded from evaluation targets.

The image information comparison unit 19 stores image information H1l of the model M.
"..." and the image information H1l of the object H"... are compared to determine the degree of agreement between them.

For example, when the image information H11'' and H11'' are compared, it is detected that the sides indicated by solid lines in FIG. 8 match.

Here, in the same figure, the sides indicated by broken lines indicate the sides that are predicted not to be imaged by the hidden line processing,
The side P indicated by the two-dot chain line is predicted to be imaged, but indicates that it is not included in the image information H1l''.

Similarly, when comparing the image information H12'・HI3''' and HI3''・HI3'', as shown in FIGS. 9 and 10, the image information H12'''・HI3''
” but not included in the image information H12' and HI3' is detected.

Note that the comparison of the image information H1l'', etc. is not limited to the side as described above, but also compares, for example, the positions of vertices and surfaces, and the density of the surfaces, etc. This can be done for various things as long as it can be obtained from the data.

The evaluation unit 20 evaluates whether the estimation of the position and orientation of the object H performed by the three-dimensional position/orientation estimating unit 15 is correct based on the comparison result, and outputs the evaluation result output unit 2.
Evaluation results etc. are output via 1.

For example, in the above example, each side is included in one of the image information H11゛... that should include that side, so the length shown by symbols in Figures 8 to 10 is If we take the ratio of the sum of the lengths of the sides actually imaged to the sum of the lengths of the sides to be imaged, where is the unit length, we get 29.
An evaluation value of OL/29.0L=1.0 is obtained. The larger the evaluation value, the more the image information H1l'...
and HI1゛”... are the same, so
The imaged object H is recognized as being equal to the model M at the estimated position and orientation.

Here, for example, as described above, any of the image information H11
'..., it is determined that the side is being imaged, so that it can be detected that there are sides that are difficult to extract depending on the influence of lighting conditions or the positions of the TV cameras 11 to 13. However, relatively appropriate evaluation values can be obtained. Moreover, by increasing the number of television cameras,
It is also possible to easily improve evaluation accuracy. Further, by using the side to be imaged as a reference and excluding shadowed sides from the evaluation target, the accuracy of the evaluation value is improved.

Further, the above-mentioned evaluation value can be calculated based only on the comparison result of the - set of image information, for example, the image information H1l".Hll". That is, in this case, the total length of the sides actually imaged is 14.5L, and the total length of the sides to be imaged is 15.5L, so 14.5
L/15. An evaluation value of SL-〇, 935 is obtained.

In this way, even when an object is imaged with only one television camera 11, it is possible to determine whether the position and orientation estimation is correct.

In this way, by predicting the image information that will be obtained when the model M is imaged, the image information H1l
Since all the information included in ' to H13° is used for evaluation, highly accurate three-dimensional object recognition is performed. In particular, objects whose position in three-dimensional space is difficult to predict because they are far away from the television camera 11 and therefore are imaged in a reduced size, or objects that are imaged by only one television camera. Even if there is a portion that is hidden, the image information H11'... outputted from the image preprocessing unit 14 is used for evaluation as is, so highly accurate recognition can be performed.

Moreover, even when multiple objects are captured, image information corresponding to the context of each object is generated through hidden line processing, etc., making it possible to reliably recognize each object. However, in this example,
An example has been described in which data indicating the vertices and edges of the object H and model M are compared using image information H11'...
Instead of this, data representing image patterns such as grayscale images or line images may be compared. In this case, a known pattern matching method can be applied, and the correspondence relationship between vertices and edges is not recognized (as explained above, the degree of matching can be evaluated as a whole). According to the present invention, the imaging unit compares the degree of agreement between the image information that would be obtained when the model in the estimated position and orientation is imaged and the image information output from the imaging unit. Since the image information from the image can be effectively used and shadowed parts are excluded from the comparison target, it is possible to recognize three-dimensional objects with high accuracy and quickly.

Furthermore, even if part or all of the model cannot be directly imaged by a camera due to distortion, etc., the recognition accuracy can be improved by comparing shadows, reflections, refractions, etc. predicted to be generated by the model with image information. can be increased.

[Brief explanation of drawings]

Fig. 1 is a block diagram showing the configuration of a three-dimensional image evaluation device according to an embodiment of the present invention, Fig. 2 is a perspective view showing an example of the shape of a model, and Fig. 3 is a state in which an object is imaged with a television camera. FIGS. 4(a) to 4(b) are explanatory diagrams showing examples of captured gray scale images, and FIGS. 5(a) to (c) are examples of image information from which vertices and edges have been extracted. 6(a)-(c) and 7(a)-(C) are explanatory diagrams showing examples of predicted image information. FIGS. 8-10 are explanatory diagrams showing examples of predicted image information. It is an explanatory diagram showing an example of a comparison result. 11-13... Television camera, 14... Image preprocessing unit, 15... Three-dimensional position/orientation estimation unit, 16... Data input device, 17... Model information storage unit, 18...
- Predicted image information generation unit, 19... image information comparison unit, 2
0...Evaluation Department, ■...Evaluation Results Output Department Patent Applicant Ken Sugiura, Director of the Agency of Industrial Science and Technology, and 1 other agent and agent

Claims (2)

[Claims] (1) An imaging unit including an imaging unit that images a subject and outputs image information; a model information storage unit that stores three-dimensional data on a model of the subject; and a model information storage unit that stores three-dimensional data on a model of the subject, based on the image information and three-dimensional data of the model. hand,
a three-dimensional position/orientation estimation unit that estimates the position and orientation of the subject; a predicted image information generation unit that predicts image information that will be obtained when the model at the estimated position and orientation is imaged; A three-dimensional image evaluation device comprising: an image information comparison unit that compares image information output from an imaging unit and image information output from a predicted image information generation unit to determine a degree of coincidence. (2) The three-dimensional image evaluation device according to claim 1, wherein the imaging section comprises: a vertex and/or a side of a subject;
or data indicating the surface as image information, and the predicted image information generation unit generates vertices and/or edges of the model obtained when the model is imaged at the position and orientation estimated by the three-dimensional position/orientation estimation unit. and/or predicts the data indicating the surface as image information, while the image information comparison unit predicts the image information output from the imaging unit and
A three-dimensional image evaluation device, characterized in that it is configured to determine the degree of coincidence of each vertex and/or edge and/or surface with image information output from a predicted image information generation section.