JP2009216480A - Three-dimensional position and attitude measuring method and system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a three-dimensional position and attitude measuring method and an apparatus capable of accurately detecting the position and attitude of an object by one camera without having to use teaching images of the object. <P>SOLUTION: The three-dimensional position and attitude measuring method includes: a model input step S1 for previously storing model data 5 of the object 1; an image taking step S2 for photographing the object to be detected and acquiring images 2; a characteristics extraction step S3 for extracting characteristic data 6 in the images corresponding to the model data from the images 2; a matching degree evaluation step S5 for evaluating the degree of geometric matching between the model data and the characteristic data in the images; a coordinate transformation step S7 for computing a coordinate transformation formula such that the model data may move to the position of characteristic data in images having a high degree of geometric matching; and a position and attitude determination step S8 for determining the three-dimensional position and attitude of the object 1 by the coordinate transformation formula. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a method and apparatus for measuring a three-dimensional position and orientation of an object (target object) based on feature arrangement.

In handling robots, self-running robots, humanoid robots, etc., it is necessary to measure an object three-dimensionally and measure its position and orientation as the eyes of the robot.
In order to achieve this object, for example, Patent Documents 1 to 3 have already been proposed.

  The “robot device having an image processing function” in Patent Document 1 teaches in advance how to see an object by photographing an object from various directions with a single camera. The position and orientation of the object are estimated by pattern matching with the appearance of the object.

Patent Document 2 “Article Position Detection Method and Device” detects a three-dimensional position of an object by stereo measurement. First, the left and right images are collated with the two-dimensional appearance model, and the position of the feature portion of the object on each image is determined. The three-dimensional position of the object is determined by performing stereo viewing in which the left and right feature parts are associated.
The collation between the two-dimensional appearance model and the image is performed by, for example, extracting a feature portion (for example, the center of a circle on the image) on the image and minimizing the sum of the angular differences of the vertices of the polygon connecting the plurality of feature portions. The combination of features is determined.

  Patent Document 3 discloses a “three-dimensional object recognition method and apparatus”, which is data that approximates a contour or the like obtained from an input image to a straight line or an arc, and has three-dimensional position data obtained by a stereo measurement, and a model. The three-dimensional position and orientation of the object is obtained by data matching. Here, a feature group that is a group of features that can form three points necessary for obtaining a three-dimensional position and orientation is used.

Japanese Patent No. 3300682 “Robot Device with Image Processing Function” Japanese Patent No. 3738456, “Article Position Detection Method and Apparatus” Japanese Patent Laid-Open No. 9-212463, “Three-dimensional object recognition method and apparatus”

  The above-described apparatus of Patent Document 1 requires a large amount of image teaching for each object. Therefore, the matching processing time becomes long. Further, the resolution of the posture depends on the number of teaching images, and there is a problem that the accuracy is poor.

  The means of Patent Document 2 requires that the appearance of an article be specified in advance. It cannot be used when there are multiple objects on the screen or when the background is complicated. In addition, the final position / orientation detection requires stereo viewing by two cameras.

  Since the means of Patent Document 3 collates for each simple group (such as two line segments), a large amount of erroneous detection occurs. Therefore, it is necessary to reduce false detections by narrowing down the group while verifying with distance data obtained by stereo viewing by two cameras. In addition, it cannot deal with complicated objects, multiple objects on the screen, or messy backgrounds.

  The present invention has been developed to solve the above-described problems. That is, an object of the present invention is to provide a three-dimensional position / orientation measurement method and apparatus that can accurately detect the position and orientation of an object with one camera without teaching an object (object). is there.

According to the present invention, a model input step for storing in advance model data of an object to be detected;
An image capturing step of capturing an image by capturing an object;
A feature extraction step of extracting feature data on the image corresponding to the model data from the image;
A degree of matching evaluation step for evaluating the degree of geometric matching between the model data and feature data on the image;
A coordinate conversion step of calculating a coordinate conversion formula so that the model data moves to the position of the feature data on the image having a high degree of geometric matching;
There is provided a three-dimensional position / orientation measurement method comprising: a position / orientation determination step for determining a three-dimensional position / orientation of an object by the coordinate conversion formula.

  According to a preferred embodiment of the present invention, the model data is composed of three or more feature data including at least one circle, which are located on the same plane.

In addition, when there are multiple feature data on the same image,
After the feature extraction step, there is a feature correspondence creation step for creating a combination of model data and a plurality of feature data on the image,
In the matching degree evaluation step, a geometric matching degree is evaluated for the combination,
A grouping step for grouping the combinations having a high degree of geometric matching;
In the coordinate conversion step, a coordinate conversion formula is calculated for each group.
Further, in the grouping step, the groups are rearranged in descending order of geometric matching degree, and when the same feature data on the image is included in a plurality of groups, the feature of the group having the low geometric matching degree is removed. It is preferable to re-evaluate the group.

According to the present invention, an imaging device that captures an image of an object to be detected and acquires an image;
An arithmetic device for determining a three-dimensional position and orientation of the object from the image,
The arithmetic device extracts a feature data on an image corresponding to the model data from the image, a feature extraction step;
A degree of matching evaluation step for evaluating the degree of geometric matching between the model data and feature data on the image;
A coordinate conversion step of calculating a coordinate conversion formula so that the model data moves to the position of the feature data on the image having a high degree of geometric matching;
A three-dimensional position / orientation measurement apparatus is provided that executes a position / orientation determination step of determining a three-dimensional position / orientation of an object using the coordinate conversion formula.

  According to the present invention, the feature data on the image corresponding to the model data is extracted, and the coordinate conversion formula is calculated so that the model data moves to the position of the feature data on the image. Without teaching, the position and orientation of an object can be accurately detected with a single camera.

  Further, since the model data is composed of shape features such as a circle, a line segment, and a corner, the model data is simple and teaching of a large amount of image data is unnecessary. Therefore, it can easily cope with a wide variety.

  Further, the degree of geometric coincidence between the model data and the feature data on the image is evaluated, and the feature data on the image having the highest degree of geometric coincidence is extracted. It is possible to cope with the inclination with respect to the gaze direction. Therefore, it is not necessary that the appearance of the article is specified in advance.

  In addition, when there are a plurality of feature data on the same image, the geometric coincidence between the model data and the plurality of feature data on the image is calculated. Since the group is detected, erroneous correspondence can be removed by grouping based on the geometric matching degree, so that it is possible to deal with a case where there are a plurality of objects on the screen and a messy background.

  In addition, since feature data on the image is extracted based on the degree of geometric matching, it is possible to detect the position and orientation based on some feature data even if not all feature data corresponding to the model data can be detected. Yes, it can cope with partial hiding of objects.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol is attached | subjected to the common part in each figure, and the overlapping description is abbreviate | omitted.

FIG. 1 is an overall configuration diagram of a three-dimensional position / orientation measurement apparatus according to the present invention.
In this figure, a three-dimensional position / orientation measuring apparatus 10 of the present invention captures an object 1 to be detected and acquires an image 2 and an operation for determining the three-dimensional position / orientation of the object 1 from the image 2. Device 14.

  The imaging device 12 (camera in this example) is mounted on the hand 4 of the robot 3 and acquires the image 2 by photographing the object 1 to be detected. In addition, the camera may be fixed to the ceiling or the like. Further, the position of the camera may be configured to be controllable separately from the robot hand.

  The computing device 14 is, for example, a computer, and in this example, includes a robot control unit 15, an object measurement processing unit 16, and a model database unit 17. The model database unit 17 stores model data 5 of the object 1 in advance.

The computing device 14 is configured to execute a feature extraction step S3, a coincidence degree evaluation step S5, a coordinate conversion step S7, and a position and orientation determination step S8, which will be described later.
In the feature extraction step S <b> 3, the feature data 6 on the image corresponding to the model data 5 is extracted from the image 2 by the object measurement processing unit 16.
In the coincidence evaluation step S5, the geometric coincidence between the model data 5 and the feature data 6 on the image is evaluated.
In the coordinate conversion step S7, a coordinate conversion formula is calculated so that the model data 5 moves to the position of the feature data 6 on the image having a high degree of geometric matching.
In the position / orientation determination step S8, the three-dimensional position / orientation of the object 1 is determined by a coordinate conversion formula.

The robot 3 provided with the above-described three-dimensional position / orientation measuring apparatus 10 of the present invention operates as follows.
1. Perform camera calibration (camera internal parameter calculation).
2. 2. Calibrate the camera coordinate system and hand coordinate system. The model data 5 of the object 1 is read from the model database unit 17. If there is no model data 5, it is newly created separately.
4). The hand 4 of the robot 3 is moved to a certain position, and the image 2 is taken. The hand position / posture in the robot coordinates at this time is recorded.
5. The three-dimensional position / orientation measurement of the object 1 is performed by the arithmetic unit 14 to obtain the position / orientation of the object 1 in the camera coordinates.
6). The position and orientation of the object 1 in the robot coordinates are calculated from the hand position and orientation in the robot coordinates and the position and orientation of the object 1 in the camera coordinates.
7). Based on the above result, the robot 3 takes an action such as gripping the object 1.

FIG. 2 is an overall flowchart of the three-dimensional position and orientation measurement method according to the present invention.
First, a case where a single feature data exists on the same image will be described with reference to FIG.

  As shown in FIG. 2, the three-dimensional position / orientation measuring method of the present invention includes a model input step S1, an image photographing step S2, a feature extraction step S3, a matching degree evaluation step S5, a coordinate conversion step S7, and a position / orientation determination step S8. .

In the model input step S1, the model data 5 of the object 1 is stored in the model database unit 17 in advance.
The model data 5 is composed of three or more feature data including at least one circle. In addition, these shall be located on the same plane.

  In the image capturing step S2, the object 1 to be detected is captured by the imaging device 12 (camera) to acquire the image 2.

In the feature extraction step S3, feature data 6 on the image corresponding to the model data 5 is extracted from the image 2.
In the coincidence evaluation step S5, the geometric coincidence between the model data 5 and the feature data 6 on the image is evaluated.

FIG. 3 shows a method for evaluating the degree of geometric matching when the two features of the model data 5 and the two features of the feature data 6 on the image are both circles.
The geometric coincidence evaluation value x is defined by Equation (1).

FIG. 4 shows a geometric coincidence when two features of the model data 5 and two features of the feature data 6 on the image are one circle and one corner, or one circle and one line segment. The evaluation method of degree is shown. In this figure, (A) is when the feature point is a corner, and (B) is when the feature point is a line segment.
In this case, in both (A) and (B), the geometric coincidence evaluation value x is defined by Equation (2) of Equation 1.

  The geometric coincidence evaluation value x according to the expressions (1) and (2) is not easily affected by the position and orientation of the object 1. Further, the smaller the geometric matching score evaluation value x, the more accurately the model data 5 and the feature data 6 correspond to each other, and “the geometric matching score is higher”.

  In the coordinate conversion step S7, a coordinate conversion formula is calculated so that the model data 5 moves to the position of the feature data 6 on the image having a high degree of geometric matching. That is, the model data 5 is rotated and moved three-dimensionally so that the model data 5 and the feature data 6 on the image coincide with each other, and the 6 degrees of freedom data of the rotation amount and the movement amount is acquired.

  In the position / orientation determination step S8, the three-dimensional position / orientation of the object 1 is determined based on the calculated coordinate conversion formula, that is, the acquired 6-degree-of-freedom data.

Next, a case where a plurality of feature data 6 exists on the same image 2 will be described.
FIG. 5 is an example of the model data 5 in this case, and includes three circles m1, m2, and m3.
FIG. 6 is an example of the feature data 6 on the image in this case, and is composed of nine ellipses s1 to s9.

  When a plurality of feature data 6 exists on the same image 2, as shown in FIG. 2, the method of the present invention further includes a feature correspondence creation step S4 and a grouping step S6.

In the feature correspondence creating step S4, a combination of the model data 5 and the plurality of feature data 6 on the image is created after the feature extracting step S3.
FIG. 7 shows an example of this combination. When the model data 5 is three circles m1, m2, and m3 and the feature data 6 is nine ellipses s1 to s9, the combination is c1 (m1, s1). , C2 (m1, s2), c3 (m1, s3),..., C27 (m3, s9).

Next, in the coincidence evaluation step S5, the geometric coincidence evaluation value x is evaluated for the combinations (27 types).
FIG. 8 is a diagram showing a list of geometric matching degrees for the combinations (27 types). The geometric coincidence evaluation value x indicated by xij is a positive value according to the equations (1) and (2).

Next, in the grouping step S6, among the combinations of the model data and the feature data on the image, those having a high degree of geometric matching are grouped, and the groups are rearranged in the descending order of the degree of geometric matching.
FIG. 9A shows a state in which three groups are detected from the plurality of feature data 6 of FIG. 6 in the grouping step S6. The geometrical matching degree x of the three groups is assumed to be high in the order of a, b, and c.
As shown in FIG. 9B, when the same feature data 6 on the image is included in different groups, the feature of the group having a low geometric matching degree x is removed, and the group is re-evaluated. .

  Next, in coordinate conversion step S7, a coordinate conversion formula is calculated for each group.

FIG. 10 is a specific flowchart of the matching degree evaluation step S5.
In this figure, the coincidence degree evaluation step S5 includes steps S51 to S58.
In this coincidence evaluation step S5, two feature data Ci of the feature data group of the model data 5 and two of the feature data groups of the image are obtained for the model data 5 and the plurality of feature data 6 on the image. A geometric matching score evaluation value xij of the combination of the feature data Cj is calculated.
The geometric matching score evaluation value xij is calculated by all combinations necessary for object detection.

FIG. 11 is a specific flowchart of the grouping step S6.
In this figure, the grouping step S6 includes steps S61 to S69.
In this grouping step S6, the corresponding Ci of the model data 5 and the feature data 6 on the image is extracted and added to the group as the first one. Another correspondence Cj having a high degree of geometric matching with this correspondence Ci is added to the group. After that, if there is a new correspondence with a high degree of geometric matching for any of the correspondences included in the group, the correspondence is added to the group in order, resulting in a high degree of geometric matching overall. Get a group. If there are a plurality of objects on the screen, a plurality of groups are generated.

  According to the present invention described above, the feature data 6 on the image corresponding to the model data 5 is extracted, and the coordinate conversion formula is calculated so that the model data 5 moves to the position of the feature data 6 on the image. Without the image teaching of (target 1), the position and orientation of the object can be accurately detected with one camera (imaging device 12).

  Since the model data 5 is composed of shape features such as a circle, a line segment, and a corner, the model data is simple, and teaching of a large amount of image data is unnecessary. Therefore, it can easily cope with a wide variety.

  Further, the degree of geometric coincidence between the model data 5 and the feature data 6 on the image is evaluated, and the feature data 6 on the image having the highest degree of geometric coincidence is extracted. In addition, it can cope with the tilt of the camera with respect to the line-of-sight direction. Therefore, it is not necessary that the appearance of the article is specified in advance.

  In addition, when there are a plurality of feature data 6 on the same image, a geometric coincidence evaluation value x between the model data and the plurality of feature data on the image is calculated. Since a plurality of groups are detected in descending order, erroneous correspondence can be removed by grouping based on the geometric coincidence, so that it is possible to deal with a case where there are a plurality of objects on the screen and a messy background.

  In addition, since feature data on the image is extracted based on geometric matching, it is possible to detect the position and orientation based on some feature data even if not all feature data corresponding to the model data can be detected. Yes, it can cope with partial hiding of objects.

  In addition, this invention is not limited to embodiment mentioned above, Of course, it can change variously in the range which does not deviate from the summary of this invention.

1 is an overall configuration diagram of a three-dimensional position / orientation measurement apparatus according to the present invention. It is a whole flow figure of the three-dimensional position and orientation measuring method by the present invention. It is a figure which shows the case where the model data 5 and the feature data 6 are two circles. It is a figure which shows the case where the model data 5 and the feature data 6 are one circle and two or more feature points. This is an example of the model data 5 when a plurality of feature data 6 exists on the same image 2. This is an example of feature data 6 on an image when a plurality of feature data 6 exists on the same image 2. This is an example of all combinations of model data 5 and a plurality of feature data 6 on an image. It is a figure which shows the list | wrist of a geometric coincidence about all the combinations (27 types). In the grouping step S6, a plurality of groups are detected in descending order of geometric matching. It is a specific flowchart of coincidence degree evaluation step S5. It is a specific flowchart of grouping step S6.

Explanation of symbols

1 object, 2 images, 3 robots, 4 hands,
5 model data, 6 image feature data,
10 three-dimensional position and orientation measurement device, 14 arithmetic device (computer),
15 robot control unit, 16 object measurement processing unit,
17 Model database section

Claims (5)

A model input step for preliminarily storing model data of an object to be detected;
An image capturing step of capturing an image by capturing an object;
A feature extraction step of extracting feature data on the image corresponding to the model data from the image;
A degree of matching evaluation step for evaluating the degree of geometric matching between the model data and feature data on the image;
A coordinate conversion step of calculating a coordinate conversion formula so that the model data moves to the position of the feature data on the image having a high degree of geometric matching;
A three-dimensional position / orientation measurement method comprising: a position / orientation determination step for determining a three-dimensional position / orientation of an object by the coordinate conversion formula.   The three-dimensional position / orientation measurement method according to claim 1, wherein the model data includes three or more feature data including at least one circle, which are located on the same plane. When there are multiple feature data on the same image,
After the feature extraction step, there is a feature correspondence creation step for creating a combination of model data and a plurality of feature data on the image,
In the matching degree evaluation step, a geometric matching degree is evaluated for the combination,
A grouping step for grouping the combinations having a high degree of geometric matching;
The three-dimensional position and orientation measurement method according to claim 1, wherein, in the coordinate conversion step, a coordinate conversion formula is calculated for each group.   In the grouping step, the groups are rearranged in descending order of the geometric matching degree, and when the same feature data is included in the plurality of groups on the image, the feature of the group having the low geometric matching degree is removed, The three-dimensional position and orientation measurement method according to claim 3, wherein group re-evaluation is performed. An imaging device that captures an image of an object to be detected and acquires an image;
An arithmetic device for determining a three-dimensional position and orientation of the object from the image,
The arithmetic device extracts a feature data on an image corresponding to the model data from the image, a feature extraction step;
A degree of matching evaluation step for evaluating the degree of geometric matching between the model data and feature data on the image;
A coordinate conversion step of calculating a coordinate conversion formula so that the model data moves to the position of the feature data on the image having a high degree of geometric matching;
A three-dimensional position / orientation measurement apparatus that performs a position / orientation determination step of determining a three-dimensional position / orientation of an object using the coordinate conversion formula.

Images