JPH04182710A - Relative positioning system - Google Patents

Abstract

PURPOSE:To enable highly accurate relative positioning of an arbitrary object located in a 3-D space by providing an image input means serving as an adjacent visual sensor and a target mark. CONSTITUTION:Positioning operation is performed by utilizing an operation input means 1, a processing area setting means 2, an image input means 3, a preliminary processing means 4, a corresponding point deciding means 5, a positional attitude calculation means 6, a moving amount calculation means 7, a moving operation means 8 and a state judgement means 9. The target mark consists of marks A(11), B12, C13, D14 and E15 arranged on a plane 17, and the mark A(11) has a larger area than the other three marks. The relaive position between the target mark and a camera is obtained, the calculation result and information of whether or not a mark F16 is controlled are transmitted to the moving amount calculation means 7 and the moving amount to move the camera or the object to the object relative position between the target mark and the camera is calculated. Thus, the highly accurate relative positioning is automatically executed.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention relates to a relative positioning system between a robot and an object, which is necessary when a robot or the like is used to perform object gripping, assembly, docking, etc. It is.

[Conventional technology]

Conventionally, a method has been used in which a mark is attached to a target during assembly, tokinder work, etc., and the image of the mark is input to a camera and image processing is performed to determine the position.

In the first example, the target marks for position measurement are four circular marks placed at the vertices of a quadrilateral and a ball on the ball raised from the center, as shown in Figure 5(a). Five marks consisting of one circular mark are used. This example uses an image processing function that uses noise processing to remove large bright areas and continuously detected bright spots.
- Provided as a code. Regarding this example, Machida, Uenohara et al.
``Prototype of 6-axis proximity position/attitude sensor for space robots''.

33rd Space Science and Technology Union Lecture Collection pp284-p
p285 (1989).

In addition, as a second example, the GFT shown in FIG.
There is a method that uses a three-dimensional target mark called a (grapple fixture target) pattern. G
The FT pattern consists of rectangles superimposed on concentric stripes, and a cut is provided in a part of the concentric ring. or,
A pole is set up from the center of the ring, and a mark is also placed on the tip of the ball.

GF that stores the GFT mark image input into the camera in advance
Estimating the position and orientation of an object by comparing it with the pattern at the center of the T mark, determining the inclination and magnification of the mark, and determining the coordinates of feature points included in the mark using affine transformation and pattern matching. I can do it. Regarding this second example, see, for example, Yamawaki et al., "Study of image-based position detection system of manipulator system," Space Artificial Intelligence Robot Automation Symposium, pp. 193-
ppi 96 (1987).

[Problem to be solved by the invention]

In the first example of the conventional relative positioning method described above, if the target's posture exceeds a certain range, it is difficult to match the image data of the four marks on the plane with each actual mark, and the pole There is also a possibility that the mark on the top and the mark on the plane overlap.

Therefore, there is a problem that the range in which the position and orientation can be detected is considerably narrowed.

In the second example, it has traditionally been used when an operator manually operates a robot while performing work, and humans can understand relative positional relationships to a certain extent and can easily identify marks, but image processing When automatically measuring relative position using methods such as
There have been problems with edge detection, etc., which are susceptible to errors.

[Means to solve the problem]

The relative positioning method of the present invention includes a ring-shaped mark, and four first circular marks each arranged surrounding the ring-shaped mark on a plane on which the ring-shaped mark is arranged and having at least one size different from the other sizes. , and a target mark having a second circular mark arranged outside the plane so that the orthogonal projection onto the plane is located inside the ring-shaped mark, and the relative position of the object to be relatively positioned is known. an image input means for capturing an image of the target mark placed at a certain position; an operation input means for specifying an image processing area in which an image signal from the image input means is to be processed; processing area setting means for setting the image processing area, and binarizing the image signal from the image inputting means within the image processing area set by the processing area setting means, and for each image area in the binarized image. a preprocessing means for removing the image area due to noise using feature quantities including at least area information and an index indicating circularity, and determining barycenter coordinate data of the remaining image area; a corresponding point determining means for associating each of the remaining image areas with each of the first and second circular marks and the cylinder mark using the feature amount and the center of gravity coordinate data; Selecting different calculation processing methods depending on whether or not there is the barycenter coordinate data corresponding to the second circular mark based on the correlated barycenter coordinate data and the geometric data of the target mark stored in advance. position and orientation calculation means for calculating relative position and orientation data between the target mark and the image input means; and the relative position and orientation data from the position and orientation calculation means and the barycenter coordinate data corresponding to the second circular mark. a movement amount calculation means for calculating the amount of movement to be performed on the object or the image input means based on information on whether or not the object or the image input means should be moved; a movement operation means for moving the input means; and a movement operation means for causing the processing area setting means to determine the image processing area again based on information as to whether or not the center of gravity coordinate data corresponding to the second circular mark exists. and a situation judgment means for judging whether to finish after moving.

[Effect]

The principle and operation of relative positioning will be explained in the case where relative positioning of a target mark and a camera is performed using the relative positioning method according to the present invention.

Figure 1 shows a functional block diagram of this system. In this method,
Operation input means 1. Processing area setting means 2゜Image input means 3
．． Pretreatment means 4. Corresponding point determining means 5, position and orientation calculating means 6. Movement amount calculation means 7. A positioning operation is performed using the movement operation means 8 and the situation judgment means 9. FIG. 2 shows an example of the structure of a target mark used in this method. Figure 2 (
a) is a view of the target mark seen from the front;
Figure (b) is a cross-sectional view of the target mark.

As shown in FIG. 2(a), the target marks are mark A11 and mark B1 arranged in a square shape on the plane 17.
2. There are mark C13 and mark D14, and mark Al
Only mark l has a larger area than the other three marks. Furthermore, there is a ring-shaped mark E15 inside the square formed by these four marks. The inside of the ring marked E15 has a conical taper as shown in FIG. 2(b). These marks are white, and the rest of Figure 2 (a)
The hatched areas are painted with black paint. The origin of the target mark is set at the center of gravity of the mark E15 as shown in FIG.

Take two axes. Further, it is assumed that the mark shape dimensions, that is, the three-dimensional positions of the six marks 11 to 160 in the mark coordinate system are known.

FIG. 3 shows an example in which a target mark image is captured from the image input means 3, for example, a camera.

In this figure, the measurement area is area A21. Area B2
2. Area C23, Area D24. Area E25. Area F26. Area G27 and area H28 are shown. The coordinate system is the image plane 2 of the camera as shown in Figure 3.
The X and Y axes are taken on 0, and two axes are taken in the direction perpendicular to the image plane 20 from the center of the image.

When processing area setting data is sent from the operation input means 1 to the processing area setting means 2, the processing area is set according to the data and positioning processing is started.

First, when an image is captured by the image input means 3, the image data is sent to the preprocessing means 4, where an appropriate threshold value is set and it is binarized as shown in FIG. Furthermore, feature quantities such as barycenter coordinates, area, and perimeter are measured for each measurement area from the binarized image. At this time, the center of gravity coordinates are X+, Y as shown in FIG. 3 with mark E15 as the image origin.
+ Based on the coordinate system. It is also assumed that some areas of noise other than marks are included. Therefore, noise is removed using an index that evaluates the area of the area and the circularity of the mark. As this index d, for example, the circumference length ρ is taken, and when the area a and the circumference π are taken, d=n2/(a·4π). This index is independent of the area; if the measurement area is close to a circle, this index approaches l, and the more it deforms from the circle, the larger the value becomes. Area E25 and Area H28) can be distinguished from each other. It also performs processing such as deleting areas with extremely large or small areas.

The remaining area feature amounts are sent to the corresponding point determining means 5, and the correspondence between the mark and the measurement area is determined. As for the determination method, for example, when comparing the areas, the ring-shaped mark E15 has the largest area, so the mark E1
5 corresponding point area C23 can be determined. Also, mark F1
6 is inside mark E15, so area F2 is the barycenter coordinate point closest to the barycenter coordinate point of mark E15.
6 corresponds to mark 16. Mark A out of the remaining 4 points
Since l'l has the largest area, the corresponding point area B22 of mark A11 can also be determined. Regarding the remaining three points, if the coordinates of mark All are in the first quadrant, mark B12
The coordinates of mark C13 are in area A21 which is the second quadrant.
The coordinates are area G27 in the third quadrant, the remaining mark D
14 corresponds to area D24, etc., it is possible to determine the correspondence between all marks and the measured coordinate values of their centers of gravity by determining conditions such as whether mark 14 corresponds to area D24.

Next, when the measured coordinate data is sent to the position and orientation calculation means 6, the position and orientation calculation means 6 first calculates the relative position using the data of the four circular marks on the plane based on the geometric data of the target mark that has been measured in advance. Calculate the posture. This calculation was performed by Takasaki, "A few considerations regarding the inverse matrix of projection transformation"
, the method described in IE79-15 of the Institute of Electronics and Communication Engineers study group material can be used. However, with this calculation alone, the accuracy of pressure detection for pitch and yaw angles will be lower than for roll angles. Therefore, if the coordinates of the center mark F16 have been measured, this mark information can be used to perform repeated convergence calculations using Newton's method, etc.
Estimate pitch and yaw angles with higher accuracy. Mark F16
If cannot be observed on the observation screen, this correction will not be made.

Through the above measurement processing, the relative position between the target mark and the camera is determined, so these calculation results and information on whether or not mark F16 was observed are sent to the movement amount calculation means 7.
and calculates the amount of movement to be performed so that the camera or object moves to the target relative position between the target mark and the camera.

Furthermore, if the mark F16 cannot be observed, the amount of movement is calculated by calculating the amount of movement from the target position to a position offset in ten directions in only two ways. In addition to sending information on whether or not Marri F16 has been measured to the situation judgment means 9,
This calculation result is sent to the movement operation means 8, and movement is executed.

When the mark F16 is observed, the situation determining means 9 assumes that the final positioning has been completed and waits for the next command to be input to the operation input means 1. In addition, if mark F16 is not observed, the measurement accuracy is not sufficient and the -
Perform positioning operations. Therefore, the relative position data after the movement is sent from the situation determining means 9 to the processing area setting means 2. The processing area setting means 2 calculates and sets a minimum processing area within a range in which the target mark can be sufficiently observed from the sent relative position data. In accordance with this setting, positioning is performed according to the operations described above, and the positioning process is completed.

The above method makes it possible to automatically perform high-precision relative positioning. Furthermore, if even higher accuracy is required, the movement and measurement may be repeated several more times.

〔Example〕

Next, specific embodiments of the present invention will be described with reference to the drawings.

FIG. 4 is a configuration diagram of a robot remote control object grasping system which is an embodiment of the present invention.

A target mark 31 and a grip 32 are installed on the object to be gripped 30, a hand-eye camera 41 is used as the image input means 3, and a robot arm 4 is used as the movement operation means 8.
0. A case will be described in which the robot hand 42 is used to grasp the object 30 to be grasped.

The operator uses the robot operating device 4 as the operation input means 1.
6 to input a robot operation command, and the robot drive device 43 controls the robot. The image data of the handheld camera 41 is input to the robot operating device 46 to be presented to the operator, and the image processing device 44 as the preprocessing device 4 extracts the feature amount of the image, and the processing area setting device 2. Corresponding point determining means 52 position and orientation calculating means 6. Movement amount calculation means 7. A system control device 45 having the function of the situation determining means 9 calculates relative positions, generates robot drive commands, and the like.

As a condition, the position where the gripping clip 32 is gripped by the robot hand 42 is the target position, and at the target position, the relative position between the target mark 31 and the hand-eye camera 41 has a certain value only in the Z-axis direction, and the other degrees of freedom are Regarding the direction, target mark 31. is set so that there is no difference in relative position and posture. Grasping grip 32. Hand-eye camera 41
Place. The coordinate system of the hand-eye camera 41 at this time is the same as that in FIG. Also, the relative position and shape from the target mark 31 to the gripping clip 32, and the relative position and shape from the hand-eye camera 41 to the robot hand 4.
The relative positions and dimensions of up to 2 are known.

First, the operator operates the robot arm 40 using the robot operating device 46 and searches for the object to be grasped 30 while viewing the image of the hand-eye camera 41 . Grasped object 3
If 0 is found, the target mark 31 set on the grasped object 30 is further identified, and the robot arm 40 is remotely operated to change the camera viewpoint position so that at least marks A11 to E1.5 can be identified within the field of view of the camera. Make adjustments. Once the above-mentioned mark image is obtained, the processing area is designated, and then the relative position from the hand-eye camera 41 to the target mark 31 is measured by the operation described in the operation section, the robot arm 40 is moved, and the process is performed again. By remeasuring, it is possible to automatically achieve highly accurate positioning.

Finally, if you move to the target relative position in the two axis directions,
Since the positioning of the robot hand 42 is completed, the object 30 to be gripped can be gripped by the gripping operation of the robot hand 42. Further, at this time, if the current image of the target mark 31 and a reference image such as the outline of the target mark 31 at the time of completion of bonding can be superimposed, a person can visually confirm the positioning status.

In order to further improve measurement accuracy, first, based on the measurement take, the robot arm 40 is moved in a direction that eliminates relative position errors in other degrees of freedom while keeping only the Z-axis direction constant. Therefore, the relative position is measured again by image measurement. If even higher accuracy is required, the relative position measurement may be repeated each time by moving closer to the Z-axis direction in steps.

〔Effect of the invention〕

As explained above, according to the present invention, by adopting a configuration consisting of an image input means and a target mark as a proximity visual sensor, it is possible to perform high-precision relative positioning of an arbitrary object placed in a three-dimensional space. be. In image processing according to the present invention, since the image processing area is limited, it is less susceptible to the influence of surrounding noise, and furthermore, by introducing an evaluation index indicating circularity, mixed noise is removed, making it easy to determine corresponding points. . Even if the second circular mark outside the plane where the five marks cannot be observed, the relative position can be measured, so by repeatedly moving and measuring positioning, the positioning range can be expanded beyond the conventional target mark. It can be expanded significantly compared to the method used. Furthermore, it is possible to further improve the positioning accuracy by changing the observation position and performing repeated measurements.

In the above processing operations, the operator first guides the camera etc. to a position where the target mark can be observed, sets the processing area first, and then everything else is processed automatically, reducing the burden of operations on the operator. It can be significantly reduced.

[Brief explanation of drawings]

FIG. 1 is a functional block diagram of the relative positioning method of the present invention.
FIG. 2(a) is a front view of the target mark according to the present invention, FIG. 2(b) is a sectional view thereof, FIG. 3 is a diagram showing an example of an image captured by the image input means according to the present invention, and FIG.
The figure is a block diagram of a robot remote control object grasping system which is an embodiment of the present invention, and FIGS. 5(a) and 5(b) are diagrams showing two examples of conventional target marks. 1... Operation input means, 2... Processing area setting means, 3... Image input means, 4...
Preprocessing means, 5... Corresponding point determining means, 6...
. . . position and orientation calculation means, 7 . . . movement amount calculation means, 8 . . . movement operation means, 9. . Situation judgment means, 30...Gripped object, 31...
...Target mark, 32...Gripping clip, 40...Robot arm, 41...Hand-eye camera, 42...Robot hunt, 43...
...Robot driving device, 44... Image processing device, 45... System control device, 46...
...Robot operating device. Agent Patent Attorney Susumu Uchihara

Claims (1)

[Claims] a ring-shaped mark, four first circular marks each arranged surrounding the ring-shaped mark on a plane on which the ring-shaped mark is disposed and having at least one size different from the other sizes; and an orthogonal projection onto the plane. a target mark having a second circular mark placed outside the plane such that the second circular mark is located inside the ring-shaped mark; and the target placed at a position where the relative position of the object to be relatively positioned is known. an image input means for capturing an image of the mark; an operation input means for specifying an image processing area in which the image signal from the image input means is to be processed; and an operation input means for setting the image processing area specified by the operation input means. processing area setting means; and two image signals from the image inputting means within the image processing area set by the processing area setting means.
Before removing the image area due to noise using feature quantities including at least area information and an index indicating circularity for each image area in the digitized and binarized image, and obtaining barycentric coordinate data of the remaining image area. processing means;
Corresponding point determining means for associating each of the image areas left without being removed by the preprocessing means with each of the first and second circular marks and ring-shaped marks using the feature amount and the barycenter coordinate data. and, based on the barycenter coordinate data associated by the corresponding point determination means and the geometric data of the target mark stored in advance, the barycenter coordinate data corresponding to the second circular mark is different depending on whether or not there is the barycenter coordinate data corresponding to the second circular mark. position and orientation calculation means for calculating relative position and orientation data between the target mark and the image input means by selecting a calculation processing method, and the relative position and orientation data from the position and orientation calculation means and the second circular mark; a movement amount calculation means for calculating the amount of movement to be performed on the object or the image input means based on information on whether or not the center of gravity coordinate data corresponding to the barycentric coordinate data exists; and the amount of movement calculated by the movement amount calculation means. a moving operation means for moving the object or the image input means, and a moving operation means for moving the object or the image input means; and a moving operation means for moving the object or the image input means; and a situation determining means for determining whether to repeat the relative positioning operation or to end the relative positioning operation using the image signal from the image input means after the movement is made by the moving operation means. Relative positioning method.