JP2678002B2 - Coordinate system calibration method for a robot with vision - Google Patents

Description

The present invention relates to a method for calibrating a coordinate system between a visual coordinate system and a robot coordinate system in a vision robot.

[Conventional technology]

Conventionally, as a coordinate system calibration method between different coordinate systems such as a visual coordinate system and a robot coordinate system in a robot with vision, as described in JP-A-61-131887, a reference recognition target is used. Prepare and make the robot's hand reference point coincide with its recognition target by operating the robot manually or by remote control, and read the position of the hand at that time, while recognizing the same recognition target with a visual device. The position is calculated, and the visual coordinate system and the robot coordinate system are associated from these results.

[Problems to be solved by the invention]

However, in the conventional technique, an error is likely to occur when guiding and matching the hand reference point of the robot, that is, a specific position such as a hand or a tool that actually acts on the object to the reference recognition target, and Since the difference is generated, the accuracy of the coordinate conversion coefficient varies each time calibration work is performed, and the coordinate conversion coefficient cannot be obtained with high accuracy and stability. In addition to this, a lot of time is required for the calibration work, which is unsuitable as a coordinate system calibration method for a robot with a fingertip vision having an image inputting means at the fingertip.

An object of the present invention is to provide a coordinate system calibrating method for a robot with a vision capable of obtaining coordinate conversion coefficients quickly with high accuracy and stability.

[Means for solving the problem]

The above-mentioned purpose is basically a relative system between a coordinate system in a visual device that recognizes the position and orientation of an object after capturing an image of the object and a coordinate system in a robot that acts on the object. When calibrating the positional relationship, the robot is moved based on the reference data by a predetermined amount from the movement start point in two different directions to a movement end point, so that the movement start point and the movement end point are within the visual field of the visual device. Is formed as the reference recognition target, the relative positional relationship of the recognition target is recognized on the coordinate system of the device by the visual device, and the recognition result of the visual device by the visual device and the recognition result of the robot. Based on the relative positional relationship of the reference recognition target previously known from the reference data on the coordinate system, the visual device and the robot Target system is achieved by being associated as the coordinate transformation coefficients to each other. In the calibration, it is desirable that the robot be moved so as to be substantially orthogonal to each other and parallel to the coordinate axes of the coordinate system of the robot. Moreover, from the coordinate axes of the visual device to the coordinate axes of the robot. The coordinate conversion coefficient to the robot may be calculated independently, and in consideration of the application to the robot with a hand vision, the coordinate system in the visual device can be calculated by moving the device with the device attached to the robot. The relative positional relationship between the robot and the coordinate system is variable. by the way,
The reference recognition target is, specifically, a recognition mark placed at each of the movement start point and the movement end point by the robot, or a line drawn from the movement start point to the movement end point by the robot. Minutes, or starting point of movement,
It may be easily formed as a point drawn at each of the movement end points.

[Action]

In calibrating the relative positional relationship between the coordinate system in the visual device for imaging an object and the coordinate system in the robot that acts on the object, the robot is first in two different directions based on the reference data. It is possible to move a predetermined amount from the movement start point to a certain movement end point, but specifically,
The robots can be moved in directions substantially orthogonal to each other and as parallel as possible to the coordinate axes of the coordinate system of the robot. By this movement, the movement start point and the movement end point in the visual field of the visual device are the reference recognition objects (specifically, the recognition marks arranged at the movement start point and the movement end point, respectively, or the movement start point to the movement end point). Is formed as a line segment drawn up to, or a point drawn at each of the movement start point and the movement end point). In this state, the relative positional relationship of the recognition target is determined by the visual device to the coordinate system of the device. It can be easily recognized as above. On the other hand, the relative positional relationship with respect to the reference recognition target is previously known (recognized) separately on the coordinate system of the robot. Therefore, the recognition result on the visual device coordinate system for the same recognition target,
From that on the robot coordinate system, the coordinate system in each of the visual device and the robot is easily associated with the coordinate conversion coefficient calculated independently from each coordinate axis in the visual device to each coordinate axis in the robot. When the relative positional relationship between the coordinate system in the visual device and that in the robot is variable, the inter-coordinate system calibration can be easily performed in a robot with a hand-eye vision in which the field of view changes with the movement of the robot itself. Is.

〔Example〕

 Hereinafter, the present invention will be described with reference to FIGS. 1 to 6.

First, the visual robot according to the present invention will be described. FIG. 2 shows an example of the configuration of the visual robot and its periphery. As shown in the figure, in addition to the robot 1 itself, a tool 2 provided at the hand of the robot 1, a camera 3 as a visual input device, a stand 4 for fixing the camera 3 at a predetermined position, and an object to be worked around the robot 1 itself. A conveyor 6 for carrying 5 is arranged. Further, a robot control device 21 for controlling the operation of the robot 1 itself and an image recognition device 22 for processing the image data captured by the camera 3 are provided. Incidentally, the robot 1 itself is a multi-joint type including a base portion 7, a swivel portion 8, an upper arm portion 9, a forearm portion 10, a wrist portion 11 having two degrees of freedom of bending and twisting. Robots 1 having different configurations can be used as long as they have flexibility.

Now, FIG. 1 shows an overall outline flow when the coordinate system calibration process is performed. In this case, the reference recognition target is first formed or created.
When forming or creating the reference recognition target, the robot 1 sequentially takes out the recognition marks 12a to 12c from the stocker 13 beside the conveyor 6 by means of the tool (in this example, the gripping mechanism) 2 and then onto the conveyor 6 in the stopped state. It is designed to be put in place. Although the recognition marks 12a to 12c are disc-shaped, the recognition marks 12a to 12c are not limited to this, and may have any shape and combination as long as they have a color having a contrast with the upper surface of the conveyor 6, and the height thereof is taken by the camera 3. The work object 5 should have the same magnifying power as that of the work object 5.
It is the same as the work surface of.

As described above, the recognition marks 12a to 12c are placed on the conveyor 6, and their positions are based on the reference data given to the robot 1. Recognition mark in this example
12a is at an arbitrary reference position, and other recognition marks 12b and 12c are from the arbitrary reference position, respectively, at X and Y of the robot coordinate system 14.
It is arranged at a predetermined position which is separated by ΔX and ΔY in the axial direction. An example of an operation control program for causing the robot 1 to execute such an operation is shown in FIG. 3, in which “MOVE” is a hand movement command and the subsequent “P01”.
And “P02” indicate the symbol of the point to which the hand moves.
Furthermore, "P01 + (0,0, ΔZ ₁ )" etc. from the destination point,
It shows the point where the hand moves in the directions of the X, Y, and Z axes in the robot coordinate system 14. If "P01 + (0,0, ΔZ ₁ )" is taken as an example, the hand is from the point P01 to the Z axis. It is shown that it is moved by ΔZ ₁ minute in the direction. Furthermore, "CLOSE", "OPEN"
Indicate the closing (grasping) and opening (release) commands of the hand, respectively. Therefore, in this example, "P01" is the recognition mark.
12a to 12c shows the take-out position (position of stocker 13),
The recognition mark 12a first gripped at this position is “P02 + (0,
The grip is released at 0, -ΔZ ₂ ) ", and then" P01 "again.
The recognition mark 12b gripped with "P02 + (ΔX, 0, -Δ
The grip is released by "Z ₂ )". Similarly, the recognition mark 12C is also released by "P02 + (0, ΔY, -ΔZ ₂ )".

Now, the recognition marks 12a to 12c placed on the conveyor 6 as described above are all within the visual field of the camera 3 when the hand of the robot 1 is moved to an appropriate position, and are defined by the camera 3. The positions can be recognized by the visual coordinate system 15 that has been created. FIG. 4 is a plan view showing the positional relationship of the recognition marks 12a to 12c in the robot coordinate system 14 and the visual coordinate system 15, respectively. Although the visual coordinate system 15 is positioned in the robot coordinate system 14 as shown in the figure, by determining the positions of the recognition marks 12a to 12c in the visual coordinate system 15, the visual coordinate system 15 and the robot coordinate system 14 are The coordinate conversion coefficient that defines the relative positional relationship between the two is obtained.

First, the positions of the recognition marks 12a to 12c in the visual coordinate system 15 are
Recognition mark as one screen imaged by the camera 3
It is obtained by binarizing the images 12a to 12c and then performing image processing by the image recognition device 22. The positions of the recognition marks 12a to 12c are determined by extracting the contours of the recognition marks 12a to 12c included in the binarized image as boundaries between the bright and dark parts and determining the barycentric position inside the contours. It is what is sought. Specifically, the center of gravity is the visual coordinate system 15 for each pixel that makes up the contour.
The coordinate values at are calculated by averaging. Unit recognition mark 12 obtained in this way
Recognition marks 12a, 12 from the position of a to 12c in visual coordinate system 15
The distances and directions between b, 12a and 12c are known, and these correspond to the predetermined movement amount of the robot 1 in the predetermined direction. That is, as shown in FIG. 4, although the lengths ΔX and ΔY of line segments indicated by arrows are the predetermined movement amounts in the predetermined direction given to the robot 1 in the robot coordinate system 14,
A straight line ▲ P ^V _X ▼ connecting the center positions of the recognition marks 12a and 12b is a line segment visually recognizing ΔX, and similarly, the recognition mark 12
A straight line ▲ P ^V _Y ▼ connecting the center positions of a and 12c is also a visual recognition of ΔY. In visual coordinate system 15, ▲ P ^V _X ▼, ▲ P
^V _Y ▼ is extracted as the X and Y axis components respectively, and these and Δ
The relationship between the coordinate systems is obtained from the relationship with X and ΔY. Specifically, the recognition marks 12a to 1 in the visual coordinate system 15
The coordinate value of 2c is transferred to the robot controller 21, and the robot controller 21 determines the coordinate conversion coefficient by associating the robot movement amounts ΔX and ΔY with the coordinate values.

To specifically explain how to obtain the coordinate conversion coefficient, the X-axis component of the visual coordinate system 15 of ▲ P ^V _X ▼ is expressed as _X ▲ P ^V _X ▼,
The Y-axis component as _{^{Y ▲ P V X ▼, X}} ▲ P V X ▼ of X in the robot coordinate system 14, each mapping to Y-axis _X ▲ P ^V _X ▼ · cos
α ・ A _XX , _X ▲ P ^V _X ▼ sin α ・ A _XY , and _Y ▲ P ^V _X ▼ for _Y ▲ P ^V _X ▼ ・ sin α ・ A _YX・ A _YX , _Y ▲ P ^V _X
▼ ・ cos α ・ A _YY is required. Therefore, the component of ΔX in the robot coordinate system 14 in the X and Y axis directions is Δ
Since X, 0, the following relationship holds.

Here, α is the angle formed by the visual coordinate system 15 with respect to the robot coordinate system 14 (provided that the counterclockwise direction is positive) is A _XX , A _XY ,
A _YX , A _YY is the X, Y between robot coordinate system 14 and visual coordinate system 15.
It is a coefficient indicating the scale ratio of the axes, and the first subscript indicates the axis of the visual coordinate system 15, and the second subscript indicates the axis of the robot coordinate system 14.

Further, in the same manner as described above, the following relationship holds for the recognition result ΔP ^V _Y ▼ of ΔY.

Therefore, the expressions (5) to (8) are obtained from the expressions (1) to (4).

Furthermore, since the ratio of the X component and the Y component is constant in the visual coordinate system 15, the following expression (9) is established.

A _XX / A _YX = A _XY / A _YY (9) Therefore, the formula (10) is obtained by substituting the formulas (5) to (8) into the formula (9).

As described above, the angle formed by the scale ratio between the coordinate axes is obtained. Calculation of such coordinate conversion coefficient
It may be calculated by the image recognition device 22 based on the robot movement amount from the robot control device 21.

Next, another example will be described with reference to FIG. In this example, the camera 3'is attached to the hand part of the robot 1 ', and the field of view is changed by the movement of the camera 3'as the robot 1'moves. belongs to. The calibration procedure between coordinate systems in this case is the same as in the previous case. That is, the procedure for causing the robot 1'to perform a predetermined operation and placing the recognition marks 12a to 12c on the conveyor 6 is the same as the previous example, and a control program for the robot 1'for that purpose is illustrated in FIG. Things can be used. After the recognition marks 12a to 12c are arranged at the predetermined positions as described above,
The camera 3'attached to the hand by the operation of the robot 1 '
Is moved, and the camera 3'is stopped at a position where the recognition marks 12a to 12c are in the visual field and can be recognized. At that time, the positions of the recognition marks 12a to 12c are represented in the visual coordinate system 15 defined by the camera 3 ', as shown in FIG. In the case of this example, the visual coordinate system 15 moves in the robot coordinate system 14 according to the motion of the robot 1 ', but since the mounting positional relationship of the camera 3'to the robot 1'is unchanged, the robot hand does not change. By knowing the current position of, the coordinate value of the origin 0 _V of the visual coordinate system 15 in the robot coordinate system 14 can be known. This is because the operation control of the robot 1'is based on the position servo control for returning the robot position information, so that the current position of the robot hand can be easily known from the position feedback information. Hereinafter, the recognition marks 12a to 1 in the visual coordinate system 15
The relationship between the coordinate systems is derived from the distance between the center positions of 2c and the direction thereof and the predetermined movement amount of the robot 1'in the robot coordinate system 14, and this procedure is performed by the above-mentioned equations (1) to (10). It has become a thing.

Finally, another example will be described with reference to FIGS. 6 (a) and 6 (b). In this example, the recognition target as the reference data is not the recognition mark described above, and as shown in the figure, Using the writing instrument 17 provided on the hand, the line segment 16 or the point sequence 19,1 on the reference surface 18 as the work object mounting surface.
It is formed by drawing 9 '. The operation of drawing the line segment 16 and the point sequence 19, 19 'is automatically executed by programming the robot 1, but the reference plane 18
After drawing these patterns on the top, the camera 3 draws a line
By recognizing the length and direction of 16 or the interval and the direction of the point sequences 19 and 19 ', ΔX and ΔY shown in FIG. 4 can be obtained. In this example, the camera 3 is fixed to the stand 4, but as shown in FIG. 5, the camera 3 is attached to the hand of the robot 1 as shown in the example of the attachment destination. It is also possible to perform the recognition process on the reference recognition target imaged by the camera 3 after moving to and stopping.

〔The invention's effect〕

As described above, according to the present invention, the reference recognition target used for calibration of the visual coordinate system and the robot coordinate system is formed by the robot itself moving along the robot coordinate system. Error is included, and the error is canceled when the position of the robot is corrected along the robot coordinate system. Further, since the operation of matching the robot hand reference to the reference recognition object is not required, the coordinate conversion coefficient can be obtained with high accuracy, stability, and promptly while eliminating alignment errors and individual differences. Become. Further, since the formation of the reference recognition object is executed by the description of the operation program of the robot, it is possible to automatically perform the operation required for the calibration, and it is necessary to frequently perform the calibration because the visual field is constantly changing like the robot with the hand vision. Even if there is a valid one.

[Brief description of the drawings]

FIG. 1 is a diagram showing an overall outline flow of coordinate system calibration processing according to the present invention, FIG. 2 is a diagram showing a robot with vision according to the present invention and its surroundings, and FIG. 3 is a reference recognition target. Showing an example of a robot operation control program for placing a plurality of recognition marks as positions at positions in a predetermined direction, and FIG.
FIG. 5 is a plan view showing the positional relationship between the recognition marks in each of the robot coordinate system and the visual coordinate system. FIG. 5 is a view showing a robot with vision and its surroundings in another example according to the present invention.
(A) and (b) are diagrams showing a robot with vision and its surroundings when a drawn line segment or point sequence is used as a reference recognition target. 1,1 ′ …… Robot, 3,3 ′ …… Camera, 12a to 12c …… Recognition mark (reference recognition target), 14 …… Robot coordinate system, 15
…… Visual coordinate system, 16 …… Line segment (reference recognition target), 17 ……
Writing instruments, 19,19 '... Point sequence (reference recognition target), 21 ... Robot control device, 22 ... Image recognition device.

Claims (6)

(57) [Claims] 1. A relative positional relationship between a coordinate system in a visual device for recognizing the position and orientation of an object after capturing an image of the object and a coordinate system in a robot acting on the object is calibrated. A method of calibrating a coordinate system for moving a robot based on reference data by moving a predetermined amount from a movement start point in two different directions to a movement end point in the visual field of the visual device. And the movement end point is formed as a reference recognition target, the relative positional relationship of the recognition target is recognized on the coordinate system of the device by the visual device, and the recognition result of the recognition target by the visual device is obtained. And the relative positional relationship of the reference recognition target previously known from the reference data on the coordinate system of the robot, the visual device, the robot Coordinate system calibration method of vision with a robot which is adapted coordinate system is associated as the coordinate transformation coefficients to each other in. 2. When moving from a certain movement start point to a certain movement end point in two different directions based on the reference data, the directions are substantially orthogonal to each other and the coordinate axes of the coordinate system of the robot are used. A method for calibrating the coordinate system of a robot with a vision in which the robot is moved so as to be parallel. 3. The coordinate system calibration method for a robot with vision according to claim 1, wherein the coordinate conversion coefficient from each coordinate axis in the visual device to each coordinate axis in the robot is independently calculated. 4. The coordinate system of the visual device according to claim 1, wherein a relative positional relationship with the coordinate system of the robot is changed by moving the coordinate system of the visual device while being attached to the robot. A method for calibrating the coordinate system of a robot with vision. 5. The coordinate system calibrating method for a robot with vision according to claim 1, wherein the reference recognition target is formed by recognition marks which are arranged at a movement start point and a movement end point by the robot. 6. The reference recognition object according to claim 1, wherein the reference recognition target is drawn by a robot at a line segment drawn from a movement start point to a movement end point, or at each of a movement start point and a movement end point. A coordinate system calibration method for a robot with vision formed as points.