JPS6257892A - Camera coordinate calibrating method of robot with visual sense - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] Industrial Application Field In a robot with a two-dimensional visual function that can move with a camera attached to its hand, it is necessary to establish a relationship between the XY coordinate system and the absolute coordinate system on the image input from the camera. There are some things that need to be done frequently. Based on visual recognition data such as the position of the reference pattern specified by instructions or programs, and data obtained from visual recognition R during the robot's actual work, the relative position of the workpiece from the reference pattern during work is determined. When working by correcting the robot's operating position and posture by determining displacement, etc., it is necessary to know which direction and size of the vector in the camera coordinate system correspond to the direction and size of the vector in the absolute coordinate system. If possible, it is often possible to establish a sufficient relationship between the two coordinate systems.

The present invention relates to a calibration method for obtaining parameters for converting a vector in a camera coordinate system as described above into a vector in an absolute coordinate system.

2. Description of the Related Art Conventionally, as a calibration method for converting between a camera coordinate system and an absolute coordinate system, the camera is fixed at an appropriate position, and the camera coordinates of multiple points within the camera's field of view are determined using the entire monitor TV.
N', +it measurements were made in the absolute coordinate system of the plurality of points, and the conversion parameters were determined using the coordinate values of both.

Furthermore, in order to automate the above method, two specific marks arranged at a constant distance are placed along the specific pressure machine axis direction of the total absolute coordinate system, and the positions of the two specific marks in camera coordinates are There was a method of visually recognizing the marks, and then determining the parameters for converting between the camera coordinate system and the absolute coordinate system from the direction and magnitude of the vector in the camera coordinates between the marks.

Problems to be Solved by the Invention With conventional techniques, it is necessary to accurately measure multiple points within the camera's field of view, or to accurately arrange a pattern of specific marks in line with one coordinate system of the robot. there were. However, in some cases, it may be difficult to actually measure the object within the field of view of the camera, or the object may be uneven relative to the camera, making it impossible to arrange a pattern of specific marks. Furthermore, when using a specific mark sequence, it may be necessary to create a specific mark sequence depending on the size of the camera field of view, the resolution of the recognition device, etc.

Means for Solving the Problems In order to solve the above problems, in the camera coordinate calibration method for a robot with vision according to the present invention, a plane input with a camera is specified, and a mark indicating a point within the plane is Recognition is performed using a plurality of camera positions obtained by moving the hand parallel to the plane, and a parameter for converting from the camera coordinate system to the absolute coordinate system is calculated from the camera coordinate value of the mark and the amount of hand movement in each recognition. This depends on the method of finding the parameters.

Effect: The method of the present invention recognizes the same fixed point and uses the amount of movement of the hand to determine the parameters for conversion, so it is not necessary to measure the recognition target or to specify the specific arrangement of multiple marks. The relationship between the camera coordinate system and the absolute coordinate system can be found from the hand movement amount and camera coordinate values as follows.

As shown in FIG. 1, the camera coordinate value of the mark 5 when recognized at the first camera position 7 is (x4.yl), and when the hand is moved parallel to the plane to be recognized, the second camera position 6 is The camera coordinate value of mark 5 when recognized is (x2.
y2), the coordinate values of the first and second camera positions in the absolute coordinate system are (ul, v4.wl), (u 2 ,
V2, VV2), the vector (xl-x2.yl-y2) in the camera coordinate system corresponds to the vector (u2-ul, v2-vl, W2-Wl) in the absolute coordinate system. Recognize.

Embodiment FIG. 2 shows the configuration of a visual robot system in an embodiment. 9 is an articulated robot having an axis configuration with six degrees of freedom that allows the position and posture of the hand to be freely moved within a predetermined range of motion. Camera 11 on the hand
is installed. 1Q is a robot controller and a visual recognition device, and the robot can appropriately move its hand and make the camera 11 take an arbitrary position and posture according to commands from the controller, which has a built-in programmed subtraction machine. , the visual recognition device inputs an image from the camera 11, performs pattern recognition, and returns camera coordinate values and the like at the target position to the controller. The controller can change the robot's subsequent behavior based on the data returned from the visual recognition device. 8 is a plane to be visually recognized.

First, the orientation of the plane 8 is taught or set. For teaching and setting the orientation of a plane, there are cases where the robot is actually moved using a teaching pendant, etc., and taught by reading the angles of each joint of the robot when three points on the plane are pointed at the tip of the robot. Settings may be made by directly inputting numerical values specifying the orientation of the plane into the controller using a keyboard or the like. For convenience, an appropriate two-dimensional orthogonal coordinate system on a plane is considered based on the orientation of the plane that has been taught and set, and its coordinate axes are defined as the ~ axis and the Yw axis. When unit vectors in the xw-axis and 7w-axis directions are expressed as Wx and Wy in the absolute coordinate system, the orientation of the plane is specified by Wx and Wy.

Next, a mark that can be recognized by a visual recognition device is placed at an arbitrary point on the plane, or a point that can be considered as a mark that can be recognized by a visual recognition device is selected on the plane. Appropriate data is set and taught to the visual recognition device so that the visual recognition device can recognize the mark or a point that can be regarded as a mark.

Then, the mark position is recognized and the camera posture remains the same.
The mark is moved parallel to the recognition plane, the same mark position is recognized again, and a conversion parameter from the camera coordinate system to the absolute coordinate system is determined based on the camera coordinate value of the recognized mark and the amount of camera movement.

This will be explained based on the flowchart shown in FIG. Recognizing the mark position in the first camera position and orientation;
The obtained camera coordinate value is 01= (xl + yl)T
shall be. Next, the camera is moved to a second camera position and orientation in which the camera orientation is the same as the first camera position and orientation, and the camera position is moved parallel to the recognition plane.

Moving the camera parallel to the recognition plane can be achieved by keeping the hand posture constant and moving the hand in accordance with the orientation of the plane, since the orientation of the plane has been previously taught and set. When the amount of movement of the hand at this time is expressed in the coordinate system (xw, Yw) on the recognition plane,
r a) Suppose that it is T. Next, the mark position is recognized again and the obtained camera coordinate value is set to 02"("2F72)". The conversion parameters from the camera coordinate system to the absolute coordinate system are determined as follows.

The transformation from the vector V, = (X,, yC) in the camera coordinate system to the vector VW-("W l yW) in the coordinate system on the recognition plane is y = = A * V.

however Based on this relationship, θ is determined as a conversion parameter.

k = l RwI/I C2-01'a, =k (
xl-x2) C2=k (y, -72) b=a12+a22 θ=ATAN2 ((-C2・rx+a1-ry)/b
.

(al・rx+a2・ry)/b) However, ATAN2(I), CI), p ==si
Let nθ be a function that gives θ such that q=cO3θ.

Furthermore, suppose that the unit vectors in the xw-axis and 7w-axis directions of the coordinate system on the recognition plane are respectively Wx=(wxl, Wx2.Wx3)T Wy= (wyl, Wy2.Wy3) in the absolute coordinate system. , a vector in the camera coordinate system■. , to the vector ■ in the absolute coordinate system, V=B-A,
It can be found as a relationship called VC.

The above embodiment is the simplest example in which the same mark position is recognized at two camera positions. In the above case, when determining the second camera position, a position that is a constant displacement from the first camera position is taken. Therefore, if the displacement is small, the accuracy of the required conversion parameter may deteriorate due to the relationship between the amount of displacement and the resolution of the visual recognition device. Conversely, if the displacement is large, the mark may not be within the field of view of the camera at the second camera position. As shown in the flowchart in Figure 4, first, the camera position is determined by a small displacement that does not cause the mark to come out from within the camera field of view, the approximate conversion parameters are determined, and then the camera field of view is The position where the amount of displacement is large at the two points entered in the
The third and fourth camera positions are obtained by inversely converting the camera coordinate values, and by moving the hand to the third and fourth camera positions and recognizing the mark position, highly accurate conversion parameters can be determined. Sometimes it is requested.

Further, in the embodiment, a series of procedures including mark position recognition, camera position movement and mark position recognition, and conversion parameter calculation are automatically performed by a programmed controller.

Effects of the Invention As described above, the present invention is a method in which a fixed point within the camera field of view is recognized from multiple camera positions by moving the hand, and conversion parameters are obtained. Vector transformation parameters from camera coordinates to absolute coordinates can be determined without the need to measure by means of any means, and without the need to arrange special patterns along an absolute coordinate system.

[Brief explanation of drawings]

Fig. 1 is a detailed explanatory diagram of the present invention, Fig. 2 is a configuration diagram of a visual robot system according to an embodiment of the present invention, and Fig. 3 is a diagram of the same embodiment, which performs mark recognition at two camera positions. FIG. 4 is a flowchart of a method for recognizing marks at four camera positions in the same embodiment. 1... Robot hand at first camera position, 2... Robot hand at second camera position, 3... Camera at first camera position, 4... ...camera at the second camera position,
5... Mark still recognition target that can be considered as a mark, 6... First camera position, 7...
Second camera position, o, x, y, z... Origin and coordinate axes of absolute coordinate system, ol, xl, yl...
- The origin and coordinate axes of the first camera coordinate system at the first camera position, o2°x2. Y2... Origin and coordinate axis of the second camera coordinate system at the second camera position, "1+3'1... Mark position coordinates in the first camera coordinate system, X2. y2... ... Mark position coordinates in the second camera coordinate system, 8 ... Recognition target plane, 9 ...
Robot with axis configuration of 6 degrees of freedom, 1o...controller and visual recognition device 1.11...camera attached to robot hand, x7. Yw
...Coordinate axes of the coordinate system on plane 8. Name of agent: Patent attorney Toshio Nakao and 1 other person1-
--% + 噌η Mera Shin Lrh cheer Nararo・4te"y)hen y
2---Abusive 2 Brown 3---
-1Iqm voice filter 7--Chi211 Hl O,X,Y=1-Mfl&4N, o+x+, Yt---%+mtrpy degree certainty D9. XJ
t-$24 - Figure 2 Figure 3

Claims (1)

[Claims] It consists of a two-dimensional visual recognition device, a robot with a camera attached to its hand to input images used for recognition, and a controller that includes a computer programmed so that the robot can move freely in an absolute coordinate system. In a robot system with vision, the orientation of a plane that is a target of two-dimensional visual recognition is taught or set, and a mark that can be accurately recognized is placed on the plane within the camera field of view, or
An object that can be accurately recognized on the plane within the field of view of the camera is regarded as a mark, and the robot hand is moved parallel to the plane to recognize the position of the same mark from multiple camera positions. A method for calibrating camera coordinates of a robot with vision, characterized in that a conversion parameter between a camera coordinate system and an absolute coordinate system of the robot is obtained from the absolute coordinate value of the mark position and the camera coordinate value of the recognized mark position.