JPS62214403A - Calibration method of robot system with visual sensor - Google Patents

Abstract

PURPOSE:To make the hand-eye operation possible in a wide range by using another camera and a work for calibration though a camera is not capable of calibration because of an obstacle or the like. CONSTITUTION:When a work 5 is handled with a robot 6 in the calibration method in a robot system with visual sensor, cameras 1 and 2 and their picture processing devices are used to obtain coordinates of centers of holes 41 and 43 in both ends of the work 5, and a gripping point of the robot 6 is obtained in accordance with these two coordinates to handle the work 5. With respect to a camera 3, a work 4 for calibration is so moved that holes 41 and 42 are within the visual field of the camera 2 and the hole 43 is within the visual field of the camera 3, and calibration is performed with the camera 2.

Description

Detailed Description of the Invention [Field of Industrial Application] The present invention provides a robot system with a visual sensor that includes a plurality of industrial television cameras and their image processing devices as visual sensors. This paper relates to a method of calculating a coordinate system with a coordinate system.

[Conventional technology]

Conventionally, in this type of robot system with a visual sensor, an object with a circular shape is placed within the field of view of the camera, the center coordinates of the circle are determined by an image processing device, and then the tip of the robot's arm (grip center position) is Normally, the center coordinates of the circle in the robot coordinates are obtained by calibrating the robot and camera from the two coordinate values, and when there are multiple industrial television cameras. , the above steps were performed for each camera to perform calibration with each camera.

[The interlayer point that the invention attempts to solve]

The above-described conventional method has the disadvantage that the tip of the robot's arm (grip center position) needs to reach within the field of view of all cameras, and it is not possible to calibrate cameras that do not.

[Means for solving problems]

In the calibration method of the robot system with a visual sensor of the present invention, the tip of the arm reaches the field of view of the camera, and the calibrated camera and the camera that cannot be directly calibrated are placed in the first and second positions, respectively.
camera, the first and second objects are within the field of view of the first camera, and the third object is within the field of view of the second camera. A calibration work having an object is placed between the first and second cameras so that the first and second objects are within the field of view of the first camera, and the third object is within the field of view of the second camera. Place it under the first,
The camera coordinate values of the first, second, and third objects are obtained using the second camera and the image processing device, and the first
, determine the robot coordinate values of the second object, and determine the robot coordinate values of the third object from the first, second, and third positional relationships and the determined robot coordinate values of the first and second objects. Then, the second camera is calibrated by slightly moving the calibration work and similarly determining the robot coordinate values of the third object.

[Effect]

Generally, calibrating a camera and a robot means finding constants for converting camera coordinate values obtained from the camera and its image processing device into robot coordinate values. To do this, there must be two points within the camera field of view where the robot coordinate values and camera coordinate values can be seen at the same time. In Figure 3, if we obtain a point 'I+P2 where the robot coordinates and camera coordinates are known, we can convert the camera coordinates (x, y) to the robot coordinates (x
, y) is obtained as follows.

In FIG. 3, the transformation from a point (x, y) on the camera coordinate system to a point (X, Y) on the robot coordinate system is a combination of parallel movement and rotational movement. Therefore, if the angle between the X-axis of the camera coordinate system and the X-axis of the robot coordinate system is T, then C and D are constants of parallel movement, and cos(T)=A 5in(T)J. Therefore, next, constants A, B, C, and D in the above equation are determined.

In Fig. 3, two points P1 (XI *
yl ) * P2 (×2 + y2) are the two points P+(XI, Yl) and P2 on the robot coordinate system, respectively.
(X21 Y2)k: If they correspond, these should be (
By substituting into equation 1), the following equation (2) is obtained.

Xl =Axl +Byl + CY2 =Bx2-Ay2 + D From this, solving the simultaneous equations yields A'' ((XI X2) (XI-X2)-(Y+
−'h)(Vt −Vz))B” ((XI >h)
(Vt-V2)+(Xt-X2)(XI X2))C
= X1AxI By+ D= Yl -Bxl +Ay+ However, Z=(xI-x2)2+(yl-y2)2.

In FIG. 1, it is assumed that camera 1 is a calibrated camera in which the tip of the robot's arm can reach within the field of view, and cameras 2 and 3 are calibrated cameras whose arms do not reach within the field of view. Three holes 41+ 42+ ' whose positional relationship is known as shown in Figure 2! A calibration work 4 having a hole 41+'2 is within the field of view of the camera 1,
When the hole 43 is placed under the camera 1.2 so as to be within the field of view of the camera 2, the camera coordinate values of the hole 'In'2+ 43 can be obtained by the camera 1, the camera 2 and its image processing device. Furthermore, since the camera 1 has been calibrated, the robot coordinate value of the hole '++'2 can also be obtained from the conversion equation (1). If the respective robot coordinate values are (xrI, Yrl) and (xr2.Y'2), the robot coordinate values (x, y) of the hole 43 are, from Figure 2, X = Xr2 - (XrI - Xr2) ・12
/f), 1Y = Yr2 − (Yrl −Y
r2) - JZ2/421, which means that the robot coordinate values for hole 43 have also been obtained. If the above steps are repeated by moving the calibration work 4 a little, and the camera coordinate values and robot coordinate values at the two points of the hole 43 are determined, the camera 2 has been calibrated.

For camera 3, move the calibration work 4 so that hole 'l+'2 is within the field of view of camera 2 and hole 43 is within the field of view of camera 3, and perform calibration in the same way using camera 2. be able to.

In addition, in the above explanation, the hole made in the calibration work is captured with a camera, but instead of the hole, a disc-shaped object may be pasted or painted. Its shape does not necessarily have to be circular.

〔Example〕

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 4 is an embodiment of a calibration method in a robot system with a visual sensor according to the present invention, and is a diagram showing an example in which a workpiece is handled by a robot. in this case,
The workpiece 5 is sent by a belt conveyor, and the center coordinates of the holes at both ends of the workpiece 5 are obtained using the cameras 1 and 2 and their image processing devices, and from these two coordinates, the gripping point of the robot 6 is determined and the workpiece is removed. 5 shall be handled. In this case, it is necessary to calibrate both camera 1 and camera z in advance. However, the tip of the arm of the robot 6 can reach the camera 1 but not the camera 2. Therefore, by using the method described above, the camera 2 can also be calibrated.

[Effects of the Invention] As explained above, the present invention has the advantage that when a large workpiece is handled by a robot using multiple cameras, the camera position may be too far from the robot or there may be obstacles. Even if there is a camera that cannot be calibrated, it can be calibrated by using another calibrated camera and a calibration work, which has the effect of making hand-eye operation possible over a wide range.

[Brief explanation of drawings]

Fig. 1 is a diagram explaining the calibration method of the present invention, Fig. 2 is a plan view of the calibration work 4 in Fig. 1, and Fig. 3 is a diagram showing points PI+P2 in the camera coordinate system and robot coordinate system, respectively. FIG. 4 is a diagram showing an example of handling a work using a robot and a camera. 1, 2. 3-axis camera, 4... Calibration work, 'I+ 42+
'3... Hole, ×I+ YIn ×2+ Y2''・Robot coordinate value,
×l, yI, ×2. y2...Camera coordinate value.

Claims (1)

[Claims] In a robot system with a visual sensor that includes a plurality of industrial television cameras and their image processing devices as visual sensors, the tip of the arm can reach within the field of view of at least one camera, but the other cameras This is a calibration method when the tip of the robot's arm cannot reach the field of view of the robot and calibration with the robot cannot be performed directly. Cameras that cannot be directly calibrated are respectively designated as first and second cameras,
The first and second objects are within the field of view of the first camera, and the third object is within the field of view of the first camera.
The object falls within the field of view of the second camera.The calibration work has a first, second, and third object whose positional relationship is known and the first and second objects fall within the field of view of the first camera. the third object is placed under the first and second cameras so that it is within the field of view of the second camera, and the first, second, and third objects are captured by the first and second cameras and the image processing device. The camera coordinate values of the object No. 3 are determined, the robot coordinate values of the first and second objects are determined using a predetermined conversion formula, and the positional relationships of the first, second, and third objects and the determined first and second objects are determined. The robot coordinate values of the third object are determined from the robot coordinate values of the object, and then the calibration work is moved slightly and the robot coordinate values of the third object are determined in the same manner. A calibration method for a robot system with a visual sensor that performs calibration.