JPH04100117A - Detection of three-dimensional positional deviation of edge for industrial robot with visual sensor - Google Patents

Abstract

PURPOSE:To easily obtain the extent of deviation with a high precision by operating a vector on camera coordinates to a deviated position based on the extent of deviation from a reference position on camera coordinates which is detected for deviation of an edge and operating the extent of deviation on robot coordinates in accordance with a vector on robot coordinates corresponding to this vector. CONSTITUTION:Coordinates of a point 13 are denoted as (VX,VY), and the distance from a point 7 (VX0,VY0) of a picture is expressed with VX=C 1X(alphax)+C2X(betax) and VY=C1X(alphay)+C2X(betay), and C1 and C2 are obtained. alphax, alphay, betax, and betay are respective elements of vectors Valpha(alphax,alphay) and Vbeta(betax,betay) on a visual sensor and are obtained by alphax=VX1-VX0, betax=VX2-VX0, alphay= VY1-VY0, and betay=VY2-VY0. C1 and C2 are used to obtain the extent of deviation from a reference point 8 (X0, Y0, Z0) on robot coordinates. Thus, the extent of deviation is obtained with a high precision.

Description

[Detailed description of the invention] [Industrial application field] The present invention relates to an industrial robot with a visual sensor.

[Conventional technology]

As a method of detecting a three-dimensional positional shift of an edge using an industrial robot equipped with a visual sensor, there is a method of using laser light, and the following two methods are known.

(1) The workpiece is irradiated with laser light as a point, and the distance from the irradiated point to the workpiece is calculated based on the principle of triangulation from the reflected laser light, and the laser is irradiated while continuously moving the position. This method continuously determines the distance from the irradiated point to the workpiece, and recognizes the position where the distance changes discontinuously as the edge position.

(2) There is a method of detecting the edge position by irradiating a laser slit beam and finding the position where the laser beam bends at the edge.

Method (1) usually uses a light-receiving element whose voltage changes depending on the intensity of the reflected light, and measures the distance between the workpiece and the irradiation point and the voltage of the light-receiving element by directing the irradiation onto the workpiece. , find the relationship between the voltage of the photodetector and the distance,
The distance is calculated from the detected voltage of the light receiving element. In this case, when moving the point continuously, the direction of the laser beam is changed using a mirror, or the module itself that integrates the laser beam irradiation part and the light receiving part is moved.

Method (2) uses a projector that emits laser slit light and a CCD camera that receives reflected light to detect the point where the laser light bends at the edge on the camera coordinates, and then The position is determined by calibrating the relationship between the camera and the plane, and the depth direction is calculated using the principle of triangulation. To calibrate the camera and plane, find out how many millimeters the pixel size of the camera corresponds to in real space, and find a matrix that represents the relationship between the camera coordinates and the robot coordinates.

[Problem to be solved by the invention]

Works targeted by industrial robots, particularly articulated industrial robots, are three-dimensional works such as the body of a car, and are multicolored.

For such a workpiece, in the method (1), it is difficult to apply the irradiation directly to the workpiece, and if the workpiece is tilted, the amount of reflected laser light changes, resulting in poor accuracy.

Furthermore, since the amount of reflected laser light changes depending on the color of the workpiece, it is necessary to separately determine the distance relationship depending on the color.

Furthermore, in order to continuously move the point, a mechanical drive unit is required, which complicates the structure.

On the other hand, in the method (2) above, the CC of the laser emitter and the light receiver is
If the positional relationship between the D camera section and the workpiece is kept constant,
It is possible to carry out a plane plane by - degrees, but for three-dimensional workpieces such as the body of an automobile, it is difficult to maintain a constant positional relationship with the workpiece. In other words, if the color of the workpiece changes, the amount of reflected light changes, so for example, if you set the optical settings for a black workpiece, the image may blur and cannot be detected for a white workpiece. It may be necessary to change the slope of the color depending on the color, and in this case, it is necessary to recalibrate separately. However, performing calibration in three-dimensional space is
This is a very complex process, and it is difficult to achieve accuracy.

In addition, with either method, in order for the industrial robot to work based on the detected data, it is necessary to convert the obtained data back into values on the robot coordinates.

Since the calibration is performed by an operator, there is a problem that accuracy varies due to individual differences.

The present invention has been made in view of the above-mentioned problems, and 3.
To provide a method for easily and accurately determining the positional deviation of an edge on a dimensionally inclined workpiece even if the relationship between a camera, a laser projector, and the workpiece changes.

[Means to solve the problem]

In order to achieve this object, the present invention uses a teaching playback type industrial robot, in which a CCD camera and a device for projecting laser slit light are attached to the end effector of the industrial robot, to control the three-dimensional positional deviation of edges. In the method to detect
The laser slit light is emitted using a laser slit projector attached to the end effector, and at this time, the position of the robot control point on the robot coordinates is memorized, and the image is captured using a CCD camera attached to the end effector. The image is analyzed by an image processing device, and the bending position due to the edge of the laser slit light is detected and memorized on the camera coordinates. Next, the industrial robot is moved in two directions, and at each point after the movement, The position of the control point of the industrial robot i on the robot coordinates and the position on the camera coordinates of the bending position of the laser slit light obtained by processing the image captured by the CCD camera attached to the end effector are detected. , The positions of the three control points on the detected robot coordinates and the positions of the three points on the visual sensor are memorized, and the robot coordinates are determined based on the detected points, starting from the starting point. Form vectors in two directions with the detected point, and then form vectors in two directions corresponding to the vectors of the robot) and coordinates -[- on the camera coordinates. , find the coefficient of the vector that allows the position detected on the camera coordinates to reach the point detected by the vectors in two directions on the camera coordinates formed with no positional deviation, and calculate the coefficients of the vectors. 2 on the camera coordinates with the sign reversed
This method is characterized in that the combined value of the vectors obtained by adding a coefficient to the vector on the robot coordinates corresponding to the vector in the direction is used as the amount of positional deviation of the workpiece on the robot coordinates.

[Effect]

By forming vectors corresponding to each of the robot coordinates and camera coordinates based on this memorized position,
Based on the amount of deviation from the reference position on the camera coordinates that is detected when the edge shifts during actual work, calculate the vector of the camera coordinates that will reach the shifted position, and move the robot corresponding to this vector. The amount of deviation on the robot coordinates can be calculated from the vector on the coordinates.

[Example] The present invention will be specifically described below based on an example shown in the drawings.

First, as shown in Fig. 1, C
Attach the CD camera 2 and laser projector 3, and place the work 4
CC is the bending point 6 of the laser slit light irradiated close to the surface of the edge 5 to be detected with no deviation.
Captured with D camera 2. It is assumed that the position of the robot is set so that the bending position of the laser slit light is near the center of the CCD camera screen.

Point 7 at the camera coordinates -1- of the bending position obtained by image processing FIG. 2 showing the image captured by the CCD camera 2 and point 8 of the control point of the robot at that time are stored. At this time, parameters necessary for image processing are set for the image captured by the CCD camera, and
Perform image training if necessary.

Then, while performing linear interpolation to prevent the robot's wrist from rotating, the robot is moved almost perpendicular to the edge to a position where the bending position of the laser slit light detected earlier does not come out of the field of view of the sensor. Point 9 on the camera coordinates of the bending position in FIG. 4, which shows the image captured by the CCD camera at , and point 10 in FIG. 3, which is the control point of the robot, are stored.

Next, the robot is moved in the depth direction (perpendicular to the previous direction) with respect to the edge using linear interpolation so that the robot's wrist does not rotate, and the bending position of the laser slit light detected earlier is detected by the sensor. 5th position in a position that does not come out of the field of view.
Set point 12 in the diagram.

After setting (teaching) these points, the robot and CCD
When the visual sensor that controls the camera and laser slit projector is activated, the robot moves to point 8, and the robot control point at that time is point 8 (
Point 7 (VXO,
Find VY[)).

Next, the robot moves to point 10 in Figure 3,
Robot coordinates of the robot control point at that time - point 1 of L
-10 (XI, Yl, Zl) and the visual sensor performs image processing to find the detected point 9 (VXl, VYI) on the visual sensor coordinates in FIG.

Next, the robot moves to point 12 in Figure 5,
Point 1 on the robot coordinates of the robot control point at that time
2 (X2.Y2.Z2) and point 11 on the visual sensor coordinates in Figure 6 detected by image processing with the visual sensor.
(VX2.VY2) and register the coordinates of the points found during the series of operations in the file.

If an error occurs during image processing of the visual sensor during this series of operations, the image processing settings are incorrect, so return to the starting reference position, reset the image processing parameters, and retrain the image. .

After the above-mentioned work is completed, point 13 in Fig. 7 detected by the visual sensor is set to (V
X, VY) and VX, VY from point 7 (VXO, VYO) of the image in Figure 262, VX = CIX (αx) + C2X (βX) VY = CIX
Set (αy) + C2X (βy) and find C1 and 02.

However, αX, αy, βX, βy are the vectors Vα (αX, αy) on the visual sensor shown in Fig. 8, ■ β (βX, β
Each element of y) is determined by the following formula.

αx=VX ] −VX o, βx=VX 2
−VX Oαy =VY 1−VY o, βy=
vy2-vy.

During teaching, the robot moves, but during actual work, the workpiece moves, so the moving direction of the image is reversed, so the signs of the obtained CI and C2 are reversed.

This obtained C1. , C2, the robot coordinates −
Find the amount of deviation of F from the reference point (XO, YO, ZO) (=point 8 in FIG. 3).

The amount of deviation is RX=CI X (Rax)+c2X (RBx)RY
=CI X (Ray) 102X (RBx)RZ=
CI X (Rax) 10C2x (Raz) However,
Rax, Ray, Raz, Rflx.

RBx, Raz are vectors on the robot coordinates shown in Figure 9. Ra CRax, Ray, Raz'), Rβ (RB
x, RBx, Raz) is calculated using the formula.

Rαx=X1-XO, Rβx=X2-XORαy=
Y 1-Y O, RB y = Y2-Y 0Raz
=Zl-ZO, Rβz=Z2-ZO Shift the work that has been taught in advance using the determined deviation amount RX in the X-axis direction, deviation amount RY in the Y-axis direction, and deviation amount RZ in the Z-axis direction. Make corrections accordingly.

〔Effect of the invention〕

As described above, according to the present invention, even if the target edge is on an inclined plane, or even if the position of the camera and laser projector is not fixed, it can be easily and accurately processed. The amount of deviation can be determined.

Therefore, it is highly effective in detecting the positional deviation of a workpiece that has many three-dimensional changes, such as the edges of an automobile body.

[Brief explanation of drawings]

Fig. 1 is a diagram showing the reference position of the device and robot used to carry out the present invention, Fig. 2 is a diagram showing the reference position of the bending position of the laser slit light, and Fig. 3 is a diagram showing the position of the robot after movement. Figure 4 is a diagram showing the bending position of the laser slit light after the robot has moved, Figure 5 is a diagram showing the position after the robot has moved, and Figure 6 is a diagram showing the bending position of the laser slit light after the robot has moved. , FIG. 7 is a diagram showing the bending position of the laser slit light during actual work, FIG. 8 is a diagram showing vectors on visual sensor coordinates, and FIG. 9 is a diagram showing vectors on robot coordinates. 1... Robot wrist tip 2... CCD camera 3.
...Laser projector 4...Workpiece 5...Edge 6...Bending point 7.9, 11.13...
Bending position 8.10.12...Robot control point Patent applicant Yasuyo Electric Manufacturing Co., Ltd. Figure 4 Figure 13 Bending position No. 1

Claims (2)

[Claims] (1) A method for detecting a three-dimensional positional shift of an edge using a teaching playback type industrial robot equipped with a CCD camera and a device that emits laser slit light on the end effector of the industrial robot. A laser slit beam is projected almost perpendicularly to the edge without deviation using a laser slit projector attached to an end effector, and at this time, the position of the robot control point on the robot coordinates is memorized, and , the image is captured by a CCD camera attached to the end effector and processed, the image is analyzed by a device, the bending position due to the edge of the laser slit light is detected and memorized on the camera coordinates, and then the industrial robot is moved in two directions. After moving,
At each position, the position of the control point of the industrial robot on the robot coordinates and the position on the camera coordinates of the bending position of the laser slit light obtained by image processing of the image captured by the CCD camera attached to the end effector are determined. Detect and store the positions of the three detected control points on the robot coordinates and the three points on the visual sensor, and align the starting point on the robot coordinates based on the detected points. When detecting edge position deviation by forming vectors in two directions with the point detected at the beginning of the sequence, and forming vectors in two directions on the camera coordinates corresponding to the vectors on the robot coordinates, The coefficients of the vectors that allow the position detected on the camera coordinates to reach the point detected by the vectors in two directions on the camera coordinates formed with no positional deviation are calculated, and the sign of the coefficients is reversed. A visual sensor characterized in that a composite value of vectors obtained by multiplying a vector in two directions on the camera coordinates by a vector on the robot coordinates corresponding to a vector in two directions on the camera coordinates is used as the amount of positional deviation of the workpiece on the robot coordinates. A method for detecting three-dimensional positional deviation of edges in industrial robots with attached robots. (2) Detection of three-dimensional positional deviation of an edge in an industrial robot with a visual sensor according to claim 1, wherein the series of operations is registered as a JOB of the industrial robot and is automatically executed. Method.