JPH0511822A - Cooperative operation system for robot - Google Patents

Abstract

PURPOSE:To easily adjust settings of a robot system in a short time even if the robot or a visual sensor shifts in position. CONSTITUTION:When the robot shifts, the quantity DELTACD of deviation between the robot and visual sensor 3 is found and a correction matrix CMI is found according to the deviation quantity DELTACD, etc. A conversion matrix correcting means 33 corrects a previously set conversion matrix CM1 for calibration with the correction matrix CMI to find a corrected conversion matrix CMC; and work detection point data WD from the visual sensor 3 are converted by this corrected conversion matrix CMC and sent out to the robot side. The work detection point data WD, on the other hand, are converted by a previously set conversion matrix CM2 for calibration and sent out to the other robot side. Namely, the setting adjustments are only made between the robot which shifts and the visual sensor 3 to perform cooperative operation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot collaborative work system that causes a robot system consisting of a plurality of robots and visual sensors to perform a collaborative work, and in particular, there is a deviation in the arrangement of either the robot or the visual sensor. On the cooperative work method of the robot at the time.

[0002]

2. Description of the Related Art A robot system has been put into practical use in which a robot and a visual sensor such as a camera are combined to give the robot a visual function, thereby recognizing the position of the work, and performing assembly work, palletizing work and the like. . In this robot system, the robot and the visual sensor are
The coordinate system is calibrated among the coordinate systems and has a common coordinate system, so that the robot can perform accurate work at the position of the work recognized by the visual sensor. When the robot system includes a plurality of robots and a visual sensor, the coordinate systems are calibrated so that they have a common coordinate system as a whole. This allows a plurality of robots in the robot system to perform a collaborative work on a common work.

[0003]

However, in the conventional robot system, even if such a common coordinate system is set, if even one of them is misaligned, the entire system is recalibrated. Had to do. For example, if one robot fails and is replaced with a new robot, the position of that robot will shift, so setting adjustment (calibration) is required to match the coordinate system between that robot and other robots. Furthermore, the setting adjustment is necessary between the visual sensor and the robot, and the setting adjustment must be performed again over the entire robot system. This is exactly the same even when a position shift or the like occurs on the visual sensor side.

Therefore, in the conventional robot system,
Once the position of the robot or the visual sensor is displaced, it takes a lot of time for the setting and adjusting work, and the maintainability thereof is extremely poor. The present invention has been made in view of the above circumstances, and provides a robot collaborative work method capable of easily performing setting adjustment of a robot system in a short time even if a position of a robot or a visual sensor is displaced. The purpose is to do.

[0005]

According to the present invention, in order to solve the above-mentioned problems, in a robot collaborative work system that causes a robot system consisting of a plurality of robots and visual sensors to perform collaborative work, a set of robot and visual sensor is used. Of these, when a displacement or the like occurs in the placement of the robot, a displacement amount calculating unit that obtains the displacement amount using each imaged data from the visual sensor before and after the displacement,
A correction matrix calculation unit that obtains a correction matrix based on the shift amount and the imaged data from the visual sensor after the shift has occurred, and the correction matrix of each coordinate system of the robot and the visual sensor. A conversion matrix correction means for correcting the calibration conversion matrix and a detection point correction for converting the detection point data on the work object detected by the visual sensor using the correction conversion matrix obtained by the conversion matrix correction means. A conversion unit, a robot control unit that causes the robot to perform a correction operation based on the detection point correction conversion data sent from the detection point correction conversion unit, a robot other than the robot, and the visual sensor. Using the conversion matrix for calibration of each coordinate system of Based on the detection point conversion means for converting the detection point data and the conversion data of the detection points sent from the detection point conversion means, the operation of the other robot is controlled, and the cooperative work with the robot is performed. There is provided another robot control means for performing the robot coordinated work method.

[0006]

When the position of the robot and the like among the set of robot and the visual sensor is deviated, the amount of deviation is obtained by using the visual sensor. The deviation amount is obtained by comparing the imaging data of the calibration jig attached to the robot hand before the deviation occurs and the imaging data of the calibration jig attached to the robot hand after the deviation occurs. To be

The correction matrix calculating means obtains the correction matrix based on the shift amount and the image data from the visual sensor after the shift has occurred. The conversion matrix correction means corrects the calibration conversion matrix of each coordinate system of the set of robot and the visual sensor by the correction matrix to obtain the corrected conversion matrix. The detection point correction conversion means uses this correction conversion matrix to convert the detection point data on the work object detected by the visual sensor, and sends the correction conversion data of the detection point to the robot side. The robot control means on the robot side, based on the correction conversion data of the sent detection points,
Make the robot perform corrective action.

On the other hand, the detection point converting means converts the detection point data by using the calibration conversion matrix of each coordinate system of the robot other than the robot and the visual sensor, and converts the conversion data of the detection point. Send to other robot side. The other robot control means controls the operation of the other robot based on the sent conversion data of the detection points, and causes the robot to cooperate with the robot.

That is, when a displacement or the like occurs in the arrangement of the robot, the robot corrects the operation according to the displacement detected by the visual sensor. On the other hand, the other robots do not undergo any change in their mutual positional relationship with the visual sensor, and the calibration that is preset between the other robot and the visual sensor. The conversion matrix for use is also stored as it is. Therefore, the other robot can operate in the conventional procedure using the calibration conversion matrix. Therefore, even if a deviation occurs on the robot side, the setting adjustment only needs to be performed between the robot and the visual sensor where the deviation has occurred, and is unnecessary for another robot.

Further, when there is a displacement in the arrangement of the robot and the visual sensor on the side of the visual sensor, the amount of displacement of the visual sensor is obtained in the same manner as when the side of the robot is displaced. That is, the image data obtained by capturing the calibration jig at the predetermined position by the visual sensor before the displacement occurs, and the visual sensor after the displacement image the calibration jig at the predetermined position. It is obtained by comparing the obtained imaging data. When the above-mentioned deviation occurs on the robot side, the robot is made to perform the correction operation according to the deviation amount, but when the deviation occurs on the visual sensor side, the deviation is applied to all robots. Therefore, the correction operation according to the deviation amount is performed for all the robots. But even then,
The setting adjustment for obtaining the deviation amount may be performed only between the pair of robots and the visual sensor, and the other robots have their movements automatically corrected using the obtained deviation amount, Can work together with each other. That is, even if a deviation occurs on the visual sensor side, setting adjustment may be performed only between the robot and the visual sensor where the deviation has occurred, and it is not necessary between other robots.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 2 is a diagram showing the overall configuration of a robot system to which the present invention is applied. The robot system includes robots 1 and 2 and a visual sensor 3. The robots 1 and 2 and the visual sensor 3 are calibrated with each other in their respective coordinate systems, and cooperate with each other on a work (for example, a vehicle body) 4 on a line based on the common coordinate system. The visual sensor 3 sends the detection point data of the detection point 5 of the work 4 to the visual sensor control device 30. The visual sensor control device 30 recognizes the accurate position of the work 4 from the detection point data, and sends the position data to the robot control devices 10 and 20. The robot control devices 10 and 20 correct the positions and orientations of the robots 1 and 2 according to the position data, and perform accurate motions with respect to the work point of the work 4. The visual sensor 3 is composed of a camera 3A and a laser sensor (length measuring sensor) 3B, and the two-dimensional position of the detection point 5 is determined by the camera 3A.
Since the position of the detection point 5 in the height direction is detected by B, respectively, the three-dimensional position of the detection point 5 can be recognized by both of them.

The calibration between the coordinate systems of the robots 1 and 2 and the visual sensor 3 is performed by the visual sensor 3 capturing the calibration plates 8 and 9 attached to the robot hands 6 and 7 of each robot. Done by That is, the visual sensor control device 30 is
Calibration of each coordinate system is performed using the imaged data of the calibration plates 8 and 9. As a result of the calibration, the robot 1 and the visual sensor 3
Conversion matrix CM for calibration between
1 and the calibration conversion matrix CM2 between the robot 2 and the visual sensor 3 are stored in the memory (not shown) of the visual sensor control device 30.

It is assumed that the position of the robot 1 is changed from X to XA as a result of replacing the robot 1 with another robot 1A due to a failure of the robot 1. In this case,
Since the coordinate systems of the robot 1A and other robots and visual sensors do not match, it becomes impossible to perform cooperative work. Hereinafter, a procedure for returning the cooperative work in this robot system will be described.

FIG. 1 is a block diagram of a robot cooperative work system according to the present invention. The shift amount calculation means 31 of the visual sensor control device 30 obtains the shift amount of the robot 1 using the image pickup data SO and SN sent from the visual sensor 3. The imaging data SN corresponds to the hand 6 of the robot 1A.
Calibration plate 8A attached to A
(FIG. 2) is imaging data when the visual sensor 3 images. It is assumed that the robot 1A at the time of imaging has the same posture as the robot 1 at the time of the previous calibration. Therefore, the imaging data SO and the imaging data SN
From this, the deviation amount ΔCD of the robot 1 is obtained.

The correction matrix calculating means 32 obtains the correction matrix CMI based on the deviation amount ΔCD and the image pickup data SN. This correction matrix CM
I is a matrix for correcting the calibration conversion matrix CM1 between the robot 1 and the visual sensor 3 according to the deviation amount ΔCD. The conversion matrix correction unit 33 corrects the calibration conversion matrix CM1 by the correction matrix CMI, and the corrected conversion matrix CMC (= [CM1] ×
[CMI]). The detection point correction conversion means 34 uses the correction conversion matrix CMC to detect the visual sensor 3
The detection point data WD of the detection point 5 on the workpiece 4 detected by the robot controller 1 is converted into the corrected conversion data θ1.
Send to 0. The robot controller 10 causes the robot 1A to perform a correction operation based on the correction conversion data θ1 sent.

On the other hand, the detection point converting means 35 is the robot 2
Also, the detection point data WD is converted using the calibration conversion matrix CM2 of each coordinate system of the visual sensor 3, and the conversion data θ2 is sent to the robot controller 20. The robot control device 20 controls the operation of the robot 2 based on the transmitted conversion data θ2, and causes the robot 1A to perform a collaborative work.

That is, when the position of the robot 1 is displaced, the robot 1A is independent of the robot 2 and
It is possible to perform operation correction according to the shift amount ΔCD detected by the visual sensor 3. On the other hand, the robot 2 has no change in its mutual positional relationship with the visual sensor 3, and the calibration conversion matrix preset between the robot 2 and the visual sensor 3 is set. The CM2 is also stored as it is. Therefore, the robot 2 can operate in the conventional procedure using the calibration conversion matrix CM2. Therefore, even if a deviation occurs on the robot 1 side, the setting adjustment only needs to be performed between the robot 1A and the visual sensor 3 where the deviation has occurred, and is not necessary between the robot 2 and the robot.

FIG. 3 is a diagram showing a flow chart for executing the calibration which is the premise of the present invention.
In the figure, the numerical value following S indicates a step number. [S1] The visual sensor 3 images the calibration plate attached to the hand of each robot. [S2] Using the image data from the visual sensor 3, calibration conversion matrices CM1 and CM2 between the coordinate systems of the visual sensor 3 and the robots 1 and 2 are obtained.

The calibration conversion matrices CM1 and CM2 are stored in the memory of the visual sensor control device 30. FIG. 4 is a diagram showing a flowchart for executing the robot cooperative work system of the present invention, and shows a case where the robot 1 is displaced. [S3] First, the image pickup data SN from the visual sensor 3 is stored. [S4] The image pickup data SO and the image pickup data SN of this time are compared to obtain the deviation amount ΔCD. [S5] Based on the imaging data SN and the shift amount ΔCD,
Obtain the correction matrix CMI. [S6] Calibration conversion matrix CM1
Is converted by the correction matrix CMI to obtain a correction conversion matrix CMC. [S7] The work detection point data WD is converted using the correction conversion matrix CMC to obtain the correction conversion data θ1 and sent to the robot controller 10. [S8] The work detection point data WD is converted using the calibration conversion matrix CM2 to obtain converted data θ2, which is sent to the robot controller 20.

As a result, the robot 1A has a deviation amount ΔC.
The motion correction is performed according to D, and the robot 2 can operate in the conventional procedure using the calibration conversion matrix CM2 stored as it is. FIG. 5 is a block diagram showing a second embodiment of the present invention. Similar to the first embodiment, a case where the robot 1 is displaced will be described.

The shift amount calculating means 41 of the visual sensor control device 30 is the image pickup data SO sent from the visual sensor 3.
And SN are used to find the deviation amount ΔCD of the robot 1 in the same manner as in the first embodiment. Correction data calculation means 42
Calculates the correction data θM according to this deviation amount ΔCD. This correction data θM corrects the position and orientation of the robot 1 and causes the robot 1 to operate as if the displacement amount ΔCD is zero. The detection point conversion means 44 is
Using the calibration conversion matrix CM1 between the robot 1 and the visual sensor 3, the detection point data WD is converted to obtain the conversion data θD. The conversion detection point correction means 43 corrects the conversion data θ2 using the correction data θM to obtain corrected conversion data θ1A (= θD + θM) and sends it to the robot controller 10. Robot controller 10
Causes the robot 1A to perform a correction operation based on the correction conversion data θ1A sent.

On the other hand, the detection point converting means 45 is the robot 2
And the conversion matrix CM2 for calibration between the visual sensors 3 is used to convert the detection point data WD to obtain conversion data θ2A, and the conversion data θ2A is sent to the robot controller 20. The robot controller 20 is
Based on the conversion data θ2A sent, the robot 2
To control the operation of the robot 1 and perform a cooperative work with the robot 1A.

FIG. 6 is a diagram showing a flow chart for executing the second embodiment of the present invention. [S11] The image pickup data SN from the visual sensor 3 is stored. [S12] The deviation amount ΔCD is obtained by comparing the imaging data SO with the imaging data SN of this time. [S13] The correction data θM corresponding to the deviation amount ΔCD is obtained. [S14] The work detection point data WD is converted using the calibration conversion matrix CM1 to obtain conversion data θD. [S15] The correction data θM is added to the conversion data θD to obtain the correction conversion data θ1A.
Send to 0. [S16] The work detection point data WD is converted using the calibration conversion matrix CM2 to obtain conversion data θ2, and the conversion data θ2 is obtained and sent to the robot controller 20.

As described above, since the operation of the robot 1A having the deviation is corrected by using the correction data θM according to the deviation amount ΔCD, the robot 1A can be corrected in the same manner as in the first embodiment. Independent of the robot 2, the amount of deviation Δ
It is possible to perform operation correction according to the CD. On the other hand, the robot 2 can operate in a conventional procedure by using the calibration conversion matrix CM set in advance.

FIG. 7 is a block diagram showing a third embodiment of the present invention. In this embodiment, a case where the visual sensor 3 is displaced is shown. In this case, the shift amount of the visual sensor 3 is
It is obtained in the same manner as when the robot side is displaced. That is, it is possible to make each robot take the same position and posture as in the previous calibration, image the calibration plate attached to each robot, and obtain it from the previous and present imaging data.

This embodiment is different from the first embodiment in that correction conversion data θ2B is also sent to the robot 2. The difference from the first embodiment is the conversion matrix correction. Means 36 and detection point correction conversion means 37
That is the point. The conversion matrix correction means 36 is
The correction matrix CMI obtained by the correction matrix calculation means 32 is used to correct the calibration conversion matrix CM2 to obtain a correction conversion matrix CMD (= [CM2] × [CMI]). The detection point correction conversion means 37 uses the correction conversion matrix CMD.
Is used to convert the detection point data WD and send it to the robot controller 20 as corrected conversion data θ2B.

As described above, when a deviation occurs on the visual sensor side, the deviation occurs for all robots, and therefore the correction operation according to the deviation amount ΔCD is performed for all robots. However, even in that case, the setting adjustment for obtaining the deviation amount ΔCD may be performed only between the pair of the robot 1 and the visual sensor 3.
Are automatically corrected in operation using the calculated deviation amount ΔCD, and they can cooperate with each other. That is, even if a shift occurs on the visual sensor side, the setting adjustment only needs to be performed between the pair of robots and the visual sensor, and is unnecessary between other robots.

FIG. 8 is a block diagram showing a fourth embodiment of the present invention. Similar to the third embodiment, a case where the visual sensor 3 is displaced will be described. In this embodiment, correction conversion data θ2C is further sent to the robot 2 as compared with the second embodiment. The difference from the second embodiment is the conversion detection point correction means 46. That is the point. The conversion detection point correction means 46 corrects the conversion data θE of the work detection point data WD converted by the detection point conversion means 45 using the correction data θM, and the corrected conversion data θ2C.
(= ΘE + θM) is calculated and sent to the robot controller 20.

As described above, even if a deviation occurs on the visual sensor side, as in the third embodiment, the correction operation according to the deviation amount ΔCD is performed for all robots. The setting adjustment for correcting the deviation may be performed only between the pair of the robot 1 and the visual sensor 3, and the operation of the robot 2 is automatically corrected by using the calculated deviation amount ΔCD. .

In the above description, the robot system is a combination of one visual sensor and two robots, but a plurality of visual sensors and robots may be provided.

[0031]

As described above, according to the present invention, when a deviation occurs in the robot or the visual sensor, the operation of each robot is performed according to the deviation amount obtained between the pair of robots and the visual sensor. Configured to control. That is, the setting adjustment only needs to be performed between the pair of robots in which the deviation has occurred and the visual sensor, and is not necessary between other robots. Therefore, even if the positions of the robot and the visual sensor are displaced, the setting adjustment of the robot system can be easily performed in a short time. As a result, the maintainability of the robot system is greatly improved.

[Brief description of drawings]

FIG. 1 is a block diagram of a robot cooperative work system according to the present invention.

FIG. 2 is a diagram showing an overall configuration of a robot system to which the present invention is applied.

FIG. 3 is a diagram showing a flowchart for executing calibration, which is a premise of the present invention.

FIG. 4 is a diagram showing a flow chart for executing a robot cooperative work system according to the present invention.

FIG. 5 is a block diagram showing a second embodiment of the present invention.

FIG. 6 shows a flowchart for carrying out the second embodiment of the present invention.

FIG. 7 is a block diagram showing a third embodiment of the present invention.

FIG. 8 is a block diagram showing a fourth embodiment of the present invention.

[Explanation of symbols]

1,1A, 2 robot 3 Visual sensor 4 work 5 detection points 8,8A, 9 Calibration plate 10, 20 Robot controller 30 Visual sensor control device 31, 41 Deviation amount calculation means 32 correction matrix calculation means 33,36 conversion matrix correction means 34, 37 Detection point correction conversion means 35, 44, 45 detection point conversion means 42 Correction data calculation means 43,46 conversion detection point correction means

Claims (9)

[Claims] 1. A robot collaborative work method for causing a robot system including a plurality of robots and visual sensors to perform collaborative work, when a displacement or the like occurs in the robot or the like among a set of robots and visual sensors. A deviation amount calculation means for obtaining a deviation amount by using each imaging data from the visual sensor before and after the deviation, and the deviation amount and the imaging data from the visual sensor after the deviation Correction matrix calculation means for obtaining a correction matrix based on the above, conversion matrix correction means for correcting the calibration conversion matrix of each coordinate system of the set of robot and visual sensor by the correction matrix, and the conversion matrix correction means. Using the correction transformation matrix obtained by the means, A detection point correction conversion means for converting the detection point data on the work object detected by the detection point correction processing, and a correction operation for the robot based on the correction conversion data of the detection point sent from the detection point correction conversion means. From the detection point conversion means, a robot control means for causing the detection point conversion means for converting the detection point data by using a conversion matrix for calibration of coordinate systems of robots other than the robot and the visual sensor. Based on the sent conversion data of the detection point, control the operation of the other robot,
A robot collaborative work method, comprising: another robot control means for performing a collaborative work with the robot. 2. Each image pickup data from the visual sensor is
Imaging data obtained by the visual sensor imaging the calibration jig attached to the robot hand before the deviation occurs, and the calibration data attached to the robot hand after the deviation occurs 2. The robot collaborative work system according to claim 1, wherein the calibration jig is image pickup data obtained by the visual sensor taking an image of the jig. 3. The collaborative work system of a robot according to claim 1, wherein the visual sensor comprises a camera and a length measuring sensor. 4. The robot collaborative work system according to claim 1, wherein the other robots are plural in number. 5. A robot collaborative work method for causing a robot system including a plurality of robots and a visual sensor to perform a collaborative work, when a displacement or the like occurs in the robot or the like among a set of the robot and the visual sensor. A deviation amount calculation means for obtaining a deviation amount using each imaged data from the visual sensor before and after the deviation occurs, and correction data for correcting the position and orientation of the robot according to the deviation amount. A first detection point conversion for converting the detection point data on the work object detected by the visual sensor using the correction data calculation means to be obtained and the calibration conversion matrix of each coordinate system of the robot and the visual sensor. Means and conversion detection for correcting the detection point data converted by the first detection point conversion means using the correction data Correction means, robot control means for causing the robot to perform a correction operation based on the correction conversion data of the detection points sent from the conversion detection point correction means, a robot other than the robot, and the visual sensor Second detection point conversion means for converting the detection point data by using the calibration conversion matrix of each coordinate system, and conversion data of the detection points sent from the second detection point conversion means. Based on the above, another robot control means for controlling the operation of the other robot and performing a collaborative work with the robot is provided. 6. A robot collaborative work method for causing a robot system including a plurality of robots and a visual sensor to perform a collaborative work, when a displacement or the like occurs in the arrangement of the visual sensor among a set of the robot and the visual sensor. In addition, a deviation amount calculating unit that obtains a deviation amount by using each imaging data from the visual sensor before and after the deviation occurs, and the deviation amount and the imaging from the visual sensor after the deviation occur Correction matrix calculation means for obtaining a correction matrix based on the data, and first conversion matrix correction means for correcting the calibration conversion matrix of each coordinate system of the pair of robot and visual sensor by the correction matrix, The first obtained by the conversion matrix correction means
Of the detection points on the work object detected by the visual sensor, and the first detection points sent from the detection point correction conversion means. Robot control means for causing the robot to perform a correction operation based on correction conversion data; and a correction matrix for correcting a calibration conversion matrix of each coordinate system of a robot other than the previous robot and the visual sensor. Second conversion matrix correction means, and a second conversion matrix correction means obtained by the conversion matrix correction means.
Second detection point correction conversion means for converting the detection point data by using the correction conversion matrix of and the second correction conversion data of the detection points sent from the second detection point correction conversion means. Based on the above, there is provided another robot control means for controlling the correction operation of the other robot and performing a collaborative work with the robot. 7. Each image pickup data from the visual sensor comprises:
Imaging data obtained by imaging the calibration jig at a predetermined position by the visual sensor before the displacement occurs,
7. The robot collaborative work method according to claim 6, wherein the visual sensor after the deviation is imaged data obtained by imaging the calibration jig at the predetermined position. 8. The robot collaborative work system according to claim 7, wherein the calibration jig at the predetermined position is a calibration jig attached to a hand of the robot. 9. In a robot collaborative work method for causing a robot system including a plurality of robots and a visual sensor to perform a collaborative work, when a displacement or the like occurs in the arrangement of the visual sensor among a set of the robot and the visual sensor. In addition, a deviation amount calculation unit that obtains a deviation amount using each imaged data from the visual sensor before and after the deviation, and correction data that corrects the position and orientation of the robot according to the deviation amount. A first detection point for converting the detection point data on the work object detected by the visual sensor by using the correction data calculation unit for obtaining the coordinate value and the conversion matrix for calibration of each coordinate system of the robot and the visual sensor. Conversion means, and a first detection point conversion means for correcting the detection point data converted by the first detection point conversion means using the correction data. Replacement detection point correction means, robot control means for causing the robot to perform a correction operation based on the correction conversion data of the detection points sent from the first conversion detection point correction means, and a robot other than the robot And second detection point conversion means for converting the detection point data using a conversion matrix for calibration of each coordinate system of the visual sensor, and the detection point data converted by the second detection point conversion means. Based on the correction conversion data of the detection point sent from the second conversion detection point correction means, and a correction of the other robot. Another robot control means for controlling an operation and performing a collaborative work with the robot, a collaborative work system for a robot.