WO2020121396A1

WO2020121396A1 - Robot calibration system and robot calibration method

Info

Publication number: WO2020121396A1
Application number: PCT/JP2018/045423
Authority: WO
Inventors: 信夫大石; 岡本　実幸; 正樹櫻井
Original assignee: 株式会社Ｆｕｊｉ
Priority date: 2018-12-11
Filing date: 2018-12-11
Publication date: 2020-06-18
Also published as: JPWO2020121396A1; JP7153085B2

Abstract

This invention is provided with: a hand camera (40) attached to an arm of a robot (11); a fixed camera (51) that is fixed facing downward at a site higher than the area in which the robot arm can move; a coordinate conversion parameter calculation unit (45) for calculating coordinate conversion parameters for converting coordinate values in the vision coordinate system of the fixed camera to coordinate values in a global coordinate system, which is a coordinate system of the robot; and a calibration tool (55) disposed at a prescribed position. The coordinate conversion parameters include an origin correction parameter for correcting the origin of the vision coordinate system of the fixed camera and a coordinate axis tilt correction parameter for correcting the tilt of the coordinate axes of the vision coordinate system of the fixed camera. The origin correction parameter and the coordinate axis tilt correction parameter are calculated on the basis of the relationship between information relating to the position and tilt angle of the calibration tool in the vision coordinate system of the hand camera and information relating to the position and tilt angle of the calibration tool in the vision coordinate system of the fixed camera.

Description

Robot calibration system and robot calibration method

This specification calibrates the correspondence between the coordinate system of a robot that works by recognizing the position of the work by imaging the work supplied to the work area from above with a fixed camera and the coordinate system of the fixed camera. The technology regarding the robot calibration system and the robot calibration method is described.

In recent years, as described in Japanese Patent Laid-Open No. 2013-215866, a fixed camera is fixed downward above an arm movable area of a robot, and a work supplied to the arm movable area is fixed by the fixed camera. 2. Description of the Related Art There is a robot control system that controls an operation of an arm of a robot while capturing an image and recognizing the position of the work, and includes a calibrator that calibrates a relationship between a coordinate system of a fixed camera and a coordinate system of the robot. According to the calibration method of Patent Document 1, a mark is attached to an arm of a robot, the mark attached to the arm of the robot is housed in the visual field of the fixed camera, an image is taken, and the position of the mark is set in a coordinate system of the fixed camera. By recognizing with, the correspondence between the coordinate system of the fixed camera and the coordinate system of the robot is calibrated.

JP, 2013-215866, A

By the way, in order to improve the accuracy of calibration, it is necessary to make the directions of the coordinate axes (X and Y axes) of the coordinate system of the fixed camera exactly match the directions of the coordinate axes (X and Y axes) of the robot coordinate system. is there. However, in reality, due to the mounting error when mounting the fixed camera on the fixed structure above the robot arm movable area (such as the ceiling of the robot protection fence), the direction of the coordinate axis of the fixed camera coordinate system and the robot coordinate system It is inevitable that some deviation from the direction of the coordinate axes will occur.

In the calibration method of the above-mentioned Patent Document 1, a mark is attached to the robot arm, the mark attached to the robot arm is housed within the field of view of the fixed camera, the image is taken, and the position of the mark is set to the coordinate system of the fixed camera. By recognizing with, it is possible to measure the deviation amount and deviation direction of the origin of the fixed camera coordinate system with respect to the origin of the robot coordinate system, so it is possible to correct the deviation of the origin of the fixed camera coordinate system with respect to the origin of the robot coordinate system. .. However, the tilt angle of the coordinate axis of the coordinate system of the fixed camera with respect to the coordinate axis of the coordinate system of the robot cannot be measured only by imaging the mark attached to the arm of the robot with the fixed camera. Therefore, it is impossible to correct the inclination of the coordinate axis of the coordinate system of the fixed camera caused by an attachment error or the like, and the inclination of the coordinate axis deteriorates the accuracy of calibration.

In order to solve the above problems, a robot that performs a predetermined work on a work supplied to a work area, a hand camera attached to an arm of the robot, and a downward facing position higher than an arm movable area of the robot. A fixed camera that is fixed to the above and that images the work supplied from above to the work area from above, and a coordinate system that processes the image captured by the hand camera and sets the position of the imaging target to the origin of the reference point of the image (hereinafter It is recognized by the coordinate values of the "hand camera's vision coordinate system", the image captured by the fixed camera is processed, and the position of the image capturing target is set as the origin of the reference point of the image (hereinafter referred to as "the fixed camera An image processing unit that recognizes the coordinate values of the "vision coordinate system") and converts the coordinate values of the fixed camera's vision coordinate system to the coordinate values of the world coordinate system, which is the coordinate system that controls the operation of the robot arm. A coordinate conversion parameter calculation unit that calculates a coordinate conversion parameter for: a coordinate conversion unit that uses the coordinate conversion parameters to convert the coordinate values of the vision coordinate system of the fixed camera into the coordinate values of the world coordinate system; A control unit for controlling the position of the arm with coordinate values of the world coordinate system, and a calibration jig arranged at a predetermined position within the visual field of the fixed camera, wherein the coordinate conversion parameter is the fixed camera. Of the vision coordinate system and a coordinate axis tilt correction parameter that corrects the tilt of the coordinate axis of the vision coordinate system of the fixed camera, and the coordinate conversion parameter calculation unit calculates the coordinate conversion parameter. At this time, the control unit controls the operation of the arm of the robot to move the hand camera so that the calibration jig fits within the visual field of the hand camera, and the image processing unit causes the hand camera to move. The image taken by the calibration jig is processed to recognize the information about the position and the tilt angle of the calibration jig in the vision coordinate system of the hand camera, and the fixed camera takes an image of the calibration jig. The image of the hand camera recognized by the image processing unit by processing the processed image and recognizing information about the position and the tilt angle of the calibration jig in the vision coordinate system of the fixed camera. Information about the position and tilt angle of the calibration jig in the coordinate system and the position of the calibration jig in the fixed camera vision coordinate system. The origin correction parameter and the coordinate axis tilt correction parameter are calculated based on the relationship between the position and the tilt angle information.

In this way, the calibration jig arranged at the predetermined position is imaged by both the fixed camera and the hand camera, and the information regarding the position and tilt angle of the calibration jig is recognized in each vision coordinate system by image processing. By doing so, the origin correction parameter can be calculated based on the positional relationship between the calibration jigs in both coordinate systems, and the coordinate axis tilt correction parameter can be calculated based on the relationship between the tilt angles of the calibration jigs in both coordinate systems. be able to. As a result, even if the coordinate axis of the vision coordinate system of the fixed camera is inclined with respect to the coordinate axis of the world coordinate system that is the robot coordinate system, the inclination of the coordinate axis of the vision coordinate system and the deviation of the origin are used as the coordinate axis inclination correction parameter. The coordinate value of the fixed camera's vision coordinate system can be accurately converted to the coordinate value of the world coordinate system by accurately correcting with the origin correction parameter.

FIG. 1 is a front view showing the appearance of the robot control system. FIG. 2 is a block diagram showing the electrical configuration of the robot control system. FIG. 3 is a plan view of the calibration jig. FIG. 4 is a flow chart showing the flow of processing in the first half of the calibration execution program. FIG. 5 is a flowchart showing the flow of processing in the latter half of the calibration execution program.

An embodiment disclosed in the present specification will be described below with reference to the drawings.
First, the configuration of the robot 11 will be described with reference to FIG.

The robot 11 is, for example, a 5-axis vertical multi-joint robot, and includes a fixed base 13 installed on the factory floor 12 and a fixed base 13 rotatably provided on the fixed base 13 about a first joint shaft 14 (J1). One arm 15, a second arm 17 rotatably provided at the tip of the first arm 15 by a second joint shaft 16 (J2), and a third joint shaft 18 (J3) at the tip of the second arm 17. The third arm 19 rotatably provided by the third arm 19, the wrist portion 21 rotatably provided by the fourth joint shaft 20 (J4) at the tip of the third arm 19, and the fifth joint shaft 21 by the wrist portion 21. 22 (J5) and an end effector 23 which is attached so as to be rotatable and replaceable. As a result, the end effector 23 attached to the wrist portion 21 is configured to rotate by the fourth joint shaft 20 which is the joint shaft of the wrist portion 21.

In this case, the end effector 23 may be, for example, a suction nozzle, a hand, a gripper, a welding machine, or the like. The first to

fifth joint shafts

14, 16, 18, 20, 22 of the robot 11 are driven by servomotors 25 to 29 (see FIG. 2), respectively. As shown in FIG. 2, each of the servo motors 25 to 29 is provided with an encoder 31 to 35 for detecting a rotation angle, and information on the rotation angle detected by each of the encoders 31 to 35 is transmitted via a servo amplifier 36. It is fed back to the control unit 37. As a result, the controller 37 feeds back the servo motors 25 to 29 via the servo amplifier 36 so that the rotation angles of the servo motors 25 to 29 detected by the encoders 31 to 35 match the respective target rotation angles. By controlling, the positions of the

arms

15, 17, 19 of the robot 11, the wrist part 21, and the end effector 23 are feedback-controlled to the respective target positions.

In the configuration example of FIG. 2, the servo amplifier 36 is a multi-axis amplifier that feedback-controls a plurality of servo motors 25 to 29, but the servo motors 25 to 29 are feedback-controlled by separate servo amplifiers one by one. Is also good.

A work (not shown) as a work target is supplied to a work area 38 at a constant height at a predetermined position of an arm movable region of the robot 11 (a region where the end effector 23 on the tip side of the wrist 21 can move). A supply device 39 is installed. The supply device 39 may be configured by a conveyor, or a parts feeder having any structure such as a vibration type parts feeder may be used. In short, the height position of the work area 38 is known. It may be a constant height position.

As shown in FIG. 1, a wrist camera 21, which is the arm tip of the robot 11, has a hand camera 40 for picking up an image of a work supplied to the work area 38 or a calibration jig 55 arranged at a predetermined position from above. It is installed. The hand camera 40 is a two-dimensional camera that captures a two-dimensional image, the optical axis of which is parallel to the fifth joint axis 22, and the lens 41 of the hand camera 40 has a tip end side of the fifth joint axis 22 ( It is fixed so that it is located on the arm tip side).

Further, a fixed camera 51 is fixed downward at a predetermined position of a fixed structure 50 (for example, the ceiling of a robot protection fence) installed at a position higher than the arm movable area of the robot 11, and its lens 52 side is directed downward. Has been. The fixed camera 51 is used as a two-dimensional camera for taking an image of the work supplied to the work area 38 or the calibration jig 55 arranged at a predetermined position from above.

As shown in FIG. 3, the calibration jig 55 is formed of a resin plate, a glass plate, a metal plate, or the like into a quadrangular plate shape, and a small circular calibration mark 56 is formed on the upper surface of the calibration jig 55 at a predetermined pitch in a matrix shape (lattice pattern). It is formed in a dot shape.

As will be described later, when the coordinate conversion parameter for converting the coordinate value of the vision coordinate system of the fixed camera 51 into the coordinate value of the world coordinate system, which is the coordinate system for controlling the arm position of the robot 11, is predetermined. The calibration jig 55 arranged at the position is sequentially imaged by both the fixed camera 51 and the hand camera 40. However, when imaging the work supplied to the work area 38, the required resolution and the robot 11 are required. The hand camera 40 and the fixed camera 51 are selectively used depending on the position of the work area 38 in the arm movable area. Here, the visual field of the fixed camera 51 and the visual field of the hand camera 40 are narrower than the arm movable region of the robot 11. Comparing the hand camera 40 and the fixed camera 51, the fixed camera 51 has a wider field of view, but the hand camera 40 has a finer resolution (actual length of the imaging target per pixel), and a high-precision image. It can be processed. Therefore, for example, when the size of the work on the work area 38 is small and a fine resolution is required, the hand camera 40 is used, and the work area 38 installed outside the field of view of the fixed camera 51 is used. For, the hand camera 40 may be used. Further, when a plurality of work areas 38 are contained within the field of view of the fixed camera 51, if the plurality of work areas 38 are housed within the field of view of the fixed camera 51 and imaged, image processing of the work on the plurality of work areas 38 is performed. Can be done efficiently. Alternatively, when there are a plurality of work areas 38, the hand camera 40 may be moved for each work area 38 to image the work.

The robot control unit 42 that controls the operation of the robot 11 configured as described above, as shown in FIG. 2, has an image processing unit 43, a coordinate conversion parameter calculation unit 45, a coordinate conversion unit 44, a control unit 37, and a servo amplifier 36. And so on. The image processing unit 43 processes the two-dimensional image captured by the hand camera 40 or the fixed camera 51 to determine the position of the work on the work area 38 or the position of the specific calibration mark 56 of the calibration jig 55. It is recognized by the coordinate value of a two-dimensional orthogonal coordinate system (hereinafter referred to as "vision coordinate system") having the reference point (for example, the center of the image) of the image as the origin. The coordinate axes of this vision coordinate system are the Xv axis and the Yv axis that are orthogonal to each other on the image. When the hand camera 40 takes an image of the work on the work area 38 or the calibration jig 55, the optical axis of the hand camera 40 is directed vertically downward, so the image taken by the hand camera 40 is a fixed camera. Similar to the image captured by 51, the image is obtained by capturing a horizontal plane, and the Xv axis and the Yv axis of the vision coordinate system are orthogonal coordinate axes on the horizontal plane. The image processing unit 43 recognizes the position of the work or the position of the specific calibration mark 56 of the calibration jig 55 by the image processing by the coordinate value of each pixel in the vision coordinate system.

On the other hand, while the robot 11 is in operation, the coordinate conversion unit 44 converts the coordinate values of the two-dimensional vision coordinate system recognized as the position of the work by the image processing of the image processing unit 43 into the

arms

15, 17, It is converted into coordinate values of the world coordinate system for controlling the positions of 19, the wrist 21, and the end effector 23. This world coordinate system is a three-dimensional orthogonal coordinate system with the reference point as the origin, and the coordinate axes of the world coordinate system are the orthogonal coordinate axes (X axis and Y axis) on the horizontal plane and the vertically upward coordinate axis (Z axis). is there. The unit of the coordinate value of this world coordinate system is the unit of length (for example, μm unit). The origin (reference point) of this world coordinate system is, for example, the center of the arm movable region of the robot 11 (the region where the end effector 23 on the tip side of the wrist 21 can move).

In this case, when the coordinate conversion unit 44 converts the coordinate values of the fixed camera 51 in the vision coordinate system into the coordinate values of the world coordinate system during the operation of the robot 11, the coordinate conversion calculated by the coordinate conversion parameter calculation unit 45. Using the parameters, the coordinate values of the fixed camera 51 in the vision coordinate system are converted into coordinate values in the world coordinate system. Here, the coordinate conversion parameter includes an origin correction parameter that corrects the origin of the vision coordinate system of the fixed camera 51 and a coordinate axis tilt correction parameter that corrects the tilt of the coordinate axis of the fixed camera 51 in the vision coordinate system. A method of calculating the origin correction parameter and the coordinate axis tilt correction parameter of the fixed camera 51 will be described later.

On the other hand, when converting the coordinate value of the vision coordinate system of the hand camera 40 to the coordinate value of the world coordinate system, the inclination angle Rz and the origin of the coordinate axis of the vision coordinate system of the hand camera 40 with respect to the coordinate axis of the world coordinate system are corrected. Thus, the coordinate value of the hand camera 40 in the vision coordinate system is converted into the coordinate value of the world coordinate system. This correction is performed by obtaining the rotation matrix indicating the inclination angle Rz of each coordinate axis and the translation vector to the position (X1w, Y1w) of the vision coordinate origin on the world coordinate system, and thus the coordinates of the vision coordinate system of the hand camera 40. The value (Xav, Yav) can be corrected by the following equation to be converted into the coordinate value (Xw', Yw') in the world coordinate system.

In order to perform coordinate conversion using the above formula, it is necessary to measure the tilt angle Rz of the coordinate axis of the vision coordinate system of the hand camera 40 and the coordinate value of the origin.

Since the hand camera 40 is fixed to the wrist 21 of the robot 11, the relative positional relationship between the arm position of the robot 11 and the hand camera 40 is always maintained constant. Therefore, the tilt angle Rz of the coordinate axis of the vision coordinate system of the hand camera 40 with respect to the XY coordinate axes of the world coordinate system is calculated by the following equation.

Rz=[installation angle of the hand camera 40]+[rotation angle of the first joint shaft 14]
Here, the installation angle of the hand camera 40 is the installation angle of the hand camera 40 with respect to the wrist 21 of the robot 11, and when the hand camera 40 is installed on the wrist 21 of the robot 11, the installation of the hand camera 40 with respect to the wrist 21. The installation angle is measured and the value stored in the non-volatile memory (not shown) of the robot control unit 42 is referred to.

Regarding the method of detecting the rotation angle of the first joint shaft 14, the output of the encoder 31 of the servo motor 25 of the first joint shaft 14 when the arm position of the robot 11 (position of the hand camera 40) is stopped at the imaging position The control unit 37 of the robot control unit 42 acquires the value in real time to calculate the rotation angle of the first joint shaft 14.

On the other hand, the coordinate value of the origin of the vision coordinate system of the hand camera 40 is handed to the robot 11 so that the control unit 37 of the robot control unit 42 matches the coordinate value of the world coordinate system designated as the imaging position of the hand camera 40. The camera 40 is left in a calibrated state when assembled. Since the relative positional relationship between the arm position of the robot 11 and the hand camera 40 is always constant, the distance between the reference point of the robot 11 and the center of the optical axis of the hand camera 40 is measured, and this is used as a correction value from the controller 37. When the robot 11 is instructed to the position (X1w , Y1w ) on the world coordinate system by reflecting it on the coordinate command value, the origin of the vision coordinate system of the hand camera 40 moves to the position (X1w , Y1w ) on the world coordinate system. To be controlled.

The control unit 37 of the robot control unit 42 uses the rotation angles of the servo motors 25 to 29 detected by the encoders 31 to 35 and the relative positional relationship between the hand camera 40 and the wrist unit 21 (end effector 23) in real time. Then, the coordinate value (X1w, Y1w) of the origin of the vision coordinate system of the hand camera 40 is calculated.

The control unit 37 of the robot control unit 42 recognizes each

arm

15, 17, 19 of the robot 11 based on the position of the work converted into the coordinate value of the three-dimensional world coordinate system by the coordinate conversion unit 44 during the operation of the robot 11. The target positions of the wrist 21 and the end effector 23 are set by the coordinate values of the world coordinate system, and the positions of the

arms

15, 17, 19 and the wrist 21 and the end effector 23 are controlled by the coordinate values of the world coordinate system.

Next, the calculation processing of the origin correction parameter, the coordinate axis tilt correction parameter, and the resolution used when converting the coordinate value of the fixed camera 51 in the vision coordinate system into the coordinate value of the world coordinate system will be described. The calculation processing of the origin correction parameter, the coordinate axis tilt correction parameter, and the resolution is automatically executed by the robot control unit 42 according to the calibration execution program of FIGS. 4 and 5 when the installation work of the robot 11 is completed.

When this program is started, first, in step 101, each

arm

15, 17, 19 of the robot 11 is operated to a position where the hand camera 40 is out of the field of view of the fixed camera 51, and the hand camera 40 is retracted. After that, the process proceeds to step 102, and guidance and/or voice guidance for instructing the operator to set the calibration jig 55 at a predetermined position in the work area 38 located at the center of the visual field of the fixed camera 51 is displayed and/or voiced. Notify the worker. By this guidance, the worker performs the work of setting the calibration jig 55 at a predetermined position in the work area 38.

At the next step 103, the robot control unit 42 waits until the worker completes the work of setting the calibration jig 55 at a predetermined position in the work area 38. The determination of the completion of the setting work of the calibration jig 55 may be automatically detected by a sensor or the like, or when the setting work is completed, the operator manually inputs the information of the setting work completion to the robot control unit 42. You may input it.

After that, when the setting operation of the calibration jig 55 is completed, the process proceeds to step 104, the fixed camera 51 captures an image of the calibration jig 55, and the next step 105 processes the image captured by the fixed camera 51. Then, the position of the central calibration mark 56 (A), which is one specific point of the calibration jig 55, is recognized by the coordinate value of the vision coordinate system of the fixed camera 51, and the two specific points 2 The position of the calibration mark 56 (B, C) at the corner portion of the location is recognized by the coordinate value of the fixed camera 51 in the vision coordinate system. Further, the tilt angle of the straight line connecting the calibration marks 56 (B, C) at the two corners of the fixed camera 51 in the vision coordinate system is calculated. Further, the positions of the two calibration marks 56 (A, D) near the center of the calibration jig 55 are recognized by the coordinate values of the vision coordinate system of the fixed camera 51, and the calibration marks 56 (A , D), the actual length dimension between the positions is divided by the number of pixels between the positions on the image to calculate the resolution of the fixed camera 51.

After that, the process proceeds to step 106 in FIG. 5, and the arm position of the robot 11 is controlled so that the calibration jig 55 fits within the visual field of the hand camera 40, and the hand camera 40 is moved to one position of the calibration jig 55. It is moved to a predetermined position above the central calibration mark 56(A) which is a specific point. After that, the process proceeds to step 107, the hand jig 40 takes an image of the calibration jig 55, and in the next step 108, the image picked up by the hand camera 40 is processed to obtain a calibration mark at the center of the calibration jig 55. The position of 56 (A) is recognized by the coordinate value of the vision coordinate system of the hand camera 40.

After that, the process proceeds to step 109, where the hand camera 40 is moved above the calibration mark 56 (B) at one corner of the calibration jig 55, and then the process proceeds to step 110 where the calibration of the hand camera 40 is corrected. The tool 55 is imaged, and in the next step 111, the image captured by the hand camera 40 is processed so that the position of the calibration mark 56 (B) at one corner of the calibration jig 55 is determined by the vision of the hand camera 40. Recognize with the coordinate values of the coordinate system.

After that, the process proceeds to step 112, where the hand camera 40 is moved above the calibration mark 56(C) at the other corner portion of the calibration jig 55, and then the process proceeds to step 113, where the calibration of the hand camera 40 is corrected. The tool 55 is imaged, and in the next step 114, the image captured by the hand camera 40 is processed so that the position of the calibration mark 56 (C) at the other corner portion of the calibration jig 55 is determined by the vision of the hand camera 40. Recognize with the coordinate values of the coordinate system.

After that, the process proceeds to step 115, and the origin correction parameter and the coordinate axis tilt correction parameter for converting the coordinate value of the fixed camera 51 in the vision coordinate system into the coordinate value in the world coordinate system are calculated as follows.

When calculating the origin correction parameter, the coordinate value of the position of the calibration mark 56 (A) at the center of the calibration jig 55, which is recognized by processing the captured image of the fixed camera 51, and the captured image of the hand camera 40. The difference value (ΔX, ΔY) from the coordinate value of the position of the calibration mark 56(A) at the center of the calibration jig 55 which is recognized by processing is calculated as the origin correction parameter.

When calculating the coordinate axis tilt correction parameter, the tilt angle of a straight line connecting the calibration marks 56 (B, C) at the two corners of the calibration jig 55 recognized by processing the image captured by the hand camera 40. And the inclination angle of the straight line connecting the calibration marks 56 (B, C) at the two corners in the vision coordinate system of the fixed camera 51 calculated in step 105 and the vision coordinate system of the hand camera 40. The difference value from the tilt angle of the straight line connecting the calibration marks 56 (B, C) at the two corners of is calculated as the coordinate axis tilt correction parameter.

By the way, in the vision coordinate system of the fixed camera 51, the coordinate value of the recognition target is recognized in pixel units, whereas the unit of the coordinate value in the world coordinate system is a unit of length (for example, a unit of [μm]). is there. For this reason, it is necessary to convert the coordinate value of the fixed camera 51 in the pixel unit of the vision coordinate system into the coordinate value of the unit of the same length as the coordinate value of the world coordinate system (for example, the unit of [μm]).

Therefore, by multiplying the resolution [μm/pixel] of the fixed camera 51 calculated in the step 105 by the coordinate value of the fixed camera 51 in the vision coordinate system in pixel units, the coordinates of the fixed camera 51 in the vision coordinate system in pixel units are multiplied. The value is converted into a coordinate value of the same length unit as the coordinate value of the world coordinate system (for example, a unit of [μm]).

After the calibration execution program shown in FIGS. 4 and 5, the calibration work is completed when the operator takes out the calibration jig 55 set at a predetermined position in the work area 38.

In the calibration execution programs of FIGS. 4 and 5, the calibration mark 56(A) at the center of the calibration jig 55 is set as one specific point for origin correction, but one position other than this is set. The calibration mark 56 may be used as one specific point for origin correction, and the point is that the information about the position of the calibration jig 55 can be acquired by image processing.

Further, in the calibration execution programs of FIGS. 4 and 5, the calibration marks 56 (B, C) at the two corners of the calibration jig 55 are set as the two specific points for correcting the coordinate axis inclination. Two calibration marks 56 at other positions may be used as the two specific points for correcting the coordinate axis inclination. In short, the inclination angle of the straight line connecting the two specific points is the inclination angle of the calibration jig 55. It may be possible to obtain it as information regarding. Alternatively, a straight line may be formed on the upper surface of the calibration jig 55 and the inclination angle of this straight line may be measured by image processing. Alternatively, the tilt angle of the calibration jig 55 may be measured by recognizing one edge of the calibration jig 55 by image processing. The point is that the angle information about the tilt angle of the calibration jig 55 may be acquired by image processing.

In addition, in the calibration execution program of FIGS. 4 and 5, after the fixed camera 51 performs the image capturing/image processing (steps 104 to 105), the hand camera 40 performs the image capturing/image processing (steps 106 to 114). Since the coordinate values of the calibration marks (A), (B), and (C) that move in

steps

106, 109, and 112 in the world coordinate system have already been recognized in the image processing of the fixed camera in step 105, since they have been performed. The coordinate value of this world coordinate system is automatically instructed by the control unit 37. That is, it is considered that the operator does not need to re-indicate the coordinate values of the calibration marks (A), (B), and (C) in the world coordinate system depending on the arrangement position of the calibration jig 55.

The above-described calibration execution programs of FIGS. 4 and 5 are executed by the robot control unit 42 when the installation work of the robot 11 is completed. However, when the frequency of occurrence of a work error of the robot 11 increases, the calibration is performed again. When it is determined that is necessary, the robot control unit 42 re-executes the calibration execution program of FIGS. 4 and 5 to re-measure each correction parameter, or to inspect the robot 11. The robot control unit 42 is operated by the robot control unit 42 as shown in FIG. The calibration execution program of FIG. 5 may be re-executed to re-measure each correction parameter.

In this case, each time the correction parameters are remeasured, the correction parameters may be updated with the measured values, or the correction parameters measured at the completion of the installation work of the robot 11 may be updated by the robot control unit 42. Stored in the non-volatile memory, and at the time of re-measurement of each correction parameter, each re-measured correction parameter is compared with the stored value of the non-volatile memory, and when the difference between the two exceeds a predetermined threshold value. Alternatively, each correction parameter may be updated to the remeasured value.

In any case, for example, the relative position/angle relationship between the fixed camera 51 and the robot 11 changes with time, the installation angle of the fixed camera 51 changes with time, the fixed camera 51 and the work area 38, and the like. When the relative position/angle relationship with (set position of the calibration jig) changes over time, and calibration is required again, each correction parameter is remeasured and each correction parameter is remeasured. Can be corrected to an appropriate value.

According to the present embodiment described above, the calibration jig 55 arranged at a predetermined position is imaged by both the fixed camera 51 and the hand camera 40, and the calibration jig 55 is image-processed in each vision coordinate system. Since the information about the position and the tilt angle of the coordinate system is recognized, the origin correction parameter can be calculated based on the positional relationship between the calibration jigs 55 of both coordinate systems, and the tilt angle of the calibration jig 55 of both coordinate systems can be calculated. The coordinate axis tilt correction parameter can be calculated based on the relationship Accordingly, even if the coordinate axis of the vision coordinate system of the fixed camera 51 is inclined with respect to the coordinate axis of the world coordinate system which is the coordinate system of the robot 11, the inclination of the coordinate axis of the vision coordinate system and the deviation of the origin are corrected for the coordinate axis inclination correction. The coordinate values of the vision coordinate system of the fixed camera 51 can be accurately converted into the coordinate values of the world coordinate system by accurately correcting the parameters and the origin correction parameter.

In this embodiment, the calibration marks 56 arranged on the upper surface of the calibration jig 55 have a circular shape, but may have another shape such as a quadrangle.

Needless to say, the present invention can be implemented by variously changing the configuration of the robot 11 as appropriate without departing from the scope of the invention.

11... Robot, 14... 1st joint axis, 15... 1st arm, 16... 2nd joint axis, 17... 2nd arm, 18... 3rd joint axis, 19... 3rd arm, 20... 4th joint axis, 21... wrist (arm tip), 22... fifth joint axis, 23... end effector, 25-29... servo motor, 31-35... encoder, 36... servo amplifier, 37... control section, 38... work area, 39... Supply device, 40... Hand camera, 41... Lens, 42... Robot control unit, 43... Image processing unit, 44... Coordinate conversion unit, 45... Coordinate conversion parameter calculation unit, 50... Fixed structure, 51... Fixed camera , 52... Lens, 55... Calibration jig, 56... Calibration mark

Claims

A robot that performs a predetermined work on the work supplied to the work area,
A hand camera attached to the robot arm,
A fixed camera that is fixed downward at a position higher than the arm movable region of the robot and that images the work supplied to the work area from above;
The image captured by the hand camera is processed to recognize the position of the image capturing target with the coordinate value of the coordinate system (hereinafter referred to as "hand camera vision coordinate system") having the reference point of the image as the origin, and the fixed camera An image processing unit that processes the captured image and recognizes the position of the imaging target with coordinate values of a coordinate system whose origin is the reference point of the image (hereinafter referred to as "fixed camera vision coordinate system").
A coordinate conversion parameter calculation unit that calculates coordinate conversion parameters for converting the coordinate values of the vision coordinate system of the fixed camera into the coordinate values of the world coordinate system that is a coordinate system that controls the operation of the arm of the robot;
A coordinate conversion unit that converts the coordinate value of the vision coordinate system of the fixed camera into the coordinate value of the world coordinate system using the coordinate conversion parameter;
A control unit that controls the position of the robot arm with coordinate values of the world coordinate system;
A calibration jig disposed at a predetermined position within the visual field of the fixed camera,
The coordinate conversion parameters include an origin correction parameter that corrects the origin of the fixed camera vision coordinate system, and a coordinate axis tilt correction parameter that corrects the tilt of the coordinate axis of the fixed camera vision coordinate system,
When the coordinate conversion parameter calculation unit calculates the coordinate conversion parameter, the control unit controls the operation of the arm of the robot so that the calibration jig fits within the visual field of the hand camera. The camera is moved, and the image processing unit processes the image of the calibration jig captured by the hand camera to recognize information about the position and tilt angle of the calibration jig in the vision coordinate system of the hand camera. At the same time, the image of the calibration jig captured by the fixed camera is processed to recognize information about the position and tilt angle of the calibration jig in the vision coordinate system of the fixed camera,
The coordinate conversion parameter calculation unit includes information regarding the position and tilt angle of the calibration jig in the vision coordinate system of the hand camera recognized by the image processing unit, and the calibration correction in the vision coordinate system of the fixed camera. A robot calibration system for calculating the origin correction parameter and the coordinate axis tilt correction parameter based on a relationship with the position and tilt angle information of the tool.
When the coordinate conversion parameter calculation unit calculates the origin correction parameter, the control unit controls the operation of the arm of the robot so that the calibration jig fits within the visual field of the hand camera. The camera is moved, and the image processing unit processes the image of the calibration jig captured by the hand camera to determine the position of one specific point of the calibration jig as information regarding the position of the calibration jig. As the coordinate value of the vision coordinate system of the hand camera, and the image of the calibration jig taken by the fixed camera is processed to determine the position of the one specific point of the calibration jig. Recognized by the coordinate values of the vision coordinate system of the fixed camera as information about the position of the jig,
The coordinate conversion parameter calculation unit calculates the coordinate value of the position of the one specific point in the vision coordinate system of the hand camera recognized by the image processing unit and the one specific point in the vision coordinate system of the fixed camera. The robot calibration system according to claim 1, wherein the origin correction parameter is calculated based on a positional relationship with a coordinate value of a position.
When the coordinate conversion parameter calculation unit calculates the coordinate axis tilt correction parameter, the control unit controls the operation of the robot arm so that the calibration jig fits within the visual field of the hand camera. The hand camera is moved, and the image processing unit processes an image obtained by capturing the calibration jig by the hand camera to determine the positions of two specific points of the calibration jig as the vision coordinate system of the hand camera. The coordinate values of the fixed camera are processed, and the images of the calibration jig captured by the fixed camera are processed to determine the positions of the two specific points of the calibration jig as coordinate values in the vision coordinate system of the fixed camera. Recognized in
The coordinate conversion parameter calculation unit calculates the tilt angle of the calibration jig in the vision coordinate system of the hand camera based on the coordinate values of the positions of the two specific points in the vision coordinate system of the hand camera. In addition, based on the coordinate values of the positions of the two specific points in the vision coordinate system of the fixed camera, information regarding the tilt angle of the calibration jig in the vision coordinate system of the fixed camera is calculated, and The coordinate axis tilt correction parameter is calculated based on a difference value of information regarding tilt angles of the two calibration jigs calculated in a vision coordinate system of a hand camera and a vision coordinate system of the fixed camera. Robot calibration system described.
4. The calibration jig according to claim 1, wherein an operator sets the calibration jig at a predetermined position within a field of view of the fixed camera when the image is taken by the hand camera and the fixed camera, and takes out after the imaging is completed. The robot calibration system described in.
A robot that performs a predetermined work on the work supplied to the work area,
A hand camera attached to the arm of the robot,
A fixed camera that is fixed downward at a position higher than the arm operation range of the robot and that images the work supplied to the work area from above,
A control unit for controlling the operation of the arm of the robot with coordinate values in the world coordinate system;
And a calibration jig arranged at a predetermined position within the visual field of the fixed camera,
An image obtained by processing the calibration jig with the fixed camera is processed to obtain information about the position and the tilt angle of the calibration jig from a coordinate system having a reference point of the image as an origin (hereinafter, “fixed camera vision coordinate system”). "))
The operation of the arm of the robot is controlled so that the calibration jig fits within the field of view of the hand camera, the hand camera is moved, and an image obtained by capturing the calibration jig with the hand camera is processed. Recognizing the information about the position and the tilt angle of the calibration jig in a coordinate system whose origin is the reference point of the image (hereinafter referred to as "hand camera vision coordinate system"),
Based on the relationship between the information about the position and the tilt angle of the calibration jig in the vision coordinate system of the fixed camera and the information about the position and the tilt angle of the calibration jig in the vision coordinate system of the hand camera, Calculating a coordinate conversion parameter for converting the coordinate value of the fixed camera vision coordinate system into the coordinate value of the world coordinate system.