JP2012091280A - Coordinate system calibration method and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a coordinate system calibration method which facilitates the calibration of a parameter as relevant information describing the relative relation between a finger coordinate system of a robot and a sensor coordinate system of a visual sensor.SOLUTION: One mark 10 is disposed on a robot coordinate system 35 of a robot 30, a finger of the robot 30 attached with a visual sensor 20 is moved, and then the mark 10 is observed at least at three observation locations from first to third locations while moving the visual sensor 20 in parallel without changing its attitude, and also at least at fourth and fifth observation locations in a direction different from the observation directions at the three locations from the first to third locations relative to the mark 10 and facing the mark 10 from mutually different directions. The images at each observation location are image-processed to obtain mark position recognition data, and the attitude data and mark position recognition data of the robot 30 at each observation location are obtained by associating both with each other to calibrate relevant information based on both data.

Description

  The present invention relates to a coordinate system calibration method for a robot system that combines a robot and a visual sensor.

  Conventionally, in a system combining a robot and a visual sensor, a method called a hand eye has been proposed as one of the installation forms of the visual sensor. Because the visual sensor is fixed to the robot's hand (flange) and the visual sensor moves the same as the robot's hand, it is possible to measure at multiple positions without increasing the number of visual sensors. Great effect in terms of cost and work space saving.

  In a system employing a hand-eye method, the position of an object is measured using a visual sensor, and the movement (position) of the robot is corrected based on the measured position. For this purpose, it is necessary to convert the position data of the object obtained by the visual sensor into data on a coordinate system in which the robot operates. That is, the relationship between the coordinate system of the visual sensor (sensor coordinate system) and the coordinate system (robot coordinate system) in which the robot operates must be obtained as information.

  Generally, a flange for attaching various tools is provided at the tip of the robot arm. Parameters (rotation matrix and translation vector) for converting the coordinate system (flange coordinate system / hand coordinate system) defined for the flange into a robot coordinate system are stored in the storage unit of the robot controller. Therefore, when a visual sensor is attached to the flange, the parameters for converting the coordinate system of the visual sensor, that is, the sensor coordinate system to the flange coordinate system, may be calibrated. Thereby, the sensor coordinate system of the target object detected by the visual sensor can be converted into the flange coordinate system, and further converted into the robot coordinate system.

  Patent Document 1 below relates to a calibration method when a visual sensor is attached to the tip of a robot arm. Four or more marks that can be detected by the visual sensor are provided on the same plane, and the robot coordinate of the mark is obtained by positioning the control point of the robot on the mark. On the other hand, the sensor coordinates of the mark are acquired by fixing the hand of the robot at a predetermined position and detecting the mark with a visual sensor. Thereafter, three marks are selected by a constant evaluation, and the relationship between the two coordinate systems is determined from the correspondence between the robot coordinates of the three marks and the sensor coordinates.

  Non-Patent Document 1 below relates to a calibration method in the case where a visual sensor is attached to the tip of a robot arm. In this document, a method of expressing the rotation and translation of the coordinate system as a 4 × 4 matrix is used. However, if the calibration mark is observed from two directions by changing the posture of the arm tip, that is, the posture of the visual sensor, the sensor coordinate system is used. It is shown that an equation of the form of AX = XB is established as a matrix for converting X in the flange coordinate system. Here, A is a matrix related to two postures of the arm when the calibration mark is detected by the visual sensor, and is obtained from the output of the robot controller. On the other hand, B is a matrix related to the sensor system coordinates of the calibration mark detected by the visual sensor at each posture of the arm, and is obtained from the detection result of the calibration mark. The subject of this document is how to solve this equation to find the matrix X. Here, in order to obtain the matrix B, at least four calibration marks that are not on the same plane are required.

  Further, Non-Patent Document 2 below is a visual sensor attached to the hand of a robot, and a single mark fixed to the robot coordinate system is changed in various directions (one by changing the position and posture of the hand of the robot. In one experimental example, observation is performed from 27 directions), and the relationship between the coordinate system of the hand of the robot and the coordinate system of the sensor is calculated by using a complex algorithm based on the nonlinear optimization method using the observed data.

Japanese Patent No. 3541980

Y. Shiu and S. Ahmad: "Calibration of Wrist-Mounted Robotic Sensors by Solving Homogeneous Transform Equations of the Form AX = XB", IEEE Trans. On Robotics and Automation, vol. 5, no. 1, pp.16-29 , August 1989. Satoshi Kawabata, Kazuyuki Nagata, Yoshihiro Kawai: "Hand-eye calibration using markers fixed to the environment", "Image Recognition and Understanding Symposium (MIRU2009)", pp. 825-832, July 2009

  However, the calibration method disclosed in Patent Document 1 is a calibration method in which the visual sensor is attached to the tip of the arm of the robot, and four or more marks that can be detected by the visual sensor are provided on the same plane. The robot coordinate of the mark is obtained by positioning the control point of the robot on the mark, and four or more calibration marks are required, which increases the cost.

  Further, the calibration method disclosed in Non-Patent Document 1 requires four or more calibration marks that are not in the same plane, that is, three-dimensionally arranged. Must be detectable. For this reason, there has been a problem that the shape and arrangement need to be devised, and the production requires cost and time.

  In addition, the calibration method disclosed in Non-Patent Document 2 requires only one calibration mark, but it is necessary to observe the mark from a very large number of directions, and a nonlinear optimization method is used. There was a problem that there was no guarantee that the calculation would converge to a true value.

  The present invention has been made in view of the above, and in a robot system that combines a robot and a visual sensor fixed to the hand of the robot, the relative relationship between the robot's hand coordinate system and the sensor coordinate system of the visual sensor. It is an object of the present invention to provide a coordinate system calibration method and a robot system that can calibrate related information (rotation matrix and translation vector) describing a relationship easily and inexpensively.

  In order to solve the above-described problems and achieve the object, the coordinate system calibration method of the present invention is a system that combines a robot and a visual sensor fixed to the hand of the robot. This is a method to calibrate the related information describing the relative relationship between the coordinate system and the sensor coordinate system of the visual sensor. One mark is placed in the robot coordinate system of the robot and the hand of the robot with the visual sensor attached is moved. And at least the first to third observation locations translated without changing the attitude of the visual sensor toward the mark, and a direction different from the first to third observation directions with respect to the mark In addition, the mark is observed at least at the fourth and fifth observation points facing the mark from different directions, and the image of each observation point is processed to obtain the mark position recognition data. Te, acquires the posture data and the mark position recognition data of the robot at each observation point in association with, and wherein the calibrating the relevant information based on both data.

  In order to solve the above-described problems and achieve the object, a robot system of the present invention operates a robot, a visual sensor fixed to the hand of the robot, an image processing apparatus that performs image processing of the visual sensor, and a robot A robot operation means, a storage unit that stores relative information describing a relative relationship between the robot's hand coordinate system and the sensor coordinate system of the visual sensor, and a calculation unit that calibrates the related information. One mark is arranged in the system, and the robot operation means translates at least the first to third three of the robot's hand attached with the visual sensor without changing the posture of the visual sensor toward the mark. Observation points and at least the fourth and fifth observation points in directions different from and different from the first to third observation directions with respect to the mark. The image processing device performs image processing on the image of the visual sensor at each observation point and outputs mark position recognition data, and the calculation unit associates the robot posture data with the mark position recognition data at each observation point. The related information is calibrated and the related information is calibrated based on this data.

  According to the present invention, there is an effect that related information describing the relative relationship between the hand coordinate system of the robot and the sensor coordinate system of the visual sensor can be calibrated easily and inexpensively.

FIG. 1 is a schematic diagram illustrating the robot system according to the first embodiment. FIG. 2 is a conceptual diagram showing the positions and orientations of the five observation positions. FIG. 3 is an observation image of the mark at each observation position. FIG. 4 is a schematic diagram illustrating the robot system according to the third embodiment.

  Embodiments of a coordinate system calibration method and a robot system according to the present invention will be described below in detail with reference to the drawings. The present invention is not limited to the embodiments.

Embodiment 1 FIG.
In the first embodiment, one mark that can be recognized by the visual sensor is arranged in the coordinate system of the robot, the hand of the robot to which the visual sensor is attached is controlled, and the postures of at least three visual sensors are Move to the same and different observation positions (that is, only parallel movement), and move the marks from different directions in which the attitude of the visual sensor is different from each other to observe the marks. The robot posture data and the mark position recognition data obtained by the visual system are acquired in association with each other. Based on the data, related information (parameters) representing the relationship between the coordinate system of the robot hand and the coordinate system of the visual sensor is obtained. Calibrate.

  One calibration mark may be prepared, and the coordinates of the arrangement position may be unknown. There are five mark observation positions, and three of them may be positions where the visual sensor is translated, so it is inexpensive and can easily calibrate the relationship between the coordinate system of the robot hand and the sensor coordinate system. it can. The calculation is performed by solving simultaneous equations, but since the solution can be obtained analytically, there is no concern about whether or not it converges to a true value unlike the nonlinear optimization algorithm.

  FIG. 1 is a schematic diagram illustrating the robot system according to the first embodiment. In FIG. 1, a robot system 101 according to the first embodiment includes a robot 30, a robot controller 31, a manual operation panel 37, an image processing device 21, and a visual sensor 20. The robot controller 31 is connected to the robot 30, the image processing device 21, and the manual operation panel 37.

  The robot 30 is a multi-joint robot with 6 degrees of freedom, and a flange 32 is provided at the tip of the robot arm. For example, a robot hand or the like is attached to the flange 32 as a working tool, which is omitted in FIG. The visual sensor 20 is attached to the flange 32 along with a tool such as a robot hand.

  In general, the origin of the robot coordinate system 35 is on the bottom surface of the base of the robot 30 and is defined with the Z axis as the vertical direction. In FIG. 1, the robot coordinate system 35 is drawn with a slight shift for convenience. The flange coordinate system (hand coordinate system) 36 is a coordinate system defined for the flange 32 of the hand of the robot 30 and is similarly drawn slightly shifted in FIG. The robot controller 31 controls the position and orientation of the flange coordinate system (hand coordinate system) 36. Only one calibration mark is arranged in the robot coordinate system 35, and its coordinate position is unknown.

  In order to measure the position of the object using the visual sensor 20 and to operate the robot 30 based on the measured position, the position data of the object obtained by the visual sensor 20 is converted into data on the coordinate system on which the robot 30 operates. To do. That is, the relationship between the coordinate system of the visual sensor 20 (referred to as “sensor coordinate system 25”) and the coordinate system in which the robot 30 operates (referred to as “robot coordinate system 35”) is obtained in advance. Parameters for converting the flange coordinate system (hand coordinate system) 36 defined in the flange 32 at the tip of the robot arm into the robot coordinate system 35 are stored in advance in the storage unit 31 c of the robot controller 31. Since the visual sensor 20 is attached to the flange 32, parameters (rotation matrix and translation vector) for converting the sensor coordinate system 25 to the flange coordinate system 36 may be calibrated. Thereby, the sensor coordinate system 25 of the mark 10 detected by the visual sensor 20 can be converted into the flange coordinate system 36 and further converted into the robot coordinate system 35.

  The robot controller 31 includes a central processing unit (CPU) composed of a microprocessor, a ROM memory, a RAM memory, a nonvolatile memory, a digital servo circuit for robot axis control, a servo amplifier, A signal input / output device and a communication interface connected to the image processing device 21 are provided. These electronic components functionally configure a control unit 31a, a calculation unit 31b, and a storage unit 31c.

  Of the memories constituting the storage unit 31c, the ROM memory stores a system program for controlling the system, and the RAM memory is used for temporary storage of data for processing performed by the CPU. The nonvolatile memory stores operation program data and various setting values necessary for system operation. These various set values include parameters for converting the coordinate system (flange coordinate system / hand coordinate system) 36 defined in the flange 32 of the robot 30 into the robot coordinate system 35, that is, the flange coordinate system (hand coordinate system) 36. Also included are parameters (rotation matrix and translation vector) which are related information describing the relative relationship between the sensor coordinate system 25 and the sensor coordinate system 25. The robot controller 31 controls the position and orientation of the flange 32 and at the same time updates the data in real time and can output it. In addition, although the memory | storage part 31c of this Embodiment is provided in the inside of the robot controller 31, the memory | storage device provided outside may be sufficient.

  The manual operation panel 37 is a panel for manually operating the robot 30, and the robot 30 can be manually operated by operating buttons, joysticks, and the like provided on the panel.

  In the operation of the first embodiment, the robot controller 31 and the manual operation panel 37 constitute a robot operation unit that sequentially moves the visual sensor 20 from the first observation position 1 to the fifth observation position 5. When the operator operates the robot by a button operation or the like using the manual operation panel 37 and sequentially moves the visual sensor 20 from the first observation position 1 to the fifth observation position 5, the manual operation panel 37 is used. Constitutes robot operating means. On the other hand, when the visual sensor 20 automatically moves sequentially from the first observation position 1 to the fifth observation position 5 by the operation program stored in the storage unit 31c of the robot controller 31 (Embodiment 2). The robot controller 31 constitutes a robot operation means.

  The visual sensor 20 is a visual sensor having a visual field 22 and capable of obtaining three-dimensional position data of the mark 10, and is depicted as one camera in FIG. It may be a sensor. The visual sensor 20 is connected to an image processing device 21 that processes information of an image captured by the visual sensor 20. In general, the origin of the sensor coordinate system 25 is set at the center of the lens and the Z-axis is set in the direction of the optical axis.

Next, before describing the operation, the relationship of the coordinate system is organized. The purpose of calibrating the relationship between the hand coordinate system (flange coordinate system) 36 of the robot 30 and the sensor coordinate system 25 is to use the sensor system coordinates (x, y, z) shown in the following equation as flange system coordinates ( 3 × 3 rotation matrix R _h and translation vector T _h to be converted to X _F , Y _F , Z _F ).

The flange system coordinates 36 are converted into robot system coordinates 35 by the following equation.

となる。すなわち、この式がセンサ系座標（ｘ、ｙ、ｚ）をロボット系座標（Ｘ，Ｙ，Ｚ）に変換する式である。フランジ３２の姿勢をＲ_ｆｃに固定して、つまり視覚センサ２０の姿勢を同じに保ったまま、２つの位置Ｔ_ｆ１およびＴ_ｆ２に移動して校正用マークを観測したときのマーク１０のセンサ系座標２５をそれぞれ（ｘ_１、ｙ_１、ｚ_１）、（ｘ_２、ｙ_２、ｚ_２）とすると、ロボット座標系３５におけるマーク１０の座標は同じであるから次の等式が成り立つ。

Here, R _f and T _f are a rotation matrix and a translation vector representing the position and orientation of the flange 32 held by the robot controller 31. And combining (1) and (2),

It becomes. That is, this expression is an expression for converting the sensor system coordinates (x, y, z) to the robot system coordinates (X, Y, Z). The sensor system of the mark 10 when the position of the flange 32 is fixed to R _fc , that is, the calibration mark is observed by moving to the two positions T _f1 and T _f2 while keeping the posture of the visual sensor 20 the same. If the coordinates 25 are (x ₁ , y ₁ , z ₁ ) and (x ₂ , y ₂ , z ₂ ), respectively, the coordinates of the mark 10 in the robot coordinate system 35 are the same, so the following equation holds.

If you transform the expression

が得られる。（５）（６）式において未知数はＲ_ｈのみである。Ｒ_ｈは３×３の行列で９要素からなるが、回転行列であることから、各列（各行）の要素の２乗和が１になるという制約条件があり、（ｘ_１、ｙ_１、ｚ_１）、（ｘ_２、ｙ_２、ｚ_２）、（ｘ_３、ｙ_３、ｚ_３）が互いに異なっていれば、（５）（６）の連立方程式を解いてＲ_ｈをユニークに決定できる。 Similarly, when the same calibration mark is observed by moving to the third position T _f3 while keeping the posture of the visual sensor 20 the same,

Is obtained. (5) (6) unknown it is only _{R h} in formulas. R _h is a 3 × 3 matrix consisting of 9 elements, but since it is a rotation matrix, there is a constraint that the sum of squares of the elements in each column (each row) is 1, and (x ₁ , y ₁ , _{z 1), (x 2,} y 2, z 2), (x 3, y 3, z 3) long as it is different from each other, determines uniquely the _{R h} by solving the simultaneous equations (5) (6) it can.

が言える。整理をすると

Next, if the position and orientation of the flange 32 are moved to T _f4 and R _f4 to observe the calibration mark, and the sensor system coordinates of the mark 10 are (x ₄ , y ₄ , z ₄ ), (4) Like the formula

I can say. When organizing

を得る。回転行列Ｒ_ｆ４、Ｒ_ｆ５の回転軸が平行でなければ、言い換えれば回転方向が異なっていれば、（８）（９）式を連立一次方程式として解いてＴ_ｈを求めることが出来る。 If been determined that _{R h} in the manner described above, (8) unknowns is only _{T h} of the right _side, seems _{T h} is obtained by multiplying the left the inverse matrix of [R f4 _{-R fc]} It is. However, since the difference matrix of the rotation matrix is degenerated to rank 2, there is no inverse matrix. Therefore, when the position and orientation of the flange 32 are further moved to T _f5 and R _f5 to observe the calibration mark, and the sensor system coordinates of the mark are (x ₅ , y ₅ , z ₅ ), equation (8) Like

Get. If not parallel the axis of rotation of the rotation matrix R _f4, R _f5 is, if the rotational direction is different in other words, (8) (9) it is possible to determine the T _h by solving a system of linear equations.

  Based on the above preparation, the operation will be described below. The operator first arranges the calibration mark 10, but the position may be any position where the robot 30 can be operated and the following observation can be performed by the visual sensor 20 attached to the hand. And the coordinates may be unknown. There are no particular restrictions on the shape or pattern of the mark 10, as long as the mark can be recognized by the visual sensor and the position of one point on it can be measured by the visual sensor 20.

Next, the operator operates the manual operation panel 37 of the robot 30 to sequentially detect the mark 10 by the visual sensor 20 while controlling the position and posture of the hand of the robot 30, and each time the robot 30 is detected. attitude data R _f, recorded in association with the sensor system coordinates of T _f and mark. At this time, the mark 10 is observed at the first to third observation points where the hand is moved by the position without changing the posture, and then the fourth and fifth observation points where the position and posture of the hand are also changed. The mark 10 is detected.

  FIG. 2 is a conceptual diagram showing the positions and orientations of the five observation positions according to the first embodiment. The observation postures of observations 1, 2, and 3, which are the first to third observation points, are fixed (held in a certain direction), indicating that only the observation positions are different from each other. Observations 4 and 5, which are the fourth and fifth observation points, indicate that the position and the posture are different from each other as compared with the first observation point. When moving from the first observation point to the fourth observation point, or when moving from the first observation point to the fifth observation point, the visual sensor 20 has an arcuate path with the mark 10 as the rotation center. Move (rotate) through. Although it is difficult to illustrate at this time, as described above, the rotation axis of the arc when moving from the first observation point to the fourth observation point and the arc when moving from the first observation point to the fifth observation point The rotation axis is a straight line that passes through the vicinity of the center of the mark 10, but is controlled so that both straight lines do not become parallel.

  For each observation, the rotation matrix Rf representing the position and orientation of the flange coordinate system 36 output from the robot controller 31 and the translation vector Tf are used as the sensor system coordinates (x of the observed mark 10 output from the image processing device 21. , Y, z) are recorded in association with 25.

  An equation that does not include a translation vector is obtained from observations 1, 2, and 3 that are the first to third observation points, and a rotation matrix can be easily obtained. Observation 1 that is the first, fourth, and fifth observation points , 4 and 5 give general equations including a rotation matrix and a translation vector. However, if the rotation matrix is known, an equation of only the translation vector is obtained, and the translation vector can be easily obtained.

  Note that the observation order of the mark 10 does not need to follow the order of observation 1 to observation 5 shown in FIG. 2, and may be executed in an arbitrary order. FIG. 3 shows an example of mark observation. The mark is a pattern in which a square is divided into four by two diagonal lines, and four triangles are painted alternately in white and black. The point at which the visual sensor 20 measures the position is the intersection of the diagonal lines. If it is a point with such a clear feature, a corresponding point can be easily found with a binocular stereo type visual sensor as in the technique of Non-Patent Document 2, and the distance from the camera can be calculated.

After completing the mark observation of a given 5 points, obtains the R _h by solving equations (5) (6) in simultaneous, utilizing Motoma' was R _h, by solving equations (8) (9) in simultaneous determine the T _h. The work of solving the equation is associated with the data on the position and orientation of the flange 32 obtained at the time of mark observation, the values of T _f and R _f, and the sensor system coordinates (x, y, z) 25 of the mark output from the visual sensor 20. For example, it can be input into a spreadsheet program such as Excel and solved.

  The operation for solving the equations may be performed by solving a program stored in the storage unit 31c of the robot controller 31 by the calculation unit 31b or by a program stored in a storage unit (not shown) of the image processing device 21. In that case, it is preferable that the robot controller 31 and the image processing device 21 transmit / receive necessary data via a communication network.

  As described above, in the coordinate system calibration method according to the first embodiment, one mark 10 is arranged in the robot coordinate system 35 of the robot 30, the hand of the robot 30 to which the visual sensor 20 is attached is moved, and the visual sensor 20. At least the first to third observation points translated in parallel without changing the posture of 20 and directions different from the first to third observation directions with respect to the mark 10 and different from each other The observation of the mark 10 is performed at least at the fourth and fifth observation points facing the mark 10, and the image of each observation point is subjected to image processing to obtain mark position recognition data, and the posture of the robot 30 at each observation point Data and mark position recognition data are acquired in association with each other, and related information (parameters) is calibrated based on both data. The calculation is performed by solving simultaneous equations, but since the solution can be obtained analytically, there is no concern about whether or not it converges to a true value unlike the nonlinear optimization algorithm. An equation that does not include a translation vector is obtained from the first to third observation points, and a rotation matrix can be easily obtained. From the first, fourth, and fifth observation points, a rotation matrix and a translation vector are generally included. In this case, if the rotation matrix is known, the equation of only the translation vector is obtained, and the translation vector can be easily obtained. Thereby, the related information (parameter) describing the relative relationship between the hand coordinate system (flange coordinate system) 36 of the robot 30 and the sensor coordinate system 25 of the visual sensor 20 can be easily and inexpensively calibrated.

Embodiment 2. FIG.
In the first embodiment, the operator operates the robot 30 from the manual operation panel 37 to control the position and posture of the hand of the robot 30 and guide the robot hand to the required five observation positions. In the second embodiment, five mark observation positions and postures at each position are programmed as a robot control program in advance and stored in the storage unit 31c of the robot controller 31. After the operator arranges the mark 10, The observation data at the required five points can be obtained only by starting the program. By incorporating the observation positions and orientations of the necessary five marks in advance as a program, it is possible to respond quickly when a calibration work for the coordinate system becomes necessary again due to some trouble.

Embodiment 3 FIG.
FIG. 4 is a schematic diagram illustrating the robot system according to the third embodiment. In the configuration of the third embodiment, a robot guidance device (robot guidance means) 40 is added to the configuration of the first embodiment. In the first embodiment, the movement of the hand, that is, the visual sensor 20 to the mark observation position is by manual operation of the operator. However, in the third embodiment, the robot guidance device 40 has the initiative and the mark 10 The robot 10 is guided using the result of position recognition, and the visual sensor 20 is sequentially moved to a predetermined mark observation position. The robot guidance device 40 stores ideal observation image information 41 as guidance target data. The ideal observation image information 41 includes an ideal position on the image for observing the mark 10 for the second and third observations, and an ideal position on the image for observing the mark 10 for the fourth and fifth observations. And the ideal rotation information of the posture as viewed from observation 1 is held.

  First, the operator manually moves the hand of the robot 30 to a position where observation like the observation 1 in FIG. 3 can be performed. Next, the robot guidance device 40 is activated by switching to the automatic mode. The robot guidance device 40 sequentially guides the hand of the robot 30 to the second, third, fourth, and fifth mark observation positions and postures by a technique called visual feedback. That is, the robot 30 is guided so that the difference between the position of the mark 10 currently observed and the ideal mark position on the image described in the ideal observation image information 41 becomes small. For this purpose, first, the hand of the robot 30 is slightly moved so that the mark does not come off from the visual field 22 of the visual sensor 25, and as a result, it is checked in which direction the mark 10 has moved on the image so that the mark approaches the ideal position. Repeat trial and error. Then, if the difference between the ideal mark position and the mark position on the image is below a certain level, it is determined that the target observation position has been reached. In the second and third observations, the posture does not change the posture that the operator first moved manually. In the fourth and fifth observations, the rotation described in the ideal observation image information 41 is changed to the mark 10. In addition to the above, the guidance is performed in the same manner. The ideal observation image information 41 according to the present embodiment is stored in a storage unit provided inside the robot guidance device 40, but may be stored in a storage device provided outside.

  In this way, in the third embodiment, observation data at five locations can be automatically acquired. The visual sensor 20 generally has higher measurement accuracy in the center of the visual field than in the peripheral part of the visual field. If the ideal observation image information 41 is set to observe at the center of the visual field, the coordinate system can be calibrated based on observation data with higher accuracy than the positioning of the mark observation position by the operator's visual observation.

  As described above, the coordinate system calibration method and the robot system according to the present invention are optimal for the coordinate system calibration method of the robot system in which the visual sensor is combined with the robot hand.

10 mark 20 visual sensor 21 image processing apparatus 22 field of view 25 sensor coordinate system 30 robot 31 robot controller (robot operation means)
31a Control unit 31b Calculation unit 31c Storage unit 32 Flange 35 Robot coordinate system 36 Flange coordinate system (hand coordinate system)
37 Manual operation panel (robot operation means)
40 Robot guidance device (robot guidance means)
41 Ideal observation image information

Claims (6)

In a system combining a robot and a visual sensor fixed to the hand of the robot, related information describing a relative relationship between the hand coordinate system of the robot stored in a storage unit and the sensor coordinate system of the visual sensor is stored. A method of proofreading,
Placing one mark in the robot coordinate system of the robot,
At least first to third observation points that have been translated by moving the hand of the robot to which the visual sensor is attached and changing the posture of the visual sensor toward the mark, and the mark On the other hand, the mark is observed at least at the fourth and fifth observation points facing the mark from directions different from and different from the first to third observation directions,
Image processing of each observation point image to obtain mark position recognition data,
A coordinate system calibration method characterized in that the posture data of the robot at each observation point and the mark position recognition data are acquired in association with each other, and the related information is calibrated based on both data. The first to fifth observation points are programmed in advance in a robot controller that controls the robot, and the robot controller moves to each observation point, observes, processes images, and calibrates the related information. The coordinate system calibration method according to claim 1, wherein the coordinate system is automatically controlled. A robot guidance means for asymptotically guiding the robot so that an image observed by the visual sensor matches an ideal observation image registered in advance;
The coordinate system calibration method according to claim 1, wherein the robot guidance unit moves the visual sensor to each observation location. A robot, a visual sensor fixed to the hand of the robot, an image processing device for performing image processing of the visual sensor, a robot operating means for operating the robot, a hand coordinate system of the robot, and a sensor of the visual sensor A storage unit that stores related information describing a relative relationship of the coordinate system, and a calculation unit that calibrates the related information,
One mark is arranged in the robot coordinate system of the robot,
The robot operation means includes at least first to third observation points obtained by translating the hand of the robot to which the visual sensor is attached without changing the posture of the visual sensor toward the mark; Move to at least the fourth and fifth observation points in directions different from and different from the first to third observation directions with respect to the mark,
The image processing device performs image processing on the image of the visual sensor at each observation location and outputs mark position recognition data,
The arithmetic unit obtains the robot posture data and the mark position recognition data at each observation location in association with each other, and calibrates the related information based on the data. The robot operation means is a robot controller in which the first to fifth observation points are programmed in advance, and the robot controller automatically performs movement to each observation point, observation, and calibration of the related information. 5. The robot system according to claim 4, wherein the robot system is performed. Robot guidance means for asymptotically guiding the robot so that an image observed by the visual sensor matches an ideal observation image registered in advance;
The robot system according to claim 4, wherein the robot guiding unit guides the visual sensor so as to approach an ideal position at each observation point.

Images