WO2021210456A1 - ロボットの制御座標系における視覚センサの位置を取得する装置、ロボットシステム、方法、及びコンピュータプログラム - Google Patents

ロボットの制御座標系における視覚センサの位置を取得する装置、ロボットシステム、方法、及びコンピュータプログラム Download PDF

Info

Publication number
WO2021210456A1
WO2021210456A1 PCT/JP2021/014676 JP2021014676W WO2021210456A1 WO 2021210456 A1 WO2021210456 A1 WO 2021210456A1 JP 2021014676 W JP2021014676 W JP 2021014676W WO 2021210456 A1 WO2021210456 A1 WO 2021210456A1
Authority
WO
WIPO (PCT)
Prior art keywords
visual sensor
posture
coordinate system
index
robot
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/014676
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
恭平 小窪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fanuc Corp
Original Assignee
Fanuc Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fanuc Corp filed Critical Fanuc Corp
Priority to DE112021002301.2T priority Critical patent/DE112021002301T5/de
Priority to JP2022515324A priority patent/JP7817152B2/ja
Priority to US17/918,326 priority patent/US20230339117A1/en
Priority to CN202180027827.XA priority patent/CN115397634B/zh
Publication of WO2021210456A1 publication Critical patent/WO2021210456A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/021—Optical sensing devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1628—Program controls characterised by the control loop
    • B25J9/1653—Program controls characterised by the control loop parameters identification, estimation, stiffness, accuracy, error analysis
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1679—Program controls characterised by the tasks executed
    • B25J9/1692—Calibration of manipulator
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1602—Program controls characterised by the control system, structure, architecture
    • B25J9/1607—Calculation of inertia, jacobian matrixes and inverses
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39045—Camera on end effector detects reference pattern

Definitions

  • the present invention relates to a device, a robot system, a method, and a computer program for acquiring the position of a visual sensor in the control coordinate system of a robot.
  • Patent Documents 1 and 2 there are known devices that measure the position and orientation of a visual sensor in the control coordinate system of a robot based on image data obtained by capturing an index with a visual sensor.
  • the index may deviate from the field of view of the visual sensor.
  • a device for acquiring the position of a visual sensor in a control coordinate system for controlling a robot that relatively moves a visual sensor and an index comprises a processor, which operates the robot. Then, the posture of the visual sensor or the index is changed by the first posture change amount, and the control coordinates are based on the image data of the index captured by the visual sensor before and after the posture is changed by the first posture change amount.
  • the position of the visual sensor in the system is acquired as the test measurement position, the robot is operated, and the posture is changed in the posture change direction determined based on the test measurement position, and the second posture change is larger than the first posture change amount.
  • the position of the visual sensor in the control coordinate system is acquired as the main measurement position based on the image data of the index captured by the visual sensor before and after changing the posture by the amount and changing the posture by the second posture change amount.
  • the method of acquiring the position of the visual sensor in the control coordinate system for controlling the robot that relatively moves the visual sensor and the index is such that the processor operates the robot to obtain the visual sensor or
  • the visual sensor in the control coordinate system is based on the index image data captured by the visual sensor before and after the index posture is changed by the first posture change amount and the posture is changed by the first posture change amount.
  • the position is acquired as the test measurement position, the robot is operated, and the posture is changed in the posture change direction determined based on the test measurement position by a second posture change amount larger than the first posture change amount.
  • the position of the visual sensor in the control coordinate system is acquired as the present measurement position based on the image data of the index captured by the visual sensor before and after the posture is changed by the second posture change amount.
  • the trial measurement position of the visual sensor in the control coordinate system is estimated, and then the attitude of the visual sensor is changed to a larger attitude.
  • the main measurement position of the visual sensor in the control coordinate system is obtained. According to this configuration, it is possible to acquire the main measurement position indicating the accurate position of the visual sensor in the control coordinate system while preventing the index from being deviated from the visual field of the visual sensor after the posture change.
  • the robot system 10 includes a robot 12, a visual sensor 14, a control device 16, and a teaching device 18.
  • the robot 12 is a vertical articulated robot, and has a robot base 20, a swivel body 22, a robot arm 24, and a wrist portion 26.
  • the robot base 20 is fixed to the floor of the work cell.
  • the swivel body 22 is provided on the robot base 20 so that it can swivel around a vertical axis.
  • the robot arm 24 has a lower arm portion 28 rotatably provided on the swivel body 22 around a horizontal axis, and an upper arm portion 30 rotatably provided at the tip end portion of the lower arm portion 28.
  • the wrist portion 26 has a wrist base 32 rotatably connected to the tip of the upper arm portion 30, and a wrist flange 34 rotatably provided around the axis A on the wrist base 32.
  • the wrist flange 34 is a cylindrical member having an axis A as a central axis, and has a mounting surface 34a on the tip end side thereof. The wrist portion 26 rotates the wrist flange 34 around the axis A.
  • An end effector (not shown) that performs work on the work is detachably attached to the attachment surface 34a.
  • the end effector is a robot hand, a welding gun, a laser processing head, a coating material coating device, or the like, and performs predetermined work (work handling, welding, laser processing, coating, etc.) on the work.
  • a servomotor 36 (FIG. 2) is built in each component of the robot 12 (that is, the robot base 20, the swivel body 22, the robot arm 24, and the wrist portion 26).
  • the servomotor 36 drives each movable element of the robot 12 (that is, the swivel body 22, the robot arm 24, and the wrist portion 26) in response to a command from the control device 16.
  • the robot coordinate system C1 (FIG. 1) is set in the robot 12.
  • the robot coordinate system C1 is a control coordinate system for controlling the operation of each movable element of the robot 12, and is fixed in a three-dimensional space.
  • the robot coordinate system C1 is set with respect to the robot 12 so that its origin is located at the center of the robot base 20 and its z-axis coincides with the swivel axis of the swivel cylinder 22. There is.
  • a mechanical interface (hereinafter abbreviated as "MIF") coordinate system C2 is set on the hand (specifically, the wrist flange 34) of the robot 12.
  • the MIF coordinate system C2 is a control coordinate system for controlling the position and orientation of the wrist flange 34 (or end effector) in the robot coordinate system C1.
  • the origin of the MIF coordinate system C2 is set at the center of the mounting surface 34a of the wrist flange 34
  • the z-axis of the MIF coordinate system C2 is set at the hand of the robot 12 so as to coincide with the axis A. ..
  • the processor 40 When moving the wrist flange 34 (end effector), the processor 40 sets the MIF coordinate system C2 in the robot coordinate system C1 and sets the wrist flange 34 (end effector) in the position and orientation represented by the set MIF coordinate system C2. Each servomotor 36 of the robot 12 is controlled so as to arrange the robot 12. In this way, the processor 40 can position the wrist flange 34 (end effector) at an arbitrary position and posture in the robot coordinate system C1.
  • the visual sensor 14 is, for example, a camera or a three-dimensional visual sensor, which is an imaging sensor (CCD, CMOS, etc.) that receives a subject image and performs photoelectric conversion, and optics that collects the subject image and focuses it on the imaging sensor. It has a lens (condensing lens, focus lens, etc.).
  • the visual sensor 14 captures an object and transmits the captured image data to the control device 16.
  • the visual sensor 14 is fixed at a predetermined position with respect to the wrist flange 34.
  • the sensor coordinate system C3 is set in the visual sensor 14.
  • the sensor coordinate system C3 is a coordinate system that defines the coordinates of each pixel of the image data captured by the visual sensor 14, and its origin is at the center of the light receiving surface (or optical lens) of the image sensor of the visual sensor 14. Arranged so that the x-axis and y-axis are arranged parallel to the horizontal and vertical directions of the image sensor, and the z-axis coincides with the line of sight (or optical axis) O of the visual sensor 14. It is set for the visual sensor 14.
  • the control device 16 controls the operations of the robot 12 and the visual sensor 14.
  • the control device 16 is a computer having a processor 40, a memory 42, and an I / O interface 44.
  • the processor 40 has a CPU, a GPU, or the like, and is communicably connected to the memory 42 and the I / O interface 44 via the bus 46.
  • the processor 40 sends a command to the robot 12 and the visual sensor 14 while communicating with the memory 42 and the I / O interface 44, and controls the operation of the robot 12 and the visual sensor 14.
  • the memory 42 has a RAM, a ROM, or the like, and temporarily or permanently stores various data.
  • the I / O interface 44 has, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, an HDMI (registered trademark) terminal, or the like, and wirelessly or data is transmitted to an external device under a command from the processor 40. Communicate by wire.
  • the servomotor 36 and the visual sensor 14 described above are connected to the I / O interface 44 so as to be able to communicate wirelessly or by wire.
  • the teaching device 18 is, for example, a hand-held device (teaching pendant, tablet-type terminal device, or the like) used to teach an operation for causing the robot 12 to perform a predetermined work.
  • the teaching device 18 is a computer having a processor 50, a memory 52, an I / O interface 54, an input device 56, and a display device 58.
  • the processor 50 has a CPU, a GPU, or the like, and is communicably connected to the memory 52, the input device 56, the display device 58, and the I / O interface 54 via the bus 60.
  • the memory 52 has a RAM, a ROM, or the like, and temporarily or permanently stores various data.
  • the I / O interface 54 has, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, an HDMI (registered trademark) terminal, or the like, and wirelessly or data is transmitted to an external device under a command from the processor 50. Communicate by wire.
  • the I / O interface 54 is connected to the I / O interface 44 of the control device 16 by wire or wirelessly, and the control device 16 and the teaching device 18 can communicate with each other.
  • the input device 56 has a push button, a switch, a keyboard, a touch panel, or the like, receives an operator's input operation, and transmits the input information to the processor 50.
  • the display device 58 has an LCD, an organic EL display, or the like, and displays various information under a command from the processor 50. The operator can jog the robot 12 by operating the input device 56 and teach the robot 12 the operation.
  • the positional relationship between the MIF coordinate system C2 and the sensor coordinate system C3 is not calibrated and is unknown.
  • the visual sensor in the control coordinate system for controlling the robot 12 that is, the robot coordinate system C1 and the MIF coordinate system C2. It is necessary to know the position of 14 (that is, the origin position of the sensor coordinate system C3) and the posture (that is, each axial direction of the sensor coordinate system C3).
  • the teaching device 18 data on the position and orientation of the visual sensor 14 in the control coordinate system (robot coordinate system C1, MIF coordinate system C2) based on the image data of the index ID captured by the visual sensor 14.
  • FIG. 3 shows an example of the index ID.
  • the index ID is provided on the upper surface of the structure B, and is composed of a circular line C and two straight lines D and E orthogonal to each other.
  • the index ID is provided on the structure B as a visually recognizable form such as a pattern using paint or a marking (unevenness) formed on the upper surface of the structure B.
  • FIG. 4 a method of acquiring the position and orientation data of the visual sensor 14 in the control coordinate system (robot coordinate system C1 and MIF coordinate system C2) will be described.
  • the flow shown in FIG. 4 starts when the processor 50 of the teaching device 18 receives an operation start command from the operator, the host controller, or the computer program CP.
  • the processor 50 may execute the flow shown in FIG. 4 according to the computer program CP.
  • This computer program CP may be stored in the memory 52 in advance.
  • step S1 the processor 50 executes the posture acquisition process. This step S1 will be described with reference to FIG.
  • step S11 the processor 50 operates the robot 12 to arrange the visual sensor 14 at the initial position PS 0 and the initial posture OR 0 with respect to the index ID.
  • the initial position PS 0 and an initial orientation OR 0 is determined in advance.
  • the data of the initial position PS 0 and the initial posture OR 0 (that is, the data indicating the coordinates of the origin of the MIF coordinate system C2 and the direction of each axis in the robot coordinate system C1) are defined in advance in the computer program CP and are stored in the memory 52. Stored in.
  • step S12 the processor 50 operates the visual sensor 14 to image the index ID, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 operates the visual sensor 14 arranged at the initial position PS 0 and the initial posture OR 0 , and acquires the image data JD 0 of the index ID by the visual sensor 14.
  • the processor 50 acquires the image data JD 0 from the visual sensor 14 via the control device 16 and stores it in the memory 52.
  • the processor 50 may directly acquire the image data JD 0 from the visual sensor 14 without going through the control device 16.
  • the I / O interface 54 may be communicably connected to the visual sensor 14 by wire or wirelessly.
  • the processor 50 acquires data indicating the relative position of the index ID with respect to the visual sensor 14 when the image data JD 0 is imaged.
  • the image data JD n the visual sensor 14 disposed at an arbitrary position PS n and orientation OR n which can fit an index ID to the field of view is captured, the visual sensor 14 when the captured the image data JD n It is possible to obtain the relative position data of the index ID with respect to. This method will be described below.
  • FIG. 6 shows an example of image data JD n captured by a visual sensor 14 arranged at an arbitrary position PS n and posture OR n.
  • the origin of the sensor coordinate system C3 is arranged at the center of the image data JD n (specifically, the pixel at the center).
  • the origin of the sensor coordinate system C3 may be arranged at any known position (pixel) in the image data JD n.
  • the processor 50 analyzes the image data JD n, identifies the intersection F of the straight line D and E of the index ID caught on the image data JD n. Then, the processor 50 acquires the coordinates (x n , y n ) of the intersection F in the sensor coordinate system C3 as data indicating the position of the index ID in the image data JD n.
  • the processor 50 analyzes the image data JD n, specifying the circle C of the index ID caught on the image data JD n. Then, the processor 50 sets the area of the circle C (or the number of pixels included in the image area of the circle C) in the sensor coordinate system C3 as the size IS n (unit [pixel]) of the index ID reflected in the image data JD n. It is acquired as data indicating.
  • the processor 50 has a size RS (unit [mm]) of the index ID in the real space, a focal length FD of the optical lens of the visual sensor 14, and a size SS (unit [mm / pixel]) of the image sensor of the visual sensor 14. ) To get.
  • RS unit [mm]
  • FD focal length
  • SS unit [mm / pixel]
  • the processor 50 acquires a vector (X n , Y n , Z n ) using the acquired coordinates (x n , y n ), size IS n , size RS, focal length FD, and size SS.
  • This vector (X n , Y n , Z n ) is from the visual sensor 14 (that is, the origin of the sensor coordinate system C3) when the image data JD n is imaged to the index ID (specifically, the intersection F). It is a vector and is data indicating the relative position (or the coordinates of the sensor coordinate system C3) of the index ID with respect to the visual sensor 14.
  • processor 50 acquires the relative position data (X 0 , Y 0 , Z 0 ) of the index ID with respect to the visual sensor 14 when the image data JD 0 is imaged.
  • step S13 the processor 50 operates the robot 12 to translate the visual sensor 14.
  • step S14 the processor 50 operates the visual sensor 14 to image the index ID in the same manner as in step S12 described above, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 acquires the image data JD 1 of the index ID by the visual sensor 14 arranged at the position PS 1 and the posture OR 0 , and the coordinates of the intersection F of the index ID reflected in the image data JD 1 ( Obtain x 1 , y 1 ) and size IS 1.
  • the processor 50 uses the acquired coordinates (x 1 , y 1 ) and size IS 1 with respect to the visual sensor 14 when the image data JD 1 is imaged using the above equations (1) to (3).
  • the relative position data (X 1 , Y 1 , Z 1 ) of the index ID is acquired.
  • the processor 50 returns the visual sensor 14 to the initial position PS 0 and the initial posture OR 0 by the robot 12.
  • step S16 the processor 50 operates the visual sensor 14 to image the index ID in the same manner as in step S12 described above, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 acquires the image data JD 2 of the index ID by the visual sensor 14 arranged at the position PS 2 and the posture OR 0 , and the coordinates of the intersection F of the index ID reflected in the image data JD 2 ( x 2, y 2) and obtains the size iS 2.
  • the processor 50 the acquired coordinates (x 2, y 2) and size IS 2, by using the above-mentioned formula (1) to (3), for the visual sensor 14 when the captured image data JD 2
  • the relative position data (X 2 , Y 2 , Z 2 ) of the index ID is acquired.
  • the processor 50 returns the visual sensor 14 to the initial position PS 0 and the initial posture OR 0 by the robot 12.
  • step S18 the processor 50 operates the visual sensor 14 to image the index ID in the same manner as in step S12 described above, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 acquires the image data JD 3 of the index ID by the visual sensor 14 arranged at the position PS 3 and the posture OR 0 , and the coordinates of the intersection F of the index ID reflected in the image data JD 3 ( x 3, y 3) and obtains the size iS 3.
  • the processor 50 uses the acquired coordinates (x 3 , y 3 ) and size IS 3 with respect to the visual sensor 14 when the image data JD 3 is imaged using the above equations (1) to (3).
  • the relative position data (X 3 , Y 3 , Z 3 ) of the index ID is acquired.
  • the processor 50 returns the visual sensor 14 to the initial position PS 0 and the initial posture OR 0 by the robot 12.
  • the matrix M1 shown in the formula of is acquired.
  • This matrix M1 is a rotation matrix representing the posture (W, P, R) of the visual sensor 14 (or the sensor coordinate system C3) in the MIF coordinate system C2.
  • This rotation matrix can be represented by three parameters, so-called roll, pitch, and yaw.
  • the coordinate W corresponds to the value of "yaw”
  • the coordinate P corresponds to the value of "pitch”
  • the coordinate R corresponds to the value of "roll”.
  • the coordinates W, P, and R of these postures can be obtained from the matrix M1.
  • the processor 50 acquires the posture data (W, P, R) of the visual sensor 14 in the MIF coordinate system C2 and stores it in the memory 52.
  • This attitude data (W, P, R) defines the direction of each axis of the sensor coordinate system C3 in the MIF coordinate system C2 (that is, the line of sight O). Since the coordinates of the MIF coordinate system C2 and the coordinates of the robot coordinate system C1 can be converted to each other via a known transformation matrix, the attitude data (W, P, R) in the MIF coordinate system C2 can be obtained. , Can be converted into the coordinates (W', P', R') of the robot coordinate system C1.
  • the initial position PS 0 and the initial posture OR 0 are described above so that the index ID is in the field of view of the visual sensor 14 at all the positions and postures in which the visual sensor 14 is arranged in steps S11, S13, S15, and S17.
  • the distances ⁇ x, ⁇ y and ⁇ z are defined.
  • the operator determines the initial position PS 0 and the initial posture OR 0 so that the line of sight O of the visual sensor 14 passes inside the circle C of the index ID.
  • the positional relationship between the line-of-sight O of the visual sensor 14 and the index ID at the initial position PS 0 and the initial posture OR 0 is, for example, the design value of the drawing data (CAD data, etc.) of the visual sensor 14, the robot 12, and the structure B. It can be estimated from the above. As a result, the index ID reflected in the image data JD 0 can be arranged near the origin of the sensor coordinate system C3. The distances ⁇ x, ⁇ y and ⁇ z may be different values from each other.
  • step S2 the processor 50 executes the test measurement process.
  • step S2 will be described with reference to FIG. 7.
  • step S21 the processor 50 changes the posture of the visual sensor 14 by rotating and moving the visual sensor 14. Specifically, the processor 50 first sets the reference coordinate system C4 in the MIF coordinate system C2 at this time point (initial position PS 0 and initial posture OR 0).
  • the processor 50 has the reference coordinate system C4 whose origin is arranged at the origin of the MIF coordinate system C2, and its posture (direction of each axis) is the posture (W) acquired in step S19 described above.
  • P, R are set in the MIF coordinate system C2. Therefore, the x-axis, y-axis, and z-axis directions of the reference coordinate system C4 are parallel to the x-axis, y-axis, and z-axis of the sensor coordinate system C3, respectively.
  • the processor 50 operates the robot 12 to move the visual sensor 14 (that is, the wrist flange) from the initial position PS 0 and the initial posture OR 0 to the z-axis of the reference coordinate system C4 (that is, the direction of the line of sight O).
  • the attitude change amount ⁇ 1 first attitude change amount
  • step S22 the processor 50 operates the visual sensor 14 to image the index ID in the same manner as in step S12 described above, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 acquires the image data JD 4 of the index ID by the visual sensor 14 arranged at the position PS 4 and the posture OR 1 , and the coordinates of the intersection F of the index ID reflected in the image data JD 4 ( acquires x 4, y 4) and size iS 4.
  • the processor 50 uses the acquired coordinates (x 4 , y 4 ) and size IS 4 with respect to the visual sensor 14 when the image data JD 4 is imaged using the above equations (1) to (3).
  • the relative position data (X 4 , Y 4 , Z 4 ) of the index ID is acquired.
  • the processor 50 returns the visual sensor 14 to the initial position PS 0 and the initial posture OR 0 by the robot 12.
  • step S23 the processor 50 changes the posture of the visual sensor 14 by rotating and moving the visual sensor 14.
  • the processor 50 operates the robot 12 to move the visual sensor 14 from the initial position PS 0 and the initial posture OR 0 to the x-axis or y-axis of the reference coordinate system C4 (that is, the direction of the line of sight O).
  • the posture change amount ⁇ 2 first posture change amount
  • step S24 the processor 50 operates the visual sensor 14 to image the index ID in the same manner as in step S12 described above, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 acquires the image data JD 5 of the index ID by the visual sensor 14 arranged at the position PS 5 and the posture OR 2 , and the coordinates of the intersection F of the index ID reflected in the image data JD 5 ( x 5, y 5) and obtains the size iS 5.
  • the processor 50 uses the acquired coordinates (x 5 , y 5 ) and size IS 5 with respect to the visual sensor 14 when the image data JD 5 is imaged using the above equations (1) to (3).
  • the relative position data (X 5 , Y 5 , Z 5 ) of the index ID is acquired.
  • the processor 50 returns the visual sensor 14 to the initial position PS 0 and the initial posture OR 0 by the robot 12.
  • step S25 the processor 50 acquires the test measurement position of the visual sensor 14.
  • the vector from the origin of the reference coordinate system C4 (the origin of the MIF coordinate system C2 in the present embodiment) in the MIF coordinate system C2 to the origin of the sensor coordinate system C3 whose position is unknown is ( ⁇ X 1 , If ⁇ Y 1 , ⁇ Z 1 ), the following equations (4) and (5) are established.
  • the processor 50 solves the above equations (4) and (5) to obtain a vector ( ⁇ X 1 , ⁇ Y 1) from the origin of the reference coordinate system C4 in the MIF coordinate system C2 to the origin of the unknown sensor coordinate system C3. , ⁇ Z 1 ) can be estimated.
  • This vector ( ⁇ X 1 , ⁇ Y 1 , ⁇ Z 1 ) is data indicating the approximate position of the visual sensor 14 (origin of the sensor coordinate system C3) in the MIF coordinate system C2.
  • the processor 50 acquires the trial measurement position as the coordinates (x T , y T , z T) of the MIF coordinate system C2.
  • test measurement positions (x T , y T , z T ) ( ⁇ X 1 , ⁇ Y 1 ) refers to the visual sensor 14 in step S21 above. It can be obtained from the above equation (4) by the operation of rotating in the circumferential direction.
  • the test measurement position (x T , y T ) ( ⁇ X 1 , ⁇ Y 1 ) is the approximate position of the line of sight O in the MIF coordinate system C2 (in other words, in the plane orthogonal to the line of sight O at the origin of the sensor coordinate system C3). Approximate position) is shown.
  • the processor 50 sets the test measurement positions (x T , y T , z T ) to the posture change amounts ⁇ 1 and ⁇ 2, and the image data before the posture is changed (that is, the initial posture OR 0).
  • the relative position data (X 0 , Y 0 , Z 0 ) when the JD 0 is imaged and the image data JD 4 and JD 5 are imaged after the posture is changed that is, the postures OR 1 and OR 2). It is acquired based on the relative position data (X 4 , Y 4 , Z 4 ) and (X 5 , Y 5 , Z 5) of.
  • the processor 50 updates the unknown coordinates of the MIF coordinate system C2 of the origin of the sensor coordinate system C3 to the acquired test measurement positions (x T , y T , z T ) and stores them in the memory 52.
  • step S3 the processor 50 changes the posture of the visual sensor 14 by rotating and moving the visual sensor 14.
  • the processor 50 first updated the direction DR 1 (posture change direction) for moving the visual sensor 14 in order to change the posture of the visual sensor 14 in step S31, and updated the origin position in step S25. It is defined as the direction around the z-axis of the sensor coordinate system C3. Since the origin position of the sensor coordinate system C3 in the MIF coordinate system C2 at this point is the trial measurement position (x T , y T , z T ), the z axis of the sensor coordinate system C3 is the trial measurement position (x T, y T, z T). It is an axis arranged at x T , y T , z T ) and parallel to the direction of the line of sight O. In this way, the processor 50 determines the attitude change direction DR 1 based on the test measurement position (x T , y T , z T).
  • the processor 50 operates the robot 12 to change the attitude of the visual sensor 14 from the initial position PS 0 and the initial attitude OR 0 to the attitude change direction DR 1 (direction around the z-axis of the sensor coordinate system C3).
  • the amount ⁇ 3 the amount of change in the second posture
  • step S32 the processor 50 operates the visual sensor 14 to image the index ID in the same manner as in step S12 described above, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 acquires the image data JD 6 of the index ID by the visual sensor 14 arranged at the position PS 6 and the posture OR 3 , and the coordinates of the intersection F of the index ID reflected in the image data JD 6 ( acquiring the x 6, y 6) and size iS 6.
  • the processor 50 uses the acquired coordinates (x 6 , y 6 ) and size IS 6 with respect to the visual sensor 14 when the image data JD 6 is imaged using the above equations (1) to (3).
  • the relative position data (X 6 , Y 6 , Z 6 ) of the index ID is acquired.
  • the processor 50 returns the visual sensor 14 to the initial position PS 0 and the initial posture OR 0 by the robot 12.
  • step S33 the processor 50 changes the posture of the visual sensor 14 by rotating and moving the visual sensor 14. Specifically, the processor 50 first uses the test measurement position (x T , y T , z T ) and the relative position data (X 0 , Y 0 , Z 0 ) acquired in step S12 described above. , Determine the posture reference position RP.
  • the processor 50 determines the test measurement position (x) of the origin of the sensor coordinate system C3 in the MIF coordinate system C2 set in the above-mentioned step S11 (that is, the initial position PS 0 and the initial posture OR 0).
  • Position the attitude reference position RP is determined.
  • the relative position of the attitude reference position RP with respect to the test measurement position (x T , y T , z T ) in the MIF coordinate system C2 of the initial position PS 0 and the initial attitude OR 0 is , It becomes the same as the relative position (X 0 , Y 0 , Z 0 ) of the index ID with respect to the visual sensor 14 when the image data JD 0 is imaged in step S12.
  • the attitude reference position RP can be arranged in the vicinity of the intersection G of the index ID.
  • the processor 50 sets the reference coordinate system C5 in the MIF coordinate system C2 at this time point (that is, the initial position PS 0 and the initial posture OR 0). Specifically, the processor 50 has the reference coordinate system C5 whose origin is arranged at the posture reference position RP, and the posture (direction of each axis) is the posture (W, P, R) acquired in step S19 described above. ) Is set in the MIF coordinate system C2. Therefore, the x-axis, y-axis, and z-axis directions of the reference coordinate system C5 are parallel to the x-axis, y-axis, and z-axis of the sensor coordinate system C3, respectively.
  • the processor 50 sets the direction DR 2 (posture change direction) in which the visual sensor 14 is moved in order to change the posture of the visual sensor 14 in step S33 around the x-axis or y-axis of the reference coordinate system C5. Determined as the direction.
  • the x-axis or y-axis of the reference coordinate system C5 is an axis arranged at the attitude reference position RP and orthogonal to the direction of the line of sight O.
  • the processor 50 determines the attitude reference position RP based on the test measurement position (x T , y T , z T ), and the attitude change direction is based on the reference coordinate system C5 set in the reference position RP.
  • DR 2 is defined.
  • the processor 50 operates the robot 12 to move the visual sensor 14 from the initial position PS 0 and the initial attitude OR 0 to the attitude change direction DR 2 (direction around the x-axis or y-axis of the reference coordinate system C5).
  • the posture change amount ⁇ 4 second posture change amount
  • step S34 the processor 50 operates the visual sensor 14 to image the index ID in the same manner as in step S12 described above, and acquires the relative position of the index ID with respect to the visual sensor 14 at this time. Specifically, the processor 50 acquires the image data JD 7 of the index ID by the visual sensor 14 arranged at the position PS 7 and the posture OR 4 , and the coordinates of the intersection F of the index ID reflected in the image data JD 7 ( acquiring the x 7, y 7) and size iS 7.
  • the processor 50 uses the acquired coordinates (x 7 , y 7 ) and size IS 7 with respect to the visual sensor 14 when the image data JD 7 is imaged using the above equations (1) to (3).
  • the relative position data (X 7 , Y 7 , Z 7 ) of the index ID is acquired.
  • processor 50 is a visual sensor based on relative position data (X 0 , Y 0 , Z 0 ), (X 6 , Y 6 , Z 6 ), and (X 7 , Y 7 , Z 7).
  • the z-axis of the sensor coordinate system C3 (x T , y T , z T ) from the test measurement position (x T, y T, z T) in the MIF coordinate system C2 acquired in step S25 to the origin position of the accurate sensor coordinate system C3. That is, assuming that the vector in the plane orthogonal to the line of sight O) is ( ⁇ X 2 , ⁇ Y 2 ), the following equation (6) holds.
  • the accurate sensor coordinate system is obtained from the attitude reference position RP (x T + X 0 , y T + Y 0 , z T + Z 0 ) in the MIF coordinate system C2 (that is, the origin position of the reference coordinate system C5 set in step S34).
  • the vector in the direction of the z-axis (that is, the line of sight O) of the sensor coordinate system C3 up to the origin position of C3 is ⁇ Z 2 .
  • the processor 50 can obtain the vector ( ⁇ X 2 , ⁇ Y 2 ) and the vector ⁇ Z 2 in the MIF coordinate system C2 by solving the above equations (6) and (7).
  • This vector ( ⁇ X 2 , ⁇ Y 2 ) indicates the exact position of the line of sight O in the MIF coordinate system C2 (in other words, the position of the origin of the sensor coordinate system C3 in the plane orthogonal to the line of sight O).
  • the vector ⁇ Z 2 indicates an accurate position of the visual sensor 14 (or the origin of the sensor coordinate system C3) in the MIF coordinate system C2 in the direction along the line of sight O.
  • the processor 50 sets the main measurement position (x R , y R , z R ) by the posture change amounts ⁇ 3 and ⁇ 4 , and before the posture is changed (that is, the initial posture OR 0 ).
  • Relative position data (X 0 , Y 0 , Z 0 ) when the image data JD 0 was imaged, and image data JD 6 , JD 7 after changing the posture (that is, posture OR 3 , OR 4). Is acquired based on the relative position data (X 6 , Y 6 , Z 6 ) and (X 7 , Y 7 , Z 7 ) when the image was taken.
  • the processor 50 determines the coordinates of the origin of the sensor coordinate system C3 in the MIF coordinate system from the trial measurement positions (x T , y T , z T ) estimated in step S25 to the main measurement positions (x R , y R , z R). ), And stored in the memory 52.
  • This measurement position (x R , y R , z R ) indicates the position of the visual sensor 14 in the MIF coordinate system (specifically, the origin coordinate of the sensor coordinate system C3) with high accuracy, and is the MIF coordinate.
  • the positional relationship between the system C2 and the sensor coordinate system C3 is shown.
  • the sensor coordinate system C3 can be calibrated with respect to the control coordinate system (robot coordinate system C1 and MIF coordinate system C2), and the control device 16 can recognize the position and orientation of the visual sensor 14 in the control coordinate system. .. Therefore, the control device 16 acquires the position of the work in the robot coordinate system C1 based on the image data of the work (not shown) captured by the visual sensor 14, and makes the work by the end effector attached to the hand of the robot 12. On the other hand, it is possible to work accurately.
  • the processor 50 changes the posture of the visual sensor 14 by the first posture change amounts ⁇ 1 and ⁇ 2 in the trial measurement process of step S2, and the control coordinate system ( The trial measurement position (x T , y T , z T ) of the visual sensor 14 in the MIF coordinate system C2) is estimated, and in the main measurement process of step S3, the posture of the visual sensor 14 is changed to a larger amount of posture change ⁇ 2 ,
  • the measurement position (x R , y R , z R ) is obtained by changing only ⁇ 4.
  • the posture of the visual sensor 14 in the control coordinate system is increased in the first measurement process. It is necessary to change the posture change amounts ⁇ 2 and ⁇ 4. This is because the measurement accuracy of the position of the visual sensor 14 in the control coordinate system is lowered unless the posture of the visual sensor 14 is changed significantly.
  • the index ID may deviate from the field of view of the visual sensor 14 after the posture change, and the index ID may not be imaged.
  • the process of measuring the position of the visual sensor 14 in the control coordinate system is divided into a trial measurement process and the main measurement process, and in steps S21 and S23 of the trial measurement process, the posture of the visual sensor 14 is determined.
  • the relatively small first posture change amounts ⁇ 1 and ⁇ 2 are changed.
  • the test measurement position (x T , y T , z T ) of the visual sensor 14 can be roughly estimated while preventing the index ID from deviating from the visual field of the visual sensor 14 after the posture change.
  • the posture of the visual sensor 14 is set to the posture change directions DR 1 and DR 2 determined based on the test measurement positions (x T , y T , z T ) in steps S31 and S33.
  • the larger second posture change amounts ⁇ 3 and ⁇ 4 are changed.
  • the processor 50 determines the posture reference position RP based on the test measurement position (x T , y T , z T ) in the above-mentioned step S33, and places the reference at the posture reference position RP.
  • the direction around the x-axis or y-axis of the coordinate system C5 is defined as the attitude change direction DR 2. According to this configuration, it is possible to more effectively prevent the index ID from being deviated from the field of view of the visual sensor 14 when step S33 is executed.
  • the relative position of the posture reference position RP with respect to the test measurement position (x T , y T , z T ) is the relative position (X) of the index ID with respect to the visual sensor 14 when the image data JD 0 is imaged.
  • the attitude reference position RP is set so as to coincide with 0, Y 0 , Z 0). According to this configuration, the posture reference position RP can be arranged in the vicinity of the intersection G of the index IDs, so that the index IDs are further prevented from being out of the visual field of the visual sensor 14 when step S33 is executed. Can be effectively prevented.
  • the processor 50 the relative position data (X n, Y n, Z n) acquires, said relative position data (X n, Y n, Z n) attempts measurement position on the basis of the ( x T , y T , z T ) and this measurement position (x R , y R , z R ) have been acquired.
  • a process of aligning the position (coordinates of the sensor coordinate system C3) of the index ID (intersection point F) in the image data JD n captured by the visual sensor 14 with a predetermined position (for example, the center) is required.
  • the position (test measurement position, main measurement position) of the visual sensor 14 in the control coordinate system can be acquired without any problem. Therefore, the work can be speeded up.
  • the processor 50 may set the reference coordinate system C4 with respect to the robot coordinate system C1 so that its origin is arranged at the origin of the robot coordinate system C1. Even in this case, the processor 50 can obtain the trial measurement position and the main measurement position by changing the above equations (4) to (7) according to the origin position of the reference coordinate system C4.
  • the robot coordinate system C1 and the interface coordinate system C2 are exemplified as the control coordinate system.
  • the control coordinate system other coordinate systems such as the world coordinate system C6, the work coordinate system C7, and the user coordinate system C8 may be set.
  • the world coordinate system C6 is a coordinate system that defines the three-dimensional space of the work cell in which the robot 12 works, and is fixed to the robot coordinate system C1.
  • the work coordinate system C7 is a coordinate system that defines the position and orientation of the work to be worked on by the robot 12 in the robot coordinate system C1 (or world coordinate C7).
  • the user coordinate system C8 is a coordinate system arbitrarily set by the operator for controlling the robot 12.
  • the operator can set the user coordinate system C8 to a known position and orientation of the MIF coordinate system C2. That is, the origin of the user coordinate system C8 in this case is arranged at the known coordinates (x C , y C , z C) in the MIF coordinate system C2.
  • the origin of the user coordinate system C8 is located at the center of the light receiving surface (or optical lens) of the image sensor of the visual sensor 14, that is, the origin of the sensor coordinate system C3 is located with respect to the origin of the MIF coordinate system C2. It is set with respect to the MIF coordinate system C2 so that the position is close to the position to be performed.
  • the position of the center of the light receiving surface (or optical lens) of the image sensor of the visual sensor 14 with respect to the center of the mounting surface 34a where the origin of the MIF coordinate system C2 is arranged is the specification of the visual sensor 14 and the robot 12 It can be estimated from information such as the mounting position of the visual sensor 14 with respect to (wrist flange 34). Alternatively, even if the operator acquires the design value of the position of the center of the light receiving surface of the image sensor of the visual sensor 14 with respect to the center of the mounting surface 34a from the drawing data (CAD data or the like) of the visual sensor 14 and the robot 12, for example. good.
  • drawing data CAD data or the like
  • the operator arranges the origin of the user coordinate system C8 at the center of the light receiving surface (or optical lens) of the image sensor of the visual sensor 14.
  • the coordinates (x C , y C , z C ) of the user coordinate system C8 are set in advance.
  • the processor 50 arranges the reference coordinate system C4 at the origin (x C , y C , z C ) of the user coordinate system C8, and its posture (direction of each axis). ) May be set in the MIF coordinate system C2 so as to match the posture (W, P, R) acquired in step S19.
  • the processor 50 may rotate the visual sensor 14 around the z-axis of the reference coordinate system C4 by the operation of the robot 12.
  • the processor 50 may also rotate the visual sensor 14 around the x-axis or y-axis of the reference coordinate system C4 in step S23.
  • the origin of the reference coordinate system C4 can be arranged at a position close to the exact position (x R , y R , z R ) of the origin of the sensor coordinate system C3, so that the visual sensor 14 can be arranged in steps S21 and S23. It is possible to effectively prevent the index ID from deviating from the field of view of.
  • the robot 12 may move the index ID with respect to the visual sensor 14.
  • FIG. 9 The robot system 10'shown in FIG. 9 is different from the robot system 10 described above in the arrangement of the visual sensor 14 and the index ID.
  • the processor 50 of the teaching device 18 can acquire the position of the visual sensor 14 in the control coordinate system by executing the flows shown in FIGS. 4, 5, 7, and 8.
  • step S11 the processor 50 operates the robot 12 to position the index ID (ie, the wrist flange 34) at the initial position PS 0 and the initial posture OR 0 with respect to the visual sensor 14. .. At this time, the index ID enters the field of view of the visual sensor 14.
  • step S12 the processor 50 images the index ID by the visual sensor 14 to acquire the image data JD 0, and acquires the relative position data (X 0 , Y 0 , Z 0 ) of the index ID with respect to the visual sensor 14.
  • step S13 the processor 50 translates the index ID from the initial position PS 0 and the initial posture OR 0 in the x-axis direction of the robot coordinate system C1 by a predetermined distance ⁇ x.
  • step S14 the processor 50 images the index ID by the visual sensor 14 to acquire the image data JD 1, and acquires the relative position data (X 1 , Y 1 , Z 1 ) of the index ID with respect to the visual sensor 14.
  • step S15 the processor 50 translates the index ID from the initial position PS 0 and the initial posture OR 0 in the y-axis direction of the robot coordinate system C1 by a predetermined distance ⁇ y.
  • step S16 the processor 50 images the index ID by the visual sensor 14 to acquire the image data JD 2, and acquires the relative position data (X 2 , Y 2 , Z 2 ) of the index ID with respect to the visual sensor 14.
  • step S17 the processor 50 translates the index ID from the initial position PS 0 and the initial posture OR 0 in the z-axis direction of the robot coordinate system C1 by a predetermined distance ⁇ z.
  • step S18 the processor 50 images the index ID by the visual sensor 14 to acquire the image data JD 3, and acquires the relative position data (X 3 , Y 3 , Z 3 ) of the index ID with respect to the visual sensor 14.
  • the processor 50 changes the posture of the index ID by rotating the index ID. Specifically, the processor 50 first arranges the reference coordinate system C4 at the MIF coordinate system C2 at this time point (initial position PS 0 and initial posture OR 0 ), and its origin is arranged at the origin of the MIF coordinate system C2. , The posture (direction of each axis) is set so as to match the posture (W, P, R) acquired in step S19. The processor 50 then operates the robot 12 to move the index ID from the initial position PS 0 and the initial posture OR 0 around the z-axis of the reference coordinate system C4 (ie, the axis parallel to the direction of the line of sight O). , it is rotated by the posture change amount theta 1.
  • step S22 the processor 50 operates the visual sensor 14 to image the index ID, and acquires the relative position data (X 4 , Y 4 , Z 4) of the index ID with respect to the visual sensor 14 at this time.
  • step S23 the processor 50 operates the robot 12 to shift the index ID from the initial position PS 0 and the initial posture OR 0 to the x-axis or y-axis of the reference coordinate system C4 (that is, orthogonal to the direction of the line of sight O). Around the axis), the posture change amount ⁇ 2 is rotated.
  • step S24 the processor 50 operates the visual sensor 14 to image the index ID, and acquires the relative position data (X 5 , Y 5 , Z 5) of the index ID with respect to the visual sensor 14 at this time.
  • step S25 the processor 50 acquires the test measurement position of the visual sensor 14. Specifically, the processor 50 uses the relative position data (X 0 , Y 0 , Z 0 ), (X 4 , Y 4 , Z 4 ), and (X 5 , Y 5 , Z 5 ) and the above equations. Using (4) and (5), a vector ( ⁇ X 1 , ⁇ Y 1 , ⁇ Z 1 ) from the origin of the reference coordinate system C4 in the MIF coordinate system C2 to the origin of the unknown sensor coordinate system C3 is calculated.
  • the processor 50 determines the position of the visual sensor 14 (origin of the sensor coordinate system C3) from the vector ( ⁇ X 1 , ⁇ Y 1 , ⁇ Z 1 ) by the coordinates (x T , y T , z T ) of the MIF coordinate system C2.
  • the coordinates (x T ', y T ', z T ') obtained by converting the coordinates (x T , y T , z T ) of the MIF coordinate system C2 into the robot coordinate system C1 are obtained as the robot coordinate system C1. It is acquired as a test measurement position of the visual sensor 14 in.
  • This trial measurement position (x T ', y T ', z T ') indicates the approximate position of the visual sensor 14 in the robot coordinate system C1.
  • step S31 the processor 50 changes the posture of the index ID by rotating the index ID. Specifically, the processor 50 sets the direction DR 1 (posture change direction) for moving the index ID in order to change the posture of the index ID in step S31, and the sensor coordinate system C3 whose origin position is updated in step S25. Is defined as the direction around the z-axis of.
  • the z axis of the sensor coordinate system C3 is the test. It is an axis parallel to the direction of the line of sight O, which is arranged at the measurement position (x T , y T ', z T').
  • the processor 50 determines the attitude change direction DR 1 based on the test measurement positions (x T ', y T ', z T').
  • the processor 50 operates the robot 12 to shift the index ID from the initial position PS 0 and the initial posture OR 0 to the posture change direction DR 1 (direction around the z-axis of the sensor coordinate system C3). Rotate by ⁇ 3 (second attitude change amount).
  • step S32 the processor 50 operates the visual sensor 14 to image the index ID, and acquires the relative position data (X 6 , Y 6 , Z 6) of the index ID with respect to the visual sensor 14 at this time.
  • step S33 the processor 50 changes the posture of the index ID by rotating the index ID.
  • the processor 50 first sets the direction DR 2 (posture change direction) for moving the index ID in order to change the posture of the index ID in step S33, and the sensor coordinates whose origin position is updated in step S25. It is defined as the direction around the x-axis or y-axis of the system C3. Since the origin position of the sensor coordinate system C3 in the robot coordinate system C1 at this point is the trial measurement position (x T ', y T ', z T '), the x-axis or y-axis of the sensor coordinate system C3 is , The axis orthogonal to the line of sight O, which is arranged at the test measurement position (x T , y T ', z T').
  • the processor 50 determines the attitude change direction DR 2 based on the test measurement position (x T ', y T ', z T').
  • the processor 50 operates the robot 12 to shift the index ID from the initial position PS 0 and the initial attitude OR 0 to the attitude change direction DR 2 (direction around the x-axis or y-axis of the sensor coordinate system C3).
  • the attitude change amount ⁇ 4 (second attitude change amount) is rotated.
  • step S34 the processor 50 operates the visual sensor 14 to image the index ID, and acquires the relative position data (X 7 , Y 7 , Z 7) of the index ID with respect to the visual sensor 14 at this time.
  • step S35 the processor 50 acquires the main measurement position of the visual sensor 14.
  • the processor 50 uses the relative position data (X 0 , Y 0 , Z 0 ), (X 6 , Y 6 , Z 6 ), and (X 7 , Y 7 , Z 7 ) and the above equations. From the test measurement position (x T ', y T ', z T ') in the robot coordinate system C1 obtained in step S25 using (6) and (7) to the origin of the accurate sensor coordinate system C3. Calculate the vector ( ⁇ X 2 , ⁇ Y 2 , ⁇ Z 2).
  • the processor 50 sets the position of the visual sensor 14 (origin of the sensor coordinate system C3) in the robot coordinate system C1 from the vector ( ⁇ X 2 , ⁇ Y 2 , ⁇ Z 2 ) to the present measurement position (x R ', y R '. , Z R ').
  • the processor 50 acquires the trial measurement position (x T ', y T ', z T ') and the main measurement position (x R ', y R ', z R '). do.
  • the index ID similarly to the above-described embodiment, it is possible to prevent the index ID from being deviated from the field of view of the visual sensor 14 in steps S21, S23, S31, and S33.
  • the processor 50 includes the relative position data (X 0 , Y 0 , Z 0 ) and (X 6 , Y 6 , Z 6 ) and the above equation (6).
  • the vector ( ⁇ X 2 , ⁇ Y 2 ) is obtained by using, and the main measurement position (x R , y R ) of the line of sight O in the MIF coordinate system C2 in the MIF coordinate system C2 is obtained from the vector ( ⁇ X 2 , ⁇ Y 2). You may get it.
  • the processor 50 sets the trial measurement position (x T , y T , z T ) to the trial measurement position (x R , y R , z T ) according to the main measurement position (x R , y R) of the line of sight O. Update.
  • the processor 50 receives the updated test measurement position (x R , y R , z T ) and the relative position data (X 0 , Y 0 , Z 0 ) acquired in step S12. Use to determine the attitude reference position RP.
  • processor 50 in the starting position PS 0 and MIF coordinate system C2 of the initial posture OR 0, trial measurement position of the updated (x R, y R, z T) from the vector (X 0, Y 0 , Z 0 ), the attitude reference position RP is set at a position (that is, the position of the coordinates (x R + X 0 , y R + Y 0 , z T + Z 0) of the MIF coordinate system C2).
  • the coordinates (x R , y R ) of the updated trial measurement positions (x R , y R , z T ) indicate the exact position of the line of sight O in the MIF coordinate system.
  • the posture reference position RP can be set more accurately at the intersection F of the index ID. Therefore, it is possible to more effectively prevent the index ID from deviating from the field of view of the visual sensor 14 in step S33.
  • steps S21, S23, S31 and S33 are executed with the initial position PS 0 and the initial posture OR 0 as the starting points
  • the visual sensor 14 captures an image of the initial position PS 0 and initial posture oR 0 and it is arranged in a different second initial position PS 0_2 and the second initial posture oR 0_2 indicators ID, image data Relative position data (X 0_2 , Y 0_2 , Z 0_2 ) may be acquired based on.
  • the processor 50 acquires the trial measurement position or the main measurement position based on the relative position data (X 0_2 , Y 0_2 , Z 0_2).
  • processor 50 based on the relative position (X n, Y n, Z n), has described the case of obtaining the position of the visual sensor 14 in the control coordinate system.
  • the concept of the present invention can also be applied to a form of acquiring the position of the visual sensor 14 in the control coordinate system by a method as described in Patent Documents 1 and 2, for example.
  • the processor 50 captures the index ID with the visual sensor 14 while moving the visual sensor 14 or the index ID by the robot 12, and the position of the index ID (intersection point F) in the captured image data JD n (sensor coordinate system C3).
  • the alignment process PP for aligning the coordinates) with a predetermined position is executed.
  • the processor 50 acquires the coordinate CD 1 (initial position) of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at the time when the alignment process PP is completed.
  • the processor 50 translates the visual sensor 14 or the index ID from the initial position, then images the index ID again with the visual sensor 14 and executes the above-mentioned alignment process PP, and the robot coordinates at this time.
  • the coordinate CD 2 of the origin of the MIF coordinate system C2 in the system C1 is acquired.
  • the processor 50 acquires the direction (that is, the posture) of the line of sight O of the visual sensor 14 in the robot coordinate system C1 from the coordinates CD 1 and CD 2.
  • the processor 50 rotates the visual sensor 14 or the index ID from the initial position in the direction around the axis parallel to the direction of the acquired line of sight O by the amount of change in attitude ⁇ 1 , and then visually.
  • the index ID is imaged by the sensor 14, and the above-mentioned alignment process PP is executed.
  • the processor 50 acquires the coordinate CD 3 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this time.
  • the processor 50 obtains the position TP 1 of the line of sight O in the robot coordinate system C1 from the coordinates CD 1 and CD 3.
  • the processor 50 rotates the visual sensor 14 or the index ID from the initial position in the direction around the axis orthogonal to the line of sight O arranged at the position TP 1 by the amount of change in attitude ⁇ 2.
  • the index ID is imaged by the visual sensor 14, the above-mentioned alignment process PP is executed, and the coordinate CD 4 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this time is acquired.
  • the processor 50 obtains the position TP 2 of the visual sensor 14 (the origin of the sensor coordinate system C3) in the robot coordinate system C1 in the direction along the line of sight O from the coordinates CD 1 and CD 4. From these positions TP 1 and TP 2 , the test measurement positions (x T ', y T ', z T ') of the visual sensor 14 (origin of the sensor coordinate system C3) in the robot coordinate system C1 can be obtained.
  • the processor 50 determines the posture change direction as the direction around the axis parallel to the direction of the line of sight O arranged at the test measurement position (x T ', y T ', z T'), and visually. After rotating the sensor 14 or the index ID from the initial position in the posture change direction by the posture change amount ⁇ 3 (> ⁇ 1 ), the index ID is imaged by the visual sensor 14 and the above-mentioned alignment process PP is performed. Run. Then, the processor 50 acquires the coordinate CD 5 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this time, and obtains the position TP 3 of the line of sight O in the robot coordinate system C1 from the coordinates CD 1 and CD 5 .
  • the processor 50 determines the posture change direction as the direction around the axis orthogonal to the line of sight O arranged at the test measurement position (x T ', y T ', z T'), and the visual sensor 14 Alternatively, the index ID is rotated from the initial position in the posture change direction by the posture change amount ⁇ 4 (> ⁇ 2 ), and then the above-mentioned alignment process PP is executed. Then, the processor 50 acquires the coordinate CD 6 of the origin of the MIF coordinate system C2 in the robot coordinate system C1 at this time.
  • the processor 50 obtains the position TP 4 of the visual sensor 14 (the origin of the sensor coordinate system C3) in the robot coordinate system C1 in the direction along the line of sight O from the coordinates CD 1 and CD 6. From these positions TP 3 and TP 4 , the main measurement position (x R ', y R ', z R ') of the visual sensor 14 (origin of the sensor coordinate system C3) in the robot coordinate system C1 can be acquired.
  • the processor 50 changes the posture with the image data of the index ID (image data captured by the alignment process PP for obtaining the initial position) captured by the visual sensor 14 before changing the posture.
  • the position of the visual sensor 14 in the control coordinate system is determined based on the image data of the index ID (image data captured by the alignment process PP for obtaining the coordinates CD 3 , CD 4 and CD 5) later captured by the visual sensor 14. Have acquired. Also by this method, the processor 50 can acquire the position (test measurement position, main measurement position) of the visual sensor 14 in the control coordinate system.
  • the teaching device 18 acquires the data of the position and the posture of the visual sensor 14 in the control coordinate system.
  • the control device 16 may acquire data on the position and orientation of the visual sensor 14 in the control coordinate system.
  • the processor 40 of the control device 16 executes the flow shown in FIG. 4 according to the computer program CP.
  • a device other than the teaching device 18 and the control device 16 may acquire data on the position and orientation of the visual sensor 14 in the control coordinate system.
  • the other device comprises a processor, which executes the flow shown in FIG. 4 according to the computer program CP.
  • the index ID is not limited to the artificial pattern as in the above-described embodiment, and for example, a hole, an edge, an uneven portion, a tip, or the like formed in the holding structure B or the wrist flange 34 can be visually recognized. Any visual feature may be used as an index.
  • the robot 12 is not limited to a vertical articulated robot, and may be any type of robot such as a horizontal articulated robot or a parallel link robot that can relatively move the visual sensor 14 and the index ID.

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Manipulator (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2021/014676 2020-04-13 2021-04-06 ロボットの制御座標系における視覚センサの位置を取得する装置、ロボットシステム、方法、及びコンピュータプログラム Ceased WO2021210456A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
DE112021002301.2T DE112021002301T5 (de) 2020-04-13 2021-04-06 Vorrichtung zum erhalten einer position eines visuellen sensors im steuerkoordinatensystem eines roboters, robotersystem, verfahren und computerprogramms
JP2022515324A JP7817152B2 (ja) 2020-04-13 2021-04-06 ロボットの制御座標系における視覚センサの位置を取得する装置、ロボットシステム、方法、及びコンピュータプログラム
US17/918,326 US20230339117A1 (en) 2020-04-13 2021-04-06 Device for obtaining position of visual sensor in control coordinate system of robot, robot system, method, and computer program
CN202180027827.XA CN115397634B (zh) 2020-04-13 2021-04-06 取得机器人的控制坐标系中的视觉传感器的位置的装置、机器人系统、方法以及计算机程序

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-071864 2020-04-13
JP2020071864 2020-04-13

Publications (1)

Publication Number Publication Date
WO2021210456A1 true WO2021210456A1 (ja) 2021-10-21

Family

ID=78083921

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/014676 Ceased WO2021210456A1 (ja) 2020-04-13 2021-04-06 ロボットの制御座標系における視覚センサの位置を取得する装置、ロボットシステム、方法、及びコンピュータプログラム

Country Status (5)

Country Link
US (1) US20230339117A1 (https=)
JP (1) JP7817152B2 (https=)
CN (1) CN115397634B (https=)
DE (1) DE112021002301T5 (https=)
WO (1) WO2021210456A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114986522A (zh) * 2022-08-01 2022-09-02 季华实验室 一种机械臂的定位方法、抓取方法、电子设备及存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024164288A1 (en) * 2023-02-10 2024-08-15 Abb Schweiz Ag Method and system for calibrating sensor with respect to robot
CN121492075B (zh) * 2026-01-14 2026-04-07 上海集迦电子科技有限公司 机械手的偏移校准方法和系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH054180A (ja) * 1991-06-27 1993-01-14 Toyota Autom Loom Works Ltd ハンドアイ付き多軸ロボツトの座標系整合方法
JP2018001332A (ja) * 2016-06-30 2018-01-11 セイコーエプソン株式会社 ロボット、制御装置、及びロボットシステム
JP2019014031A (ja) * 2017-07-11 2019-01-31 セイコーエプソン株式会社 ロボットの制御装置、ロボット、ロボットシステム、並びに、ロボット用カメラの校正方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003211381A (ja) * 2002-01-16 2003-07-29 Denso Wave Inc ロボット制御装置
JP4021413B2 (ja) 2004-01-16 2007-12-12 ファナック株式会社 計測装置
JP4191080B2 (ja) 2004-04-07 2008-12-03 ファナック株式会社 計測装置
JP4967858B2 (ja) 2007-06-29 2012-07-04 日産自動車株式会社 カメラとロボット間のキャリブレーション方法及びその装置
JP5574805B2 (ja) * 2010-04-28 2014-08-20 株式会社ダイヘン 視覚センサを有するマニピュレータのセンサキャリブレーション方法及びロボット制御システム
JP5516974B2 (ja) * 2010-07-07 2014-06-11 株式会社Ｉｈｉ 視覚センサのマウント装置と方法
CN102350700A (zh) * 2011-09-19 2012-02-15 华南理工大学 一种基于视觉的机器人控制方法
JP5845212B2 (ja) * 2013-06-28 2016-01-20 ファナック株式会社 視覚センサ及び力センサを備えたバリ取り装置
JP6429473B2 (ja) * 2014-03-20 2018-11-28 キヤノン株式会社 ロボットシステム、ロボットシステムの校正方法、プログラム、およびコンピュータ読み取り可能な記録媒体
US10551821B2 (en) * 2016-06-30 2020-02-04 Seiko Epson Corporation Robot, robot control apparatus and robot system
JP6568165B2 (ja) * 2017-08-14 2019-08-28 ファナック株式会社 ロボットシステム及びロボット制御装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH054180A (ja) * 1991-06-27 1993-01-14 Toyota Autom Loom Works Ltd ハンドアイ付き多軸ロボツトの座標系整合方法
JP2018001332A (ja) * 2016-06-30 2018-01-11 セイコーエプソン株式会社 ロボット、制御装置、及びロボットシステム
JP2019014031A (ja) * 2017-07-11 2019-01-31 セイコーエプソン株式会社 ロボットの制御装置、ロボット、ロボットシステム、並びに、ロボット用カメラの校正方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114986522A (zh) * 2022-08-01 2022-09-02 季华实验室 一种机械臂的定位方法、抓取方法、电子设备及存储介质
CN114986522B (zh) * 2022-08-01 2022-11-08 季华实验室 一种机械臂的定位方法、抓取方法、电子设备及存储介质

Also Published As

Publication number Publication date
CN115397634B (zh) 2025-09-12
US20230339117A1 (en) 2023-10-26
JPWO2021210456A1 (https=) 2021-10-21
JP7817152B2 (ja) 2026-02-18
DE112021002301T5 (de) 2023-03-23
CN115397634A (zh) 2022-11-25

Similar Documents

Publication Publication Date Title
JP6966582B2 (ja) ロボットモーション用のビジョンシステムの自動ハンドアイ校正のためのシステム及び方法
US8406923B2 (en) Apparatus for determining pickup pose of robot arm with camera
JP4191080B2 (ja) 計測装置
JP5815761B2 (ja) 視覚センサのデータ作成システム及び検出シミュレーションシステム
US11082621B2 (en) Object inspection device, object inspection system and method for adjusting inspection position
US20100232684A1 (en) Calibration apparatus and method for assisting accuracy confirmation of parameter for three-dimensional measurement
JP6869159B2 (ja) ロボットシステム
JP2018012184A (ja) 制御装置、ロボットおよびロボットシステム
WO2021210456A1 (ja) ロボットの制御座標系における視覚センサの位置を取得する装置、ロボットシステム、方法、及びコンピュータプログラム
US20170339335A1 (en) Finger camera offset measurement
JP2018051634A (ja) ロボット制御装置、ロボット、ロボットシステムおよび姿勢特定装置
JP2018031701A (ja) 校正方法、および校正装置
JP7481432B2 (ja) ロボットの教示位置を修正するための装置、教示装置、ロボットシステム、教示位置修正方法、及びコンピュータプログラム
JP2009269134A (ja) 視覚検査装置のシミュレーション装置
US20190197676A1 (en) Object inspection system and object inspection method
JP5198078B2 (ja) 計測装置および計測方法
JPH05318361A (ja) 物体操作方式
JP2678002B2 (ja) 視覚付きロボットの座標系較正方法
WO2022124232A1 (ja) 画像処理システム及び画像処理方法
JP2013173191A (ja) ロボット装置、ロボット制御装置、ロボット制御方法及びロボット制御プログラム
JPH0847881A (ja) ロボットの遠隔操作方法
JP7443014B2 (ja) ロボットアーム試験装置
JP7723101B2 (ja) ロボットがワークを把持する位置及び姿勢を教示する装置、ロボットシステム、及び方法
JP7509535B2 (ja) 画像処理装置、ロボットシステム、及び画像処理方法
JP2005186193A (ja) ロボットのキャリブレーション方法および三次元位置計測方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21788516

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022515324

Country of ref document: JP

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 21788516

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 202180027827.X

Country of ref document: CN