EP3974767A1 - Calibration method for computer vision system and three-dimensional reference object for use in same - Google Patents
Calibration method for computer vision system and three-dimensional reference object for use in same Download PDFInfo
- Publication number
- EP3974767A1 EP3974767A1 EP19939259.8A EP19939259A EP3974767A1 EP 3974767 A1 EP3974767 A1 EP 3974767A1 EP 19939259 A EP19939259 A EP 19939259A EP 3974767 A1 EP3974767 A1 EP 3974767A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- dimensional
- reference object
- robot
- measurement
- measurement unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000005259 measurement Methods 0.000 claims abstract description 202
- 230000009466 transformation Effects 0.000 claims abstract description 91
- 239000011159 matrix material Substances 0.000 claims abstract description 51
- 230000008859 change Effects 0.000 claims description 8
- 230000008569 process Effects 0.000 description 9
- 238000001514 detection method Methods 0.000 description 6
- 238000011960 computer-aided design Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000013519 translation Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 2
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004381 surface treatment Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B21/00—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant
- G01B21/02—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant for measuring length, width, or thickness
- G01B21/04—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant for measuring length, width, or thickness by measuring coordinates of points
- G01B21/042—Calibration or calibration artifacts
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Definitions
- the present invention relates to a calibration method for a computer vision system and a three-dimensional (3D) reference object for use with the calibration method.
- a technique for improving alignment accuracy of 3D measurement data that is obtained through 3D measurement of a target object placed in a measurement space uses 3D measurement data obtained through 3D measurement of a 3D object (hereafter also a 3D reference object) placed in the measurement space.
- Patent Literature 1 describes a technique to determine wear of a tire tread over use by accurately aligning the circumferential positions of a tire tread and the positions of the tire in the width direction for 3D measurement data about the tire before and after use.
- the technique uses a 3D reference object including three spheres.
- the 3D reference object is installed and fixed in a measurement space with rods extending from a fixed support that holds a tire in a rotatable manner.
- a coordinate transformation is determined for the coordinates of the centers of the spheres to move to predetermined target coordinates and is used to align 3D measurement data for the tire as a target object.
- Such a 3D reference object can be recognized with sufficiently high accuracy using at least a predetermined number of measurement points.
- a 3D reference object with large spheres is to be used.
- a large 3D reference object may hide a target object or may use a large proportion of area and thus cause a relatively small imaging size, thus disabling measurement of the target object.
- Another 3D reference object includes multiple planes (refer to, for example, Patent Literature 2) at undefined positions in tangent planes, thus possibly disabling accurate recognition of the position and orientation of a measurement viewpoint.
- one or more aspects of the present invention are directed to a 3D reference object that allows more accurate recognition of its position and orientation in 3D measurement, and a technique using the 3D reference object for more accurate calibration of 3D measurement in a computer vision system.
- a calibration method is a method for performing calibration in a computer vision system using a three-dimensional reference object.
- the system includes a three-dimensional measurement unit that performs three-dimensional measurement and a measurement controller that controls the three-dimensional measurement unit.
- the three-dimensional reference object is asymmetric as viewed in any direction and has predetermined dimensions to be a reference for recognition of a position and an orientation in the three-dimensional measurement.
- the method includes measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit, calculating, based on data about a three-dimensional shape of the three-dimensional reference object and three-dimensional measurement data obtained through three-dimensional measurement of the three-dimensional reference object, a position and an orientation of the three-dimensional reference object relative to a measurement unit coordinate system defined for the three-dimensional measurement unit, and calculating, based on the position and the orientation of the three-dimensional reference object relative to the measurement unit coordinate system, a reference-measurement unit transformation matrix representing a coordinate transformation between a reference coordinate system defined for the three-dimensional reference object and the measurement unit coordinate system.
- the 3D reference object shaped asymmetric as viewed in any direction and with predetermined dimensions is used as a reference for recognizing its position and orientation in the 3D measurement.
- This allows accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system defined for the 3D measurement unit.
- More accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system allows accurate calculation of the reference-measurement unit transformation matrix representing a coordinate transformation between the reference coordinate system and the measurement unit coordinate system. This allows accurate calibration for a computer vision system including a 3D measurement unit.
- the computer vision system may further include a robot including an arm, and a robot controller that controls the robot.
- the measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit may include measuring the three-dimensional reference object by three-dimensional measurement by controlling the arm holding the three-dimensional reference object to change a position and an orientation of the three-dimensional reference object relative to the three-dimensional measurement unit.
- the arm holds the 3D reference object.
- This allows calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on multiple sets of 3D measurement data for different positions and orientations of the 3D reference object relative to the 3D measurement unit. This allows more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system. This allows accurate calibration of 3D measurement in the computer vision system.
- the computer vision system may further include a robot including an arm, and a robot controller that controls the robot.
- the measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit may include measuring the three-dimensional reference object by three-dimensional measurement by controlling the arm holding the three-dimensional measurement unit to change a position and an orientation of the three-dimensional measurement unit relative to the three-dimensional reference object.
- the arm holds the 3D measurement unit.
- This allows calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on multiple sets of 3D measurement data for different positions and orientations of the 3D reference object relative to the 3D measurement unit.
- This allows still more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system.
- This allows accurate calibration of 3D measurement in the computer vision system.
- the calibration method may further include calculating, based on the reference-measurement unit transformation matrix, a robot-tool transformation matrix representing a coordinate transformation between a robot coordinate system defined for a basal end of the robot and a tool coordinate system defined for the arm, a tool-reference transformation matrix representing a coordinate transformation between the tool coordinate system and the reference coordinate system, and a robot-measurement unit transformation matrix representing a coordinate transformation between the robot coordinate system and the measurement unit coordinate system.
- the robot-measurement unit transformation matrix representing a coordinate transformation between the robot coordinate system and the measurement unit coordinate system equals the product of the robot-tool transformation matrix, the tool-reference transformation matrix, and the reference-measurement unit transformation matrix.
- the unknown robot-measurement unit transformation matrix can be calculated based on the above relational expression, the reference-measurement unit transformation matrix already calculated, and known information about, for example, a sensor detection value indicating the position and the orientation of the arm of the robot. This allows accurate calibration between the robot and the 3D measurement unit.
- the computer vision system may include a table on which the three-dimensional reference object is placeable to allow at least one of a position or an orientation of the three-dimensional reference object to be changed relative to the three-dimensional measurement unit.
- the measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit may include measuring the three-dimensional reference object by three-dimensional measurement by changing at least one of a position or an orientation of the three-dimensional reference object relative to the three-dimensional measurement unit with the table.
- the table is controlled to change at least the position or the orientation of the 3D reference object relative to the 3D measurement unit. This allows calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on multiple sets of 3D measurement data for different positions and orientations of the 3D reference object relative to the 3D measurement unit. This allows more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system. This allows accurate calibration of 3D measurement in the computer vision system.
- the three-dimensional reference object may have a shape combining three cylinders each having two hemispherical ends in an axial direction with one end of each cylinder overlapping one end of the other cylinders.
- the 3D reference object includes many curved surfaces with large curvatures, thus allowing more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on 3D measurement data from the 3D measurement unit.
- Each transformation matrix calculated as above is thus more accurate. In other words, this structure further allows more accurate calibration of 3D measurement in the computer vision system.
- a three-dimensional reference object according to another aspect of the present invention is an object for use with the calibration method.
- the 3D reference object allows accurate recognition of its position and orientation in 3D measurement. This allows more accurate calibration of 3D measurement in the computer vision system.
- the 3D reference object according to the above aspects of the present invention allows more accurate recognition of its position and orientation in 3D measurement, and the technique according to the above aspects of the present invention allows, using the 3D reference object, more accurate calibration of 3D measurement in a computer vision system.
- FIG. 1A is a view of a three-dimensional (3D) reference object 1 according to an embodiment of the present invention.
- the 3D reference object 1 is shaped asymmetric as viewed in any direction and uniquely determines the orientation of the 3D reference object 1.
- the 3D reference object 1 with known dimensions is prepared and measured by 3D measurement to obtain 3D measurement data.
- the 3D measurement data is processed to allow a 3D measurement unit such as a 3D sensor to accurately recognize the position and the orientation of the 3D reference object 1.
- the 3D reference object 1 for such 3D measurement may be used in the manner described below.
- a calibration board 25 on which the 3D reference object 1 is placed as shown in FIG. 2 is attached to a hand of a robot arm 212.
- the 3D reference object 1 is measured by 3D measurement with a sensor unit 20 that can perform 3D measurement to allow accurate recognition of the position and the orientation of the 3D reference object 1.
- the sensor unit 20 attached to the hand of the robot arm 212 may measure the 3D reference object 1 on a table 28 placed in a workspace by 3D measurement.
- this structure allows accurate recognition of the position and the orientation of the 3D reference object 1. This allows accurate calculation of an unknown transformation matrix for a transformation between the robot and the sensor.
- three 3D reference objects 1-1, 1-2, 1-3, and 1-4 are placed on a turntable 27 circumferentially about the center at intervals of 90 degrees.
- a target object is then placed at the center of the turntable 27 on which the 3D reference objects 1-1 to 1-4 are placed, and is measured by 3D measurement with a sensor.
- the recognition results of the positions and the orientations of the 3D reference objects 1-1 to 1-4 based on the 3D measurement data obtained in this manner are used to restore the 3D shape of the target object.
- the 3D reference objects 1-1 to 1-4 are arranged as above relative to the turntable 27 to prevent the 3D reference objects 1-1 to 1-4 from being hidden by the target object.
- the turntable 27 is rotated with its center as the rotation center to change the positions and the orientations of the 3D reference objects 1-1, 1-2, 1-3, and 1-4 relative to the sensor.
- Four 3D reference objects as at least one 3D reference object 1 are placed on the turntable 27.
- the turntable 27 herein corresponds to a table in an aspect of the present invention.
- 3D reference objects 1-5 to 1-8 may be placed at the four corners of a table 29 on which the target object 30 is placed. This allows accurate recognition of the position and the orientation of the target object 30. This allows accurate alignment between multiple sets of data about 3D measurement from various relative positions.
- FIG. 1A shows the shape of a 3D reference object 1 according to the present embodiment.
- the 3D reference object 1 includes many curved surfaces with large curvatures, is shaped asymmetric as viewed in any direction, and uniquely determines the orientation of the 3D reference object 1. More specifically, the 3D reference object 1 is shaped by combining three objects 10 (refer to FIG. 1B ) each having one end overlapping one end of the other objects 10.
- the object 10 includes a cylinder 10a with a radius of 12 mm and a length of 30 mm and hemispherical surfaces 10b and 10c with a radius of 12 mm located at either end of the cylinder 10a.
- FIG. 1A shows the shape of a 3D reference object 1 according to the present embodiment.
- the 3D reference object 1 includes many curved surfaces with large curvatures, is shaped asymmetric as viewed in any direction, and uniquely determines the orientation of the 3D reference object 1. More specifically, the 3D reference object 1 is shaped by combining three objects 10 (
- the 3D reference object 1 includes a first branch 11, a second branch 12, and a third branch 13.
- the first branch 11 is tilted at an angle of 23 degrees to the direction (y-direction) in which the second branch 12 extends.
- the third branch 13 is tilted at an angle of 40 degrees to z-axis.
- the second branch 12 is tilted at an angle of 7 degrees about x-axis.
- the 3D reference object 1 may be processed to have diffuse reflecting surfaces to facilitate 3D measurement.
- diffuse reflecting surfaces may be achieved by selecting a material or providing surface treatment such as coating and roughing.
- the 3D reference object 1 with a patterned surface may obstruct 3D measurement involving emission of patterned light.
- the 3D reference object 1 may thus have an unpatterned surface.
- the 3D reference object 1 may have a white surface.
- the 3D reference object 1 is shaped to uniquely determine the orientation of the 3D reference object 1 relative to any viewpoint.
- the 3D reference object 1 also has known dimensions. This allows accurate recognition of the position and the orientation of the 3D reference object 1 based on 3D measurement data obtained through 3D measurement of the 3D reference object 1.
- FIG. 4 is a view of a 3D reference object 50 for comparison.
- the 3D reference object 50 is a disk having a hollow at its center as a through-hole in the height direction.
- the 3D reference object 50 and the 3D reference object 1 remaining still are measured by 3D measurement 15 times in a row with a sensor.
- the table below shows standard deviations of the recognized positions and orientations in each direction.
- the x-translation denotes a translation in x-direction
- the x-rotation denotes a rotation about x-axis (the same applies to the other directions and angles).
- Table 1 x-translation (mm) y-translation (mm) z-translation (mm) x-rotation (degree) y-rotation (degree) 3D reference object 1 0.058 0.039 0.033 0.091 0.049 3D reference object 50 0.146 0.212 0.011 0.058 0.059
- the recognition results of the position and the orientation of the 3D reference object 50 vary greatly depending on the direction, whereas the position and the orientation of the 3D reference object 1 can be accurately recognized in any direction.
- a robot vision system 100 in the embodiment of the present invention will be described with reference to FIG. 5 .
- the robot vision system 100 is installed on a production line for, for example, product assembly or processing.
- the robot vision system 100 recognizes, based on data received from the sensor unit 20, the gripping position and orientation of a multi-finger hand or a suction pad gripping an object placed in, for example, a tray.
- the multi-finger hand or the suction pad is attached to a distal end of an arm of a robot 21.
- the robot vision system 100 herein corresponds to a computer vision system in an aspect of the present invention.
- FIG. 5 shows the robot vision system 100 with the hand of an arm 212 receiving the calibration board 25 holding a calibration target 26 for robot-sensor unit calibration to be performed.
- the robot vision system 100 mainly includes the sensor unit 20, an information processor 22, a programmable logic controller (PLC) 24, and the robot 21.
- the sensor unit 20 and the information processor 22 are connected to each other with wires or wirelessly.
- the information processor 22 receives the output from the sensor unit 20.
- the information processor 22 performs various processes using data received from the sensor unit 20. Examples of the processes performed by the information processor 22 may include distance measurement (ranging), 3D shape recognition, object recognition, and scene recognition.
- the process result from the information processor 22 is output to, for example, the PLC 24 or a display 23, and used for controlling the robot 21.
- the sensor unit 20 includes at least a camera for capturing optical images of target objects.
- the sensor unit 20 may include any component (e.g., a sensor, an illuminator, or a projector) to be used for 3D measurement of target objects.
- the sensor unit 20 includes multiple cameras.
- the sensor unit 20 For active stereo that projects a random dot pattern onto a target object, the sensor unit 20 further includes a projector for projecting structured light onto the target objects.
- the sensor unit 20 includes a projector for projecting patterned light and cameras.
- the sensor unit 20 herein is fixed at a predetermined position.
- the sensor unit 20 herein corresponds to a 3D measurement unit in an aspect of the present invention.
- the information processor 22 is, for example, a computer including a central processing unit (CPU), a random-access memory (RAM), a nonvolatile storage (e.g., a hard disk drive, or a solid-state drive or SSD), an input device, and an output device.
- the CPU loads the program stored in the nonvolatile storage into the RAM and executes the program to implement various components described later.
- the information processor 22 may have any other configuration.
- the components may be entirely or partly implemented by a dedicated circuit such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), or by cloud computing or distributed computing.
- FPGA field-programmable gate array
- ASIC application-specific integrated circuit
- the robot 21 includes the arm 212 connected to a robot base 211 with multiple joints in between and having predetermined degrees of freedom. As described above, for gripping a target object, the arm 212 receives a multi-finger hand with multiple fingers or a suction pad at the distal end.
- the robot base 211 is installed on a floor surface or a predetermined installation part.
- the calibration board 25 holding the calibration target 26 is attached to the hand of the arm 212.
- the robot base 211 herein corresponds to a basal end of a robot in an aspect of the present invention.
- the 3D reference object 1 having the shape shown in FIG. 1A and with known dimensions is used as the calibration target 26.
- FIG. 6 is a functional block diagram showing the components of the information processor 22.
- the information processor 22 includes a sensor unit controller 221, a robot control instructor 222, a storage 223, a position-orientation calculator 224, a calibration unit 225, and a calibration result output unit 226.
- the sensor unit controller 221 controls the sensor unit 20 to instruct the sensor unit 20 to, for example, perform 3D measurement and output a 3D measurement result.
- the robot control instructor 222 outputs a control instruction to a robot controller 30 including the PLC 24 and other components, and obtains a detection value of, for example, the rotation angle of a joint in the arm 212 of the robot 21 detected by a sensor and stores the detection value into a predetermined area in the storage 223. During calibration, the robot control instructor 222 outputs control information to the robot controller 30 to move the arm 212 and thus change the position of the 3D reference object 1 relative to the sensor unit 20.
- the storage 223 includes a predetermined area in the nonvolatile storage device, and stores a detection value of, for example, the rotation angle of a joint in the arm 212 of the robot 21 detected by a sensor (described above), and data including dimension data about, for example, links between the robot base 211 and the distal end of the arm 212 and 3D shape data about the 3D reference object 1 (described later).
- the position-orientation calculator 224 calculates the position and the orientation of the 3D reference object 1 based on 3D measurement data obtained through 3D measurement of the 3D reference object 1 by the sensor unit 20 and the 3D shape data about the 3D reference object 1.
- the calibration unit 225 performs a calibration process (described later).
- the calibration result output unit 226 outputs a result of calibration to an external device, or stores the result into a predetermined area in the storage 223.
- a calibration result is used for, for example, recognition of a target object gripping position performed by a robot vision system.
- Robot-sensor unit calibration calculates a transformation matrix representing a coordinate transformation between a coordinate system defined for the robot 21 and a coordinate system defined for the sensor unit 20.
- a camera coordinate system defined for the camera in the sensor unit 20 shown in FIG. 5 is denoted by ⁇ cam
- a target coordinate system defined for the calibration target 26 is denoted by ⁇ cal
- a tool coordinate system defined for the distal end of the arm 212 is denoted by ⁇ tool
- a robot coordinate system defined for the robot base 211 is denoted by ⁇ base.
- a transformation matrix for a transformation from the target coordinate system ⁇ cal to the camera coordinate system ⁇ cam is written as cam H cal .
- a transformation matrix for a transformation from the target coordinate system ⁇ cal to the tool coordinate system ⁇ tool is written as tool H cal .
- a transformation matrix for a transformation from the tool coordinate system ⁇ tool to the robot coordinate system ⁇ base is written as base H tool .
- a transformation matrix for a transformation from the robot coordinate system ⁇ base to the camera coordinate system ⁇ cam is written as cam H base .
- the relations among these transformation matrices are written by Formula 1 below.
- H cal cam H base cam ⁇ H tool base ⁇ H cal tool
- the camera coordinate system ⁇ cam herein corresponds to a measurement unit coordinate system in an aspect of the present invention
- the target coordinate system ⁇ cal corresponds to a reference coordinate system in an aspect of the present invention
- the transformation matrix cam H cal corresponds to a reference-measurement unit transformation matrix in an aspect of the present invention
- the transformation matrix base H tool corresponds to a robot-tool transformation matrix in an aspect of the present invention
- the transformation matrix tool H cal corresponds to a tool-reference transformation matrix in an aspect of the present invention
- the transformation matrix cam H base corresponds to a robot-measurement unit transformation matrix in an aspect of the present invention.
- the sensor unit 20 measures the calibration target 26 placed on the calibration board 25 attached to the hand of the arm 212 of the robot 21 by 3D measurement (step S1).
- a 3D shape model such as 3D computer-aided design (CAD) data about the calibration target 26 is fitted with 3D measurement data obtained through the 3D measurement of the calibration target 26 by the sensor unit 20.
- the position and the orientation of the calibration target 26 in the camera coordinate system ⁇ cam can thus be calculated (step S2).
- the 3D measurement data is obtained through 3D measurement and includes multiple points each having 3D information.
- the 3D measurement data may, as with point cloud data, include multiple points each having a 3D coordinate value in the camera coordinate system, or may include multiple points (pixels) in a two-dimensional image each associated with a depth value (depth information).
- point cloud data is used as the 3D measurement data.
- the position and the orientation of the calibration target 26 in the camera coordinate system ⁇ cam is recognized to identify the positional relationship between the target coordinate system ⁇ cal defined for the calibration target 26 and the camera coordinate system ⁇ cam.
- the transformation matrix cam H cal for a transformation from the target coordinate system ⁇ cal to the camera coordinate system ⁇ cam can thus be calculated (step S3).
- the position and angle of the hand of the arm 212 relative to the robot base 211 can be calculated from dimension data about links and other components between the robot base 211 and the hand of the arm 212 and detection values from a sensor that detects the amount of displacement in each degree of freedom including the rotation angle of each joint.
- the transformation matrix base H tool for a transformation from the tool coordinate system ⁇ tool to the robot coordinate system ⁇ base can be calculated (step S4).
- the attaching position and orientation of the calibration board 25 relative to the hand of the arm 212 are preset.
- the position and the orientation of the calibration target 26 to be placed relative to the calibration board 25 are preset.
- the position of the calibration target 26 relative to the hand of the arm 212 is thus known.
- the transformation matrix tool H cal for a transformation from the target coordinate system ⁇ cal to the tool coordinate system ⁇ tool can also be calculated (step S4).
- the relations among the transformation matrix tool H cal for a transformation from the target coordinate system ⁇ cal to the tool coordinate system ⁇ tool, the transformation matrix base H tool for a transformation from the tool coordinate system ⁇ tool to the robot coordinate system ⁇ base, the transformation matrix cam H cal for a transformation from the target coordinate system ⁇ cal to the camera coordinate system ⁇ cam, and the transformation matrix cam H base for a transformation from the robot coordinate system ⁇ base to the camera coordinate system ⁇ cam are written by Formula 1.
- the position and the orientation of the calibration target 26 are recognized to calculate the transformation matrix cam H cal for a transformation from the target coordinate system ⁇ cal to the camera coordinate system ⁇ cam.
- the attaching position and orientation of the calibration board 25 relative to the hand of the arm 212 are preset.
- the position and the orientation of the calibration target 26 to be placed relative to the calibration board 25 are preset.
- the position of the calibration target 26 relative to the hand of the arm 212 is known.
- the transformation matrix tool H cal for a transformation from the target coordinate system ⁇ cal to the tool coordinate system ⁇ tool can also be calculated.
- the position of the hand of the arm 212 relative to the robot base 211 can be calculated from dimension data about the links between the robot base 211 and the hand of the arm 212 and detection values from a sensor that detects a rotational angle of each joint.
- the transformation matrix base H tool for a transformation from the tool coordinate system ⁇ tool to the robot coordinate system ⁇ base can also be calculated.
- cam H base can be calculated with Formula 1.
- a transformation matrix for a transformation from the robot coordinate system ⁇ base to the camera coordinate system ⁇ cam can be calculated, thus allowing robot-sensor unit calibration (step S5).
- the calibration result is output to an external device or stored into a predetermined area in the storage 223 (step S6).
- the robot 21 is controlled to cause the calibration target 26 to have different positions and orientations relative to the sensor unit 20.
- the sensor unit 20 measures the calibration target 26 in the multiple positions and orientations by 3D measurement.
- the calibration target 26 at different positions and orientations as above has a constant relative relationship with the tool coordinate system ⁇ tool.
- the transformation matrix tool H cal for a transformation from the target coordinate system ⁇ calto the tool coordinate system ⁇ tool can be calculated with multiple Formulas 1 for the calibration target 26 at the different positions and orientations.
- a robot vision system 200 in a second embodiment of the present invention will be described with reference to FIG. 8 .
- the robot vision system 200 herein corresponds to a computer vision system in an aspect of the present invention.
- the sensor unit 20 is attached to the hand of the arm 212 of the robot 21. Except for the attaching position of the sensor unit 20, the robot vision system 200 has the same configuration as the robot vision system 100.
- the 3D reference object 1 is placed on the table 28 in a workspace in which the robot 21 is placed.
- the camera coordinate system ⁇ cam, the target coordinate system ⁇ cal defined for the 3D reference object 1 as a calibration target, the tool coordinate system ⁇ tool, and the robot coordinate system ⁇ base are the same as in the first embodiment.
- the transformation matrices between these coordinate systems, cam H Cal , tool H cal , base H tool , and cam H base are also the same as in the first embodiment.
- Formula 1 also applies to the relations among these transformation matrices.
- robot-sensor unit calibration is performed with the same procedure shown in FIG. 7 .
- the sensor unit 20 is attached to the hand of the arm 212.
- the relative positional relationship between the camera coordinate system ⁇ cam and the tool coordinate system ⁇ tool remains the same independently of the operation of the robot.
- a transformation matrix tool H cam for a transformation from the camera coordinate system to the tool coordinate system may be used for calibration.
- one 3D reference object 1 is placed on the table 28.
- the turntable 27 on which multiple (four in this example) 3D reference objects 1-1, 1-2, 1-3, and 1-4 are placed as calibration targets may be placed in a workspace.
- the multiple 3D reference objects 1-1, 1-2, 1-3, and 1-4 allow more accurate calibration.
- the 3D reference object 1 may be used in the manner described below.
- FIGs. 9A to 9E show 3D measurement data obtained through 3D measurement of the substantially square table 29 from different viewpoints, with the target object 30 placed on the center of the table 29 and 3D reference objects 1-5, 1-6, 1-7, and 1-8 placed at the four corners.
- Three-dimensional measurement of the target object 30 from one viewpoint fails to provide 3D measurement data about all the surfaces of the target object 30, which has a 3D shape.
- Three-dimensional measurement data about portions of the target object 30 shadowed when viewed from the camera viewpoint cannot be obtained.
- 3D measurement with the camera facing the table 29 provides 3D measurement data about a front surface of the target object 30 (with a front surface facing toward the front of the page of FIG. 9A , and upper, lower, right side, and left side surfaces of the target object 30 being defined based on FIG. 9A ) without providing 3D data about the upper, lower, right side, and left side surfaces of the target object 30.
- FIGs. 9B to 9E respectively show 3D measurement data obtained through 3D measurement with the camera from viewpoints facing the lower surface, upper surface, right side surface, and left side surface of the target object 30.
- the relative position of the target object 30 relative to the viewpoint of the camera is changed to obtain 3D measurement data about various portions of the target object 30.
- Three-dimensional measurement data obtained in this manner through 3D measurement of the target object 30 from different viewpoints allows more accurate restoration of the 3D shape of the target object 30.
- accurate alignment between the multiple sets of the 3D measurement data is to be performed.
- the position and the orientation of the target object 30 in the 3D measurement of the target object 30 are to be accurately recognized.
- the 3D reference objects 1-5, 1-6, 1-7, and 1-8 are each shaped asymmetric as viewed in any direction and uniquely determine their orientation, and have known dimensions. The positions and the orientations of the 3D reference objects 1-5, 1-6, 1-7, and 1-8 used in 3D measurement can thus be recognized accurately.
- the 3D reference objects 1-5, 1-6, 1-7, and 1-8 placed on the table 29 on which the target object 30 is placed allow accurate recognition of the position and the orientation of the target object 30 in 3D measurement.
- the 3D reference objects 1-5, 1-6, 1-7, and 1-8 placed at the four corners of the table 29 are prevented from being hidden.
- the recognition results of the positions and the orientations of the four 3D reference objects 1-5, 1-6, 1-7, and 1-8 are averaged to allow more accurate recognition of the position and the orientation of the target object 30.
- the number of 3D reference objects 1 and the positions of the 3D reference objects 1 relative to the target object 30 are not limited to those in the embodiment and may be set as appropriate in accordance with, for example, the shape of the target object 30 and the field of view of the camera.
- 3D CAD data of the target object 30 When no 3D CAD data of the target object 30 is provided, or 3D CAD data about the target object 30 differs greatly from the actual target object 30, 3D CAD data cannot be used or may be inappropriate for an operation such as picking in which a multi-finger hand or a suction pad attached to the distal end of the arm of the robot 21 grips the target object 30.
- the target object 30 is measured by 3D measurement to generate 3D model data about the target object 30 from the 3D measurement data.
- the 3D model data about the target object 30 generated in this manner is used to recognize the gripping position of the target object 30 in an operation such as picking. Based on the recognition result, the robot is controlled to perform picking or another operation.
- 3D measurement of the target object 30 may be performed with, instead of the calibration board 25, the table 29 on which the 3D reference objects 1-5, 1-6, 1-7, and 1-8 are placed attached to the hand of the arm 212 of the robot 21 in FIG. 5 .
- the robot 21 is controlled to change the position and the orientation of the target object 30 relative to the sensor unit 20 to perform 3D measurement.
- the table 29 on which the target object 30 and the 3D reference objects 1-5, 1-6, 1-7, and 1-8 are placed may be placed in a workspace for the robot 21, and the position and the orientation of the sensor unit 20 attached to the hand of the arm 212 of the robot 21 shown in FIG. 8 may be changed to perform 3D measurement.
- the position and orientation of either the sensor unit 20 or the target object 30 may be changed.
- the sensor unit 20 may be fixed and the position and the orientation of the target object 30 may be changed.
- 3D measurement is performed using the 3D reference object 1 shaped asymmetric as viewed in any direction and having predetermined dimensions. This allows accurate recognition of the position and the orientation of the 3D reference object 1 relative to the sensor unit. The technique according to the embodiments thus allows more accurate robot-sensor unit calibration.
- a calibration method for performing calibration in a computer vision system using a three-dimensional reference object (1) including a three-dimensional measurement unit (20) configured to perform three-dimensional measurement and a measurement controller (221) configured to control the three-dimensional measurement unit (20), the three-dimensional reference object (1) being asymmetric as viewed in any direction and having predetermined dimensions to be a reference for recognition of a position and an orientation in the three-dimensional measurement, the method comprising:
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Manipulator (AREA)
Abstract
Description
- The present invention relates to a calibration method for a computer vision system and a three-dimensional (3D) reference object for use with the calibration method.
- A technique for improving alignment accuracy of 3D measurement data that is obtained through 3D measurement of a target object placed in a measurement space uses 3D measurement data obtained through 3D measurement of a 3D object (hereafter also a 3D reference object) placed in the measurement space.
- For example,
Patent Literature 1 describes a technique to determine wear of a tire tread over use by accurately aligning the circumferential positions of a tire tread and the positions of the tire in the width direction for 3D measurement data about the tire before and after use. - The technique uses a 3D reference object including three spheres. The 3D reference object is installed and fixed in a measurement space with rods extending from a fixed support that holds a tire in a rotatable manner. A coordinate transformation is determined for the coordinates of the centers of the spheres to move to predetermined target coordinates and is used to align 3D measurement data for the tire as a target object.
- Such a 3D reference object can be recognized with sufficiently high accuracy using at least a predetermined number of measurement points. A 3D reference object with large spheres is to be used. However, a large 3D reference object may hide a target object or may use a large proportion of area and thus cause a relatively small imaging size, thus disabling measurement of the target object.
- Another 3D reference object includes multiple planes (refer to, for example, Patent Literature 2) at undefined positions in tangent planes, thus possibly disabling accurate recognition of the position and orientation of a measurement viewpoint.
-
- Patent Literature 1:
Japanese Patent No. 4956960 - Patent Literature 2:
Japanese Patent No. 4423811 - In response to the above issue, one or more aspects of the present invention are directed to a 3D reference object that allows more accurate recognition of its position and orientation in 3D measurement, and a technique using the 3D reference object for more accurate calibration of 3D measurement in a computer vision system.
- A calibration method according to an aspect of the present invention is a method for performing calibration in a computer vision system using a three-dimensional reference object. The system includes a three-dimensional measurement unit that performs three-dimensional measurement and a measurement controller that controls the three-dimensional measurement unit. The three-dimensional reference object is asymmetric as viewed in any direction and has predetermined dimensions to be a reference for recognition of a position and an orientation in the three-dimensional measurement. The method includes measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit, calculating, based on data about a three-dimensional shape of the three-dimensional reference object and three-dimensional measurement data obtained through three-dimensional measurement of the three-dimensional reference object, a position and an orientation of the three-dimensional reference object relative to a measurement unit coordinate system defined for the three-dimensional measurement unit, and calculating, based on the position and the orientation of the three-dimensional reference object relative to the measurement unit coordinate system, a reference-measurement unit transformation matrix representing a coordinate transformation between a reference coordinate system defined for the three-dimensional reference object and the measurement unit coordinate system.
- In this manner, the 3D reference object shaped asymmetric as viewed in any direction and with predetermined dimensions is used as a reference for recognizing its position and orientation in the 3D measurement. This allows accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system defined for the 3D measurement unit. More accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system allows accurate calculation of the reference-measurement unit transformation matrix representing a coordinate transformation between the reference coordinate system and the measurement unit coordinate system. This allows accurate calibration for a computer vision system including a 3D measurement unit.
- In the above aspect of the present invention, the computer vision system may further include a robot including an arm, and a robot controller that controls the robot. The measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit may include measuring the three-dimensional reference object by three-dimensional measurement by controlling the arm holding the three-dimensional reference object to change a position and an orientation of the three-dimensional reference object relative to the three-dimensional measurement unit.
- In the computer vision system used for a robot with an arm, the arm holds the 3D reference object. This allows calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on multiple sets of 3D measurement data for different positions and orientations of the 3D reference object relative to the 3D measurement unit. This allows more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system. This allows accurate calibration of 3D measurement in the computer vision system.
- In the above aspect of the present invention, the computer vision system may further include a robot including an arm, and a robot controller that controls the robot. The measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit may include measuring the three-dimensional reference object by three-dimensional measurement by controlling the arm holding the three-dimensional measurement unit to change a position and an orientation of the three-dimensional measurement unit relative to the three-dimensional reference object.
- In the computer vision system used for a robot with an arm, the arm holds the 3D measurement unit. This allows calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on multiple sets of 3D measurement data for different positions and orientations of the 3D reference object relative to the 3D measurement unit. This allows still more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system. This allows accurate calibration of 3D measurement in the computer vision system.
- The calibration method according to the above aspect of the present invention may further include calculating, based on the reference-measurement unit transformation matrix, a robot-tool transformation matrix representing a coordinate transformation between a robot coordinate system defined for a basal end of the robot and a tool coordinate system defined for the arm, a tool-reference transformation matrix representing a coordinate transformation between the tool coordinate system and the reference coordinate system, and a robot-measurement unit transformation matrix representing a coordinate transformation between the robot coordinate system and the measurement unit coordinate system.
- This allows more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system. This allows accurate calculation of the reference-measurement unit transformation matrix representing a coordinate transformation between the reference coordinate system and the measurement unit coordinate system. The robot-measurement unit transformation matrix representing a coordinate transformation between the robot coordinate system and the measurement unit coordinate system equals the product of the robot-tool transformation matrix, the tool-reference transformation matrix, and the reference-measurement unit transformation matrix. The unknown robot-measurement unit transformation matrix can be calculated based on the above relational expression, the reference-measurement unit transformation matrix already calculated, and known information about, for example, a sensor detection value indicating the position and the orientation of the arm of the robot. This allows accurate calibration between the robot and the 3D measurement unit.
- In the above aspect of the present invention, the computer vision system may include a table on which the three-dimensional reference object is placeable to allow at least one of a position or an orientation of the three-dimensional reference object to be changed relative to the three-dimensional measurement unit. The measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit may include measuring the three-dimensional reference object by three-dimensional measurement by changing at least one of a position or an orientation of the three-dimensional reference object relative to the three-dimensional measurement unit with the table.
- The table is controlled to change at least the position or the orientation of the 3D reference object relative to the 3D measurement unit. This allows calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on multiple sets of 3D measurement data for different positions and orientations of the 3D reference object relative to the 3D measurement unit. This allows more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system. This allows accurate calibration of 3D measurement in the computer vision system.
- In the above aspect of the present invention, the three-dimensional reference object may have a shape combining three cylinders each having two hemispherical ends in an axial direction with one end of each cylinder overlapping one end of the other cylinders.
- The 3D reference object includes many curved surfaces with large curvatures, thus allowing more accurate calculation of the position and the orientation of the 3D reference object relative to the measurement unit coordinate system based on 3D measurement data from the 3D measurement unit. Each transformation matrix calculated as above is thus more accurate. In other words, this structure further allows more accurate calibration of 3D measurement in the computer vision system.
- A three-dimensional reference object according to another aspect of the present invention is an object for use with the calibration method.
- The 3D reference object allows accurate recognition of its position and orientation in 3D measurement. This allows more accurate calibration of 3D measurement in the computer vision system.
- The 3D reference object according to the above aspects of the present invention allows more accurate recognition of its position and orientation in 3D measurement, and the technique according to the above aspects of the present invention allows, using the 3D reference object, more accurate calibration of 3D measurement in a computer vision system.
-
-
FIG. 1A is an overall perspective view of a 3D reference object.FIG. 1B is a perspective view of a component of the 3D reference object. -
FIG. 2 is a view of the 3D reference object placed on a calibration board. -
FIG. 3 is a view of the 3D reference objects placed on a turntable. -
FIG. 4 is a view of a 3D object for comparison. -
FIG. 5 is a schematic diagram of a robot vision system. -
FIG. 6 is a functional block diagram of an information processor. -
FIG. 7 is a flowchart showing a calibration procedure. -
FIG. 8 is a schematic diagram of another robot vision system. -
FIGs. 9A to 9E are diagrams showing 3D measurement data about the 3D reference objects and a target object placed on a table as viewed from different viewpoints. - Example uses of the present invention will now be described with reference to the drawings.
-
FIG. 1A is a view of a three-dimensional (3D)reference object 1 according to an embodiment of the present invention. The3D reference object 1 is shaped asymmetric as viewed in any direction and uniquely determines the orientation of the3D reference object 1. - The
3D reference object 1 with known dimensions is prepared and measured by 3D measurement to obtain 3D measurement data. The 3D measurement data is processed to allow a 3D measurement unit such as a 3D sensor to accurately recognize the position and the orientation of the3D reference object 1. - The
3D reference object 1 for such 3D measurement may be used in the manner described below. For example, acalibration board 25 on which the3D reference object 1 is placed as shown inFIG. 2 is attached to a hand of arobot arm 212. The3D reference object 1 is measured by 3D measurement with asensor unit 20 that can perform 3D measurement to allow accurate recognition of the position and the orientation of the3D reference object 1. This allows accurate calculation of an unknown transformation matrix for a transformation between the robot and the sensor. In other embodiments, as shown inFIG. 8 , thesensor unit 20 attached to the hand of therobot arm 212 may measure the3D reference object 1 on a table 28 placed in a workspace by 3D measurement. Similarly, this structure allows accurate recognition of the position and the orientation of the3D reference object 1. This allows accurate calculation of an unknown transformation matrix for a transformation between the robot and the sensor. - In other embodiments, as shown in
FIG. 3 , four 3D reference objects 1-1, 1-2, 1-3, and 1-4 are placed on aturntable 27 circumferentially about the center at intervals of 90 degrees. A target object is then placed at the center of theturntable 27 on which the 3D reference objects 1-1 to 1-4 are placed, and is measured by 3D measurement with a sensor. The recognition results of the positions and the orientations of the 3D reference objects 1-1 to 1-4 based on the 3D measurement data obtained in this manner are used to restore the 3D shape of the target object. The 3D reference objects 1-1 to 1-4 are arranged as above relative to theturntable 27 to prevent the 3D reference objects 1-1 to 1-4 from being hidden by the target object. An average from the recognition results of the positions and the orientations of the four 3D reference objects 1-1 to 1-4 is used for more accurate recognition. Theturntable 27 is rotated with its center as the rotation center to change the positions and the orientations of the 3D reference objects 1-1, 1-2, 1-3, and 1-4 relative to the sensor. Four 3D reference objects as at least one3D reference object 1 are placed on theturntable 27. However, the number and arrangement of the 3D reference objects are not limited to this example. Theturntable 27 herein corresponds to a table in an aspect of the present invention. - In some embodiments, as shown in
FIGs. 9A to 9E , for atarget object 30 to be measured by 3D measurement from various relative positions, four 3D reference objects 1-5 to 1-8 may be placed at the four corners of a table 29 on which thetarget object 30 is placed. This allows accurate recognition of the position and the orientation of thetarget object 30. This allows accurate alignment between multiple sets of data about 3D measurement from various relative positions. -
FIG. 1A shows the shape of a3D reference object 1 according to the present embodiment. The3D reference object 1 includes many curved surfaces with large curvatures, is shaped asymmetric as viewed in any direction, and uniquely determines the orientation of the3D reference object 1. More specifically, the3D reference object 1 is shaped by combining three objects 10 (refer toFIG. 1B ) each having one end overlapping one end of the other objects 10. Theobject 10 includes acylinder 10a with a radius of 12 mm and a length of 30 mm andhemispherical surfaces cylinder 10a. InFIG. 1A , x-axis and y-axis are parallel to the page of the figure, and z-axis extends in the direction from the page of the figure. The3D reference object 1 includes afirst branch 11, asecond branch 12, and athird branch 13. Thefirst branch 11 is tilted at an angle of 23 degrees to the direction (y-direction) in which thesecond branch 12 extends. Thethird branch 13 is tilted at an angle of 40 degrees to z-axis. Thesecond branch 12 is tilted at an angle of 7 degrees about x-axis. The dimensions and angles described above are mere examples and are not limited to these. The dimensions and angles of the branches may be changed as appropriate in accordance with the size of the measurement field of view of the 3D sensor. - The
3D reference object 1 may be processed to have diffuse reflecting surfaces to facilitate 3D measurement. Such diffuse reflecting surfaces may be achieved by selecting a material or providing surface treatment such as coating and roughing. - The
3D reference object 1 with a patterned surface may obstruct 3D measurement involving emission of patterned light. The3D reference object 1 may thus have an unpatterned surface. - For 3D measurement using emission of visible light, the
3D reference object 1 may have a white surface. - As described above, the
3D reference object 1 is shaped to uniquely determine the orientation of the3D reference object 1 relative to any viewpoint. The3D reference object 1 also has known dimensions. This allows accurate recognition of the position and the orientation of the3D reference object 1 based on 3D measurement data obtained through 3D measurement of the3D reference object 1.FIG. 4 is a view of a3D reference object 50 for comparison. The3D reference object 50 is a disk having a hollow at its center as a through-hole in the height direction. The3D reference object 50 and the3D reference object 1 remaining still are measured by 3D measurement 15 times in a row with a sensor. The table below shows standard deviations of the recognized positions and orientations in each direction. The x-translation denotes a translation in x-direction, and the x-rotation denotes a rotation about x-axis (the same applies to the other directions and angles).Table 1 x-translation (mm) y-translation (mm) z-translation (mm) x-rotation (degree) y-rotation (degree) 3D reference object 10.058 0.039 0.033 0.091 0.049 3D reference object 500.146 0.212 0.011 0.058 0.059 - As shown in the above table, the recognition results of the position and the orientation of the
3D reference object 50 vary greatly depending on the direction, whereas the position and the orientation of the3D reference object 1 can be accurately recognized in any direction. - A
robot vision system 100 in the embodiment of the present invention will be described with reference toFIG. 5 . - The
robot vision system 100 is installed on a production line for, for example, product assembly or processing. Therobot vision system 100 recognizes, based on data received from thesensor unit 20, the gripping position and orientation of a multi-finger hand or a suction pad gripping an object placed in, for example, a tray. The multi-finger hand or the suction pad is attached to a distal end of an arm of arobot 21. Therobot vision system 100 herein corresponds to a computer vision system in an aspect of the present invention. -
FIG. 5 shows therobot vision system 100 with the hand of anarm 212 receiving thecalibration board 25 holding acalibration target 26 for robot-sensor unit calibration to be performed. - The
robot vision system 100 mainly includes thesensor unit 20, aninformation processor 22, a programmable logic controller (PLC) 24, and therobot 21. Thesensor unit 20 and theinformation processor 22 are connected to each other with wires or wirelessly. Theinformation processor 22 receives the output from thesensor unit 20. Theinformation processor 22 performs various processes using data received from thesensor unit 20. Examples of the processes performed by theinformation processor 22 may include distance measurement (ranging), 3D shape recognition, object recognition, and scene recognition. The process result from theinformation processor 22 is output to, for example, thePLC 24 or adisplay 23, and used for controlling therobot 21. - The
sensor unit 20 includes at least a camera for capturing optical images of target objects. Thesensor unit 20 may include any component (e.g., a sensor, an illuminator, or a projector) to be used for 3D measurement of target objects. For measuring the depth using stereo matching (also referred to as stereo vision or a stereo camera system), for example, thesensor unit 20 includes multiple cameras. For active stereo that projects a random dot pattern onto a target object, thesensor unit 20 further includes a projector for projecting structured light onto the target objects. For 3D measurement using pattern projection with space encoding, thesensor unit 20 includes a projector for projecting patterned light and cameras. Any other method may be used to generate 3D information about the target objects, such as photometric stereo, a time-of-flight (TOF) method, or phase shifting. Thesensor unit 20 herein is fixed at a predetermined position. Thesensor unit 20 herein corresponds to a 3D measurement unit in an aspect of the present invention. - The
information processor 22 is, for example, a computer including a central processing unit (CPU), a random-access memory (RAM), a nonvolatile storage (e.g., a hard disk drive, or a solid-state drive or SSD), an input device, and an output device. In this case, the CPU loads the program stored in the nonvolatile storage into the RAM and executes the program to implement various components described later. Theinformation processor 22 may have any other configuration. The components may be entirely or partly implemented by a dedicated circuit such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), or by cloud computing or distributed computing. - The
robot 21 includes thearm 212 connected to arobot base 211 with multiple joints in between and having predetermined degrees of freedom. As described above, for gripping a target object, thearm 212 receives a multi-finger hand with multiple fingers or a suction pad at the distal end. Therobot base 211 is installed on a floor surface or a predetermined installation part. In this embodiment, thecalibration board 25 holding thecalibration target 26 is attached to the hand of thearm 212. Therobot base 211 herein corresponds to a basal end of a robot in an aspect of the present invention. - In this embodiment, the
3D reference object 1 having the shape shown inFIG. 1A and with known dimensions is used as thecalibration target 26. -
FIG. 6 is a functional block diagram showing the components of theinformation processor 22. - The
information processor 22 includes asensor unit controller 221, arobot control instructor 222, astorage 223, a position-orientation calculator 224, acalibration unit 225, and a calibrationresult output unit 226. - The
sensor unit controller 221 controls thesensor unit 20 to instruct thesensor unit 20 to, for example, perform 3D measurement and output a 3D measurement result. - The
robot control instructor 222 outputs a control instruction to arobot controller 30 including thePLC 24 and other components, and obtains a detection value of, for example, the rotation angle of a joint in thearm 212 of therobot 21 detected by a sensor and stores the detection value into a predetermined area in thestorage 223. During calibration, therobot control instructor 222 outputs control information to therobot controller 30 to move thearm 212 and thus change the position of the3D reference object 1 relative to thesensor unit 20. - The
storage 223 includes a predetermined area in the nonvolatile storage device, and stores a detection value of, for example, the rotation angle of a joint in thearm 212 of therobot 21 detected by a sensor (described above), and data including dimension data about, for example, links between therobot base 211 and the distal end of thearm - The position-
orientation calculator 224 calculates the position and the orientation of the3D reference object 1 based on 3D measurement data obtained through 3D measurement of the3D reference object 1 by thesensor unit 20 and the 3D shape data about the3D reference object 1. - The
calibration unit 225 performs a calibration process (described later). - The calibration
result output unit 226 outputs a result of calibration to an external device, or stores the result into a predetermined area in thestorage 223. A calibration result is used for, for example, recognition of a target object gripping position performed by a robot vision system. - With reference to the flowchart in
FIG. 7 , an example calibration process implementing a calibration method will now be described. - Robot-sensor unit calibration calculates a transformation matrix representing a coordinate transformation between a coordinate system defined for the
robot 21 and a coordinate system defined for thesensor unit 20. - A camera coordinate system defined for the camera in the
sensor unit 20 shown inFIG. 5 is denoted by Σcam, a target coordinate system defined for thecalibration target 26 is denoted by Σcal, a tool coordinate system defined for the distal end of thearm 212 is denoted by Σtool, and a robot coordinate system defined for therobot base 211 is denoted by Σbase. A transformation matrix for a transformation from the target coordinate system Σcal to the camera coordinate system Σcam is written as camHcal. A transformation matrix for a transformation from the target coordinate system Σcal to the tool coordinate system Σtool is written as toolHcal. A transformation matrix for a transformation from the tool coordinate system Σtool to the robot coordinate system Σbase is written as baseHtool. A transformation matrix for a transformation from the robot coordinate system Σbase to the camera coordinate system Σcam is written as camHbase. The relations among these transformation matrices are written byFormula 1 below. - The camera coordinate system Σcam herein corresponds to a measurement unit coordinate system in an aspect of the present invention, the target coordinate system Σcal corresponds to a reference coordinate system in an aspect of the present invention, and the transformation matrix camHcal corresponds to a reference-measurement unit transformation matrix in an aspect of the present invention. The transformation matrix baseHtool corresponds to a robot-tool transformation matrix in an aspect of the present invention, the transformation matrix toolHcal corresponds to a tool-reference transformation matrix in an aspect of the present invention, and the transformation matrix camHbase corresponds to a robot-measurement unit transformation matrix in an aspect of the present invention.
- As shown in
FIG. 5 , thesensor unit 20 measures thecalibration target 26 placed on thecalibration board 25 attached to the hand of thearm 212 of therobot 21 by 3D measurement (step S1). - A 3D shape model such as 3D computer-aided design (CAD) data about the
calibration target 26 is fitted with 3D measurement data obtained through the 3D measurement of thecalibration target 26 by thesensor unit 20. The position and the orientation of thecalibration target 26 in the camera coordinate system Σcam can thus be calculated (step S2). The 3D measurement data is obtained through 3D measurement and includes multiple points each having 3D information. The 3D measurement data may, as with point cloud data, include multiple points each having a 3D coordinate value in the camera coordinate system, or may include multiple points (pixels) in a two-dimensional image each associated with a depth value (depth information). In this embodiment, point cloud data is used as the 3D measurement data. - The position and the orientation of the
calibration target 26 in the camera coordinate system Σcam is recognized to identify the positional relationship between the target coordinate system Σcal defined for thecalibration target 26 and the camera coordinate system Σcam. The transformation matrix camHcal for a transformation from the target coordinate system Σcal to the camera coordinate system Σcam can thus be calculated (step S3). - The position and angle of the hand of the
arm 212 relative to therobot base 211 can be calculated from dimension data about links and other components between therobot base 211 and the hand of thearm 212 and detection values from a sensor that detects the amount of displacement in each degree of freedom including the rotation angle of each joint. Thus, the transformation matrix baseHtool for a transformation from the tool coordinate system Σtool to the robot coordinate system Σbase can be calculated (step S4). - The attaching position and orientation of the
calibration board 25 relative to the hand of thearm 212 are preset. The position and the orientation of thecalibration target 26 to be placed relative to thecalibration board 25 are preset. The position of thecalibration target 26 relative to the hand of thearm 212 is thus known. Thus, the transformation matrix toolHcal for a transformation from the target coordinate system Σcal to the tool coordinate system Σtool can also be calculated (step S4). - The relations among the transformation matrix toolHcal for a transformation from the target coordinate system Σcal to the tool coordinate system Σtool, the transformation matrix baseHtool for a transformation from the tool coordinate system Σtool to the robot coordinate system Σbase, the transformation matrix camHcal for a transformation from the target coordinate system Σcal to the camera coordinate system Σcam, and the transformation matrix camHbase for a transformation from the robot coordinate system Σbase to the camera coordinate system Σcam are written by
Formula 1. - As described above, the position and the orientation of the
calibration target 26 are recognized to calculate the transformation matrix camHcal for a transformation from the target coordinate system Σcal to the camera coordinate system Σcam. The attaching position and orientation of thecalibration board 25 relative to the hand of thearm 212 are preset. The position and the orientation of thecalibration target 26 to be placed relative to thecalibration board 25 are preset. Thus, the position of thecalibration target 26 relative to the hand of thearm 212 is known. Thus, the transformation matrix toolHcal for a transformation from the target coordinate system Σcal to the tool coordinate system Σtool can also be calculated. The position of the hand of thearm 212 relative to therobot base 211 can be calculated from dimension data about the links between therobot base 211 and the hand of thearm 212 and detection values from a sensor that detects a rotational angle of each joint. Thus, the transformation matrix baseHtool for a transformation from the tool coordinate system Σtool to the robot coordinate system Σbase can also be calculated. - The transformation matrices in
Formula 1 except camHbase on the right side can thus be calculated. Thus, camHbase can be calculated withFormula 1. In other words, a transformation matrix for a transformation from the robot coordinate system Σbase to the camera coordinate system Σcam can be calculated, thus allowing robot-sensor unit calibration (step S5). - The calibration result is output to an external device or stored into a predetermined area in the storage 223 (step S6).
- In the above process, the position and the orientation of the
calibration target 26 relative to the tool coordinate system Σtool are known, and the transformation matrix toolHcal for a transformation from the target coordinate system Σcal to the tool coordinate system Σtool can be calculated. A process performed when the position and the orientation of thecalibration target 26 relative to the tool coordinate system Σtool is unknow will now be described. In this case, therobot 21 is controlled to cause thecalibration target 26 to have different positions and orientations relative to thesensor unit 20. Thesensor unit 20 then measures thecalibration target 26 in the multiple positions and orientations by 3D measurement. Thecalibration target 26 at different positions and orientations as above has a constant relative relationship with the tool coordinate system Σtool. Thus, the transformation matrix toolHcal for a transformation from the target coordinate system Σcalto the tool coordinate system Σtool can be calculated withmultiple Formulas 1 for thecalibration target 26 at the different positions and orientations. - A
robot vision system 200 in a second embodiment of the present invention will be described with reference toFIG. 8 . - The same components as in the first embodiment are given the same numerals and will not be described in detail. The
robot vision system 200 herein corresponds to a computer vision system in an aspect of the present invention. - In the
robot vision system 200, thesensor unit 20 is attached to the hand of thearm 212 of therobot 21. Except for the attaching position of thesensor unit 20, therobot vision system 200 has the same configuration as therobot vision system 100. - In the
robot vision system 200, the3D reference object 1 is placed on the table 28 in a workspace in which therobot 21 is placed. - The camera coordinate system Σcam, the target coordinate system Σcal defined for the
3D reference object 1 as a calibration target, the tool coordinate system Σtool, and the robot coordinate system Σbase are the same as in the first embodiment. The transformation matrices between these coordinate systems, camHCal, toolHcal, baseHtool, and camHbase are also the same as in the first embodiment. -
Formula 1 also applies to the relations among these transformation matrices. - In this embodiment as well, robot-sensor unit calibration is performed with the same procedure shown in
FIG. 7 . - In the present embodiment, the
sensor unit 20 is attached to the hand of thearm 212. Thus, the relative positional relationship between the camera coordinate system Σcam and the tool coordinate system Σtool remains the same independently of the operation of the robot. Instead of the transformation matrix for a transformation between the target coordinate system Σcal and the tool coordinate system Σtool, a transformation matrix toolHcam for a transformation from the camera coordinate system to the tool coordinate system may be used for calibration. - In the example shown in
FIG. 8 , one3D reference object 1 is placed on the table 28. Instead, as shown inFIG. 3 , theturntable 27 on which multiple (four in this example) 3D reference objects 1-1, 1-2, 1-3, and 1-4 are placed as calibration targets may be placed in a workspace. The multiple 3D reference objects 1-1, 1-2, 1-3, and 1-4 allow more accurate calibration. - Other than the robot-sensor unit calibration described above, the
3D reference object 1 may be used in the manner described below. -
FIGs. 9A to 9E show 3D measurement data obtained through 3D measurement of the substantially square table 29 from different viewpoints, with thetarget object 30 placed on the center of the table 29 and 3D reference objects 1-5, 1-6, 1-7, and 1-8 placed at the four corners. - Three-dimensional measurement of the
target object 30 from one viewpoint fails to provide 3D measurement data about all the surfaces of thetarget object 30, which has a 3D shape. Three-dimensional measurement data about portions of thetarget object 30 shadowed when viewed from the camera viewpoint cannot be obtained. As shown inFIG. 9A , 3D measurement with the camera facing the table 29 provides 3D measurement data about a front surface of the target object 30 (with a front surface facing toward the front of the page ofFIG. 9A , and upper, lower, right side, and left side surfaces of thetarget object 30 being defined based onFIG. 9A ) without providing 3D data about the upper, lower, right side, and left side surfaces of thetarget object 30. -
FIGs. 9B to 9E respectively show 3D measurement data obtained through 3D measurement with the camera from viewpoints facing the lower surface, upper surface, right side surface, and left side surface of thetarget object 30. As shown inFIGs. 9A to 9E , the relative position of thetarget object 30 relative to the viewpoint of the camera is changed to obtain 3D measurement data about various portions of thetarget object 30. - Three-dimensional measurement data obtained in this manner through 3D measurement of the
target object 30 from different viewpoints allows more accurate restoration of the 3D shape of thetarget object 30. When the 3D shape of thetarget object 30 is restored by integrating 3D measurement data obtained through 3D measurement from different viewpoints, accurate alignment between the multiple sets of the 3D measurement data is to be performed. Thus, the position and the orientation of thetarget object 30 in the 3D measurement of thetarget object 30 are to be accurately recognized. - The 3D reference objects 1-5, 1-6, 1-7, and 1-8 according to the present embodiment are each shaped asymmetric as viewed in any direction and uniquely determine their orientation, and have known dimensions. The positions and the orientations of the 3D reference objects 1-5, 1-6, 1-7, and 1-8 used in 3D measurement can thus be recognized accurately. The 3D reference objects 1-5, 1-6, 1-7, and 1-8 placed on the table 29 on which the
target object 30 is placed allow accurate recognition of the position and the orientation of thetarget object 30 in 3D measurement. The 3D reference objects 1-5, 1-6, 1-7, and 1-8 placed at the four corners of the table 29 are prevented from being hidden. The recognition results of the positions and the orientations of the four 3D reference objects 1-5, 1-6, 1-7, and 1-8 are averaged to allow more accurate recognition of the position and the orientation of thetarget object 30. The number of3D reference objects 1 and the positions of the3D reference objects 1 relative to thetarget object 30 are not limited to those in the embodiment and may be set as appropriate in accordance with, for example, the shape of thetarget object 30 and the field of view of the camera. - When no 3D CAD data of the
target object 30 is provided, or 3D CAD data about thetarget object 30 differs greatly from theactual target object robot 21 grips thetarget object 30. In this case, thetarget object 30 is measured by 3D measurement to generate 3D model data about thetarget object 30 from the 3D measurement data. The 3D model data about thetarget object 30 generated in this manner is used to recognize the gripping position of thetarget object 30 in an operation such as picking. Based on the recognition result, the robot is controlled to perform picking or another operation. In this case, 3D measurement of thetarget object 30 may be performed with, instead of thecalibration board 25, the table 29 on which the 3D reference objects 1-5, 1-6, 1-7, and 1-8 are placed attached to the hand of thearm 212 of therobot 21 inFIG. 5 . Therobot 21 is controlled to change the position and the orientation of thetarget object 30 relative to thesensor unit 20 to perform 3D measurement. In some embodiments, the table 29 on which thetarget object 30 and the 3D reference objects 1-5, 1-6, 1-7, and 1-8 are placed may be placed in a workspace for therobot 21, and the position and the orientation of thesensor unit 20 attached to the hand of thearm 212 of therobot 21 shown inFIG. 8 may be changed to perform 3D measurement. - When 3D measurement is performed by changing the positions and the orientations of the
sensor unit 20 and thetarget object 30, the position and orientation of either thesensor unit 20 or thetarget object 30 may be changed. Thesensor unit 20 may be fixed and the position and the orientation of thetarget object 30 may be changed. - In the configurations and the processes described above, 3D measurement is performed using the
3D reference object 1 shaped asymmetric as viewed in any direction and having predetermined dimensions. This allows accurate recognition of the position and the orientation of the3D reference object 1 relative to the sensor unit. The technique according to the embodiments thus allows more accurate robot-sensor unit calibration. - The elements in the aspects of the present invention below are identified with reference numerals used in the drawings to show the correspondence between these elements and the components in the embodiments.
- A calibration method for performing calibration in a computer vision system using a three-dimensional reference object (1), the system including a three-dimensional measurement unit (20) configured to perform three-dimensional measurement and a measurement controller (221) configured to control the three-dimensional measurement unit (20), the three-dimensional reference object (1) being asymmetric as viewed in any direction and having predetermined dimensions to be a reference for recognition of a position and an orientation in the three-dimensional measurement, the method comprising:
- (S1) measuring the three-dimensional reference object (1) by three-dimensional measurement with the three-dimensional measurement unit (20);
- (S2) calculating, based on data about a three-dimensional shape of the three-dimensional reference object (1) and three-dimensional measurement data obtained through three-dimensional measurement of the three-dimensional reference object (1), a position and an orientation of the three-dimensional reference object relative to a measurement unit coordinate system defined for the three-dimensional measurement unit (20); and
- (S3) calculating, based on the position and the orientation of the three-dimensional reference object (1) relative to the measurement unit coordinate system, a reference-measurement unit transformation matrix representing a coordinate transformation between a reference coordinate system defined for the three-dimensional reference object (1) and the measurement unit coordinate system.
-
- 1
- 3D reference object
- 20
- sensor unit
- 21
- robot
- 27
- turntable
- 30
- robot controller
- 211
- basal end
- 212
- arm
- 221
- sensor unit controller
- 100, 200
- robot vision system
Claims (7)
- A calibration method for performing calibration in a computer vision system using a three-dimensional reference object, the system including a three-dimensional measurement unit configured to perform three-dimensional measurement and a measurement controller configured to control the three-dimensional measurement unit, the three-dimensional reference object being asymmetric as viewed in any direction and having predetermined dimensions to be a reference for recognition of a position and an orientation in the three-dimensional measurement, the method comprising:measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit;calculating, based on data about a three-dimensional shape of the three-dimensional reference object and three-dimensional measurement data obtained through three-dimensional measurement of the three-dimensional reference object, a position and an orientation of the three-dimensional reference object relative to a measurement unit coordinate system defined for the three-dimensional measurement unit; andcalculating, based on the position and the orientation of the three-dimensional reference object relative to the measurement unit coordinate system, a reference-measurement unit transformation matrix representing a coordinate transformation between a reference coordinate system defined for the three-dimensional reference object and the measurement unit coordinate system.
- The calibration method according to claim 1, whereinthe computer vision system further includesa robot including an arm, anda robot controller configured to control the robot, andthe measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit includes measuring the three-dimensional reference object by three-dimensional measurement by controlling the arm holding the three-dimensional reference object to change a position and an orientation of the three-dimensional reference object relative to the three-dimensional measurement unit.
- The calibration method according to claim 1, whereinthe computer vision system further includesa robot including an arm, anda robot controller configured to control the robot, andthe measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit includes measuring the three-dimensional reference object by three-dimensional measurement by controlling the arm holding the three-dimensional measurement unit to change a position and an orientation of the three-dimensional measurement unit relative to the three-dimensional reference object.
- The calibration method according to claim 2 or claim 3, further comprising:
calculating, based on the reference-measurement unit transformation matrix, a robot-tool transformation matrix representing a coordinate transformation between a robot coordinate system defined for a basal end of the robot and a tool coordinate system defined for the arm, a tool-reference transformation matrix representing a coordinate transformation between the tool coordinate system and the reference coordinate system, and a robot-measurement unit transformation matrix representing a coordinate transformation between the robot coordinate system and the measurement unit coordinate system. - The calibration method according to claim 1, whereinthe computer vision system includes a table on which the three-dimensional reference object is placeable to allow at least one of a position or an orientation of the three-dimensional reference object to be changed relative to the three-dimensional measurement unit, andthe measuring the three-dimensional reference object by three-dimensional measurement with the three-dimensional measurement unit includes measuring the three-dimensional reference object by three-dimensional measurement by changing at least one of a position or an orientation of the three-dimensional reference object relative to the three-dimensional measurement unit with the table.
- The calibration method according to any one of claims 1 to 5, wherein
the three-dimensional reference object has a shape combining three cylinders each having two hemispherical ends in an axial direction with one end of each cylinder overlapping one end of the other cylinders. - A three-dimensional reference object for use with the calibration method according to any one of claims 1 to 6.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2019/029529 WO2021019627A1 (en) | 2019-07-26 | 2019-07-26 | Calibration method for computer vision system and three-dimensional reference object for use in same |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3974767A1 true EP3974767A1 (en) | 2022-03-30 |
EP3974767A4 EP3974767A4 (en) | 2023-01-11 |
Family
ID=74229418
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP19939259.8A Pending EP3974767A4 (en) | 2019-07-26 | 2019-07-26 | Calibration method for computer vision system and three-dimensional reference object for use in same |
Country Status (5)
Country | Link |
---|---|
US (1) | US20220357153A1 (en) |
EP (1) | EP3974767A4 (en) |
JP (1) | JP7180783B2 (en) |
CN (1) | CN114026384B (en) |
WO (1) | WO2021019627A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111515950B (en) * | 2020-04-28 | 2022-04-08 | 腾讯科技(深圳)有限公司 | Method, device and equipment for determining transformation relation of robot coordinate system and storage medium |
US11908100B2 (en) * | 2021-03-15 | 2024-02-20 | Qualcomm Incorporated | Transform matrix learning for multi-sensor image capture devices |
JP7363841B2 (en) * | 2021-03-18 | 2023-10-18 | Jfeスチール株式会社 | Module block shape measurement method, module block shape measurement system, and furnace construction method |
US20220395981A1 (en) * | 2021-06-09 | 2022-12-15 | eBots Inc. | System and method for improving accuracy of 3d eye-to-hand coordination of a robotic system |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH04203907A (en) * | 1990-11-29 | 1992-07-24 | Sanyo Mach Works Ltd | Indication fitting for measuring apparatus of position and attitude |
JP3306781B2 (en) * | 1992-12-31 | 2002-07-24 | 株式会社ダイヘン | Teaching control device and teaching method by manual operation of industrial robot |
JP4429503B2 (en) * | 2000-08-21 | 2010-03-10 | 三洋機工株式会社 | 3D measurement method of virtual ridgeline |
BE1014137A6 (en) * | 2001-04-24 | 2003-05-06 | Krypton Electronic Eng Nv | Method and device for verification and identification of a measuring device. |
JP4423811B2 (en) | 2001-04-27 | 2010-03-03 | コニカミノルタセンシング株式会社 | Three-dimensional shape measurement system and three-dimensional shape measurement method |
US7423666B2 (en) * | 2001-05-25 | 2008-09-09 | Minolta Co., Ltd. | Image pickup system employing a three-dimensional reference object |
JP4877891B2 (en) * | 2001-08-03 | 2012-02-15 | 株式会社トプコン | Calibration subject |
JP4956960B2 (en) * | 2005-10-28 | 2012-06-20 | 横浜ゴム株式会社 | 3D shape measuring apparatus and 3D shape measuring method |
DE102006021063B3 (en) * | 2006-05-03 | 2007-09-06 | Aicon 3D Systems Gmbh | Marking body for three-dimensional photogrammetry measurement of test object, has dice surfaces providing marking using four marking units, where positions of marking units are known to each other |
JP4863006B2 (en) * | 2006-12-28 | 2012-01-25 | パルステック工業株式会社 | 3D shape measurement method |
JP6080407B2 (en) * | 2012-07-03 | 2017-02-15 | キヤノン株式会社 | Three-dimensional measuring device and robot device |
CN105026885B (en) * | 2013-01-09 | 2018-06-19 | 卡尔蔡司工业测量技术有限公司 | For asking for the inspection body of rotating device rotation error |
JP6210748B2 (en) * | 2013-06-17 | 2017-10-11 | キヤノン株式会社 | Three-dimensional position measurement apparatus and calibration deviation determination method for three-dimensional position measurement apparatus |
TWI578269B (en) * | 2015-12-14 | 2017-04-11 | 財團法人工業技術研究院 | Method for suturing 3d coordinate information and the device using the same |
JP6572262B2 (en) * | 2017-06-06 | 2019-09-04 | ファナック株式会社 | Teaching position correcting device and teaching position correcting method |
-
2019
- 2019-07-26 EP EP19939259.8A patent/EP3974767A4/en active Pending
- 2019-07-26 US US17/621,405 patent/US20220357153A1/en active Pending
- 2019-07-26 WO PCT/JP2019/029529 patent/WO2021019627A1/en unknown
- 2019-07-26 CN CN201980097798.7A patent/CN114026384B/en active Active
- 2019-07-26 JP JP2021536471A patent/JP7180783B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN114026384A (en) | 2022-02-08 |
CN114026384B (en) | 2023-12-22 |
JP7180783B2 (en) | 2022-11-30 |
EP3974767A4 (en) | 2023-01-11 |
WO2021019627A1 (en) | 2021-02-04 |
JPWO2021019627A1 (en) | 2021-02-04 |
US20220357153A1 (en) | 2022-11-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3974767A1 (en) | Calibration method for computer vision system and three-dimensional reference object for use in same | |
US11911914B2 (en) | System and method for automatic hand-eye calibration of vision system for robot motion | |
CN107292927B (en) | Binocular vision-based symmetric motion platform pose measurement method | |
JP6426725B2 (en) | System and method for tracking the location of a movable target object | |
KR101549103B1 (en) | Detection apparatus, Detection method and manipulator | |
US20120099798A1 (en) | Information processing apparatus, information processing method, and program | |
JP6067175B2 (en) | Position measuring apparatus and position measuring method | |
KR20160079756A (en) | System and method for calibration of machine vision cameras along at least three discrete planes | |
WO2020217878A1 (en) | Device, method, and program for detecting position and orientation of object | |
US9491448B2 (en) | Laser videogrammetry | |
US11446822B2 (en) | Simulation device that simulates operation of robot | |
CN104976950B (en) | Object space information measuring device and method and image capturing path calculating method | |
JP2023014351A (en) | System and method for three-dimensional calibration of vision system | |
JP2021193400A (en) | Method for measuring artefact | |
CN109773589B (en) | Method, device and equipment for online measurement and machining guidance of workpiece surface | |
JP7439410B2 (en) | Image processing device, image processing method and program | |
WO2021145304A1 (en) | Image processing system | |
CN112060083B (en) | Binocular stereoscopic vision system for mechanical arm and measuring method thereof | |
CN111028298B (en) | Convergent binocular system for rigid coordinate system space transformation calibration | |
CN111062989B (en) | Method and system for calibrating high-precision two-dimensional camera and robot hand and eye | |
JPH0843044A (en) | Measuring apparatus for three dimensional coordinate | |
KR101436097B1 (en) | Non-Contacting Method for Measuring 6-DOF Motion Based on Laser Sensor | |
EP4382258A1 (en) | Robot control device, robot control system, and robot control method | |
CN111383274B (en) | Calibration method of camera module and target for camera module calibration | |
US20190061152A1 (en) | Measuring method, program, measuring apparatus and method of manufacturing article |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20211223 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20221214 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G01B 21/04 20060101ALI20221208BHEP Ipc: B25J 9/16 20060101ALI20221208BHEP Ipc: G01B 11/24 20060101AFI20221208BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: B25J 9/16 20060101ALI20240611BHEP Ipc: G01B 21/04 20060101ALI20240611BHEP Ipc: G01B 11/24 20060101AFI20240611BHEP |
|
INTG | Intention to grant announced |
Effective date: 20240627 |