US20060023938A1 - Method of and device for re-calibrating three-dimensional visual sensor in robot system - Google Patents
Method of and device for re-calibrating three-dimensional visual sensor in robot system Download PDFInfo
- Publication number
- US20060023938A1 US20060023938A1 US11/190,946 US19094605A US2006023938A1 US 20060023938 A1 US20060023938 A1 US 20060023938A1 US 19094605 A US19094605 A US 19094605A US 2006023938 A1 US2006023938 A1 US 2006023938A1
- Authority
- US
- United States
- Prior art keywords
- visual sensor
- dimensional visual
- measurement target
- feature amount
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 204
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000005259 measurement Methods 0.000 claims abstract description 235
- 238000001514 detection method Methods 0.000 claims description 30
- 230000015654 memory Effects 0.000 description 14
- 230000014509 gene expression Effects 0.000 description 10
- 238000003384 imaging method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000000875 corresponding effect Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012951 Remeasurement Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000000116 mitigating effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B21/00—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant
- G01B21/02—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant for measuring length, width, or thickness
- G01B21/04—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant for measuring length, width, or thickness by measuring coordinates of points
- G01B21/042—Calibration or calibration artifacts
Definitions
- the present invention generally relates to an industrial robot system including a three-dimensional visual sensor, and more particularly to a method of re-calibrating a three-dimensional visual sensor and a device for carrying out the re-calibration method.
- Three-dimensional visual sensors are widely used in various applications employing robot systems. Usage of three-dimensional visual sensors can be classified roughly into the following two cases (i) and (ii).
- a three-dimensional visual sensor is used with its sensor head mounted on a robot arm.
- the sensor head denotes a sensor section having an optical system (including an optical detector) for measuring an object, and control of the sensor head and processing of the detection signal are performed by a visual sensor controller.
- the expression “mounting a three-dimensional visual sensor on a robot arm” is sometimes used in the sense of “mounting the sensor head of a three-dimensional visual sensor on a robot arm” in accordance with the practice of the technical field concerned.
- a three-dimensional visual sensor is used with its sensor head set in a fixed position external to a robot.
- Such displacement of the sensor is caused, for example, when the visual sensor mounted on the robot arm collides against an object (including a person, as is the case with the following description) around the sensor, when the visual sensor is applied with an impact as a result of the collision of the robot arm against an object around the robot arm, or when an object collides with the sensor set in the fixed position. Also, even if the apparent position of the sensor is not displaced, the arrangement of the internal optical system may possibly change from that at the time of calibration due to an impact or the like, causing lowering of the measurement accuracy.
- the “measurement target” referred to in this specification denotes a target to be recognized by the visual sensor for the purpose of calibration, and a group of dots drawn on the calibration jig, for example, is used as the measurement target.
- data obtained while a three-dimensional visual sensor is capable of normal measurement is used so that re-calibration can be performed easily without the need to repeat calibration work all over again, thereby mitigating the work load required for the re-calibration.
- the three-dimensional visual sensor while the three-dimensional visual sensor is in a state (first state) capable of normal measurement, the three-dimensional visual sensor and a measurement target are arranged in at least one optional relative positional relation, and then the measurement target is measured by the three-dimensional visual sensor to acquire predetermined “reference data”.
- the “reference data” may be either of the following two (I) and (II).
- Position/orientation data data indicative of at least one of position and orientation of the measurement target obtained using parameters relating to calibration then held by the three-dimensional visual sensor.
- the “optional relative positional relation” may be two or more in number, and in this case, the position data is acquired and stored with respect to each of the relative positional relations.
- the three-dimensional visual sensor needs to be recalibrated (second state) due to interference of a robot, for example, the relative positional relation between the three-dimensional visual sensor and the measurement target is reproduced at least approximately so as to be almost identical to that with which the predetermined data was obtained. Then, the measurement target is again measured to detect feature amount data on the measurement target. Using the data obtained by the remeasurement and the reference data already acquired, re-calibration is performed.
- the present invention provides a re-calibration method for a three-dimensional visual sensor of a robot system, the three-dimensional visual sensor holding a plurality of parameters relating to calibration.
- the re-calibration method comprises the steps of:
- step (a) wherein arranging the three-dimensional visual sensor and the measurement target in the at least one relative positional relation, in the aforesaid step (a), and arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately, in the aforesaid step (b), are carried out by moving a robot which supports at least one of the three-dimensional visual sensor and the measurement target, and
- the robot is moved to reproduce a robot position where the at least one relative positional relation was established in the aforesaid step (a), in order to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- the re-calibration method comprises the steps of:
- step (a) wherein arranging the three-dimensional visual sensor and the measurement target in the at least one relative positional relation, in the aforesaid step (a), and arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately, in the aforesaid step (b), are carried out by moving a robot which supports at least one of the three-dimensional visual sensor and the measurement target, and
- the robot is moved to reproduce a robot position where the at least one relative positional relation was established in the aforesaid step (a), in order to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- One of the three-dimensional visual sensor and the measurement target may be mounted on an arm of the robot while the other may be placed at a predetermined position other than the arm of the robot, so as to be arranged in the at least one relative positional relation in the aforesaid step (a), and to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- the three-dimensional visual sensor may be mounted on the arm of the robot and the measurement target may be placed at a position different from the arm of the robot, so as to be arranged in the at least one relative positional relation in the aforesaid step (a), and to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- the three-dimensional visual sensor may be mounted on the arm of the robot and the measurement target may be placed on an arm of a different robot, so as to be arranged in the at least one relative positional relation in the aforesaid step (a), and to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- the present invention further provides a device for re-calibrating a three-dimensional visual sensor with a measurement target mounted on a robot.
- the re-calibration device comprises:
- target position information detection means for detecting position information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- the target position information detection means responsive to a reference data acquisition command, for causing the target position information detection means to detect position information on the measurement target, and storing the detected position information as position/orientation data indicative of at least one of position and orientation of the measurement target by using the held parameters relating to calibration of the three-dimensional visual sensor;
- the target feature amount information detection means responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data;
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data.
- the re-calibration device comprises:
- target position information detection means for detecting position information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- the target position information detection means responsive to a reference data acquisition command, for causing the target position information detection means to detect position information on the measurement target, and storing the detected position information as position/orientation data indicative of at least one of position and orientation of the measurement target by using the held parameters relating to calibration of the three-dimensional visual sensor;
- the target feature amount information detection means responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data;
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data.
- the re-calibration device comprises:
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- the target feature amount information detection means responsive to a reference data acquisition command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and storing the detected feature amount information as reference feature amount data;
- the target feature amount information detection means responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data;
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data, the reference feature amount data, and the parameters relating to calibration of the three-dimensional visual sensor held at the time of reception of the reference data acquisition command.
- the re-calibration device comprises:
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- the target feature amount information detection means responsive to a reference data acquisition command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and storing the detected feature amount information as reference feature amount data;
- the target feature amount information detection means responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data;
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data, the reference feature amount data, and the parameters relating to calibration of the three-dimensional visual sensor held at the time of reception of the reference data acquisition command.
- the three-dimensional visual sensor can be recalibrated with ease. Specifically, at the time of re-calibration, it is unnecessary to use a calibration jig for accurately reproducing the relative positional relation between the three-dimensional visual sensor and the measurement target, and the re-calibration is achieved using the position data (obtained during normal measurement) and feature amounts (obtained during re-calibration) of the measurement target acquired at the same robot position. Accordingly, the re-calibration does not require the labor and time for attaching and detaching the calibration jig, unlike the conventional techniques, and the three-dimensional visual sensor can be restored to its normal state in a very short time.
- FIG. 1 shows the configuration of a principal part of a three-dimensional visual sensor used in an embodiment of the invention
- FIG. 2 illustrates a method of calibrating the three-dimensional visual sensor by using a calibration jig
- FIG. 3 illustrates details of the calibration
- FIG. 4 illustrates an arrangement 1 according to the embodiment
- FIG. 5 illustrates an arrangement 2 according to the embodiment
- FIG. 6 illustrates a position data acquisition step of the embodiment
- FIG. 7 illustrates a re-calibration step of the embodiment
- FIG. 9 is a diagram explaining that the positions of the individual dots of the dot pattern can be calculated from the combination of information on the arrangement of the dot pattern and information on the position/orientation of a measurement target;
- FIG. 10 is a diagram summarizing the relationship of processes explained with reference to the embodiment.
- FIG. 11 is a diagram illustrating a modification of the relationship shown in FIG. 10 ;
- FIG. 12 is a diagram summarizing the relationship of processes which are modifications of the processes explained with reference to the embodiment of the present invention.
- FIG. 1 shows the configuration of a principal part of a three-dimensional visual sensor (hereinafter also referred to merely as “visual sensor”) used in an embodiment of the invention.
- the illustrated visual sensor itself is a typical one well known in the art, and comprises a sensor head 30 including an optical system and optical detection means for measuring an object, and a three-dimensional visual sensor controller 10 for controlling the sensor head 30 and processing information related to three-dimensional measurement.
- a laser beam emitter 34 In the sensor head 30 are arranged a laser beam emitter 34 , a laser driving section 32 for driving the emitter, a mirror 35 for scanning an object with the laser beam, and a mirror scanning section 33 for driving the mirror.
- the sensor head is also equipped with a camera 31 for acquiring a two-dimensional image of an object 40 and receiving the reflection of the laser beam irradiated onto the object 40 .
- the three-dimensional visual sensor controller 10 includes a CPU 11 , a group of memories such as a ROM 12 , a RAM 13 , a nonvolatile memory 14 and a frame memory 17 , a communication I/F (interface) 15 for exchanging data with a robot controller etc., an input/output device I/F (interface) 16 for exchanging data with external devices such as a monitor, a camera I/F (interface) 18 for driving the camera 31 to acquire images therefrom, and a laser control I/F (interface) 19 for exchanging data with the laser driving section 32 and the mirror scanning section 33 .
- a CPU 11 a group of memories such as a ROM 12 , a RAM 13 , a nonvolatile memory 14 and a frame memory 17 , a communication I/F (interface) 15 for exchanging data with a robot controller etc., an input/output device I/F (interface) 16 for exchanging data with external devices such as a monitor, a camera I/F (inter
- a program stored in the ROM 12 or the like is executed by the CPU 11 , whereupon the laser beam is irradiated onto the object 40 via the mirror 35 set at a predetermined angle ⁇ (usually, the mirror is set successively at a plurality of angles), and with respect to each set angle ⁇ , an image of the laser beam is acquired through the camera 31 and stored in the frame memory 17 . Then, feature amounts are extracted from the acquired images by image processing, and using the feature amounts and calibration data previously stored in the nonvolatile memory 14 or the like (calibration data then stored), information such as the position and orientation of the object 40 is calculated.
- the present invention is applied to re-calibration of a three-dimensional visual sensor and thus is naturally premised on the assumption that proper calibration is executed prior to the re-calibration to obtain a plurality of parameters representative of the contents of the calibration (hereinafter also referred to merely as “calibration parameters”).
- the calibration parameters obtained by the calibration prior to the re-calibration are also necessary for the “acquisition of position data (in this embodiment, dot pattern position data)”, described later.
- a calibration jig 50 having a measurement target 51 for calibration is attached to the sensor head 30 and the measurement target 51 is set in a predetermined position.
- the measurement target is set in a single position or is sequentially set in a plurality of positions, depending on the calibration method used. In the illustrated example, the measurement target is sequentially set in two positions. At each of the set positions, predetermined measurement is carried out by using the sensor head 30 and the visual sensor controller 10 in accordance with the calibration method employed, whereby calibration parameters are obtained.
- an exemplary calibration method will be explained.
- the measurement target 51 has a dot pattern or the like printed thereon, and the calibration jig 50 is configured such that when the measurement target 51 is set in each of the predetermined positions, each dot of the dot pattern 52 is located at a certain known position relative to the sensor head 30 . Details of the calibration will be explained with reference to FIG. 3 .
- reference signs 60 A and 60 B denote the measurement target ( 51 in FIG. 2 ) positioned in the respective set positions.
- the signs 60 A and 60 B are used to indicate not only the measurement target but the respective set positions. Namely, 60 A and 60 B represent the measurement targets (though single in number) set in the respective positions shown in FIG. 3 . Similarly, reference signs 61 and 62 are used to represent the positions which the dot pattern ( 52 in FIG. 2 ) assumes when the measurement target is set in the respective positions shown in FIG. 3 .
- each dot of the dot pattern 61 , 62 printed thereon is located at the known position.
- a coordinate system 80 is defined, as shown in the figure, with respect to the measurement target 60 A as a sensor measurement reference coordinate system (hereinafter referred to as sensor coordinate system)
- the measurement target 60 A, 60 B set in the corresponding position is imaged according to the normal imaging using the camera 31 , whereupon an image of the measurement target is formed on the image plane on the extension of a line of sight 71 , 72 .
- Vai, Hai can be regarded as a kind of feature amount detected with respect to the dot pattern 61
- (Vbi, Hbi) can be regarded as a kind of feature amount detected with respect to the dot pattern 62 .
- a calibration calculation technique known in the art is applied to the combination of the feature amounts, whereby, for a given optional image feature amount (V, H), an intersection point (Xa, Ya, Za) where the line of sight determined by the image feature amount intersects with the measurement target 60 A and an intersection point (Xb, Yb, Zb) where the line of sight intersects with the measurement target 60 B can be calculated.
- a brilliant point 63 , 64 of the laser beam irradiated with the mirror 35 directed at a desired angle ⁇ is imaged by the camera 31 , whereupon an image of the brilliant point is formed on the image plane on the extension of a line of sight 73 , 74 .
- the brilliant points 63 and 64 are obtained as three-dimensional positions on the coordinate system 80 by using the relationship of the above expressions (1).
- the laser beam is a slit beam
- two straight brilliant lines are formed on the respective measurement targets, in place of the brilliant points 63 and 64 .
- three-dimensional positions of a plurality of brilliant points are calculated, whereby a plane (hereinafter referred to as laser slit beam plane) which contains the laser slit beam forming the two brilliant lines can be uniquely determined.
- the laser slit beam plane at each angle is obtained, whereby the laser slit beam plane corresponding to a desired ⁇ can be calculated.
- the equation for calculating the laser slit beam plane is expressed simply by the following expression (3): ⁇ “Equation of Laser Slit Beam Plane” (3)
- equations (1) to (3) are derived, whereupon the calibration is completed. Specifically, the contents of calculations of the individual equations (1) to (3) (e.g., polynomials) and a plurality of specified parameters (e.g., coefficients and constants of individual terms of the polynomials) are stored in the memory of the visual sensor.
- the individual equations (1) to (3) e.g., polynomials
- a plurality of specified parameters e.g., coefficients and constants of individual terms of the polynomials
- the aforementioned calibration method is carried out with the measurement target set at two positions.
- the calibration may alternatively be performed with the measurement target set at a single position.
- An example of such calibration will be briefly explained below.
- the measurement target 60 A shown in FIG. 3 is assumed to be the “measurement target set at a single position” and the coordinate system 80 defined on the measurement target 60 A is referred to as “measurement target coordinate system”.
- the imaging plane can be assumed to be located at a position spaced at the focal distance from the center point of the lens of the camera.
- a coordinate system three-dimensional coordinate system whose XY plane coincides with the imaging plane; hereinafter referred to as imaging plane coordinate system
- imaging plane coordinate system three-dimensional coordinate system whose XY plane coincides with the imaging plane
- the brilliant point 63 of the laser beam irradiated with the mirror 35 directed at a desired angle ⁇ is imaged by the camera 31 , whereupon an image of the brilliant point is formed on the image plane on the extension of the line of sight 73 .
- the brilliant point 63 is obtained as a three-dimensional position on the measurement target coordinate system 80 by using the relationship of the above expressions (1′).
- the laser beam is a slit beam
- a straight brilliant line is formed on the measurement target 60 A, in place of the brilliant point 63 .
- the laser beam emitter 34 is a point light source
- the point light source and a straight line (brilliant line) represented by the brilliant point 63 form one plane.
- the angle ⁇ of the mirror 35 is changed successively to different angles each by ⁇ (small angle) and the plane derived at each angle is correlated with the corresponding relative angle varied by ⁇ , whereby the position of the point light source can be obtained. Consequently, for a given optional angle ⁇ , the laser slit beam plane for that angle can be calculated.
- equations (1′) to (3′) are derived, whereupon the calibration by the “single-position setting” method is completed. Specifically, the contents of calculations of the individual equations (1′) to (3′) (e.g., polynomials) and a plurality of specified parameters (e.g., coefficients and constants of individual terms of the polynomials) are stored in the memory of the visual sensor, as in the case of the “two-position setting”.
- specified parameters e.g., coefficients and constants of individual terms of the polynomials
- an object is measured in the following manner. First, based on the angle ⁇ of the mirror during the measurement, the laser slit beam plane for that angle is obtained according to the equation (3) or (3′), and then based on the image feature amount (V, H) indicative of the brilliant point of the laser beam, the line of sight from the camera 31 to the brilliant point is derived according to the equations (1) and (2) or the equations (1′) and (2′). Subsequently, the equation of laser slit beam plane and the line-of-sight equation are simultaneously solved, thereby obtaining the three-dimensional position of the brilliant point.
- the three-dimensional visual sensor which is now capable of normal measurement, is detached from the jig 50 and then is arranged on the robot arm or at a fixed position external to the robot, depending on the application.
- the following describes exemplary cases of “arrangement 1 ” and “arrangement 2 ” wherein, for convenience' sake, reference numeral 103 is used to denote the measurement target ( 51 , 60 A, 60 B) explained above in connection with the calibration.
- a sensor head 102 (identical with the one shown in FIG. 1 , as is the case with other elements) is mounted on the arm of a robot 101 while the measurement target 103 is fixed at a suitable position (close to the robot 101 but free from interference).
- the measurement target 103 is fixed to the base of the robot by a fixing member.
- the robot 101 is connected to a robot controller 100 , and a three-dimensional visual sensor controller 104 is connected with the sensor head 102 .
- the robot controller 100 and the three-dimensional visual sensor controller 104 are connected to each other by, for example, a communication line 105 .
- the sensor head 102 is arranged at a fixed position external to the robot 101 (in the illustrated example, a fixed position on a support 106 ).
- the measurement target 103 is attached to the arm of the robot 101 .
- the robot 101 is connected to the robot controller 100 , and the three-dimensional visual sensor controller 104 is connected with the sensor head 102 . Also, the robot controller 100 and the three-dimensional visual sensor controller 104 are connected to each other by the communication line 105 , for example.
- FIG. 6 is a flowchart summarizing the series of processes. In the following, the individual steps will be explained.
- Step S 1 The robot is moved to the position P 1 , whereby the visual sensor 102 and the measurement target 103 are arranged in a “first relative positional relation”.
- Step S 2 A command (including a position data acquisition command) for causing the visual sensor 102 to measure the measurement target 103 is output to the three-dimensional visual sensor controller 104 by means of manual operation of the keyboard of the robot controller 100 , a programmed instruction, etc.
- Step S 3 The dot pattern position in the sensor coordinate system (or measurement target coordinate system) 80 is calculated.
- the calibration parameters saved at the time of the aforementioned calibration are used.
- the sensor coordinate system (or measurement target coordinate system) 80 is not necessarily limited to the one established on one measurement target, unlike the case illustrated in FIG. 3 .
- Step S 4 The data obtained in Step S 3 is stored in the nonvolatile memory 14 of the visual sensor as “dot pattern position A”.
- Step S 5 The robot is moved to the position P 2 , whereby the visual sensor 102 and the measurement target 103 are arranged in a “second relative positional relation”.
- Step S 6 A command (including a position data acquisition command) for causing the visual sensor 102 to measure the measurement target 103 is output to the three-dimensional visual sensor controller 104 by means of manual operation of the keyboard of the robot controller 100 , a programmed instruction, etc.
- Step S 7 The dot pattern position in the sensor coordinate system (or measurement target coordinate system) 80 is calculated again.
- the calibration parameters saved at the time of the aforementioned calibration are used.
- the sensor coordinate system (or measurement target coordinate system) 80 is not necessarily limited to the one established on one measurement target, unlike the case illustrated in FIG. 3 .
- Step S 8 The data obtained in Step S 7 is stored in the nonvolatile memory 14 of the visual sensor as “dot pattern position B”.
- FIG. 7 is a flowchart summarizing the re-calibration, and in the following, the individual steps will be explained.
- Step S 11 The robot is moved to the position P 1 . Consequently, the “first relative positional relation” between the visual sensor 102 and the measurement target 103 is reproduced at least approximately. (Generally, the position/orientation of the visual sensor may presumably have changed, and thus the reproduced relation involves corresponding inaccuracy.)
- Step S 12 A command (including a re-calibration command) for causing the visual sensor 102 to measure the measurement target 103 is output to the three-dimensional visual sensor controller 104 by means of manual operation of the keyboard of the robot controller 100 , a programmed instruction, etc.
- Step S 13 Image coordinates of the dot pattern on the measurement target 103 are extracted.
- Step S 14 The image coordinates extracted in Step S 13 are temporarily saved in the nonvolatile memory 14 of the visual sensor as image coordinates A.
- Step S 15 The robot is moved to the position P 2 . Consequently, the “second relative positional relation” between the visual sensor 102 and the measurement target 103 is reproduced at least approximately. (Also in this case, generally, the position/orientation of the visual sensor may presumably have changed, and thus the reproduced relation involves corresponding inaccuracy.)
- Step S 16 A command (including a re-calibration command) for causing the visual sensor 102 to measure the measurement target 103 is output to the three-dimensional visual sensor controller 104 by means of manual operation of the keyboard of the robot controller 100 , a programmed instruction, etc.
- Step S 17 Image coordinates of the dot pattern on the measurement target 103 are extracted.
- Step S 18 The image coordinates extracted in Step S 17 are temporarily saved in the nonvolatile memory 14 of the visual sensor as image coordinates B.
- Step S 19 Using the data “dot pattern position A”, “dot pattern position B”, “image coordinates A” and “image coordinates B”, calibration calculation for the re-calibration is performed. Namely, the aforementioned equations (1) and (2) or the equations (1′) and (2′) can be updated (recalculated and stored) by using these items of data. Also, by performing the aforementioned laser beam calibration, it is possible to update (recalculate and store) the equation (3) or (3′). This completes the re-calibration.
- Step S 20 The calibration parameters obtained in Step S 19 are saved in the memory of the visual sensor, in place of the previous calibration parameters, whereupon the process ends.
- the three-dimensional visual sensor and the measurement target are brought into two relative positional relations by the robot.
- the aforementioned calibration method in which the measurement target is set in a single position is employed, however, only one relative positional relation needs to be determined.
- the measurement target 103 may be arranged on the arm of a different robot so as to establish each of the relative positional relations and to reproduce the relations at least approximately.
- the measurement target may be attached to the arm of a “different robot”, and the position P 3 may be used as a substitute for the aforementioned “fixed position”.
- the measurement target is measured by the three-dimensional visual sensor at the time of acquiring reference data and at the time of re-calibration, to obtain “position” data of the measurement target.
- the dot pattern position of the measurement target with reference to the sensor coordinate system is detected and saved as the dot pattern position A
- Steps S 7 and S 8 the dot postern position of the measurement target with reference to the sensor coordinate system is detected and saved as the dot pattern position B (reference data acquisition).
- Steps S 13 , S 14 , S 17 and S 18 of the flowchart of FIG. 7 the extraction of image coordinates of the dot pattern on the measurement target and the temporary storage of the extracted image coordinates A and B are performed in like manner (data acquisition for re-calibration).
- the present invention can be equally carried out by acquiring “orientation” data of the measurement target, instead of the “position” data of same.
- the reason will be briefly explained below.
- the following explanation is directed to the dot pattern position A only, because the dot pattern positions A and B of the above embodiment, though described separately, are obtained with the same measurement target set in different positions relative to the three-dimensional visual sensor, as already explained.
- the manner of storing the dot pattern position A exemplifies one method of measuring the measurement target and storing the position information relating thereto.
- the sensor coordinate system is expressed by ⁇ s
- (Xai, Yai, Zai) represents coordinates in ⁇ s.
- “orientation” of the measurement target may be stored by the method described below. If the arrangement of the dot pattern on the measurement target is already known, information equivalent to that obtained by measuring the positions of the individual dots of the dot pattern can be derived by measuring the position/orientation of the measurement target. Namely, if information on the arrangement of the dot pattern is prepared in advance, the positions of the individual dots of the dot pattern can be calculated immediately by using the arrangement information in combination with information on the position/orientation of the measurement target. This will be explained with reference to FIG. 9 .
- ⁇ t a coordinate system fixedly established on the measurement target.
- Di the arrangement of the individual dots
- Di ( xi, yi, zi )
- the measurement target is then measured and the position information relating thereto is stored as ⁇ t position/orientation (hereinafter expressed by T) as viewed from the sensor coordinate system ⁇ s.
- T is a homogeneous transformation matrix.
- positions (coordinates) on the sensor coordinate system ⁇ s can be converted to those on the coordinate system ⁇ t and vice versa by using the equation (4) below.
- Storing the position/orientation T of the measurement target is therefore equivalent to storing (Xai, Yai, Zai).
- the position information (position/orientation data) is acquired at the time of acquiring the reference data, but it will be understood that essentially the feature amount information on the measurement object (measurement target) has only to be acquired at that timing.
- the relationship of processes shown in FIG. 12 may be adopted in place of the relationship shown in FIG. 10 .
- the three-dimensional visual sensor and the measurement target (measurement object) are arranged in at least one optional relative positional relation while the three-dimensional visual sensor is in a state capable of normal measurement, and then the measurement target (measurement object) is measured by the three-dimensional visual sensor to acquire feature amount data on the measurement target (measurement object), the acquired feature amount data being stored.
- the three-dimensional visual sensor and the measurement target (measurement object) are arranged so that the at least one relative positional relation may be reproduced at least approximately, and then the measurement target (measurement object) is measured by the three-dimensional visual sensor to acquire feature amount data on the measurement target (measurement object).
- the parameters relating to calibration of the three-dimensional sensor are updated.
- arranging the three-dimensional visual sensor and the measurement object in the at least one relative positional relation at the time of the reference data acquisition and arranging the same elements so as to reproduce the at least one relative positional relation at least approximately at the time of the re-calibration are carried out by moving the robot which supports at least one of the three-dimensional visual sensor and the measurement object. Also, to reproduce the at least one relative positional relation at least approximately at the time of the re-calibration, the robot is moved to reproduce the robot position where the at least one relative positional relation was established at the time of the reference data acquisition.
- the method wherein calibration is performed with the measurement target set in a single position may be employed as mentioned above, in which case only one relative positional relation may be defined.
- the method may be employed wherein the measurement target 103 is arranged on the arm of a different robot to establish the at least one relative positional relation and to reproduce the at least one relative positional relation at least approximately.
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Manipulator (AREA)
Abstract
A re-calibration method and device for a three-dimensional visual sensor of a robot system, whereby the work load required for re-calibration is mitigated. While the visual sensor is normal, the visual sensor and a measurement target are arranged in one or more relative positional relations by a robot, and the target is measured to acquire position/orientation information of a dot pattern etc. by using calibration parameters then held. During re-calibration, each relative positional relation is approximately reproduced, and the target is again measured to acquire feature amount information or position/orientation of the dot pattern etc. on the image. Based on the feature amount data and the position information, the parameters relating to calibration of the visual sensor are updated. At least one of the visual sensor and the target, which are brought into the relative positional relation, is mounted on the robot arm. During the re-calibration, position information may be calculated using the held calibration parameters as well as the feature amount information obtained during normal operation of the visual sensor and that obtained during the re-calibration, and the calibration parameters may be updated based on the calculation results.
Description
- 1. Field of the Invention
- The present invention generally relates to an industrial robot system including a three-dimensional visual sensor, and more particularly to a method of re-calibrating a three-dimensional visual sensor and a device for carrying out the re-calibration method.
- 2. Description of Related Art
- Three-dimensional visual sensors are widely used in various applications employing robot systems. Usage of three-dimensional visual sensors can be classified roughly into the following two cases (i) and (ii).
- (i) A three-dimensional visual sensor is used with its sensor head mounted on a robot arm. The sensor head denotes a sensor section having an optical system (including an optical detector) for measuring an object, and control of the sensor head and processing of the detection signal are performed by a visual sensor controller. In this specification, the expression “mounting a three-dimensional visual sensor on a robot arm” is sometimes used in the sense of “mounting the sensor head of a three-dimensional visual sensor on a robot arm” in accordance with the practice of the technical field concerned.
- (ii) A three-dimensional visual sensor is used with its sensor head set in a fixed position external to a robot.
- In either case, it is well known that “calibration” is needed in order to perform measurement with proper accuracy by using a three-dimensional visual sensor. Calibration methods for three-dimensional visual sensors are described in detail in, for example, JP2690603B and JP2686351B and JP 08-5351A, JP08-233516A and JP08-272414A.
- Thus, various calibration methods for three-dimensional visual sensors are conventionally known, but it is often the case that after proper measurement accuracy is once obtained by calibration, the measurement accuracy lowers, making accurate measurement impossible. Such a situation arises, in the aforementioned case (i), when the mounting position (including the orientation, as is the case with the following description) of the visual sensor (sensor head) mounted on the robot arm is displaced from the position where the calibration was performed. In the above case (ii), the same situation occurs when the set position of the visual sensor (sensor head) is displaced from the fixed position where the calibration was performed.
- Such displacement of the sensor is caused, for example, when the visual sensor mounted on the robot arm collides against an object (including a person, as is the case with the following description) around the sensor, when the visual sensor is applied with an impact as a result of the collision of the robot arm against an object around the robot arm, or when an object collides with the sensor set in the fixed position. Also, even if the apparent position of the sensor is not displaced, the arrangement of the internal optical system may possibly change from that at the time of calibration due to an impact or the like, causing lowering of the measurement accuracy.
- Conventionally, when such a situation occurs or is presumed to have occurred, identical calibration work is performed all over again. To again perform the identical calibration work, however, it is necessary that the relative positional relation between a measurement target and the visual sensor should be accurately reproduced by again using a calibration jig, requiring a heavy work load and inevitably prolonging the system downtime. The “measurement target” referred to in this specification denotes a target to be recognized by the visual sensor for the purpose of calibration, and a group of dots drawn on the calibration jig, for example, is used as the measurement target.
- According to the present invention, data obtained while a three-dimensional visual sensor is capable of normal measurement is used so that re-calibration can be performed easily without the need to repeat calibration work all over again, thereby mitigating the work load required for the re-calibration.
- In accordance with the present invention, while the three-dimensional visual sensor is in a state (first state) capable of normal measurement, the three-dimensional visual sensor and a measurement target are arranged in at least one optional relative positional relation, and then the measurement target is measured by the three-dimensional visual sensor to acquire predetermined “reference data”. The “reference data” may be either of the following two (I) and (II).
- (I) Position/orientation data (data indicative of at least one of position and orientation of the measurement target) obtained using parameters relating to calibration then held by the three-dimensional visual sensor.
- (II) Feature amount data on the measurement target.
- The “optional relative positional relation” may be two or more in number, and in this case, the position data is acquired and stored with respect to each of the relative positional relations.
- If, after the above preparatory process, the three-dimensional visual sensor needs to be recalibrated (second state) due to interference of a robot, for example, the relative positional relation between the three-dimensional visual sensor and the measurement target is reproduced at least approximately so as to be almost identical to that with which the predetermined data was obtained. Then, the measurement target is again measured to detect feature amount data on the measurement target. Using the data obtained by the remeasurement and the reference data already acquired, re-calibration is performed.
- The present invention provides a re-calibration method for a three-dimensional visual sensor of a robot system, the three-dimensional visual sensor holding a plurality of parameters relating to calibration.
- According to one aspect of the present invention, the re-calibration method comprises the steps of:
- (a) arranging the three-dimensional visual sensor and a measurement target in at least one optional relative positional relation while the three-dimensional visual sensor is in a state capable of normal measurement, then measuring the measurement target by the three-dimensional visual sensor to acquire position/orientation data indicative of at least one of position and orientation of the measurement target by using the parameters relating to calibration then held by the three-dimensional visual sensor, and storing the position/orientation data;
- (b) arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately when the three-dimensional visual sensor is to be recalibrated after the three-dimensional visual sensor was in the aforesaid state, and then measuring the measurement target by the three-dimensional visual sensor to acquire feature amount data on the measurement target; and
- (c) updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data,
- wherein arranging the three-dimensional visual sensor and the measurement target in the at least one relative positional relation, in the aforesaid step (a), and arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately, in the aforesaid step (b), are carried out by moving a robot which supports at least one of the three-dimensional visual sensor and the measurement target, and
- in the aforesaid step (b), the robot is moved to reproduce a robot position where the at least one relative positional relation was established in the aforesaid step (a), in order to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- According to another aspect of the present invention, the re-calibration method comprises the steps of:
- (a) arranging the three-dimensional visual sensor and the measurement target in at least one optional relative positional relation while the three-dimensional visual sensor is in a state capable of normal measurement, then measuring the measurement target by the three-dimensional visual sensor to acquire feature amount data on the measurement target, and storing the feature amount data;
- (b) arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately when the three-dimensional visual sensor is to be recalibrated after the three-dimensional visual sensor was in the aforesaid state, and then measuring the measurement target by the three-dimensional visual sensor to acquire feature amount data on the measurement target; and
- (c) updating the parameters relating to calibration of the three-dimensional visual sensor, based on the parameters relating to calibration of the three-dimensional visual sensor held at the time of the aforesaid step (a), the feature amount data acquired in the aforesaid step (a), and the feature amount data acquired in the aforesaid step (b),
- wherein arranging the three-dimensional visual sensor and the measurement target in the at least one relative positional relation, in the aforesaid step (a), and arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately, in the aforesaid step (b), are carried out by moving a robot which supports at least one of the three-dimensional visual sensor and the measurement target, and
- in the aforesaid step (b), the robot is moved to reproduce a robot position where the at least one relative positional relation was established in the aforesaid step (a), in order to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- One of the three-dimensional visual sensor and the measurement target may be mounted on an arm of the robot while the other may be placed at a predetermined position other than the arm of the robot, so as to be arranged in the at least one relative positional relation in the aforesaid step (a), and to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- Also, the three-dimensional visual sensor may be mounted on the arm of the robot and the measurement target may be placed at a position different from the arm of the robot, so as to be arranged in the at least one relative positional relation in the aforesaid step (a), and to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- Moreover, the three-dimensional visual sensor may be mounted on the arm of the robot and the measurement target may be placed on an arm of a different robot, so as to be arranged in the at least one relative positional relation in the aforesaid step (a), and to reproduce the at least one relative positional relation at least approximately in the aforesaid step (b).
- The present invention further provides a device for re-calibrating a three-dimensional visual sensor with a measurement target mounted on a robot.
- According to one aspect of the present invention, the re-calibration device comprises:
- means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
- means for moving the robot to at least one robot position where the three-dimensional visual sensor is located close to the measurement target;
- target position information detection means for detecting position information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- means, responsive to a reference data acquisition command, for causing the target position information detection means to detect position information on the measurement target, and storing the detected position information as position/orientation data indicative of at least one of position and orientation of the measurement target by using the held parameters relating to calibration of the three-dimensional visual sensor;
- means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data.
- According to another aspect of the invention, the re-calibration device comprises:
- means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
- means for moving the robot to at least one robot position where the measurement target is located close to the three-dimensional visual sensor;
- target position information detection means for detecting position information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- means, responsive to a reference data acquisition command, for causing the target position information detection means to detect position information on the measurement target, and storing the detected position information as position/orientation data indicative of at least one of position and orientation of the measurement target by using the held parameters relating to calibration of the three-dimensional visual sensor;
- means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data.
- According to still another aspect of the invention, the re-calibration device comprises:
- means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
- means for moving the robot to at least one robot position where the three-dimensional visual sensor is located close to the measurement target;
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- means, responsive to a reference data acquisition command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and storing the detected feature amount information as reference feature amount data;
- means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data, the reference feature amount data, and the parameters relating to calibration of the three-dimensional visual sensor held at the time of reception of the reference data acquisition command.
- According to a further aspect of the invention, the re-calibration device comprises:
- means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
- means for moving the robot to at least one robot position where the measurement target is located close to the three-dimensional visual sensor;
- target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
- means, responsive to a reference data acquisition command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and storing the detected feature amount information as reference feature amount data;
- means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
- parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data, the reference feature amount data, and the parameters relating to calibration of the three-dimensional visual sensor held at the time of reception of the reference data acquisition command.
- According to the present invention, the three-dimensional visual sensor can be recalibrated with ease. Specifically, at the time of re-calibration, it is unnecessary to use a calibration jig for accurately reproducing the relative positional relation between the three-dimensional visual sensor and the measurement target, and the re-calibration is achieved using the position data (obtained during normal measurement) and feature amounts (obtained during re-calibration) of the measurement target acquired at the same robot position. Accordingly, the re-calibration does not require the labor and time for attaching and detaching the calibration jig, unlike the conventional techniques, and the three-dimensional visual sensor can be restored to its normal state in a very short time.
-
FIG. 1 shows the configuration of a principal part of a three-dimensional visual sensor used in an embodiment of the invention; -
FIG. 2 illustrates a method of calibrating the three-dimensional visual sensor by using a calibration jig; -
FIG. 3 illustrates details of the calibration; -
FIG. 4 illustrates anarrangement 1 according to the embodiment; -
FIG. 5 illustrates an arrangement 2 according to the embodiment; -
FIG. 6 illustrates a position data acquisition step of the embodiment; -
FIG. 7 illustrates a re-calibration step of the embodiment; -
FIG. 8 shows, by way of example, a dot pattern constituted by a matrix of N dots (N=12), wherein the coordinates of first and last dots are (Xa1, Ya1, Za1) and (XaN, YaN, ZaN), respectively; -
FIG. 9 is a diagram explaining that the positions of the individual dots of the dot pattern can be calculated from the combination of information on the arrangement of the dot pattern and information on the position/orientation of a measurement target; -
FIG. 10 is a diagram summarizing the relationship of processes explained with reference to the embodiment; -
FIG. 11 is a diagram illustrating a modification of the relationship shown inFIG. 10 ; and -
FIG. 12 is a diagram summarizing the relationship of processes which are modifications of the processes explained with reference to the embodiment of the present invention. -
FIG. 1 shows the configuration of a principal part of a three-dimensional visual sensor (hereinafter also referred to merely as “visual sensor”) used in an embodiment of the invention. The illustrated visual sensor itself is a typical one well known in the art, and comprises asensor head 30 including an optical system and optical detection means for measuring an object, and a three-dimensionalvisual sensor controller 10 for controlling thesensor head 30 and processing information related to three-dimensional measurement. In thesensor head 30 are arranged alaser beam emitter 34, alaser driving section 32 for driving the emitter, amirror 35 for scanning an object with the laser beam, and amirror scanning section 33 for driving the mirror. The sensor head is also equipped with acamera 31 for acquiring a two-dimensional image of anobject 40 and receiving the reflection of the laser beam irradiated onto theobject 40. - On the other hand, the three-dimensional
visual sensor controller 10 includes aCPU 11, a group of memories such as aROM 12, aRAM 13, anonvolatile memory 14 and aframe memory 17, a communication I/F (interface) 15 for exchanging data with a robot controller etc., an input/output device I/F (interface) 16 for exchanging data with external devices such as a monitor, a camera I/F (interface) 18 for driving thecamera 31 to acquire images therefrom, and a laser control I/F (interface) 19 for exchanging data with thelaser driving section 32 and themirror scanning section 33. - At the time of three-dimensional measurement, a program stored in the
ROM 12 or the like is executed by theCPU 11, whereupon the laser beam is irradiated onto theobject 40 via themirror 35 set at a predetermined angle θ (usually, the mirror is set successively at a plurality of angles), and with respect to each set angle θ, an image of the laser beam is acquired through thecamera 31 and stored in theframe memory 17. Then, feature amounts are extracted from the acquired images by image processing, and using the feature amounts and calibration data previously stored in thenonvolatile memory 14 or the like (calibration data then stored), information such as the position and orientation of theobject 40 is calculated. In the case of normal imaging of theobject 40 by thecamera 31, two-dimensional images are stored in theframe memory 17 and subjected to required image processing to extract feature amounts from the images. Also, using the calibration data as needed which is previously stored in thenonvolatile memory 14 or the like, feature amount data etc. are calculated. - The present invention is applied to re-calibration of a three-dimensional visual sensor and thus is naturally premised on the assumption that proper calibration is executed prior to the re-calibration to obtain a plurality of parameters representative of the contents of the calibration (hereinafter also referred to merely as “calibration parameters”). The calibration parameters obtained by the calibration prior to the re-calibration are also necessary for the “acquisition of position data (in this embodiment, dot pattern position data)”, described later.
- Referring also to
FIG. 2 , the “calibration prior to re-calibration” will be explained. In the following description of the present invention, the term “each” is used also to designate an element which can in practice be single in number. To perform the calibration, first, acalibration jig 50 having ameasurement target 51 for calibration, as shown inFIG. 2 , is attached to thesensor head 30 and themeasurement target 51 is set in a predetermined position. The measurement target is set in a single position or is sequentially set in a plurality of positions, depending on the calibration method used. In the illustrated example, the measurement target is sequentially set in two positions. At each of the set positions, predetermined measurement is carried out by using thesensor head 30 and thevisual sensor controller 10 in accordance with the calibration method employed, whereby calibration parameters are obtained. In the following, an exemplary calibration method will be explained. - The
measurement target 51 has a dot pattern or the like printed thereon, and thecalibration jig 50 is configured such that when themeasurement target 51 is set in each of the predetermined positions, each dot of thedot pattern 52 is located at a certain known position relative to thesensor head 30. Details of the calibration will be explained with reference toFIG. 3 . - In
FIG. 3 ,reference signs FIG. 2 ) positioned in the respective set positions. - In the following, the
signs FIG. 3 . Similarly,reference signs FIG. 2 ) assumes when the measurement target is set in the respective positions shown inFIG. 3 . - As already mentioned, when the measurement target is placed in each of the set positions by means of the
calibration jig 50, each dot of thedot pattern system 80 is defined, as shown in the figure, with respect to themeasurement target 60A as a sensor measurement reference coordinate system (hereinafter referred to as sensor coordinate system), the positions of thedot patterns
Position ofDot Pattern 61=(Xa 1,Ya 1, Za 1)
Position ofDot Pattern 62=(Xb 1,Yb 1, Zb 1)
where, in the illustrated example, Za1=0. - If the number of dots of the dot pattern on the measurement target is N and the dots are labeled 1 to N, respectively, then the positions of the individual dots can generally be expressed as follows:
Position of i-th Dot ofDot Pattern 61=(Xai, Yai, Zai)
Position of i-th Dot ofDot Pattern 62=(Xbi, Ybi, Zbi)
where i=1, 2, . . . , N, and in the illustrated example, Za1=Za2= . . . =ZaN=0. - Then, the
measurement target camera 31, whereupon an image of the measurement target is formed on the image plane on the extension of a line ofsight individual dot patterns
(Vai, Hai)
(Vbi, Hbi) (where i=1, 2, . . . , N)
(Vai, Hai) can be regarded as a kind of feature amount detected with respect to thedot pattern 61, and (Vbi, Hbi) can be regarded as a kind of feature amount detected with respect to thedot pattern 62. - Since the position of each dot on the sensor coordinate
system 80 is known as mentioned above, a calibration calculation technique known in the art is applied to the combination of the feature amounts, whereby, for a given optional image feature amount (V, H), an intersection point (Xa, Ya, Za) where the line of sight determined by the image feature amount intersects with themeasurement target 60A and an intersection point (Xb, Yb, Zb) where the line of sight intersects with themeasurement target 60B can be calculated. - The equations for calculating the intersection points on the
measurement targets
(V, H)→(Xa, Ya, Za)
(V, H)→(Xb, Yb, Zb) (1) - Once the two intersection points (Xa, Ya, Za) and (Xb, Yb, Zb) are obtained, the equation of a straight line passing through these two points can be derived by a simple calculation. The equation representing the straight line is a “line-of-sight equation”, which is expressed simply by the following expression (2):
(Xa, Ya, Za), (Xb, Yb, Zb)→“Line-of-sight Equation” (2) - Subsequently, a
brilliant point mirror 35 directed at a desired angle θ is imaged by thecamera 31, whereupon an image of the brilliant point is formed on the image plane on the extension of a line ofsight system 80 by using the relationship of the above expressions (1). - In cases where the laser beam is a slit beam, two straight brilliant lines are formed on the respective measurement targets, in place of the
brilliant points - Similarly, with the angle θ of the
mirror 35 changed to different angles, the laser slit beam plane at each angle is obtained, whereby the laser slit beam plane corresponding to a desired θ can be calculated. The equation for calculating the laser slit beam plane is expressed simply by the following expression (3):
θ→“Equation of Laser Slit Beam Plane” (3) - Thus, the equations (1) to (3) are derived, whereupon the calibration is completed. Specifically, the contents of calculations of the individual equations (1) to (3) (e.g., polynomials) and a plurality of specified parameters (e.g., coefficients and constants of individual terms of the polynomials) are stored in the memory of the visual sensor.
- The aforementioned calibration method is carried out with the measurement target set at two positions. The calibration may alternatively be performed with the measurement target set at a single position. An example of such calibration will be briefly explained below. In the following, the
measurement target 60A shown inFIG. 3 is assumed to be the “measurement target set at a single position” and the coordinatesystem 80 defined on themeasurement target 60A is referred to as “measurement target coordinate system”. - If the pixel size of the
camera 31 and the focal distance of the lens used in thecamera 31 are already known, then the imaging plane can be assumed to be located at a position spaced at the focal distance from the center point of the lens of the camera. When an optional image feature amount (V, H) is given, a coordinate system (three-dimensional coordinate system whose XY plane coincides with the imaging plane; hereinafter referred to as imaging plane coordinate system) is defined with respect to the imaging plane of the camera, whereby the position (Xc, Yc, Zc) of the object image can be determined. Namely, with respect to the pattern of N dots on the measurement target, the following combination -
- Image Feature amount: (Vai, Hai)
- Object Image Position on Imaging Plane Coordinate System: (Xci, Yci, Zci)
- Object Position on Measurement Target Coordinate System: (Xai, Yai, Zai) (where i=1, 2, . . . , N)
is obtained. Calibration calculation is then performed on the combination, whereby relative positional relation between the camera imaging plane coordinate system and the measurement target coordinate system is obtained. Consequently, when an optional image feature amount (V, H) is given, the object image position (Xc, Yc, Zc) on the imaging plane coordinate system and the intersection point (Xa, Ya, Za) where the line of sight intersects with themeasurement target 60A can be calculated.
- The equations for calculating these values are expressed simply by the following set of expressions (1′):
(V, H)→(Xc, Yc, Zc)
(V, H)→(Xa, Ya, Za) (1′) - Once the two points (Xc, Yc, Zc) and (Xa, Ya, Za) are obtained, the line-of-sight equation can be obtained by a simple calculation.
- The calculation equation is expressed simply by the following expression (2′):
(Xc, Yc, Zc), (Xa, Ya, Za)→“Line-of-sight Equation” (2′) - Subsequently, as in the case of the aforementioned “two-position setting”, the
brilliant point 63 of the laser beam irradiated with themirror 35 directed at a desired angle θ is imaged by thecamera 31, whereupon an image of the brilliant point is formed on the image plane on the extension of the line ofsight 73. Thebrilliant point 63 is obtained as a three-dimensional position on the measurement target coordinatesystem 80 by using the relationship of the above expressions (1′). - In cases where the laser beam is a slit beam, a straight brilliant line is formed on the
measurement target 60A, in place of thebrilliant point 63. Where thelaser beam emitter 34 is a point light source, the point light source and a straight line (brilliant line) represented by thebrilliant point 63 form one plane. Thus, the angle θ of themirror 35 is changed successively to different angles each by δθ (small angle) and the plane derived at each angle is correlated with the corresponding relative angle varied by δθ, whereby the position of the point light source can be obtained. Consequently, for a given optional angle θ, the laser slit beam plane for that angle can be calculated. - The calculation equation is expressed simply by the following expression (3′):
θ→“Equation of Laser Slit Beam Plane” (3′) - Thus, the equations (1′) to (3′) are derived, whereupon the calibration by the “single-position setting” method is completed. Specifically, the contents of calculations of the individual equations (1′) to (3′) (e.g., polynomials) and a plurality of specified parameters (e.g., coefficients and constants of individual terms of the polynomials) are stored in the memory of the visual sensor, as in the case of the “two-position setting”.
- Using the three-dimensional visual sensor which has been calibrated by the “one-position setting” or “two-position setting” method as described above, an object is measured in the following manner. First, based on the angle θ of the mirror during the measurement, the laser slit beam plane for that angle is obtained according to the equation (3) or (3′), and then based on the image feature amount (V, H) indicative of the brilliant point of the laser beam, the line of sight from the
camera 31 to the brilliant point is derived according to the equations (1) and (2) or the equations (1′) and (2′). Subsequently, the equation of laser slit beam plane and the line-of-sight equation are simultaneously solved, thereby obtaining the three-dimensional position of the brilliant point. - The three-dimensional visual sensor, which is now capable of normal measurement, is detached from the
jig 50 and then is arranged on the robot arm or at a fixed position external to the robot, depending on the application. The following describes exemplary cases of “arrangement 1” and “arrangement 2” wherein, for convenience' sake,reference numeral 103 is used to denote the measurement target (51, 60A, 60B) explained above in connection with the calibration. - Arrangement 1: As shown in
FIG. 4 , a sensor head 102 (identical with the one shown inFIG. 1 , as is the case with other elements) is mounted on the arm of arobot 101 while themeasurement target 103 is fixed at a suitable position (close to therobot 101 but free from interference). In the illustrated example, themeasurement target 103 is fixed to the base of the robot by a fixing member. - The
robot 101 is connected to arobot controller 100, and a three-dimensionalvisual sensor controller 104 is connected with thesensor head 102. Therobot controller 100 and the three-dimensionalvisual sensor controller 104 are connected to each other by, for example, acommunication line 105. - Arrangement 2: As shown in
FIG. 5 , thesensor head 102 is arranged at a fixed position external to the robot 101 (in the illustrated example, a fixed position on a support 106). Themeasurement target 103 is attached to the arm of therobot 101. - Like the
arrangement 1, therobot 101 is connected to therobot controller 100, and the three-dimensionalvisual sensor controller 104 is connected with thesensor head 102. Also, therobot controller 100 and the three-dimensionalvisual sensor controller 104 are connected to each other by thecommunication line 105, for example. - Whichever arrangement is adopted, while the
visual sensor 102 is capable of normal measurement, a series of processes, which includes moving the robot arm to two different optional positions P1 and P2 previously set with respect to therobot 101, measuring themeasurement target 103 and storing data on the dot pattern, is executed for future re-calibration.FIG. 6 is a flowchart summarizing the series of processes. In the following, the individual steps will be explained. - Step S1: The robot is moved to the position P1, whereby the
visual sensor 102 and themeasurement target 103 are arranged in a “first relative positional relation”. - Step S2: A command (including a position data acquisition command) for causing the
visual sensor 102 to measure themeasurement target 103 is output to the three-dimensionalvisual sensor controller 104 by means of manual operation of the keyboard of therobot controller 100, a programmed instruction, etc. - Step S3: The dot pattern position in the sensor coordinate system (or measurement target coordinate system) 80 is calculated. In this case, the calibration parameters saved at the time of the aforementioned calibration are used. It should be noted here that the sensor coordinate system (or measurement target coordinate system) 80 is not necessarily limited to the one established on one measurement target, unlike the case illustrated in
FIG. 3 . - Step S4: The data obtained in Step S3 is stored in the
nonvolatile memory 14 of the visual sensor as “dot pattern position A”. - Step S5: The robot is moved to the position P2, whereby the
visual sensor 102 and themeasurement target 103 are arranged in a “second relative positional relation”. - Step S6: A command (including a position data acquisition command) for causing the
visual sensor 102 to measure themeasurement target 103 is output to the three-dimensionalvisual sensor controller 104 by means of manual operation of the keyboard of therobot controller 100, a programmed instruction, etc. - Step S7: The dot pattern position in the sensor coordinate system (or measurement target coordinate system) 80 is calculated again. In this case, the calibration parameters saved at the time of the aforementioned calibration are used. It should also be noted here that the sensor coordinate system (or measurement target coordinate system) 80 is not necessarily limited to the one established on one measurement target, unlike the case illustrated in
FIG. 3 . - Step S8: The data obtained in Step S7 is stored in the
nonvolatile memory 14 of the visual sensor as “dot pattern position B”. - The above completes the preparatory process (position data acquisition step) for re-calibration. If re-calibration of the
visual sensor 102 is required thereafter due to the various causes mentioned in the BACKGROUND OF THE INVENTION section, a re-calibration step is executed.FIG. 7 is a flowchart summarizing the re-calibration, and in the following, the individual steps will be explained. - Step S11: The robot is moved to the position P1. Consequently, the “first relative positional relation” between the
visual sensor 102 and themeasurement target 103 is reproduced at least approximately. (Generally, the position/orientation of the visual sensor may presumably have changed, and thus the reproduced relation involves corresponding inaccuracy.) - Step S12: A command (including a re-calibration command) for causing the
visual sensor 102 to measure themeasurement target 103 is output to the three-dimensionalvisual sensor controller 104 by means of manual operation of the keyboard of therobot controller 100, a programmed instruction, etc. - Step S13: Image coordinates of the dot pattern on the
measurement target 103 are extracted. - Step S14: The image coordinates extracted in Step S13 are temporarily saved in the
nonvolatile memory 14 of the visual sensor as image coordinates A. - Step S15: The robot is moved to the position P2. Consequently, the “second relative positional relation” between the
visual sensor 102 and themeasurement target 103 is reproduced at least approximately. (Also in this case, generally, the position/orientation of the visual sensor may presumably have changed, and thus the reproduced relation involves corresponding inaccuracy.) - Step S16: A command (including a re-calibration command) for causing the
visual sensor 102 to measure themeasurement target 103 is output to the three-dimensionalvisual sensor controller 104 by means of manual operation of the keyboard of therobot controller 100, a programmed instruction, etc. - Step S17: Image coordinates of the dot pattern on the
measurement target 103 are extracted. - Step S18: The image coordinates extracted in Step S17 are temporarily saved in the
nonvolatile memory 14 of the visual sensor as image coordinates B. - Step S19: Using the data “dot pattern position A”, “dot pattern position B”, “image coordinates A” and “image coordinates B”, calibration calculation for the re-calibration is performed. Namely, the aforementioned equations (1) and (2) or the equations (1′) and (2′) can be updated (recalculated and stored) by using these items of data. Also, by performing the aforementioned laser beam calibration, it is possible to update (recalculate and store) the equation (3) or (3′). This completes the re-calibration.
- Step S20: The calibration parameters obtained in Step S19 are saved in the memory of the visual sensor, in place of the previous calibration parameters, whereupon the process ends.
- In the above description of the embodiment, the three-dimensional visual sensor and the measurement target are brought into two relative positional relations by the robot. Where the aforementioned calibration method in which the measurement target is set in a single position is employed, however, only one relative positional relation needs to be determined.
- Also, as a modification of the
arrangement 1 or 2, themeasurement target 103 may be arranged on the arm of a different robot so as to establish each of the relative positional relations and to reproduce the relations at least approximately. For example, the measurement target may be attached to the arm of a “different robot”, and the position P3 may be used as a substitute for the aforementioned “fixed position”. - Further, in the above description of the embodiment, the measurement target is measured by the three-dimensional visual sensor at the time of acquiring reference data and at the time of re-calibration, to obtain “position” data of the measurement target. More specifically, in Steps S3 and S4 of the flowchart of
FIG. 6 , the dot pattern position of the measurement target with reference to the sensor coordinate system is detected and saved as the dot pattern position A, and in Steps S7 and S8, the dot postern position of the measurement target with reference to the sensor coordinate system is detected and saved as the dot pattern position B (reference data acquisition). Then, in Steps S13, S14, S17 and S18 of the flowchart ofFIG. 7 , the extraction of image coordinates of the dot pattern on the measurement target and the temporary storage of the extracted image coordinates A and B are performed in like manner (data acquisition for re-calibration). - The present invention can be equally carried out by acquiring “orientation” data of the measurement target, instead of the “position” data of same. The reason will be briefly explained below. The following explanation is directed to the dot pattern position A only, because the dot pattern positions A and B of the above embodiment, though described separately, are obtained with the same measurement target set in different positions relative to the three-dimensional visual sensor, as already explained.
- The manner of storing the dot pattern position A according to the foregoing embodiment exemplifies one method of measuring the measurement target and storing the position information relating thereto. In accordance with the expression used in this specification, the position data (Xai, Yai, Zai) (where i=1, 2, . . . , N) is stored. Provided the sensor coordinate system is expressed by Σs, (Xai, Yai, Zai) represents coordinates in Σs.
FIG. 8 shows an example of a dot pattern constituted by a matrix of N dots (N=12) and indicates that the coordinates of first and last dots are (Xa1, Ya1, Za1) and (XaN, YaN, ZaN), respectively. - As another method of measuring the measurement target and storing the position information relating thereto, “orientation” of the measurement target may be stored by the method described below. If the arrangement of the dot pattern on the measurement target is already known, information equivalent to that obtained by measuring the positions of the individual dots of the dot pattern can be derived by measuring the position/orientation of the measurement target. Namely, if information on the arrangement of the dot pattern is prepared in advance, the positions of the individual dots of the dot pattern can be calculated immediately by using the arrangement information in combination with information on the position/orientation of the measurement target. This will be explained with reference to
FIG. 9 . - In the following, a coordinate system fixedly established on the measurement target is referred to as Σt. Assuming that the arrangement of the dot pattern on the measurement target is already known, the arrangement of the individual dots is expressed by
Di(where i=1, 2, . . . , N)
where Di indicates coordinates in the coordinate system Σt. More specifically, the arrangement of the individual dots is expressed by
Di=(xi, yi, zi) (where i=1, 2, . . . , N) - The measurement target is then measured and the position information relating thereto is stored as Σt position/orientation (hereinafter expressed by T) as viewed from the sensor coordinate system Σs. As is known in the art, T is a homogeneous transformation matrix. Thus, if Di is previously known, positions (coordinates) on the sensor coordinate system Σs can be converted to those on the coordinate system Σt and vice versa by using the equation (4) below. Storing the position/orientation T of the measurement target is therefore equivalent to storing (Xai, Yai, Zai).
- The processes explained above with reference to the embodiment can be summarized as shown in the relationship diagram of
FIG. 10 . In the figure, “measurement object” is used as a more generic term for the “measurement target” referred to in the foregoing description of the embodiment. According to the present invention, it is also possible to modify the illustrated relationship. In the relationship diagram ofFIG. 10 , the position information on the measurement object (measurement target) is calculated from the feature amount information on the measurement object (measurement target) and the parameters relating to calibration of the three-dimensional visual sensor already held by the same sensor (as data before updating). Accordingly, the relationship shown inFIG. 10 can be rewritten as shown inFIG. 11 . - Here let us consider the timing of acquiring the position information. In the above embodiment (
FIG. 10 ), the position information (position/orientation data) is acquired at the time of acquiring the reference data, but it will be understood that essentially the feature amount information on the measurement object (measurement target) has only to be acquired at that timing. - In view of this, the relationship of processes shown in
FIG. 12 may be adopted in place of the relationship shown inFIG. 10 . Namely, to acquire the reference data, the three-dimensional visual sensor and the measurement target (measurement object) are arranged in at least one optional relative positional relation while the three-dimensional visual sensor is in a state capable of normal measurement, and then the measurement target (measurement object) is measured by the three-dimensional visual sensor to acquire feature amount data on the measurement target (measurement object), the acquired feature amount data being stored. - Then, at the time of re-calibrating the three-dimensional visual sensor after the three-dimensional visual sensor was in the state capable of normal measurement, the three-dimensional visual sensor and the measurement target (measurement object) are arranged so that the at least one relative positional relation may be reproduced at least approximately, and then the measurement target (measurement object) is measured by the three-dimensional visual sensor to acquire feature amount data on the measurement target (measurement object).
- Subsequently, based on the parameters relating to calibration obtained while the three-dimensional visual sensor was in the state capable of normal measurement, the feature amount data obtained at the time of the reference data acquisition and the feature amount data obtained at the time of the re-calibration data acquisition, the parameters relating to calibration of the three-dimensional sensor are updated.
- Like the foregoing embodiment, arranging the three-dimensional visual sensor and the measurement object in the at least one relative positional relation at the time of the reference data acquisition and arranging the same elements so as to reproduce the at least one relative positional relation at least approximately at the time of the re-calibration are carried out by moving the robot which supports at least one of the three-dimensional visual sensor and the measurement object. Also, to reproduce the at least one relative positional relation at least approximately at the time of the re-calibration, the robot is moved to reproduce the robot position where the at least one relative positional relation was established at the time of the reference data acquisition.
- Further, instead of defining the two relative positional relations between the three-dimensional visual sensor and the measurement target by means of the robot, the method wherein calibration is performed with the measurement target set in a single position may be employed as mentioned above, in which case only one relative positional relation may be defined. Moreover, the method may be employed wherein the
measurement target 103 is arranged on the arm of a different robot to establish the at least one relative positional relation and to reproduce the at least one relative positional relation at least approximately.
Claims (12)
1. A method of re-calibrating a three-dimensional visual sensor holding a plurality of parameters relating to calibration in a robot system, said method comprising the steps of:
(a) arranging the three-dimensional visual sensor and a measurement target in at least one optional relative positional relation while the three-dimensional visual sensor is in a state capable of normal measurement, measuring the measurement target by the three-dimensional visual sensor to acquire position/orientation data indicative of at least one of position and orientation of the measurement target by using the parameters relating to calibration then held by the three-dimensional visual sensor, and storing the position/orientation data;
(b) arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately when the three-dimensional visual sensor is to be recalibrated after the three-dimensional visual sensor was in said state, and then measuring the measurement target by the three-dimensional visual sensor to acquire feature amount data on the measurement target; and
(c) updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data,
wherein arranging the three-dimensional visual sensor and the measurement target in the at least one relative positional relation, in said step (a), and arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately, in said step (b), are carried out by moving a robot which supports at least one of the three-dimensional visual sensor and the measurement target, and
in said step (b), the robot is moved to reproduce a robot position where the at least one relative positional relation was established in said step (a), in order to reproduce the at least one relative positional relation at least approximately in said step (b).
2. A method of re-calibrating a three-dimensional visual sensor according to claim 1 , wherein one of the three-dimensional visual sensor and the measurement target is mounted on an arm of the robot while the other is placed at a predetermined position other than the arm of the robot, so as to be arranged in the at least one relative positional relation in said step (a), and to reproduce the at least one relative positional relation at least approximately in said step (b).
3. A method of re-calibrating a three-dimensional visual sensor according to claim 1 , wherein the three-dimensional visual sensor is mounted on an arm of the robot and the measurement target is placed at a position different from the arm of the robot, so as to be arranged in the at least one relative positional relation in said step (a), and to reproduce the at least one relative positional relation at least approximately in said step (b).
4. A method of re-calibrating a three-dimensional visual sensor according to claim 1 , wherein the three-dimensional visual sensor is mounted on an arm of the robot and the measurement target is placed on an arm of a different robot, so as to be arranged in the at least one relative positional relation in said step (a), and to reproduce the at least one relative positional relation at least approximately in said step (b).
5. A method of re-calibrating a three-dimensional visual sensor holding a plurality of parameters relating to calibration in a robot system, said method comprising the steps of:
(a) arranging the three-dimensional visual sensor and a measurement target in at least one optional relative positional relation while the three-dimensional visual sensor is in a state capable of normal measurement, then measuring the measurement target by the three-dimensional visual sensor to acquire feature amount data on the measurement target, and storing the feature amount data;
(b) arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately when the three-dimensional visual sensor is to be recalibrated after the three-dimensional visual sensor was in said state, and then measuring the measurement target by the three-dimensional visual sensor to acquire feature amount data on the measurement target; and
(c) updating the parameters relating to calibration of the three-dimensional visual sensor, based on the parameters relating to calibration of the three-dimensional visual sensor held at the time of said step (a), the feature amount data acquired in said step (a), and the feature amount data acquired in said step (b),
wherein arranging the three-dimensional visual sensor and the measurement target in the at least one relative positional relation, in said step (a), and arranging the three-dimensional visual sensor and the measurement target so as to reproduce the at least one relative positional relation at least approximately, in said step (b), are carried out by moving a robot which supports at least one of the three-dimensional visual sensor and the measurement target, and
in said step (b), the robot is moved to reproduce a robot position where the at least one relative positional relation was established in said step (a), in order to reproduce the at least one relative positional relation at least approximately in said step (b).
6. A method of re-calibrating a three-dimensional visual sensor according to claim 5 , wherein one of the three-dimensional visual sensor and the measurement target is mounted on an arm of the robot while the other is placed at a predetermined position other than the arm of the robot, so as to be arranged in the at least one relative positional relation in said step (a), and to reproduce the at least one relative positional relation at least approximately in said step (b).
7. A method of re-calibrating a three-dimensional visual sensor according to claim 5 , wherein the three-dimensional visual sensor is mounted on an arm of the robot and the measurement target is placed at a position different from the arm of the robot, so as to be arranged in the at least one relative positional relation in said step (a), and to reproduce the at least one relative positional relation at least approximately in said step (b).
8. A method of re-calibrating a three-dimensional visual sensor according to claim 5 , wherein the three-dimensional visual sensor is mounted on an arm of the robot and the measurement target is placed on an arm of a different robot, so as to be arranged in the at least one relative positional relation in said step (a), and to reproduce the at least one relative positional relation at least approximately in said step (b).
9. A device for re-calibrating a three-dimensional visual sensor mounted on an arm of a robot, comprising:
means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
means for moving the robot to at least one robot position where the three-dimensional visual sensor is located close to a measurement target;
target position information detection means for detecting position information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
target feature-amount information detection means for detecting feature-amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
means, responsive to a reference data acquisition command, for causing the target position information detection means to detect position information on the measurement target, and storing the detected position information as position/orientation data indicative of at least one of position and orientation of the measurement target by using the held parameters relating to calibration of the three-dimensional visual sensor;
means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data.
10. A device for re-calibrating a three-dimensional visual sensor using a measurement target mounted on a robot, comprising:
means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
means for moving the robot to at least one robot position where the measurement target is located close to the three-dimensional visual sensor;
target position information detection means for detecting position information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
means, responsive to a reference data acquisition command, for causing the target position information detection means to detect position information on the measurement target, and storing the detected position information as position/orientation data indicative of at least one of position and orientation of the measurement target by using the held parameters relating to calibration of the three-dimensional visual sensor;
means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data and the position/orientation data.
11. A device for re-calibrating a three-dimensional visual sensor mounted on an arm of a robot, comprising:
means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
means for moving the robot to at least one robot position where the three-dimensional visual sensor is located close to a measurement target;
target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
means, responsive to a reference data acquisition command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and storing the detected feature amount information as reference feature amount data;
means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data, the reference feature amount data, and the parameters relating to calibration of the three-dimensional visual sensor held at the time of reception of the reference data acquisition command.
12. A device for re-calibrating a three-dimensional visual sensor using a measurement target mounted on a robot, comprising:
means for holding a plurality of parameters relating to calibration of the three-dimensional visual sensor;
means for moving the robot to at least one robot position where the measurement target is located close to the three-dimensional visual sensor;
target feature amount information detection means for detecting feature amount information on the measurement target in the at least one robot position, through measurement by the three-dimensional visual sensor;
means, responsive to a reference data acquisition command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and storing the detected feature amount information as reference feature amount data;
means, responsive to a re-calibration command, for causing the target feature amount information detection means to detect feature amount information on the measurement target, and obtaining the detected feature amount information as feature amount data; and
parameter updating means for updating the parameters relating to calibration of the three-dimensional visual sensor, based on the feature amount data, the reference feature amount data, and the parameters relating to calibration of the three-dimensional visual sensor held at the time of reception of the reference data acquisition command.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2004220251A JP3946716B2 (en) | 2004-07-28 | 2004-07-28 | Method and apparatus for recalibrating a three-dimensional visual sensor in a robot system |
JP220251/2004 | 2004-07-28 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20060023938A1 true US20060023938A1 (en) | 2006-02-02 |
US7359817B2 US7359817B2 (en) | 2008-04-15 |
Family
ID=35229893
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/190,946 Expired - Fee Related US7359817B2 (en) | 2004-07-28 | 2005-07-28 | Method of and device for re-calibrating three-dimensional visual sensor in robot system |
Country Status (4)
Country | Link |
---|---|
US (1) | US7359817B2 (en) |
EP (1) | EP1621297A1 (en) |
JP (1) | JP3946716B2 (en) |
CN (1) | CN100368765C (en) |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050162420A1 (en) * | 2004-01-19 | 2005-07-28 | Fanuc Ltd | Three-dimensional visual sensor |
US7359817B2 (en) * | 2004-07-28 | 2008-04-15 | Fanuc Ltd | Method of and device for re-calibrating three-dimensional visual sensor in robot system |
US20100259608A1 (en) * | 2007-09-03 | 2010-10-14 | Antti Knuuttila | Recording machine vision system |
US20140148949A1 (en) * | 2012-11-29 | 2014-05-29 | Fanuc America Corporation | Robot system calibration method |
CN104808490A (en) * | 2015-03-10 | 2015-07-29 | 浙江工业大学 | Uncalibrated visual servoing control method for estimating image Jacobian matrix based on echo state network facing mold protection |
US20150237308A1 (en) * | 2012-02-14 | 2015-08-20 | Kawasaki Jukogyo Kabushiki Kaisha | Imaging inspection apparatus, control device thereof, and method of controlling imaging inspection apparatus |
KR20160079756A (en) * | 2013-02-25 | 2016-07-06 | 코그넥스코오포레이션 | System and method for calibration of machine vision cameras along at least three discrete planes |
US20170151670A1 (en) * | 2015-11-30 | 2017-06-01 | Delta Electronics, Inc. | Tool calibration apparatus of robot manipulator |
US9969090B2 (en) * | 2015-10-05 | 2018-05-15 | Fanuc Corporation | Robot system equipped with camera for capturing image of target mark |
US10307912B2 (en) * | 2013-07-15 | 2019-06-04 | Lg Electronics Inc. | Robot cleaner and method for auto-correcting 3D sensor of the robot cleaner |
US10357879B2 (en) * | 2016-07-20 | 2019-07-23 | Fanuc Corporation | Robot zero-point calibration device and method |
US10614565B2 (en) * | 2017-09-27 | 2020-04-07 | Fanuc Corporation | Inspection device and inspection system |
CN112013761A (en) * | 2019-05-28 | 2020-12-01 | 帝肯贸易股份公司 | Position detector and method for 3D position determination |
US20210354299A1 (en) * | 2018-12-25 | 2021-11-18 | Ubtech Robotics Corp | External parameter calibration method for robot sensors and apparatus, robot and storage medium with the same |
CN114404041A (en) * | 2022-01-19 | 2022-04-29 | 上海精劢医疗科技有限公司 | C-shaped arm imaging parameter calibration system and method |
US20220168902A1 (en) * | 2019-03-25 | 2022-06-02 | Abb Schweiz Ag | Method And Control Arrangement For Determining A Relation Between A Robot Coordinate System And A Movable Apparatus Coordinate System |
US11358283B2 (en) * | 2019-02-08 | 2022-06-14 | Fanuc Corporation | Robot calibration method and robot calibration device |
US20230023844A1 (en) * | 2021-07-23 | 2023-01-26 | Chroma Ate Inc. | Position calibration system and method |
US11571816B2 (en) | 2019-04-12 | 2023-02-07 | Mujin, Inc. | Method and control system for updating camera calibration for robot control |
CN116539068A (en) * | 2023-07-03 | 2023-08-04 | 国网山西省电力公司电力科学研究院 | Flexible self-checking adjusting device and method for vision measurement system |
Families Citing this family (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4267005B2 (en) * | 2006-07-03 | 2009-05-27 | ファナック株式会社 | Measuring apparatus and calibration method |
JP4969279B2 (en) * | 2007-03-22 | 2012-07-04 | 本田技研工業株式会社 | Position detection method and position detection apparatus |
JP5139740B2 (en) * | 2007-07-23 | 2013-02-06 | 大成建設株式会社 | Tail clearance automatic measurement system and tail clearance automatic measurement method |
JP5136108B2 (en) * | 2008-02-18 | 2013-02-06 | トヨタ自動車株式会社 | 3D shape measuring method and 3D shape measuring apparatus |
JP5233601B2 (en) * | 2008-11-07 | 2013-07-10 | セイコーエプソン株式会社 | Robot system, robot control apparatus, and robot control method |
TWI408486B (en) * | 2008-12-30 | 2013-09-11 | Ind Tech Res Inst | Camera with dynamic calibration and method thereof |
CN201852793U (en) * | 2009-09-21 | 2011-06-01 | Abb技术有限公司 | System for producing manufacture part |
US11699247B2 (en) * | 2009-12-24 | 2023-07-11 | Cognex Corporation | System and method for runtime determination of camera miscalibration |
US9393694B2 (en) * | 2010-05-14 | 2016-07-19 | Cognex Corporation | System and method for robust calibration between a machine vision system and a robot |
TWI408037B (en) | 2010-12-03 | 2013-09-11 | Ind Tech Res Inst | A position method and a calibrating method for the robot arm |
CN102135761B (en) * | 2011-01-10 | 2013-08-21 | 穆科明 | Fuzzy self-adaptive control system for parameters of visual sensor |
US9266241B2 (en) * | 2011-03-14 | 2016-02-23 | Matthew E. Trompeter | Robotic work object cell calibration system |
US8643703B1 (en) | 2011-03-30 | 2014-02-04 | Amazon Technologies, Inc. | Viewer tracking image display |
US9449427B1 (en) | 2011-05-13 | 2016-09-20 | Amazon Technologies, Inc. | Intensity modeling for rendering realistic images |
US9041734B2 (en) * | 2011-07-12 | 2015-05-26 | Amazon Technologies, Inc. | Simulating three-dimensional features |
US9852135B1 (en) | 2011-11-29 | 2017-12-26 | Amazon Technologies, Inc. | Context-aware caching |
US9713869B2 (en) * | 2012-03-07 | 2017-07-25 | Matthew E. Trompeter | Calibration of robot work paths |
JP6025386B2 (en) * | 2012-05-02 | 2016-11-16 | キヤノン株式会社 | Image measuring apparatus, image measuring method, and image measuring program |
US9120233B2 (en) * | 2012-05-31 | 2015-09-01 | Toyota Motor Engineering & Manufacturing North America, Inc. | Non-contact optical distance and tactile sensing system and method |
CN102818524A (en) * | 2012-07-31 | 2012-12-12 | 华南理工大学 | On-line robot parameter calibration method based on visual measurement |
US8485017B1 (en) * | 2012-08-01 | 2013-07-16 | Matthew E. Trompeter | Robotic work object cell calibration system |
JP6108860B2 (en) | 2013-02-14 | 2017-04-05 | キヤノン株式会社 | Robot system and control method of robot system |
CN103292709B (en) * | 2013-05-24 | 2015-09-09 | 深圳市华星光电技术有限公司 | Gauging machine routine testing and automatic compensation method |
JP5897532B2 (en) * | 2013-11-05 | 2016-03-30 | ファナック株式会社 | Apparatus and method for picking up articles placed in three-dimensional space by robot |
US9857869B1 (en) | 2014-06-17 | 2018-01-02 | Amazon Technologies, Inc. | Data optimization |
US9211643B1 (en) * | 2014-06-25 | 2015-12-15 | Microsoft Technology Licensing, Llc | Automatic in-situ registration and calibration of robotic arm/sensor/workspace system |
CN104180753A (en) * | 2014-07-31 | 2014-12-03 | 东莞市奥普特自动化科技有限公司 | Rapid calibration method of robot visual system |
KR101616366B1 (en) * | 2014-11-27 | 2016-04-29 | 삼성중공업(주) | Apparatus and method for compensating measuring device |
CN105128009A (en) * | 2015-08-10 | 2015-12-09 | 深圳百思拓威机器人技术有限公司 | Bionic robot with object shape and pose accurate sensing function and sensing method of bionic robot |
CN105844670B (en) * | 2016-03-30 | 2018-12-18 | 广东速美达自动化股份有限公司 | Horizontal machine people moves camera Multipoint movable scaling method |
CN105773661B (en) * | 2016-03-30 | 2018-08-21 | 广东速美达自动化股份有限公司 | Workpiece translational motion rotates scaling method under horizontal machine people's fixed camera |
US10076842B2 (en) | 2016-09-28 | 2018-09-18 | Cognex Corporation | Simultaneous kinematic and hand-eye calibration |
US20200016757A1 (en) * | 2017-03-09 | 2020-01-16 | Mitsubishi Electric Corporation | Robot control apparatus and calibration method |
JP6301045B1 (en) * | 2017-03-09 | 2018-03-28 | 三菱電機株式会社 | Robot control apparatus and calibration method |
CN107167072A (en) * | 2017-05-05 | 2017-09-15 | 华南理工大学 | A kind of vision detection system corrected for LED filament spot welding and detection method |
US10331728B2 (en) | 2017-05-30 | 2019-06-25 | General Electric Company | System and method of robot calibration using image data |
EP3698609B1 (en) * | 2017-10-17 | 2021-02-17 | Signify Holding B.V. | Occupancy sensor calibration and occupancy estimation |
CN109421050B (en) * | 2018-09-06 | 2021-03-26 | 北京猎户星空科技有限公司 | Robot control method and device |
KR102577448B1 (en) * | 2019-01-22 | 2023-09-12 | 삼성전자 주식회사 | Hand eye calibration method and system |
DE102019107964B3 (en) | 2019-03-28 | 2020-08-06 | Franka Emika Gmbh | Projection device for a robot manipulator |
CN109986541A (en) * | 2019-05-06 | 2019-07-09 | 深圳市恒晟智能技术有限公司 | Manipulator |
CN114945450A (en) * | 2020-01-14 | 2022-08-26 | 发那科株式会社 | Robot system |
TW202234184A (en) | 2021-02-25 | 2022-09-01 | 日商發那科股份有限公司 | Simulation device using three-dimensional position information obtained from output from vision sensor |
CN117177845A (en) * | 2021-04-20 | 2023-12-05 | 发那科株式会社 | robot |
CN113778414A (en) * | 2021-11-11 | 2021-12-10 | 深圳市越疆科技有限公司 | Machine vision communication script generation method and device based on graphical programming |
TWI788134B (en) * | 2021-12-08 | 2022-12-21 | 財團法人工業技術研究院 | Calibration device and calibration method for autonomous control equipment |
CN114918928B (en) * | 2022-07-22 | 2022-10-28 | 杭州柳叶刀机器人有限公司 | Method and device for accurately positioning surgical mechanical arm, control terminal and storage medium |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US2896501A (en) * | 1953-05-28 | 1959-07-28 | Faximile Inc | Apparatus for outlining contours |
US2964644A (en) * | 1957-11-14 | 1960-12-13 | Hunting Survey Corp Ltd | Method and apparatus for locating corresponding areas of two similar images |
US3078042A (en) * | 1959-09-23 | 1963-02-19 | Gilbert R Grado | Coordinate transformation computer |
US5671056A (en) * | 1995-05-11 | 1997-09-23 | Technology Research Association Of Medical & Welfare Apparatus | Three-dimensional form measuring apparatus and method |
US5706419A (en) * | 1995-02-24 | 1998-01-06 | Canon Kabushiki Kaisha | Image capturing and processing apparatus and image capturing and processing method |
US6114824A (en) * | 1990-07-19 | 2000-09-05 | Fanuc Ltd. | Calibration method for a visual sensor |
US6321137B1 (en) * | 1997-09-04 | 2001-11-20 | Dynalog, Inc. | Method for calibration of a robot inspection system |
US20030187548A1 (en) * | 2002-03-29 | 2003-10-02 | Farhang Sakhitab | Methods and apparatus for precision placement of an optical component on a substrate and precision assembly thereof into a fiberoptic telecommunication package |
US6798527B2 (en) * | 2001-04-27 | 2004-09-28 | Minolta Co., Ltd. | Three-dimensional shape-measuring system |
US20070009149A1 (en) * | 2003-09-26 | 2007-01-11 | Micro-Epsilon Messtechnik Gmbh & Co. Kg | Method and device for the contactless optical determination of the 3D position of an object |
US7209161B2 (en) * | 2002-07-15 | 2007-04-24 | The Boeing Company | Method and apparatus for aligning a pair of digital cameras forming a three dimensional image to compensate for a physical misalignment of cameras |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH02183102A (en) | 1989-01-10 | 1990-07-17 | Fanuc Ltd | Automatic calibration adjusting system for visual sensor |
JP2690603B2 (en) | 1990-05-30 | 1997-12-10 | ファナック株式会社 | Vision sensor calibration method |
JPH0516083A (en) | 1991-02-14 | 1993-01-26 | Komatsu Ltd | Calibration device of robot |
JPH085351A (en) | 1994-06-20 | 1996-01-12 | Fanuc Ltd | Slit beam calibration method of three-dimensional visual sensor |
JPH08233516A (en) | 1995-03-01 | 1996-09-13 | Kobe Steel Ltd | Calibration method of three-dimensional visual sensor |
JPH08272414A (en) | 1995-03-29 | 1996-10-18 | Fanuc Ltd | Calibrating method for robot and visual sensor using hand camera |
JPH10336701A (en) | 1997-05-28 | 1998-12-18 | Sony Corp | Calibration method and its device |
JP2001349707A (en) * | 2000-06-12 | 2001-12-21 | Asia Air Survey Co Ltd | Three-dimensional position measuring system of moving body |
JP2004193947A (en) | 2002-12-11 | 2004-07-08 | Fuji Xerox Co Ltd | Three-dimensional imaging apparatus and method |
JP2005172610A (en) * | 2003-12-11 | 2005-06-30 | Mitsutoyo Corp | Three-dimensional measurement apparatus |
JP4021413B2 (en) * | 2004-01-16 | 2007-12-12 | ファナック株式会社 | Measuring device |
JP3946716B2 (en) * | 2004-07-28 | 2007-07-18 | ファナック株式会社 | Method and apparatus for recalibrating a three-dimensional visual sensor in a robot system |
-
2004
- 2004-07-28 JP JP2004220251A patent/JP3946716B2/en not_active Expired - Fee Related
-
2005
- 2005-07-28 CN CNB2005100873653A patent/CN100368765C/en not_active Expired - Fee Related
- 2005-07-28 US US11/190,946 patent/US7359817B2/en not_active Expired - Fee Related
- 2005-07-28 EP EP05254725A patent/EP1621297A1/en not_active Withdrawn
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US2896501A (en) * | 1953-05-28 | 1959-07-28 | Faximile Inc | Apparatus for outlining contours |
US2964644A (en) * | 1957-11-14 | 1960-12-13 | Hunting Survey Corp Ltd | Method and apparatus for locating corresponding areas of two similar images |
US3078042A (en) * | 1959-09-23 | 1963-02-19 | Gilbert R Grado | Coordinate transformation computer |
US6114824A (en) * | 1990-07-19 | 2000-09-05 | Fanuc Ltd. | Calibration method for a visual sensor |
US5706419A (en) * | 1995-02-24 | 1998-01-06 | Canon Kabushiki Kaisha | Image capturing and processing apparatus and image capturing and processing method |
US5671056A (en) * | 1995-05-11 | 1997-09-23 | Technology Research Association Of Medical & Welfare Apparatus | Three-dimensional form measuring apparatus and method |
US6321137B1 (en) * | 1997-09-04 | 2001-11-20 | Dynalog, Inc. | Method for calibration of a robot inspection system |
US6798527B2 (en) * | 2001-04-27 | 2004-09-28 | Minolta Co., Ltd. | Three-dimensional shape-measuring system |
US20030187548A1 (en) * | 2002-03-29 | 2003-10-02 | Farhang Sakhitab | Methods and apparatus for precision placement of an optical component on a substrate and precision assembly thereof into a fiberoptic telecommunication package |
US7209161B2 (en) * | 2002-07-15 | 2007-04-24 | The Boeing Company | Method and apparatus for aligning a pair of digital cameras forming a three dimensional image to compensate for a physical misalignment of cameras |
US20070009149A1 (en) * | 2003-09-26 | 2007-01-11 | Micro-Epsilon Messtechnik Gmbh & Co. Kg | Method and device for the contactless optical determination of the 3D position of an object |
Cited By (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050162420A1 (en) * | 2004-01-19 | 2005-07-28 | Fanuc Ltd | Three-dimensional visual sensor |
US7202957B2 (en) * | 2004-01-19 | 2007-04-10 | Fanuc Ltd | Three-dimensional visual sensor |
US7359817B2 (en) * | 2004-07-28 | 2008-04-15 | Fanuc Ltd | Method of and device for re-calibrating three-dimensional visual sensor in robot system |
US20100259608A1 (en) * | 2007-09-03 | 2010-10-14 | Antti Knuuttila | Recording machine vision system |
US10012500B2 (en) * | 2007-09-03 | 2018-07-03 | Oy Mapvision Ltd. | Storing data for re-measurement of objects using a recording machine vision system |
US20150237308A1 (en) * | 2012-02-14 | 2015-08-20 | Kawasaki Jukogyo Kabushiki Kaisha | Imaging inspection apparatus, control device thereof, and method of controlling imaging inspection apparatus |
US9774827B2 (en) * | 2012-02-14 | 2017-09-26 | Kawasaki Jukogyo Kabushiki Kaisha | Imaging inspection apparatus for setting one or more image-capturing positions on a line that connects two taught positions, control device thereof, and method of controlling imaging inspection apparatus |
US9417625B2 (en) * | 2012-11-29 | 2016-08-16 | Fanuc America Corporation | Robot system calibration method |
US20140148949A1 (en) * | 2012-11-29 | 2014-05-29 | Fanuc America Corporation | Robot system calibration method |
US11544874B2 (en) | 2013-02-25 | 2023-01-03 | Cognex Corporation | System and method for calibration of machine vision cameras along at least three discrete planes |
KR20160079756A (en) * | 2013-02-25 | 2016-07-06 | 코그넥스코오포레이션 | System and method for calibration of machine vision cameras along at least three discrete planes |
KR102129103B1 (en) * | 2013-02-25 | 2020-07-01 | 코그넥스코오포레이션 | System and method for calibration of machine vision cameras along at least three discrete planes |
US10664994B2 (en) | 2013-02-25 | 2020-05-26 | Cognex Corporation | System and method for calibration of machine vision cameras along at least three discrete planes |
US10307912B2 (en) * | 2013-07-15 | 2019-06-04 | Lg Electronics Inc. | Robot cleaner and method for auto-correcting 3D sensor of the robot cleaner |
CN104808490A (en) * | 2015-03-10 | 2015-07-29 | 浙江工业大学 | Uncalibrated visual servoing control method for estimating image Jacobian matrix based on echo state network facing mold protection |
US9969090B2 (en) * | 2015-10-05 | 2018-05-15 | Fanuc Corporation | Robot system equipped with camera for capturing image of target mark |
US20170151670A1 (en) * | 2015-11-30 | 2017-06-01 | Delta Electronics, Inc. | Tool calibration apparatus of robot manipulator |
US10065319B2 (en) * | 2015-11-30 | 2018-09-04 | Delta Electronics, Inc. | Tool calibration apparatus of robot manipulator |
US10357879B2 (en) * | 2016-07-20 | 2019-07-23 | Fanuc Corporation | Robot zero-point calibration device and method |
US10614565B2 (en) * | 2017-09-27 | 2020-04-07 | Fanuc Corporation | Inspection device and inspection system |
US11590655B2 (en) * | 2018-12-25 | 2023-02-28 | Ubtech Robotics Corp Ltd | External parameter calibration method for robot sensors and apparatus and robot with the same |
US20210354299A1 (en) * | 2018-12-25 | 2021-11-18 | Ubtech Robotics Corp | External parameter calibration method for robot sensors and apparatus, robot and storage medium with the same |
US11358283B2 (en) * | 2019-02-08 | 2022-06-14 | Fanuc Corporation | Robot calibration method and robot calibration device |
US20220168902A1 (en) * | 2019-03-25 | 2022-06-02 | Abb Schweiz Ag | Method And Control Arrangement For Determining A Relation Between A Robot Coordinate System And A Movable Apparatus Coordinate System |
US11571816B2 (en) | 2019-04-12 | 2023-02-07 | Mujin, Inc. | Method and control system for updating camera calibration for robot control |
US11707847B2 (en) * | 2019-05-28 | 2023-07-25 | Tecan Trading Ag | Position detector and method for 3D position determination |
US20200376672A1 (en) * | 2019-05-28 | 2020-12-03 | Tecan Trading Ag | Position detector and method for 3D position determination |
CN112013761A (en) * | 2019-05-28 | 2020-12-01 | 帝肯贸易股份公司 | Position detector and method for 3D position determination |
US20230023844A1 (en) * | 2021-07-23 | 2023-01-26 | Chroma Ate Inc. | Position calibration system and method |
CN114404041A (en) * | 2022-01-19 | 2022-04-29 | 上海精劢医疗科技有限公司 | C-shaped arm imaging parameter calibration system and method |
CN116539068A (en) * | 2023-07-03 | 2023-08-04 | 国网山西省电力公司电力科学研究院 | Flexible self-checking adjusting device and method for vision measurement system |
Also Published As
Publication number | Publication date |
---|---|
US7359817B2 (en) | 2008-04-15 |
CN100368765C (en) | 2008-02-13 |
EP1621297A1 (en) | 2006-02-01 |
CN1727839A (en) | 2006-02-01 |
JP2006035384A (en) | 2006-02-09 |
JP3946716B2 (en) | 2007-07-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7359817B2 (en) | Method of and device for re-calibrating three-dimensional visual sensor in robot system | |
EP1875991B1 (en) | Measuring system and calibration method | |
EP1607194B1 (en) | Robot system comprising a plurality of robots provided with means for calibrating their relative position | |
US7532949B2 (en) | Measuring system | |
US10357879B2 (en) | Robot zero-point calibration device and method | |
EP0493612B1 (en) | Method of calibrating visual sensor | |
EP1584426B1 (en) | Tool center point calibration system | |
US7853359B2 (en) | Calibration device and method for robot mechanism | |
US9199379B2 (en) | Robot system display device | |
US20180345493A1 (en) | Teaching position correction device and teaching position correction method | |
US20080252248A1 (en) | Device and Method for Calibrating the Center Point of a Tool Mounted on a Robot by Means of a Camera | |
US20110029131A1 (en) | Apparatus and method for measuring tool center point position of robot | |
US20220410375A1 (en) | Coordinate system alignment method, alignment system, and alignment device for robot | |
JPH08132373A (en) | Coordinate system coupling method in robot-sensor system | |
US10569418B2 (en) | Robot controller for executing calibration, measurement system and calibration method | |
JPH08272425A (en) | Method to teach coordinate system to robot in non-contact | |
KR101972432B1 (en) | A laser-vision sensor and calibration method thereof | |
JPH1097311A (en) | Correction and setting method for robot tool coordinate system and end effector used for the method | |
KR20060012936A (en) | Robot calibration system using perceptron sensor | |
CN117226856A (en) | Robot self-calibration method and system based on vision |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: FANUC LTD, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BAN, KAZUNORI;KANNO, ICHIRO;REEL/FRAME:016823/0343 Effective date: 20050630 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
CC | Certificate of correction | ||
FPAY | Fee payment |
Year of fee payment: 4 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20160415 |