CN113561182A - Configurable hand-eye calibration method and device - Google Patents
Configurable hand-eye calibration method and device Download PDFInfo
- Publication number
- CN113561182A CN113561182A CN202110917173.XA CN202110917173A CN113561182A CN 113561182 A CN113561182 A CN 113561182A CN 202110917173 A CN202110917173 A CN 202110917173A CN 113561182 A CN113561182 A CN 113561182A
- Authority
- CN
- China
- Prior art keywords
- calibration
- pose
- calibration plate
- robot
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/0095—Means or methods for testing manipulators
Abstract
The invention discloses a configurable hand-eye calibration method and a configurable hand-eye calibration device, which comprise an off-line configuration stage and an on-line automatic calibration stage; the off-line configuration comprises the following steps: the off-line teaching can acquire a plurality of robot poses of clear images and record the poses to be acquired in a pose queue; generating at least two calibration plate images with different poses according to requirements; the on-line automatic calibration comprises coarse calibration and fine calibration. The invention can reduce manual intervention, avoid errors caused by manually recording data, save time of an on-line hand-eye calibration process, avoid the problems of collision and inaccessible path caused by fully automatic path planning, and solve the problem of poor general applicability of different application scenes.
Description
Technical Field
The invention relates to the field of hand-eye calibration, in particular to a configurable hand-eye calibration method and device.
Background
With the rapid development of science and technology, the application of industrial robots is becoming more and more popular, and in the research of industrial robots, the hand-eye calibration technology is an important component of a visual guidance system. The method aims to measure and calculate a posture transformation matrix from a camera coordinate system carried on an executing element at the tail end of a mechanical arm to a coordinate system of an end effector, and is an important premise for ensuring that a visual servo mechanical arm accurately positions and grabs a target.
In modern intelligent factories, for efficient utilization of production facilities, it is desirable that robots can participate in the work of multiple work stations or that the actuators need to be replaced periodically. This requires frequent calibration of the hand-eye system of an industrial robot. In such application scenarios, the hand-eye calibration should be fast and can be automated.
The hand-eye calibration problem can be mathematically expressed as solving a homogeneous equation AX XB, where A, B and X both represent a rotation-translation matrix in homogeneous form, a and B are known observations, and X is the hand-eye transformation matrix to be solved. There are many classical algorithms that can solve this equation, which can be reduced to two stages: the first stage, solving the rotation axis and rotation angle of the X matrix according to the rotation axis and rotation angle of the A and B matrixes by a linear method; and in the second stage, linearly solving the translation part in the X matrix by using the result of the rotation part of the X matrix obtained in the first stage. The method can convert a complex nonlinear problem into a linear equation with a simple form and higher calculation speed for solving. However, errors generated by the calculation of the rotating part in the first stage can be diffused in the solution of the translating part in the second stage, so that the calibration result is easily influenced by the noise of the observed data. Of course, if enough observation data can be collected, the influence of the observation error can be eliminated to some extent.
To conveniently obtain the relative pose relationship between the robot and the vision system, a feasible hand-eye transformation matrix solving algorithm is required, and a set of system capable of being automatically executed is required so as to obtain enough observation data. In the automatic execution process, a series of robot poses and corresponding calibration plate images need to be acquired, and the important part is the motion planning of the acquired image poses. To improve the accuracy of the hand-eye calibration result, four important criteria are followed: firstly, the included angle between the rotating shafts of the relative movement is increased as much as possible; secondly, the rotation angle of the relative movement is increased as much as possible; thirdly, the distance from the calibration plate to the center of the optical axis of the camera is reduced as much as possible; fourthly, the distance between the tail end centers in different postures is reduced as much as possible.
In the prior hand-eye calibration schemes, one type of the prior hand-eye calibration schemes is purely manually acquired data, so that the requirements on professional knowledge of operators are high, errors are easy to occur in data recording, and the time consumed for once calibration is long. The other type is automatic data acquisition, and in the scheme, a method for directly planning the motion path of the robot on line is mostly adopted, so that two problems are easy to occur:
(1) collision or inaccessible path problems, interfering with other devices on the station;
(2) because the calibration result is limited by the data collected during calibration, the actual accuracy is not as ideal as calculated in different application scenarios.
Disclosure of Invention
In order to solve the problems, the invention provides a novel hand-eye calibration method and a novel hand-eye calibration device so as to avoid the problems of collision and inaccessible path and improve the precision of actual positioning and grabbing.
In order to achieve the purpose, the technical scheme of the invention is as follows:
a configurable hand-eye calibration method comprises an off-line configuration stage and an on-line automatic calibration stage;
the off-line configuration comprises the following steps:
the off-line teaching can acquire a plurality of robot poses of clear images and record the poses to be acquired in a pose queue;
generating at least two calibration plate images with different poses according to requirements;
the on-line automatic calibration comprises coarse calibration and fine calibration;
the rough calibration comprises the following steps:
a, the robot moves to an offline configuration result to initialize the first position and pose queue to be acquired;
b, collecting images of the calibration plate, and removing the pose from the queue;
step c, judging the image quality of the calibration plate, if the image quality meets the requirements, storing the current actual pose and the image of the calibration plate, otherwise, executing the step a;
d, judging whether the queue to be acquired is empty, if so, finishing the movement of the robot, and executing the step e, otherwise, executing the step a;
step e, judging whether at least N effective images exist, if so, calculating a hand-eye relation matrix and a reprojection error thereof according to the collected robot pose and calibration plate image data, calibrating internal parameters of a camera, calculating the pose of the calibration plate under a robot coordinate system, and if not, resetting off-line configuration after calibration fails;
the fine calibration comprises the following steps:
step f, calculating the corresponding pose of the tail end of the mechanical arm according to the rough calibrated camera internal parameter, the hand-eye calibration result and the pose of the calibration plate in the robot coordinate system, and sequentially adding the pose of the tail end of the mechanical arm into a robot pose queue;
step g, the robot moves to the pose and collects the image of the calibration plate;
h, judging whether the calibration plate image can be correctly identified or not, if so, saving the current actual pose and the actual calibration plate image, and otherwise, executing the step g;
step i, judging whether the calibration board image queue is empty, if so, executing step j, otherwise, executing step g;
j, judging whether the number of the effective images is larger than or equal to N, if so, calculating the internal parameter and hand-eye transformation matrix of the camera and the reprojection error of the internal parameter and hand-eye transformation matrix of the camera, and if not, directly ending the fine calibration.
Preferably, the off-line configuration teaching robot posture comprises the following steps;
s1, initializing a pose queue and setting the minimum length of the pose queue;
s2, moving the robot to a pose, and collecting a calibration plate image;
s3, judging whether the image is clear, if so, adding the current pose to a pose queue, otherwise, executing the step S2;
and S4, judging whether the pose queue length is equal to the minimum length, if so, finishing off-line configuration of the teaching robot, and if not, executing the step S2.
Preferably, generating calibration plate images with different poses comprises the following steps:
step one, initializing camera internal parameters, a Sudoku area and a mark;
step two, randomly generating a camera gesture;
step three, generating a calibration plate image;
judging whether the center of the calibration plate is in the central area of the nine-square grid or not; if yes, saving the image of the calibration board, otherwise, executing the step two;
and fifthly, judging whether each area is covered by at least two images with different poses, if so, finishing, and otherwise, executing the second step.
Preferably, in the fine calibration stage, the formula for calculating the pose of the end of the mechanical arm in the step f is as follows:
in the formula (I), the compound is shown in the specification,
is the pose of the tool end under the robot base coordinate system;
is the pose of the calibration plate under the coordinate system of the robot base;
calculating the pose of the calibration plate under a camera coordinate system according to internal parameters of the camera and a physical model of the calibration plate;
is the pose of the tool tip under the camera coordinate system.
Preferably, N is greater than or equal to 9.
In order to realize the hand-eye calibration method, the invention also provides a configurable hand-eye calibration device, which comprises an image acquisition module, a display screen I, a display screen II, a calibration plate with a backlight light source, a light source controller, a mechanical arm, an embedded control system and a control system based on a PC;
the image acquisition module, the embedded control system and the control system based on the PC are connected through the Ethernet; the calibration plate with the backlight source is connected with the light source controller; the mechanical arm is connected with the embedded control system; the first display screen is connected with the embedded control system, and parameter setting is carried out through the first display screen; and the second display screen is connected with a control system based on a PC, and parameter setting is carried out through the second display screen.
Compared with the prior art, the invention has the beneficial effects that:
the invention can reduce manual intervention, avoid errors caused by manually recording data, save the time of an online hand-eye calibration process, and control the time for completing one-time online hand-eye calibration within three minutes after offline configuration; secondly, the problems of collision and unreachable path caused by completely automatic path planning can be avoided, and the problem of poor general applicability of different application scenes is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart of an offline configuration teaching robot pose;
FIG. 2 is a flow chart for randomly generating calibration plate images;
FIG. 3 is a flow chart of coarse calibration;
FIG. 4 is a fine calibration flow chart;
fig. 5 is a schematic structural diagram of the configurable automatic hand-eye calibration device.
In the figure: 100. an image acquisition module; 110. a first display screen; 120. a second display screen; 130. a calibration plate with a backlight source; 140. a source controller; 150. a mechanical arm; 160. an embedded control system; 170. a PC based control system.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
As shown in fig. 1-4, the embodiment discloses a configurable hand-eye calibration method, which includes an offline configuration phase and an online automatic calibration phase;
firstly, the off-line configuration comprises the following steps:
(1) according to the flow shown in the attached drawing 1, a group of robot poses capable of acquiring clear images are taught, the poses are selected from 18-24 and are different from one another, the corresponding images cover nine-grid subareas of the whole visual field, different rotation angles are arranged in each subarea, and after the teaching is finished, the poses are recorded to a pose queue to be acquired and are off-line. Specifically, the off-line configuration teaching robot posture comprises the following steps:
and S1, initializing a pose queue and setting the minimum length of the pose queue.
And S2, moving the robot to a pose, and acquiring the image of the calibration plate.
And S3, judging whether the image is clear, if so, adding the current pose to the pose queue, and otherwise, executing the step S2.
And S4, judging whether the pose queue length is equal to the minimum length, if so, finishing off-line configuration of the teaching robot, and if not, executing the step S2.
(2) According to the actual application scene requirement, a group of calibration plate images with large pose difference is generated according to the flow shown in the attached figure 2 by knowing the physical size of the calibration plate, and at least two calibration plate images are generated. Specifically, the step of generating calibration plate images with different poses comprises the following steps:
step one, initializing camera internal parameters, a Sudoku area and a mark.
And step two, randomly generating a camera gesture.
And step three, generating a calibration plate image.
Judging whether the center of the calibration plate is in the central area of the nine-square grid or not; if yes, the calibration board image is saved, otherwise, the step two is executed.
And fifthly, judging whether each area is covered by at least two images with different poses, if so, finishing, and otherwise, executing the second step.
In the first step, the initialized internal parameters do not need to be the same as the actual internal parameters, a group of external parameters are set according to the internal parameters, the coordinates of the calibration plate under a camera coordinate system are calculated through projection transformation, then the pixel coordinates are calculated according to the internal parameters, and the calculation formula is as follows:
wherein the content of the first and second substances,
is an external parameter of the camera,
is to scale the world homogeneous coordinates of a point on the board,
is the camera homogeneous coordinate of a point on the calibration plate.
Wherein dx and dy represent the pixel sizes in x and y directions respectively, and u0、v0Is the pixel coordinate of the center point of the image, f is the focal length, and γ is the warping factor, where it is set to 0 according to the ideal model.
The above process is automatically completed by a program, and as an extension, a group of manual swinging calibration plate images meeting the requirements can be configured.
Secondly, the on-line automatic calibration comprises coarse calibration and fine calibration;
(1) the rough calibration comprises the following steps:
step a, the robot moves to an offline configuration result to initialize the first position and pose queue to be collected.
And b, acquiring a calibration plate image, and removing the pose from the queue.
And c, judging the image quality of the calibration plate, if the image quality meets the requirements, storing the current actual pose and the image of the calibration plate, and if not, executing the step a.
And d, judging whether the queue to be acquired is empty, if so, finishing the movement of the robot, and executing the step e, otherwise, executing the step a.
And e, judging whether at least N effective images exist, if so, calculating a hand-eye relation matrix and a reprojection error thereof according to the collected robot pose and calibration plate image data, calibrating internal parameters of a camera, calculating the pose of the calibration plate in a robot coordinate system, otherwise, failing to calibrate, and performing off-line configuration again. The calculation process of the hand-eye relationship matrix is a process of solving AX ═ XB, which is not described herein again.
(2) The fine calibration comprises the following steps:
step f, calculating the corresponding end pose of the mechanical arm according to the rough calibrated camera internal reference, the hand-eye calibration result and the pose of the calibration plate in the robot coordinate system, and sequentially adding the end poses into a robot pose queue, wherein the calculation formula is as follows:
in the above-mentioned formula,
is the pose of the tool end under the robot base coordinate system;
is the pose of the calibration plate under the coordinate system of the robot base;
calculating the pose of the calibration plate under a camera coordinate system according to internal parameters of the camera and a physical model of the calibration plate;
with the tool end in phaseAnd (4) pose under a machine coordinate system, namely hand-eye transformation relation.
And g, moving the robot to the pose, and acquiring an image of the calibration plate.
And h, judging whether the calibration plate image can be correctly identified or not, if so, saving the current actual pose and the actual calibration plate image, and otherwise, executing the step g.
And i, judging whether the calibration board image queue is empty, if so, executing a step j, and otherwise, executing a step g.
J, judging whether the number of the effective images is larger than or equal to N, if so, calculating the internal parameter and hand-eye transformation matrix of the camera and the reprojection error of the internal parameter and hand-eye transformation matrix of the camera, and if not, directly ending the fine calibration.
In order to implement the above hand-eye calibration method, the invention further provides a configurable hand-eye calibration device, which comprises an image acquisition module 100, a first display screen 110, a second display screen 120, a calibration board 130 with a backlight light source, a light source controller 140, a mechanical arm 150, an embedded control system 160, and a control system 170 based on a PC;
the image acquisition module 100, the embedded control system 160 and the PC-based control system 170 are connected through Ethernet; the calibration board with backlight source 130 is connected with the light source controller 140; the robotic arm 150 is connected to an embedded control system 160; the first display screen 110 is connected with the embedded control system 160, and parameter setting is carried out through the first display screen 110; the second display screen 120 is connected with a PC-based control system 170, and parameter setting is carried out through the second display screen 120.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be able to cover the technical scope of the present invention and the equivalent alternatives or modifications according to the technical solution and the inventive concept of the present invention within the technical scope of the present invention.
Claims (6)
1. A configurable hand-eye calibration method is characterized by comprising an off-line configuration stage and an on-line automatic calibration stage;
the off-line configuration comprises the following steps:
the off-line teaching can acquire a plurality of robot poses of clear images and record the poses to be acquired in a pose queue;
generating at least two calibration plate images with different poses according to requirements;
the on-line automatic calibration comprises coarse calibration and fine calibration;
the rough calibration comprises the following steps:
a, the robot moves to an offline configuration result to initialize the first position and pose queue to be acquired;
b, collecting images of the calibration plate, and removing the pose from the queue;
step c, judging the image quality of the calibration plate, if the image quality meets the requirements, storing the current actual pose and the image of the calibration plate, otherwise, executing the step a;
d, judging whether the queue to be acquired is empty, if so, finishing the movement of the robot, and executing the step e, otherwise, executing the step a;
step e, judging whether at least N effective images exist, if so, calculating a hand-eye relation matrix and a reprojection error thereof according to the collected robot pose and calibration plate image data, calibrating internal parameters of a camera, calculating the pose of the calibration plate under a robot coordinate system, and if not, resetting off-line configuration after calibration fails;
the fine calibration comprises the following steps:
step f, calculating the corresponding pose of the tail end of the mechanical arm according to the rough calibrated camera internal parameter, the hand-eye calibration result and the pose of the calibration plate in the robot coordinate system, and sequentially adding the pose of the tail end of the mechanical arm into a robot pose queue;
step g, the robot moves to the pose and collects the image of the calibration plate;
h, judging whether the calibration plate image can be correctly identified or not, if so, saving the current actual pose and the actual calibration plate image, and otherwise, executing the step g;
step i, judging whether the calibration board image queue is empty, if so, executing step j, otherwise, executing step g;
j, judging whether the number of the effective images is larger than or equal to N, if so, calculating the internal parameter and hand-eye transformation matrix of the camera and the reprojection error of the internal parameter and hand-eye transformation matrix of the camera, and if not, directly ending the fine calibration.
2. The configurable hand-eye calibration method according to claim 1, wherein the off-line configuration teaching robot poses comprises the following steps;
s1, initializing a pose queue and setting the minimum length of the pose queue;
s2, moving the robot to a pose, and collecting a calibration plate image;
s3, judging whether the image is clear, if so, adding the current pose to a pose queue, otherwise, executing the step S2;
and S4, judging whether the pose queue length is equal to the minimum length, if so, finishing off-line configuration of the teaching robot, and if not, executing the step S2.
3. The configurable hand-eye calibration method according to claim 2, wherein generating calibration plate images with different poses comprises the following steps:
step one, initializing camera internal parameters, a Sudoku area and a mark;
step two, randomly generating a camera gesture;
step three, generating a calibration plate image;
judging whether the center of the calibration plate is in the central area of the nine-square grid or not; if yes, saving the image of the calibration board, otherwise, executing the step two;
and fifthly, judging whether each area is covered by at least two images with different poses, if so, finishing, and otherwise, executing the second step.
4. The configurable hand-eye calibration method according to claim 1, wherein in the fine calibration stage, the formula for calculating the pose of the end of the mechanical arm in step f is as follows:
in the formula (I), the compound is shown in the specification,
is the pose of the tool end under the robot base coordinate system;
is the pose of the calibration plate under the coordinate system of the robot base;
calculating the pose of the calibration plate under a camera coordinate system according to internal parameters of the camera and a physical model of the calibration plate;
is the pose of the tool tip under the camera coordinate system.
5. The configurable method of claim 3, wherein N is greater than or equal to 9.
6. A configurable hand-eye calibration device is characterized by comprising an image acquisition module (100), a display screen I (110), a display screen II (120), a calibration board (130) with a backlight light source, a light source controller (140), a mechanical arm (150), an embedded control system (160) and a control system (170) based on a PC (personal computer);
the image acquisition module (100), the embedded control system (160) and the PC-based control system (170) are connected through the Ethernet; the calibration plate (130) with the backlight light source is connected with the light source controller (140); the mechanical arm (150) is connected with the embedded control system (160); the display screen I (110) is connected with the embedded control system (160), and parameter setting is carried out through the display screen I (110); and the second display screen (120) is connected with a control system (170) based on a PC, and parameter setting is carried out through the second display screen (120).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110917173.XA CN113561182A (en) | 2021-08-11 | 2021-08-11 | Configurable hand-eye calibration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110917173.XA CN113561182A (en) | 2021-08-11 | 2021-08-11 | Configurable hand-eye calibration method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113561182A true CN113561182A (en) | 2021-10-29 |
Family
ID=78171172
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110917173.XA Pending CN113561182A (en) | 2021-08-11 | 2021-08-11 | Configurable hand-eye calibration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113561182A (en) |
-
2021
- 2021-08-11 CN CN202110917173.XA patent/CN113561182A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109483516B (en) | Mechanical arm hand-eye calibration method based on space distance and polar line constraint | |
| CN110238849B (en) | Robot hand-eye calibration method and device | |
| TWI672206B (en) | Method and apparatus of non-contact tool center point calibration for a mechanical arm, and a mechanical arm system with said calibration function | |
| CN111801198B (en) | Hand-eye calibration method, system and computer storage medium | |
| JP6429473B2 (en) | Robot system, robot system calibration method, program, and computer-readable recording medium | |
| CN109658460A (en) | A kind of mechanical arm tail end camera hand and eye calibrating method and system | |
| CN111369625B (en) | Positioning method, positioning device and storage medium | |
| CN113379849B (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| CN110148187B (en) | High-precision Hand-Eye calibration method and system for SCARA manipulator Eye-in-Hand | |
| CN110246193B (en) | Industrial robot end camera online calibration method | |
| JP2005300230A (en) | Measuring instrument | |
| CN110253574B (en) | Multi-task mechanical arm pose detection and error compensation method | |
| CN111452048B (en) | Calibration method and device for relative spatial position relation of multiple robots | |
| CN114643578B (en) | Calibration device and method for improving robot vision guiding precision | |
| CN114519738A (en) | Hand-eye calibration error correction method based on ICP algorithm | |
| CN114310901B (en) | Coordinate system calibration method, device, system and medium for robot | |
| EP3602214B1 (en) | Method and apparatus for estimating system error of commissioning tool of industrial robot | |
| CN115446847A (en) | System and method for improving 3D eye-hand coordination accuracy of a robotic system | |
| Hvilshøj et al. | Calibration techniques for industrial mobile manipulators: Theoretical configurations and best practices | |
| CN114029982A (en) | Hand-eye calibration device and calibration method of camera outside robot arm | |
| CN110533727B (en) | Robot self-positioning method based on single industrial camera | |
| JP6912529B2 (en) | How to correct the visual guidance robot arm | |
| CN110421565B (en) | Robot global positioning and measuring system and method for practical training | |
| CN215701709U (en) | Configurable hand-eye calibration device | |
| CN112809668A (en) | Method, system and terminal for automatic hand-eye calibration of mechanical arm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |