CN107808401B - Hand-eye calibration method for single camera at tail end of mechanical arm - Google Patents
Hand-eye calibration method for single camera at tail end of mechanical arm Download PDFInfo
- Publication number
- CN107808401B CN107808401B CN201711038822.9A CN201711038822A CN107808401B CN 107808401 B CN107808401 B CN 107808401B CN 201711038822 A CN201711038822 A CN 201711038822A CN 107808401 B CN107808401 B CN 107808401B
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- camera
- tail end
- calibration
- hand
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Manipulator (AREA)
Abstract
The invention relates to a hand-eye calibration method of a single camera at the tail end of a mechanical arm. The hand-eye calibration method of the single camera at the tail end of the mechanical arm comprises the following steps: providing a calibration plate; placing the calibration plate in the visual field of a camera, controlling the camera to take a picture and controlling the tail end of the mechanical arm to rotate to obtain a first pixel coordinate of the centers of the marks; before the tail end of the mechanical arm rotates, a first pixel coordinate of the center of the mark is a first pixel coordinate; fitting the first pixel coordinates into a circle, and calculating a circle center coordinate corresponding to the circle, namely the pixel coordinate of the tail end of the mechanical arm; according to the hand-eye calibration method of the single camera at the tail end of the mechanical arm, manual operation of the mechanical arm is not needed, the whole calibration process is simple, the calibration time is short, the calibration precision is less influenced by human factors, and the calibration efficiency of the calibration method is high due to the fact that the calibration time of the calibration method is short.
Description
Technical Field
The invention relates to the technical field of camera calibration, in particular to a hand-eye calibration method of a single camera at the tail end of a mechanical arm.
Background
With the proposal of industry 4.0, the application of mechanical arms and machine vision in industrial production is more and more extensive, the technology of guiding the mechanical arms to carry out precision machining by visual positioning instead of the traditional manual precision machining becomes a trend, and the machining efficiency of products can be greatly improved.
The visual positioning guiding mechanical arm is used for carrying out precision machining, firstly, coordinates of machine vision and coordinates of the mechanical arm are calibrated, target coordinates of visual positioning are converted into mechanical arm coordinates, and visual guiding is finished, wherein the hand-eye calibration is used as a key point of the visual guiding mechanical arm. The traditional hand-eye calibration method needs to install a calibration needle and move a manual operation manipulator to nine angular points of a calibration plate, and as the precision of the calibration needle affects the calibration precision, the precision requirement of the manual operation manipulator moving the calibration needle to the nine angular points is high, the operation difficulty is high, the calibration precision is greatly affected by human factors, and the calibration time is long, the traditional hand-eye calibration method has the problems that the calibration process is complex, the calibration efficiency is low, and the calibration precision is greatly affected by the human factors.
Disclosure of Invention
Therefore, it is necessary to provide a hand-eye calibration method for a single camera at the tail end of a mechanical arm, aiming at the problems that the hand-eye calibration method is complex in calibration process, low in calibration efficiency and greatly influenced by human factors in calibration precision.
A hand-eye calibration method of a single camera at the tail end of a mechanical arm comprises the following steps:
providing a calibration plate, wherein a mark is arranged on the calibration plate;
placing the calibration plate in the visual field of a camera, controlling the camera to take a picture and controlling the tail end of a mechanical arm to rotate to obtain a first pixel coordinate of the centers of the marks; wherein, before the rotation of the end of the mechanical arm, a first pixel coordinate of the center of the mark is a first pixel coordinate;
fitting the first pixel coordinates into a circle, and calculating a circle center coordinate corresponding to the circle, namely a pixel coordinate of the tail end of the mechanical arm;
calculating a first offset vector of the center of the mark relative to the tail end of the mechanical arm in camera coordinates according to a first pixel coordinate and the pixel coordinate of the tail end of the mechanical arm;
controlling the mechanical arm to respectively translate nine positions according to a preset rule, and controlling the camera to take a picture to obtain corresponding mechanical arm coordinates and corresponding second pixel coordinates of the nine positions in the center of the mark;
calculating a scaling rotation matrix between a camera coordinate system and a mechanical arm coordinate system according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates;
calculating a second offset vector of the center of the mark relative to the center of the tail end of the mechanical arm according to the scaling rotation matrix and the first offset vector; and
and calculating a corresponding hand-eye calibration matrix according to the second pixel coordinate and the corresponding mechanical arm coordinate.
According to the hand-eye calibration method of the single camera at the tail end of the mechanical arm, the calibration process is as follows: firstly, providing a calibration plate; then placing the calibration plate in the visual field of a camera, controlling the camera to photograph and controlling the tail end of the mechanical arm to rotate to obtain first pixel coordinates of the centers of the marks, wherein before the tail end of the mechanical arm rotates, the first pixel coordinates of the centers of the marks are first pixel coordinates, and other first pixel coordinates are photographed in the tail end rotation process of the mechanical arm; then, fitting the first pixel coordinates into a circle, and calculating a circle center coordinate corresponding to the circle, namely the pixel coordinate of the tail end of the mechanical arm; then, calculating a first offset vector of the center of the mark relative to the tail end of the mechanical arm in the camera coordinate according to the first pixel coordinate and the pixel coordinate of the tail end of the mechanical arm; then, controlling the mechanical arm to respectively translate nine positions according to a preset rule, and controlling the camera to take a picture to obtain mechanical arm coordinates corresponding to the centers of the nine positions positioned in the mark and corresponding second pixel coordinates, so as to obtain second pixel coordinates of the centers of the mark of the nine positions of the calibration plate and mechanical arm coordinates corresponding to the nine positions; then, calculating a scaling rotation matrix between a camera coordinate system and a mechanical arm coordinate system according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates; then, calculating a second offset vector of the center of the mark relative to the center of the tail end of the mechanical arm according to the scaling rotation matrix and the first offset vector; and finally, calculating a corresponding hand-eye calibration matrix according to the first pixel coordinate and the corresponding mechanical arm coordinate. According to the hand-eye calibration method, the calibration needle does not need to be installed in the calibration process, and the problem that the precision of the calibration needle affects the calibration precision does not exist; in addition, in the calibration process, the rotation and the translational movement of the tail end of the mechanical arm can be realized through the control of the control end of the mechanical arm, the movement of the mechanical arm is not required to be manually operated, the whole calibration process is simple, the calibration time is short, the calibration precision is less influenced by human factors, and the calibration efficiency of the calibration method is high due to the fact that the calibration time of the calibration method is short.
In one embodiment, the step of placing the calibration plate in the field of view of a camera, and the step of controlling the camera to take a picture and controlling the end of the robot arm to rotate comprises:
placing the calibration plate in the visual field of a camera to enable the camera to photograph the calibration plate;
controlling the camera to shoot to obtain the first pixel coordinate, and enabling the camera to shoot the first pixel coordinate when the calibration plate is static;
and controlling the tail end of the mechanical arm to rotate, and simultaneously controlling the camera to shoot to obtain other first pixel coordinates, so that the first pixel coordinates when the calibration plate rotates are acquired and shot.
In one embodiment, the step of controlling the end of the mechanical arm to rotate and controlling the camera to take a picture includes:
and controlling the tail end of the mechanical arm to rotate, and simultaneously controlling the camera to take a picture once at each rotation preset angle value of the tail end of the mechanical arm.
In one embodiment, the predetermined angle value is 10 °.
In one embodiment, an image processing algorithm template is integrated in the camera, so that when the camera takes a picture of the calibration plate, coordinate pixels corresponding to the center of the mark are calculated according to the image processing algorithm template in a matching mode; when the camera takes a picture of the calibration plate to acquire image data, the image processor in the camera processes the image data according to the image processing algorithm module to obtain coordinate data corresponding to the center of the mark.
In one embodiment, a plurality of the first pixel coordinates are fitted to a circle according to a least squares method.
In one embodiment, the first offset vector is equal to a difference between pixel coordinates of the end of the arm and the first one of the first pixel coordinates.
In one embodiment, the step of calculating a scaled rotation matrix between the camera coordinate system and the robot arm coordinate system according to the robot arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates includes:
calculating a hand-eye transformation matrix between the camera coordinate system and the mechanical arm coordinate system by adopting a nine-point calibration algorithm according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates;
and calculating a scaling rotation matrix according to the hand-eye transformation matrix.
In one embodiment, the camera is mounted at the end of the robotic arm.
In one embodiment, after calculating the corresponding hand-eye calibration matrix according to the second pixel coordinates and the corresponding arm coordinates, the method further includes the following steps: and correcting the movement displacement of the tail end of the mechanical arm according to the hand-eye calibration matrix, so that the tail end of the mechanical arm can accurately act according to a preset path.
Drawings
FIG. 1 is a schematic view of the end of a robotic arm connected to a camera;
FIG. 2 is a flowchart of a hand-eye calibration method for a single camera at the end of a robotic arm according to an embodiment;
FIG. 3 is a flowchart of step S103 of the hand-eye calibration method of the single camera at the end of the robot arm shown in FIG. 2;
fig. 4 is a flowchart of step S111 of the hand-eye calibration method of the single camera at the end of the robot arm shown in fig. 2.
Detailed Description
In order to facilitate understanding of the present invention, a hand-eye calibration method of a single camera at the end of a robot arm will be described more fully below with reference to the accompanying drawings. The figures show a preferred embodiment of the hand-eye calibration method of a single camera at the end of a robotic arm. However, the single camera hand-eye calibration method at the end of the robotic arm may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to make the disclosure of the hand-eye calibration method for a single camera at the end of a robotic arm more thorough.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and the like as used herein are for illustrative purposes only and do not denote a unique embodiment.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the hand-eye calibration method for a single camera at the end of a robot arm herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
For example, a hand-eye calibration method for a single camera at the end of a mechanical arm includes: for example, a calibration plate is provided; for example, the calibration plate is placed in the visual field of a camera, the camera is controlled to photograph and the tail end of the mechanical arm is controlled to rotate, so that first pixel coordinates of the centers of the marks are obtained; wherein the first pixel coordinate of the center of the marker is a first pixel coordinate before the end of the robotic arm is rotated; for example, fitting a plurality of first pixel coordinates into a circle, and calculating a center coordinate corresponding to the circle, that is, a pixel coordinate of the end of the mechanical arm; for example, a first offset vector of the center of the mark relative to the robot arm end in camera coordinates is calculated according to a first one of the first pixel coordinates and the pixel coordinates of the robot arm end; for example, the mechanical arm is controlled to respectively translate nine positions according to a preset rule, the camera is controlled to take a picture, and the corresponding mechanical arm coordinates and the corresponding second pixel coordinates of the nine positions in the center of the mark are obtained; for example, a scaling rotation matrix between a camera coordinate system and a mechanical arm coordinate system is calculated according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates; for example, a second offset vector of the center of the marker relative to the center of the end of the mechanical arm is calculated from the scaled rotation matrix and the first offset vector; for example, a corresponding hand-eye calibration matrix is calculated according to the second pixel coordinates and corresponding mechanical arm coordinates. For example, a hand-eye calibration method for a single camera at the end of a mechanical arm includes: providing a calibration plate; placing the calibration plate in the visual field of a camera, controlling the camera to take a picture and controlling the tail end of a mechanical arm to rotate to obtain a first pixel coordinate of the centers of the marks; wherein, prior to rotation of the end of the robotic arm, a first pixel coordinate of the center of the marker is a first pixel coordinate; fitting the first pixel coordinates into a circle, and calculating a circle center coordinate corresponding to the circle, namely a pixel coordinate of the tail end of the mechanical arm; calculating a first offset vector of the center of the mark relative to the tail end of the mechanical arm in camera coordinates according to a first pixel coordinate and the pixel coordinate of the tail end of the mechanical arm; controlling the mechanical arm to respectively translate nine positions according to a preset rule, and controlling the camera to take a picture to obtain corresponding mechanical arm coordinates and corresponding second pixel coordinates of the nine positions in the center of the mark; calculating a scaling rotation matrix between a camera coordinate system and a mechanical arm coordinate system according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates; calculating a second offset vector of the center of the mark relative to the center of the tail end of the mechanical arm according to the scaling rotation matrix and the first offset vector; and calculating a corresponding hand-eye calibration matrix according to the second pixel coordinate and the corresponding mechanical arm coordinate.
As shown in fig. 1 and 2, a single-camera hand-eye calibration method for a distal end 100 of a robotic arm 10 according to an embodiment includes:
s101, providing a calibration plate, wherein the calibration plate is provided with marks.
S103, placing the calibration plate in the visual field of the camera 20, controlling the camera to take a picture and controlling the tail end of the mechanical arm to rotate to obtain a first pixel coordinate of the centers of the marks. Wherein the first pixel coordinate of the center of the marker is a first pixel coordinate before the end of the robotic arm is rotated. In the present embodiment, the number of the first pixel coordinates of the center of the marker obtained by camera shooting is 9, which are PixelP01(Px01, Py01), PixelP02(Px02, Py02), … …, and PixelP09(Px09, Py 09). For example, the camera is mounted at the tail end of the mechanical arm, so that the camera can better shoot the mark of the calibration plate, and the pixel coordinate of the center of the mark obtained by the camera is more accurate. For example, the end of the arm 100 is provided with a U-axis 110 to which the camera is fixed. When the tail end of the mechanical arm rotates, the tail end of the mechanical arm drives the U shaft to rotate, and the U shaft drives the camera to rotate.
In one embodiment, an image processing algorithm template is integrated in the camera, so that when the camera takes a picture of the calibration plate, coordinate pixels corresponding to the center of the mark are calculated according to the image processing algorithm template in a matching mode; when the camera takes a picture of the calibration plate to acquire image data, the image processor in the camera processes the image data according to the image processing algorithm module to obtain coordinate data corresponding to the center of the mark.
Referring to fig. 3, in one embodiment, the step S103 of placing the calibration plate in the field of view of the camera, controlling the camera to take a picture, and controlling the end of the robot arm to rotate includes:
S103A, placing the calibration board in the visual field of the camera, and enabling the camera to photograph the calibration board;
S103B, controlling the camera to take a picture to obtain the first pixel coordinate, and enabling the camera to take the first pixel coordinate when the calibration board is static;
S103C, controlling the tail end of the mechanical arm to rotate, controlling the camera to take a picture, obtaining other first pixel coordinates, and acquiring the first pixel coordinates when the calibration plate rotates. In this embodiment, the camera is controlled to take pictures at intervals while the end of the robot arm is controlled to rotate, that is, the camera is controlled to take pictures when the end of the robot arm rotates to different angles. In other embodiments, the camera is controlled to continuously photograph while the end of the mechanical arm is controlled to rotate, so as to obtain first pixel coordinate data corresponding to the instantaneous angle of rotation of the end of the mechanical arm, and the first pixel coordinate data is more accurate.
In one embodiment, the step S103C of controlling the end of the mechanical arm to rotate and controlling the camera to photograph is specifically:
and controlling the tail end of the mechanical arm to rotate, and simultaneously controlling the camera to take a picture once at each rotation preset angle value of the tail end of the mechanical arm. For example, the number of times the camera takes a picture is 9. For example, the predetermined angle value is 10 °. As another example, the predetermined angle value may also be 20 ° or 30 °.
And S105, fitting the first pixel coordinates into a circle, and calculating a circle center coordinate corresponding to the circle, namely the pixel coordinates of the tail end of the mechanical arm. In this embodiment, the number of the first pixel coordinates of the center of the marker obtained by camera shooting is 9, which are PixelP1(Px01, Py01), PixelP 2(Px02, Py02), … …, and PixelP9(Px09, Py 09). Fitting the 9 first pixel coordinates to a circle to obtain a corresponding center coordinate of PixelP0(Px00, Py 00).
In one embodiment, a plurality of the first pixel coordinates are fitted to a circle according to a least squares method.
S107, calculating a first offset vector of the center of the mark relative to the tail end of the mechanical arm in camera coordinates according to the first pixel coordinate and the pixel coordinate of the tail end of the mechanical arm. Setting the first offset vector as
In one embodiment, the first offset vector
Equal to the difference between said first pixel coordinate PixelP1(Px01, Py01) and said arm end pixel coordinate PixelP0(Px00, Py 00).
And S109, controlling the mechanical arm to respectively translate nine positions according to a preset rule, and controlling the camera to shoot to obtain the corresponding mechanical arm coordinate and the corresponding second pixel coordinate of the nine positions in the center of the mark. The mechanical arm coordinates corresponding to the nine positions are respectively Rxy00(X0, Y0), Rxy01(X0+ delta X, Y0), Rxy02 (X0-delta X, Y0), Rxy03(X0, Y0+ delta Y), Rxy04(X0, Y0-delta Y), Rxy05 (X0+ delta X, Y0+ delta Y), Rxy06 (X0-delta X, Y0-delta Y), Rxy07(X0+ delta X, Y0-delta Y) and Rxy08 (X0-delta X, Y0+ delta Y). The corresponding pixel coordinates of the center of the mark obtained by camera shooting positioning are respectively (Px01 ), … …, (Px09 ) because the mechanical arm translates to one position each time. Considering that the pixel coordinates of the center of the mark formed by moving the robot arm are reversed, the robot arm coordinates are adjusted to obtain new robot arm coordinates Rxy10(X0, Y0), Rxy11(X0- Δ X, Y0), Rxy12(X0+ Δ X, Y0), Rxy13(X0, Y0- Δ Y), Rxy14(X0, Y0+ Δ Y), Rxy15(X0- Δ X, Y0- Δ Y), Rxy16(X0+ Δ X, Y0+ Δ Y), Rxy17(X0+ Δ X, Y0- Δ Y), Rxy18(X0- Δ X, Y0+ Δ Y).
And S111, calculating a scaling rotation matrix between a camera coordinate system and a mechanical arm coordinate system according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates.
Referring to fig. 4, for example, the step S111 of calculating a scaling rotation matrix between the camera coordinate system and the robot arm coordinate system according to the robot arm coordinates corresponding to the nine positions and the second pixel coordinates corresponding to the nine positions includes:
and S111A, calculating a hand-eye transformation matrix between the camera coordinate system and the mechanical arm coordinate system by adopting a nine-point calibration algorithm according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates. The hand-eye transformation matrix is set to HandEyeCaliA1, and the scaling rotation matrix is set to HandEyeCaliA 2.
Considering that the pixel coordinates of the center of the mark formed by moving the mechanical arm are reversed, a hand-eye transformation matrix HandEyeCaliA1 is calculated by adopting a nine-point calibration algorithm according to the adjusted mechanical arm coordinates of the new nine positions and the corresponding second pixel coordinates:
and S111B, calculating a scaling rotation matrix according to the hand-eye transformation matrix. The scaling rotation matrix, HandEyeCaliA2, is:
and S113, calculating a second offset vector of the center of the mark relative to the center of the tail end of the mechanical arm according to the scaling rotation matrix and the first offset vector. Setting the second offset vector as
For example, the camera is mounted in front of the center of the flange surface at the end of the robot arm, and the robot arm coordinate corresponding to the center of the mark has a fixed offset, i.e., a second offset, with respect to the coordinate of the center of the flange surface
The center coordinates of the flange face corresponding to the Robot arm are set as Robot P1, Robot P2, Robot P3, Robot P4, Robot P5, Robot P6, Robot P7, Robot P8 and Robot P9.
The robot arm coordinates corresponding to the center of the mark are set to be robotpos mark p1, robotpos mark p2, robotpos mark p3, robotpos mark p4, robotpos mark p5, robotpos mark p6, robotpos mark p7, robotpos mark p8, and robotpos mark p9, and the relationship between the robot arm coordinates corresponding to the center of the mark and the flange face center coordinates is:
wherein RobotPosMarkP [ i ] is a mechanical arm coordinate corresponding to the center of the mark, namely a mechanical arm coordinate corresponding to the actual center of the mark; robot P [ i ] is the center coordinate of the flange face corresponding to the Robot arm, i.e., the Robot arm coordinate corresponding to the center of the virtual mark.
According to the first offset vector
And scaling the rotation matrix HandEyeCaliA2 to calculate a second offset vector
And S115, calculating a corresponding hand-eye calibration matrix according to the second pixel coordinate and the corresponding mechanical arm coordinate.
And calculating a corresponding hand-eye calibration matrix HandEyeCaliA0 according to the second pixel coordinate of the center of the mark and the corresponding mechanical arm coordinate.
HandEyeCaliA0＝affine_trans_point_2d(PixelP[i]，RobotPosMarkP[i])
Wherein, affine _ trans _ point _2d is one of HALCON operator functions, and arbitrary simplest two-dimensional transformation is carried out on point coordinates; PixelP [ i ] is the second pixel coordinate of the center of the marker; RobotPosMarkP [ i ] is the arm coordinate corresponding to the center of the mark.
In one embodiment, after calculating the corresponding hand-eye calibration matrix according to the second pixel coordinates and the corresponding arm coordinates, the method further includes the following steps: and correcting the movement displacement of the tail end of the mechanical arm according to the hand-eye calibration matrix, so that the tail end of the mechanical arm can accurately act according to a preset path.
According to the hand-eye calibration method of the single camera at the tail end of the mechanical arm, the calibration process is as follows: firstly, providing a calibration plate; then placing the calibration plate in the visual field of a camera, controlling the camera to photograph and controlling the tail end of the mechanical arm to rotate to obtain first pixel coordinates of the centers of the marks, wherein before the tail end of the mechanical arm rotates, the first pixel coordinates of the centers of the marks are first pixel coordinates, and other first pixel coordinates are photographed in the tail end rotation process of the mechanical arm; then, fitting the first pixel coordinates into a circle, and calculating a circle center coordinate corresponding to the circle, namely the pixel coordinate of the tail end of the mechanical arm; then, calculating a first offset vector of the center of the mark relative to the tail end of the mechanical arm in the camera coordinate according to the first pixel coordinate and the pixel coordinate of the tail end of the mechanical arm; then, controlling the mechanical arm to respectively translate nine positions according to a preset rule, and controlling the camera to take a picture to obtain mechanical arm coordinates corresponding to the centers of the nine positions positioned in the mark and corresponding second pixel coordinates, so as to obtain second pixel coordinates of the centers of the mark of the nine positions of the calibration plate and mechanical arm coordinates corresponding to the nine positions; then, calculating a scaling rotation matrix between a camera coordinate system and a mechanical arm coordinate system according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates; then, calculating a second offset vector of the center of the mark relative to the center of the tail end of the mechanical arm according to the scaling rotation matrix and the first offset vector; and finally, calculating a corresponding hand-eye calibration matrix according to the first pixel coordinate and the corresponding mechanical arm coordinate.
According to the hand-eye calibration method, the calibration needle does not need to be installed in the calibration process, and the problem that the precision of the calibration needle affects the calibration precision does not exist; in addition, in the calibration process, the rotation and the translation of the tail end of the mechanical arm can be realized through the control of the control end of the mechanical arm, the movement of the mechanical arm is not required to be manually operated, the whole calibration process is simple, the calibration time is short, and the calibration precision is less influenced by human factors. The hand-eye calibration method can be integrated on an industrial computer, for example, a software interface of the hand-eye calibration, the industrial computer controls the actions of the manipulator and the camera respectively, an operator only needs to set calibration related parameters on the software interface, the manipulator does not need to be manually operated to move, the whole calibration process is simple and rapid, and the influence of human factors is small.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A hand-eye calibration method of a single camera at the tail end of a mechanical arm is characterized by comprising the following steps:
providing a calibration plate, wherein a mark is arranged on the calibration plate;
placing the calibration plate in the visual field of a camera, controlling the camera to take a picture and controlling the tail end of a mechanical arm to rotate to obtain a first pixel coordinate of the centers of the marks; wherein a first pixel coordinate of a center of the marker is a first pixel coordinate before the robot arm tip is rotated, wherein the camera is mounted to the robot arm tip;
fitting the first pixel coordinates into a circle, and calculating a circle center coordinate corresponding to the circle, namely a pixel coordinate of the tail end of the mechanical arm;
calculating a first offset vector of the center of the mark relative to the tail end of the mechanical arm in camera coordinates according to a first pixel coordinate and the pixel coordinate of the tail end of the mechanical arm;
controlling the mechanical arm to respectively translate nine positions according to a preset rule, and controlling the camera to take a picture to obtain corresponding mechanical arm coordinates and corresponding second pixel coordinates of the nine positions in the center of the mark;
calculating a scaling rotation matrix between a camera coordinate system and a mechanical arm coordinate system according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates;
calculating a second offset vector of the center of the mark relative to the center of the tail end of the mechanical arm according to the scaling rotation matrix and the first offset vector; and
and calculating a corresponding hand-eye calibration matrix according to the second pixel coordinate and the corresponding mechanical arm coordinate.
2. The method for calibrating the hand-eye of the single camera at the tail end of the mechanical arm according to claim 1, wherein the steps of placing the calibration plate in the visual field of the camera, controlling the camera to take a picture and controlling the tail end of the mechanical arm to rotate comprise:
placing the calibration plate within a field of view of a camera;
controlling the camera to take a picture to obtain the first pixel coordinate;
and controlling the tail end of the mechanical arm to rotate, and simultaneously controlling the camera to shoot to obtain other first pixel coordinates.
3. The hand-eye calibration method for the single camera at the tail end of the mechanical arm as recited in claim 2, wherein the step of controlling the tail end of the mechanical arm to rotate and controlling the camera to take a picture comprises the following steps:
and controlling the tail end of the mechanical arm to rotate, and simultaneously controlling the camera to take a picture once at each preset rotation angle value of the tail end of the mechanical arm.
4. A hand-eye calibration method for a single camera of a robot arm tip according to claim 3, characterized in that the predetermined angle value is 10 °.
5. The method for calibrating the hands and eyes of the single camera at the tail end of the mechanical arm according to claim 1, wherein an image processing algorithm template is integrated in the camera, so that when the camera takes a picture of the calibration plate, the coordinate pixels corresponding to the centers of the marks are calculated according to the image processing algorithm template in a matching mode.
6. The hand-eye calibration method for a single camera at an end of a robot arm according to claim 1, wherein a plurality of the first pixel coordinates are fitted to a circle according to a least square method.
7. The method of claim 1, wherein the first offset vector is equal to a difference between a pixel coordinate of the end of the robotic arm and the first pixel coordinate.
8. The method for calibrating the hand-eye of a single camera at the end of a robot arm as claimed in claim 1, wherein the step of calculating the scaled rotation matrix between the camera coordinate system and the robot arm coordinate system according to the robot arm coordinates corresponding to nine positions and the second pixel coordinates corresponding to the nine positions comprises:
calculating a hand-eye transformation matrix between the camera coordinate system and the mechanical arm coordinate system by adopting a nine-point calibration algorithm according to the mechanical arm coordinates corresponding to the nine positions and the corresponding second pixel coordinates;
and calculating a scaling rotation matrix according to the hand-eye transformation matrix.
9. The hand-eye calibration method for the single camera at the tail end of the mechanical arm as claimed in claim 1, wherein a U-axis is arranged at the tail end of the mechanical arm, and the camera is fixed on the U-axis.
10. The method for calibrating the hand-eye of the single camera at the end of the mechanical arm according to claim 1, after calculating the corresponding hand-eye calibration matrix according to the second pixel coordinates and the corresponding mechanical arm coordinates, further comprising the steps of: and correcting the moving displacement of the tail end of the mechanical arm according to the hand-eye calibration matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711038822.9A CN107808401B (en) | 2017-10-30 | 2017-10-30 | Hand-eye calibration method for single camera at tail end of mechanical arm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711038822.9A CN107808401B (en) | 2017-10-30 | 2017-10-30 | Hand-eye calibration method for single camera at tail end of mechanical arm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107808401A CN107808401A (en) | 2018-03-16 |
| CN107808401B true CN107808401B (en) | 2020-09-22 |
Family
ID=61590952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711038822.9A Active CN107808401B (en) | 2017-10-30 | 2017-10-30 | Hand-eye calibration method for single camera at tail end of mechanical arm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107808401B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108510550A (en) * | 2018-03-29 | 2018-09-07 | 轻客智能科技（江苏）有限公司 | A kind of binocular camera automatic calibration method and device |
| CN109121309A (en) * | 2018-10-11 | 2019-01-01 | 深圳视觉龙智能传感器有限公司 | The visible sensation method and system of flexible circuit board fold contraposition |
| CN109671122A (en) * | 2018-12-14 | 2019-04-23 | 四川长虹电器股份有限公司 | Trick camera calibration method and device |
| CN109781164B (en) * | 2018-12-28 | 2021-02-05 | 长沙长泰机器人有限公司 | Static calibration method of line laser sensor |
| CN109829953B (en) * | 2019-02-27 | 2021-09-03 | 广东拓斯达科技股份有限公司 | Image acquisition device calibration method and device, computer equipment and storage medium |
| WO2021012122A1 (en) * | 2019-07-19 | 2021-01-28 | 西门子（中国）有限公司 | Robot hand-eye calibration method and apparatus, computing device, medium and product |
| CN111272067A (en) * | 2020-01-21 | 2020-06-12 | 上海悦易网络信息技术有限公司 | Mobile phone detection method and equipment |
| CN111482964A (en) * | 2020-04-17 | 2020-08-04 | 上海智殷自动化科技有限公司 | Novel robot hand-eye calibration method |
| CN111627073B (en) * | 2020-04-30 | 2023-10-24 | 贝壳技术有限公司 | Calibration method, calibration device and storage medium based on man-machine interaction |
| CN112001967A (en) * | 2020-08-14 | 2020-11-27 | 苏州华兴源创科技股份有限公司 | Method and device for guiding manipulator to carry object by camera |
| CN112548265A (en) * | 2020-10-28 | 2021-03-26 | 深圳前海瑞集科技有限公司 | Intelligent welding method and equipment for container lock seat |
| CN112658470B (en) * | 2020-12-10 | 2023-05-26 | 安徽巨一科技股份有限公司 | Visual positioning guiding and calibrating method for laser welding of stator end flat wire |
| CN113223048B (en) * | 2021-04-20 | 2024-02-27 | 深圳瀚维智能医疗科技有限公司 | Method and device for determining hand-eye calibration precision, terminal equipment and storage medium |
| CN114332249B (en) * | 2022-03-17 | 2022-05-24 | 常州铭赛机器人科技股份有限公司 | Camera vision internal segmentation type hand-eye calibration method |
| CN114700989A (en) * | 2022-04-24 | 2022-07-05 | 安吉八塔机器人有限公司 | Automatic leveling device for plane camera and control method thereof |
| CN114918926B (en) * | 2022-07-22 | 2022-10-25 | 杭州柳叶刀机器人有限公司 | Mechanical arm visual registration method and device, control terminal and storage medium |
| CN115682926B (en) * | 2022-09-07 | 2023-08-29 | 广东爱吉尔机器人科技有限公司 | One-to-many rapid hand-eye calibration method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104354167A (en) * | 2014-08-29 | 2015-02-18 | 广东正业科技股份有限公司 | Robot hand-eye calibration method and device |
| CN105014679A (en) * | 2015-08-03 | 2015-11-04 | 华中科技大学无锡研究院 | Robot hand and eye calibrating method based on scanner |
| CN105278454A (en) * | 2015-11-05 | 2016-01-27 | 合肥图迅电子科技有限公司 | Robot hand-eye positioning algorithm based on mechanical arm visual positioning system |
| CN105451461A (en) * | 2015-11-25 | 2016-03-30 | 四川长虹电器股份有限公司 | PCB board positioning method based on SCARA robot |
| CN106127722A (en) * | 2016-05-03 | 2016-11-16 | 深圳视觉龙智能传感器有限公司 | The demarcation of polyphaser and para-position applying method |
-
2017
- 2017-10-30 CN CN201711038822.9A patent/CN107808401B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104354167A (en) * | 2014-08-29 | 2015-02-18 | 广东正业科技股份有限公司 | Robot hand-eye calibration method and device |
| CN105014679A (en) * | 2015-08-03 | 2015-11-04 | 华中科技大学无锡研究院 | Robot hand and eye calibrating method based on scanner |
| CN105278454A (en) * | 2015-11-05 | 2016-01-27 | 合肥图迅电子科技有限公司 | Robot hand-eye positioning algorithm based on mechanical arm visual positioning system |
| CN105451461A (en) * | 2015-11-25 | 2016-03-30 | 四川长虹电器股份有限公司 | PCB board positioning method based on SCARA robot |
| CN106127722A (en) * | 2016-05-03 | 2016-11-16 | 深圳视觉龙智能传感器有限公司 | The demarcation of polyphaser and para-position applying method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107808401A (en) | 2018-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107808401B (en) | Hand-eye calibration method for single camera at tail end of mechanical arm | |
| CN111482959B (en) | Automatic hand-eye calibration system and method of robot motion vision system | |
| US11247340B2 (en) | Method and apparatus of non-contact tool center point calibration for a mechanical arm, and a mechanical arm system with said calibration function | |
| CN111791227B (en) | Robot hand-eye calibration method and device and robot | |
| CN111775146B (en) | Visual alignment method under industrial mechanical arm multi-station operation | |
| JP5815761B2 (en) | Visual sensor data creation system and detection simulation system | |
| CN111801198B (en) | Hand-eye calibration method, system and computer storage medium | |
| CN109658460A (en) | A kind of mechanical arm tail end camera hand and eye calibrating method and system | |
| JP6812095B2 (en) | Control methods, programs, recording media, robotic devices, and manufacturing methods for articles | |
| CN109671122A (en) | Trick camera calibration method and device | |
| US8306660B2 (en) | Device and a method for restoring positional information of robot | |
| JP6235664B2 (en) | Measuring device used to calibrate mechanism parameters of robot | |
| CN111862220B (en) | Correction method and device for UVW platform calibration, deviation correction method and alignment system | |
| JP2006110705A5 (en) | ||
| JP2019115974A (en) | Calibration and operation of vision-based manipulation systems | |
| JP7185860B2 (en) | Calibration method for a multi-axis movable vision system | |
| CN104005180A (en) | Visual positioning method and system for sewing | |
| CN109648605A (en) | Robot system | |
| CN110465946B (en) | Method for calibrating relation between pixel coordinate and robot coordinate | |
| Lee et al. | High precision hand-eye self-calibration for industrial robots | |
| EP3602214B1 (en) | Method and apparatus for estimating system error of commissioning tool of industrial robot | |
| CN109900251A (en) | A kind of robotic positioning device and method of view-based access control model technology | |
| CN114833832A (en) | Robot hand-eye calibration method, device, equipment and readable storage medium | |
| CN107756391A (en) | The bearing calibration of mechanical arm correction system | |
| CN115401688A (en) | Control method and device of mechanical arm, mechanical arm system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20180601 Address after: 518051 Dazu laser building, 9 new West Road, North Nanshan District high tech park, Shenzhen, Guangdong Applicant after: HANS LASER TECHNOLOGY INDUSTRY GROUP CO., LTD. Applicant after: Shenzhen Dazu robot Co., Ltd. Address before: 518051 Dazu laser building, 9 new West Road, North Nanshan District high tech park, Shenzhen, Guangdong Applicant before: HANS LASER TECHNOLOGY INDUSTRY GROUP CO., LTD. Applicant before: Shenzhen Hans Electric Motor Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210908 Address after: 518103 workshop 601, building 3, Zhizao Center Park, No. 12, Chongqing Road, Heping community, Fuhai street, Bao'an District, Shenzhen, Guangdong Province Patentee after: Shenzhen Dazu robot Co.,Ltd. Address before: 518051 Dazu laser building, 9 new West Road, North Nanshan District high tech park, Shenzhen, Guangdong Patentee before: HAN'S LASER TECHNOLOGY INDUSTRY GROUP Co.,Ltd. Patentee before: Shenzhen Dazu robot Co.,Ltd. |