CN111890356A - Mechanical arm coordinate system and camera coordinate system calibration method, device, equipment and medium - Google Patents
Mechanical arm coordinate system and camera coordinate system calibration method, device, equipment and medium Download PDFInfo
- Publication number
- CN111890356A CN111890356A CN202010615931.8A CN202010615931A CN111890356A CN 111890356 A CN111890356 A CN 111890356A CN 202010615931 A CN202010615931 A CN 202010615931A CN 111890356 A CN111890356 A CN 111890356A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- calibration
- depth
- mechanical arm
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Abstract
The invention discloses a calibration method of a mechanical arm coordinate system and a camera coordinate system, which comprises the steps of obtaining a color image and a depth image of a calibration plate; acquiring pixel coordinates of each calibration pattern on the color image, and calculating a depth coordinate according to the pixel coordinates and the mapping relation; obtaining the depth value of the calibration pattern, and calculating the three-dimensional coordinate under the depth camera coordinate system according to the depth coordinate and the depth value; respectively controlling the tail end of the mechanical arm to move to the position where the calibration tool aligns to each calibration pattern, and calculating a rotation and translation matrix of a local coordinate system of the tail end joint of the mechanical arm relative to a basic coordinate system of the mechanical arm according to a positive kinematics method of the mechanical arm; calculating the three-dimensional coordinate of the tail end of the calibration tool under the basic coordinate system of the mechanical arm according to the rotation translation matrix and the coordinate of the calibration tool; and calculating a rotation and translation matrix of the depth camera coordinate system and the mechanical arm basic coordinate system according to the three-dimensional coordinates of the depth camera coordinate system and the three-dimensional coordinates of the mechanical arm basic coordinate system.
Description
Technical Field
The invention relates to the field of machine vision, in particular to a method, a device, equipment and a medium for calibrating a mechanical arm coordinate system and a camera coordinate system.
Background
In order to control the mechanical arm to perform related motion according to the point cloud data acquired by the depth camera, the coordinate system of the depth camera needs to be converted into the coordinate system of the mechanical arm through calibration, namely a 4 x 4 rotation and translation matrix is solved, so that the point cloud data acquired by the depth camera can be converted into the coordinate system of the mechanical arm, and the motion of the mechanical arm is controlled. However, the calibration method adopted by the traditional mechanical arm coordinate system and the traditional camera coordinate system has the defect of low calibration precision.
Disclosure of Invention
The invention mainly aims to provide a method for calibrating a mechanical arm coordinate system and a camera coordinate system, and aims to solve the technical problem that the existing mechanical arm coordinate system and camera coordinate system are low in calibration precision.
In order to achieve the aim, the invention provides a mechanical arm coordinate system and a camera coordinate system calibration method, which comprises the steps of installing a depth camera on any joint of a mechanical arm and installing a calibration tool at the tail end of the mechanical arm; the method comprises the steps that a mechanical arm is controlled to move to the position of a calibration plate, so that a depth camera shoots a complete calibration plate image, the calibration plate image comprises a color image and a depth image, and a plurality of calibration patterns are arranged on the calibration plate; acquiring the pixel coordinate of each calibration pattern on the color image, and calculating the depth coordinate of each calibration pattern on the depth image according to the pixel coordinate and the mapping relation between the pixel coordinate and the depth coordinate; acquiring the depth value of each calibration pattern on the depth image, and calculating the three-dimensional coordinate of each calibration pattern in a depth camera coordinate system according to the depth coordinate and the depth value; respectively controlling the tail end of the mechanical arm to move to the position where the calibration tool aligns to each calibration pattern, and calculating to obtain a rotation and translation matrix of a local coordinate system of the tail end joint of the mechanical arm relative to a basic coordinate system of the mechanical arm according to a positive kinematics method of the mechanical arm; calculating the three-dimensional coordinates of the tail end of the calibration tool under a mechanical arm basic coordinate system according to the rotation and translation matrix and the coordinates of a preset calibration tool, namely the three-dimensional coordinates of each calibration pattern under the mechanical arm basic coordinate system; and calculating to obtain a rotation and translation matrix between the depth camera coordinate system and the mechanical arm basic coordinate system through a singular value decomposition method according to the three-dimensional coordinate of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinate of each calibration pattern in the mechanical arm basic coordinate system.
Preferably, the mapping relationship between the pixel coordinate and the depth coordinate is:
xs＝a*xc+b
ys＝e*yc+f
wherein, the (x)c，yc) For the pixel coordinates of a certain calibration pattern on the color image, the (x)s，ys) Is the (x)c，yc) And coordinates mapped on the depth map, wherein a, b, e and f are scaling translation parameters between the color image and the depth image respectively.
Preferably, the three-dimensional coordinates of each of the calibration patterns in the depth camera coordinate system are calculated according to the following formula:
X＝D*(xs-U0)/(fx)
Y＝D*(ys-V0)/(fy)
Z＝D
wherein (X, Y, Z) is the three-dimensional coordinate of the calibration pattern in the depth camera coordinate system, D is the depth value of the calibration pattern on the depth image, and U is the depth value of the calibration pattern0、V0、fx、fyRespectively, the internal parameters of the depth camera.
Preferably, the calibration pattern is a circular mark point, the calibration tool is a cylindrical tool, and the diameter of the cylindrical tool is the same as that of the circular mark point.
The invention also provides a calibration device for the coordinate system of the mechanical arm and the coordinate system of the camera, which comprises the following components: the image acquisition module is used for controlling the mechanical arm to move to the position of the calibration plate so that the depth camera can shoot a complete calibration plate image, the calibration plate image comprises a color image and a depth image, and a plurality of calibration patterns are arranged on the calibration plate; the depth coordinate acquisition module is used for acquiring the pixel coordinate of each calibration pattern on the color image and calculating the depth coordinate of each calibration pattern on the depth image according to the pixel coordinate and the mapping relation between the pixel coordinate and the depth coordinate; the first three-dimensional coordinate acquisition module is used for acquiring the depth value of each calibration pattern on the depth image and calculating the three-dimensional coordinate of each calibration pattern in a depth camera coordinate system according to the depth coordinate and the depth value; the first rotation and translation matrix acquisition module is used for respectively controlling the tail end of the mechanical arm to move to the position where the calibration tool aligns to each calibration pattern, and calculating to obtain a rotation and translation matrix of a local coordinate system of the tail end joint of the mechanical arm relative to a basic coordinate system of the mechanical arm according to a positive kinematics method of the mechanical arm; the second three-dimensional coordinate acquisition module is used for calculating the three-dimensional coordinates of the tail end of the calibration tool under the basic coordinate system of the mechanical arm according to the rotation and translation matrix and the coordinates of a preset calibration tool, namely the three-dimensional coordinates of each calibration pattern under the basic coordinate system of the mechanical arm; and the second rotation and translation matrix acquisition module is used for calculating to obtain a rotation and translation matrix between the depth camera coordinate system and the mechanical arm basic coordinate system through a singular value decomposition method according to the three-dimensional coordinate of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinate of each calibration pattern in the mechanical arm basic coordinate system.
Preferably, the mapping relationship between the pixel coordinate and the depth coordinate is:
xs＝a*xc+b
ys＝e*yc+f
wherein, the (x)c，yc) For the pixel coordinates of a certain calibration pattern on the color image, the (x)s，ys) Is the (x)c，yc) And coordinates mapped on the depth map, wherein a, b, e and f are scaling translation parameters between the color image and the depth image respectively.
Preferably, the three-dimensional coordinates of each of the calibration patterns in the depth camera coordinate system are calculated according to the following formula:
X＝D*(xs-U0)/(fx)
Y＝D*(ys-V0)/(fy)
Z＝D
wherein (X, Y, Z) is the three-dimensional coordinate of the calibration pattern in the depth camera coordinate system, D is the depth value of the calibration pattern on the depth image, and U is the depth value of the calibration pattern0、V0、fx、fyRespectively, the internal parameters of the depth camera.
Preferably, the calibration pattern is a circular mark point, the calibration tool is a cylindrical tool, and the diameter of the cylindrical tool is the same as that of the circular mark point.
The invention further provides a robot arm coordinate system and camera coordinate system calibration device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the robot arm coordinate system and camera coordinate system calibration method described in the foregoing embodiment when executing the computer program.
The present invention also proposes a medium storing a computer program that, when executed by a processor, implements the robot arm coordinate system and the camera coordinate system calibration method described in the foregoing embodiments.
Compared with the prior art, the embodiment of the invention has the beneficial technical effects that:
the method comprises the steps of shooting a color image and a depth image of a calibration plate through a depth camera, and obtaining three-dimensional coordinates of each calibration pattern on the calibration plate under a depth camera coordinate system according to the color image and the depth image of the calibration plate; then, keeping the calibration plate still, controlling the tail end of the mechanical arm to move to the position where each calibration pattern is aligned, and calculating a rotation translation matrix of a local coordinate system of the tail end joint of the mechanical arm relative to a basic coordinate system of the mechanical arm through a kinematics forward solution of the mechanical arm; then, according to the rotation and translation matrix and the preset coordinates of the calibration tool, calculating three-dimensional coordinates of the tail end of the calibration tool under a mechanical arm basic coordinate system, namely three-dimensional coordinates of each calibration pattern under the mechanical arm basic coordinate system; and finally, calculating to obtain a rotation and translation matrix between the depth camera coordinate system and the mechanical arm basic coordinate system through a singular value decomposition method according to the three-dimensional coordinate of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinate of each calibration pattern in the mechanical arm basic coordinate system. The mechanical arm coordinate system and the camera coordinate system calibration method provided by the embodiment of the invention can meet the conversion of an object between the camera coordinate system and the mechanical arm coordinate system, so that the mechanical arm can control the mechanical arm to accurately grab the object according to the object coordinate acquired by the depth camera. In addition, the calibration precision can be improved by controlling the placing position of the calibration plate to be close to the depth camera.
Drawings
FIG. 1 is a flowchart of an embodiment of a robot arm coordinate system and camera coordinate system calibration method of the present invention;
fig. 2 is a functional block diagram of the calibration apparatus for the robot arm coordinate system and the camera coordinate system according to the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be illustrative of the present invention and should not be construed as limiting the present invention, and all other embodiments that can be obtained by one skilled in the art based on the embodiments of the present invention without inventive efforts shall fall within the scope of protection of the present invention.
The invention provides a method for calibrating a coordinate system of a mechanical arm and a coordinate system of a camera, and in one embodiment, referring to fig. 1, the method for calibrating the coordinate system of the mechanical arm and the coordinate system of the camera comprises the following steps:
step S10, controlling the mechanical arm to move to the position of the calibration plate, so that the depth camera can shoot a complete calibration plate image, wherein the calibration plate image comprises a color image and a depth image, and a plurality of calibration patterns are arranged on the calibration plate;
in this embodiment, before calibrating the robot arm coordinate system and the camera coordinate system, a calibration tool needs to be installed at the end of the robot arm, and the three-dimensional coordinates of the end of the calibration tool in the robot arm basic coordinate system are the three-dimensional coordinates of the calibration pattern in the robot arm basic coordinate system. The basic coordinate system of the mechanical arm refers to a coordinate system of the mechanical arm, each joint of the mechanical arm has a local coordinate system, the local coordinate systems of the joints of the mechanical arm and the basic coordinate system of the mechanical arm can be mutually converted, and the conversion can be completed according to a rotation and translation matrix between the local coordinate systems and the basic coordinate system of the mechanical arm.
The calibration plate is provided with N × M calibration patterns distributed according to a preset distance, and the calibration patterns may be in the shape of a circle, an ellipse, and the like, including but not limited to this, and those skilled in the art can select the calibration patterns according to actual situations. The calibration plate is placed at a fixed position, the mechanical arm is controlled to move to the position above the calibration plate when a mechanical arm coordinate system and a camera coordinate system are calibrated, and a depth camera mounted on the mechanical arm can shoot a complete calibration plate image, wherein the calibration plate image comprises a three-channel color image and a single-channel depth image.
Step S20, acquiring the pixel coordinate of each calibration pattern on the color image, and calculating the depth coordinate of each calibration pattern on the depth image according to the pixel coordinate and the mapping relation between the pixel coordinate and the depth coordinate;
in this embodiment, the calibration patterns can be displayed on the color image, so that the pixel coordinates of each calibration pattern on the color image can be directly read, the calibration pattern cannot be seen on the depth image, and the corresponding depth value is obtained from the depth image. After the pixel coordinates of each calibration pattern on the color image are obtained, the corresponding depth coordinates are calculated according to the pixel coordinates and the mapping relation between the pixel coordinates and the depth coordinates. It will be appreciated that each time the depth camera takes a picture, a colour image and a depth image are acquired, and the pixel coordinates of a certain calibration pattern on the colour image can be mapped to the coordinates of the corresponding point on the depth image. It should be noted that, the pixel coordinates of the calibration pattern on the color image specifically refer to the pixel coordinates of the center of the calibration pattern on the color image.
The mapping relationship between the pixel coordinates on the color image and the depth coordinates on the depth image is a linear mapping relationship, and specifically includes:
xs＝a*xc+b
ys＝e*yc+f
wherein (x)c，yc) Is the pixel coordinate of a certain calibration pattern on the color image, (x)s，ys) Is (x)c，yc) And coordinates mapped on the depth map, and a, b, e and f are scaling translation parameters between the color image and the depth image respectively. It should be noted that a, b, e, and f are obtained by calibration by a depth camera manufacturer, that is, a, b, e, and f are constants and can be directly substituted into a formula during calculation.
Step S30, obtaining the depth value of each calibration pattern on the depth image, and calculating the three-dimensional coordinate of each calibration pattern in the depth camera coordinate system according to the depth coordinate and the depth value;
in this embodiment, after the depth coordinate and the corresponding depth value are obtained, the depth coordinate and the corresponding depth value are substituted into a built-in calculation formula, so that a three-dimensional coordinate of each calibration pattern in a depth camera coordinate system can be calculated, where the calculation formula specifically is:
X＝D*(xs-U0)/(fx)
Y＝D*(ys-V0)/(fy)
Z＝D
wherein (X, Y, Z) is the three-dimensional coordinate of the calibration pattern in the depth camera coordinate system, D is the depth value of the calibration pattern in the depth image, U0、V0、fx、fyRespectively, the intrinsic parameters of the depth camera. Note that U is0、V0、fx、fyAlso obtained by calibration by the depth camera manufacturer, i.e. U0、V0、fx、fyAlso a constant value, and it is sufficient to substitute it into the formula at the time of calculation.
Step S40, respectively controlling the tail end of the mechanical arm to move to the position where the calibration tool aligns to each calibration pattern, and calculating to obtain a rotation and translation matrix of the local coordinate system of the tail end joint of the mechanical arm relative to the basic coordinate system of the mechanical arm according to the positive kinematics method of the mechanical arm;
in the embodiment, after the color image and the depth image of the calibration plate are acquired by the depth camera, the calibration plate is kept still, the tail end of the mechanical arm is controlled to move, so that the tail end of the calibration tool is aligned with the calibration pattern on the calibration plate, and then the rotation angle of each joint of the mechanical arm is acquired; and then, according to the rotation angle of each joint of the mechanical arm and by a positive kinematics method of the mechanical arm, solving a rotation and translation matrix of the local coordinate system of the tail end joint of the mechanical arm relative to a basic coordinate system of the mechanical arm. The positive kinematics of the mechanical arm is to find the end pose by knowing joint variables (rotation angles), and since the positive kinematics method of the mechanical arm is well known in the art, the applicant does not describe it in detail here, and specifically refers to the records of the positive kinematics method in the prior art.
Step S50, calculating the three-dimensional coordinates of the tail end of the calibration tool under the basic coordinate system of the mechanical arm according to the rotation and translation matrix and the coordinates of a preset calibration tool, namely the three-dimensional coordinates of each calibration pattern under the basic coordinate system of the mechanical arm;
in this embodiment, the calibration tool is mounted on the end joint of the robot arm, that is, the coordinates of the calibration tool are the coordinates in the local coordinate system of the end joint of the robot arm. And the rotation and translation matrix of the local coordinate system of the tail end joint of the known mechanical arm relative to the basic coordinate system of the mechanical arm and the coordinate of the calibration tool can be converted to obtain the three-dimensional coordinate of the tail end of the calibration tool under the basic coordinate system of the mechanical arm through the rotation and translation between the coordinate systems, namely the three-dimensional coordinate of the pattern under the basic coordinate system of the mechanical arm.
And step S60, calculating to obtain a rotation and translation matrix between the depth camera coordinate system and the mechanical arm basic coordinate system through a singular value decomposition method according to the three-dimensional coordinates of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinates of each calibration pattern in the mechanical arm basic coordinate system.
In this embodiment, the three-dimensional coordinates of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinates of each calibration pattern in the robot arm base coordinate system form a 3D point pair, and a rotation and translation matrix between the depth camera coordinate system and the robot arm base coordinate system can be calculated by using an SVD (Singular Value Decomposition) method. Since SVD methods are well known to those skilled in the art, applicants will not describe them in detail herein, and refer to the prior art for their description of SVD methods.
In one embodiment, the calibration pattern is a circular mark point, and the calibration tool is a cylindrical tool having the same diameter as the circular mark point. In the embodiment, the circular calibration pattern and the cylindrical tool are adopted for calibration, so that the calibration tool at the tail end of the mechanical arm and the calibration pattern are more accurate, and the calibration precision is improved.
Based on the aforementioned calibration method for the robot arm coordinate system and the camera coordinate system, referring to fig. 2, the present invention further provides a calibration apparatus for the robot arm coordinate system and the camera coordinate system, the calibration apparatus for the robot arm coordinate system and the camera coordinate system comprising:
the image acquisition module 10 is configured to control the mechanical arm to move to a position where a calibration plate is located, so that the depth camera shoots a complete calibration plate image, the calibration plate image includes a color image and a depth image, and the calibration plate is provided with a plurality of calibration patterns;
the depth coordinate acquisition module 20 is configured to acquire a pixel coordinate of each calibration pattern on the color image, and calculate a depth coordinate of each calibration pattern on the depth image according to the pixel coordinate and a mapping relationship between the pixel coordinate and the depth coordinate;
the first three-dimensional coordinate acquisition module 30 is configured to acquire a depth value of each calibration pattern on the depth image, and calculate a three-dimensional coordinate of each calibration pattern in a depth camera coordinate system according to the depth coordinate and the depth value;
the first rotation and translation matrix acquisition module 40 is used for respectively controlling the tail end of the mechanical arm to move to the position where the calibration tool aligns to each calibration pattern, and calculating to obtain a rotation and translation matrix of the local coordinate system of the tail end joint of the mechanical arm relative to the basic coordinate system of the mechanical arm according to the positive kinematics method of the mechanical arm;
the second three-dimensional coordinate acquisition module 50 is configured to calculate a three-dimensional coordinate of the tail end of the calibration tool in the basic coordinate system of the robot arm, that is, a three-dimensional coordinate of each calibration pattern in the basic coordinate system of the robot arm, according to the rotation and translation matrix and a preset coordinate of the calibration tool;
and a second rotation and translation matrix obtaining module 60, configured to calculate, according to the three-dimensional coordinates of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinates of each calibration pattern in the mechanical arm base coordinate system, a rotation and translation matrix between the depth camera coordinate system and the mechanical arm base coordinate system by using a singular value decomposition method.
Based on the aforementioned calibration method for the mechanical arm coordinate system and the camera coordinate system, the invention further provides a calibration device for the mechanical arm coordinate system and the camera coordinate system, wherein the calibration device for the mechanical arm coordinate system and the camera coordinate system comprises:
a memory for storing a computer program;
a processor for implementing the steps of the robot arm coordinate system and camera coordinate system calibration method as shown in fig. 1 when executing the computer program.
Based on the aforementioned proposed robot arm coordinate system and camera coordinate system calibration method, the present invention also proposes a medium storing a computer program, which when executed by a processor implements each step in the robot arm coordinate system and camera coordinate system calibration method as shown in fig. 1.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules or components may be combined or integrated into another apparatus, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only a part of or preferred embodiments of the present invention, and neither the text nor the drawings should be construed as limiting the scope of the present invention, and all equivalent structural changes, which are made by using the contents of the present specification and the drawings, or any other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for calibrating a coordinate system of a mechanical arm and a coordinate system of a camera is characterized by comprising the following steps:
a depth camera is installed on any joint of the mechanical arm, and a calibration tool is installed at the tail end of the mechanical arm;
the method comprises the steps that a mechanical arm is controlled to move to the position of a calibration plate, so that a depth camera shoots a complete calibration plate image, the calibration plate image comprises a color image and a depth image, and a plurality of calibration patterns are arranged on the calibration plate;
acquiring the pixel coordinate of each calibration pattern on the color image, and calculating the depth coordinate of each calibration pattern on the depth image according to the pixel coordinate and the mapping relation between the pixel coordinate and the depth coordinate;
acquiring the depth value of each calibration pattern on the depth image, and calculating the three-dimensional coordinate of each calibration pattern in a depth camera coordinate system according to the depth coordinate and the depth value;
respectively controlling the tail end of the mechanical arm to move to the position where the calibration tool aligns to each calibration pattern, and calculating to obtain a rotation and translation matrix of a local coordinate system of the tail end joint of the mechanical arm relative to a basic coordinate system of the mechanical arm according to a positive kinematics method of the mechanical arm;
calculating the three-dimensional coordinates of the tail end of the calibration tool under a mechanical arm basic coordinate system according to the rotation and translation matrix and the coordinates of a preset calibration tool, namely the three-dimensional coordinates of each calibration pattern under the mechanical arm basic coordinate system;
and calculating to obtain a rotation and translation matrix between the depth camera coordinate system and the mechanical arm basic coordinate system through a singular value decomposition method according to the three-dimensional coordinate of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinate of each calibration pattern in the mechanical arm basic coordinate system.
2. The robot arm coordinate system and camera coordinate system calibration method according to claim 1, wherein the mapping relationship between the pixel coordinates and the depth coordinates is:
xs＝a*xc+b
ys＝e*yc+f
wherein, the (x)c，yc) For the pixel coordinates of a certain calibration pattern on the color image, the (x)s，ys) Is the (x)c，yc) And coordinates mapped on the depth map, wherein a, b, e and f are scaling translation parameters between the color image and the depth image respectively.
3. The robot arm coordinate system and camera coordinate system calibration method as claimed in claim 1, wherein the three-dimensional coordinates of each of the calibration patterns in the depth camera coordinate system are calculated according to the following formula:
X＝D*(xs-U0)/(fx)
Y＝D*(ys-V0)/(fy)
Z＝D
wherein (X, Y, Z) is the three-dimensional coordinate of the calibration pattern in the depth camera coordinate system, D is the depth value of the calibration pattern on the depth image, and U is the depth value of the calibration pattern0、V0、fx、fyRespectively, the internal parameters of the depth camera.
4. The robot arm coordinate system and camera coordinate system calibration method according to claim 1, wherein the calibration pattern is a circular marker, the calibration tool is a cylindrical tool, and the diameter of the cylindrical tool is the same as that of the circular marker.
5. The utility model provides a manipulator coordinate system and camera coordinate system calibration device which characterized in that includes:
the image acquisition module is used for controlling the mechanical arm to move to the position of the calibration plate so that the depth camera can shoot a complete calibration plate image, the calibration plate image comprises a color image and a depth image, and a plurality of calibration patterns are arranged on the calibration plate;
the depth coordinate acquisition module is used for acquiring the pixel coordinate of each calibration pattern on the color image and calculating the depth coordinate of each calibration pattern on the depth image according to the pixel coordinate and the mapping relation between the pixel coordinate and the depth coordinate;
the first three-dimensional coordinate acquisition module is used for acquiring the depth value of each calibration pattern on the depth image and calculating the three-dimensional coordinate of each calibration pattern in a depth camera coordinate system according to the depth coordinate and the depth value;
the first rotation and translation matrix acquisition module is used for respectively controlling the tail end of the mechanical arm to move to the position where the calibration tool aligns to each calibration pattern, and calculating to obtain a rotation and translation matrix of a local coordinate system of the tail end joint of the mechanical arm relative to a basic coordinate system of the mechanical arm according to a positive kinematics method of the mechanical arm;
the second three-dimensional coordinate acquisition module is used for calculating the three-dimensional coordinates of the tail end of the calibration tool under the basic coordinate system of the mechanical arm according to the rotation and translation matrix and the coordinates of a preset calibration tool, namely the three-dimensional coordinates of each calibration pattern under the basic coordinate system of the mechanical arm;
and the second rotation and translation matrix acquisition module is used for calculating to obtain a rotation and translation matrix between the depth camera coordinate system and the mechanical arm basic coordinate system through a singular value decomposition method according to the three-dimensional coordinate of each calibration pattern in the depth camera coordinate system and the three-dimensional coordinate of each calibration pattern in the mechanical arm basic coordinate system.
6. The robot arm coordinate system and camera coordinate system calibration device of claim 5, wherein the mapping relationship between the pixel coordinates and the depth coordinates is:
xs＝a*xc+b
ys＝e*yc+f
wherein, the (x)c，yc) For the pixel coordinates of a certain calibration pattern on the color image, the (x)s，ys) Is the (x)c，yc) And coordinates mapped on the depth map, wherein a, b, e and f are scaling translation parameters between the color image and the depth image respectively.
7. The robotic arm and camera coordinate system calibration device of claim 5, wherein the three-dimensional coordinates of each of the calibration patterns in the depth camera coordinate system are calculated according to the following formula:
X＝D*(xs-U0)/(fx)
Y＝D*(ys-V0)/(fy)
Z＝D
wherein (X, Y, Z) is the three-dimensional coordinate of the calibration pattern in the depth camera coordinate system, D is the depth value of the calibration pattern on the depth image, and U is the depth value of the calibration pattern0、V0、fx、fyRespectively, the internal parameters of the depth camera.
8. The mechanical arm coordinate system and camera coordinate system calibration device as claimed in claim 5, wherein the calibration pattern is a circular mark point, the calibration tool is a cylindrical tool, and the diameter of the cylindrical tool is the same as that of the circular mark point.
9. A robot arm coordinate system and camera coordinate system calibration apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the robot arm coordinate system and camera coordinate system calibration method of any one of claims 1-4 when executing the computer program.
10. A medium, characterized in that it stores a computer program which, when executed by a processor, implements the robot arm coordinate system and camera coordinate system calibration method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010615931.8A CN111890356A (en) | 2020-06-30 | 2020-06-30 | Mechanical arm coordinate system and camera coordinate system calibration method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010615931.8A CN111890356A (en) | 2020-06-30 | 2020-06-30 | Mechanical arm coordinate system and camera coordinate system calibration method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111890356A true CN111890356A (en) | 2020-11-06 |
Family
ID=73191304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010615931.8A Pending CN111890356A (en) | 2020-06-30 | 2020-06-30 | Mechanical arm coordinate system and camera coordinate system calibration method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111890356A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112643207A (en) * | 2020-12-12 | 2021-04-13 | 南京理工大学 | Laser automatic derusting system and method based on computer vision |
| CN113223048A (en) * | 2021-04-20 | 2021-08-06 | 深圳瀚维智能医疗科技有限公司 | Hand-eye calibration precision determination method and device, terminal equipment and storage medium |
| CN113263499A (en) * | 2021-04-19 | 2021-08-17 | 深圳瀚维智能医疗科技有限公司 | Mechanical arm hand-eye calibration method, device and system and computer readable storage medium |
| CN114046889A (en) * | 2021-11-18 | 2022-02-15 | 南京佗道医疗科技有限公司 | Automatic calibration method of infrared camera |
| CN114189630A (en) * | 2021-12-08 | 2022-03-15 | 湖南快乐阳光互动娱乐传媒有限公司 | Method and device for processing relative coordinates of camera |
| CN115969418A (en) * | 2022-11-07 | 2023-04-18 | 珠海市人民医院 | Method for identifying nucleic acid detection oral cavity collection points |
| CN114046889B (en) * | 2021-11-18 | 2024-04-30 | 佗道医疗科技有限公司 | Automatic calibration method for infrared camera |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2521429A (en) * | 2013-12-19 | 2015-06-24 | Canon Kk | Visual Servoing |
| CN109658460A (en) * | 2018-12-11 | 2019-04-19 | 北京无线电测量研究所 | A kind of mechanical arm tail end camera hand and eye calibrating method and system |
| CN109920006A (en) * | 2019-01-21 | 2019-06-21 | 上海大学 | A kind of blueness feeding machine throws the scaling method of system module and carriage transformation matrix automatically |
| CN110497386A (en) * | 2019-08-26 | 2019-11-26 | 中科新松有限公司 | A kind of cooperation Robot Hand-eye relationship automatic calibration device and method |
| CN110695991A (en) * | 2019-09-26 | 2020-01-17 | 南京理工大学 | Automatic hand-eye calibration method based on variation particle swarm optimization |
| CN110842914A (en) * | 2019-10-15 | 2020-02-28 | 上海交通大学 | Hand-eye calibration parameter identification method, system and medium based on differential evolution algorithm |
-
2020
- 2020-06-30 CN CN202010615931.8A patent/CN111890356A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2521429A (en) * | 2013-12-19 | 2015-06-24 | Canon Kk | Visual Servoing |
| CN109658460A (en) * | 2018-12-11 | 2019-04-19 | 北京无线电测量研究所 | A kind of mechanical arm tail end camera hand and eye calibrating method and system |
| CN109920006A (en) * | 2019-01-21 | 2019-06-21 | 上海大学 | A kind of blueness feeding machine throws the scaling method of system module and carriage transformation matrix automatically |
| CN110497386A (en) * | 2019-08-26 | 2019-11-26 | 中科新松有限公司 | A kind of cooperation Robot Hand-eye relationship automatic calibration device and method |
| CN110695991A (en) * | 2019-09-26 | 2020-01-17 | 南京理工大学 | Automatic hand-eye calibration method based on variation particle swarm optimization |
| CN110842914A (en) * | 2019-10-15 | 2020-02-28 | 上海交通大学 | Hand-eye calibration parameter identification method, system and medium based on differential evolution algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 张国良等: "《移动机器人的SLAM与VSLAM方法》", 31 October 2018, 西安交通大学出版社 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112643207A (en) * | 2020-12-12 | 2021-04-13 | 南京理工大学 | Laser automatic derusting system and method based on computer vision |
| CN112643207B (en) * | 2020-12-12 | 2022-09-30 | 南京理工大学 | Laser automatic derusting system and method based on computer vision |
| CN113263499A (en) * | 2021-04-19 | 2021-08-17 | 深圳瀚维智能医疗科技有限公司 | Mechanical arm hand-eye calibration method, device and system and computer readable storage medium |
| CN113223048A (en) * | 2021-04-20 | 2021-08-06 | 深圳瀚维智能医疗科技有限公司 | Hand-eye calibration precision determination method and device, terminal equipment and storage medium |
| CN113223048B (en) * | 2021-04-20 | 2024-02-27 | 深圳瀚维智能医疗科技有限公司 | Method and device for determining hand-eye calibration precision, terminal equipment and storage medium |
| CN114046889A (en) * | 2021-11-18 | 2022-02-15 | 南京佗道医疗科技有限公司 | Automatic calibration method of infrared camera |
| CN114046889B (en) * | 2021-11-18 | 2024-04-30 | 佗道医疗科技有限公司 | Automatic calibration method for infrared camera |
| CN114189630A (en) * | 2021-12-08 | 2022-03-15 | 湖南快乐阳光互动娱乐传媒有限公司 | Method and device for processing relative coordinates of camera |
| CN114189630B (en) * | 2021-12-08 | 2024-04-02 | 湖南快乐阳光互动娱乐传媒有限公司 | Processing method and device for relative coordinates of camera |
| CN115969418A (en) * | 2022-11-07 | 2023-04-18 | 珠海市人民医院 | Method for identifying nucleic acid detection oral cavity collection points |
| CN115969418B (en) * | 2022-11-07 | 2023-08-15 | 珠海市人民医院 | Nucleic acid detection oral cavity collection point identification method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111890356A (en) | Mechanical arm coordinate system and camera coordinate system calibration method, device, equipment and medium | |
| CN110555889B (en) | CALTag and point cloud information-based depth camera hand-eye calibration method | |
| US10984554B2 (en) | Monocular vision tracking method, apparatus and non-volatile computer-readable storage medium | |
| CN110728715B (en) | Intelligent inspection robot camera angle self-adaptive adjustment method | |
| CN110695993B (en) | Synchronous measurement method, system and device for flexible mechanical arm | |
| DE102015101710B4 (en) | A method of calibrating a moveable gripping member using a remote digital camera | |
| CN112223302B (en) | Rapid calibration method and device of live working robot based on multiple sensors | |
| CN110355755B (en) | Robot hand-eye system calibration method, device, equipment and storage medium | |
| CN109297436B (en) | Binocular line laser stereo measurement reference calibration method | |
| CN110095089B (en) | Method and system for measuring rotation angle of aircraft | |
| CN113263499B (en) | Mechanical arm hand-eye calibration method, device and system and computer readable storage medium | |
| CN111612794A (en) | Multi-2D vision-based high-precision three-dimensional pose estimation method and system for parts | |
| CN114211484B (en) | Front-end tool pose synchronization method, electronic equipment and storage medium | |
| CN112792814B (en) | Mechanical arm zero calibration method based on visual marks | |
| CN108942918A (en) | A kind of solid locating method based on line-structured light | |
| US20220230348A1 (en) | Method and apparatus for determining a three-dimensional position and pose of a fiducial marker | |
| CN110695982A (en) | Mechanical arm hand-eye calibration method and device based on three-dimensional vision | |
| CN109781164A (en) | A kind of static demarcating method of line laser sensor | |
| CN111590593A (en) | Calibration method, device and system of mechanical arm and storage medium | |
| JPWO2018043524A1 (en) | Robot system, robot system control apparatus, and robot system control method | |
| CN114310880A (en) | Mechanical arm calibration method and device | |
| CN112603542A (en) | Hand-eye calibration method and device, electronic equipment and storage medium | |
| CN115439633A (en) | Calibration method and device and electronic equipment | |
| CN115187612A (en) | Plane area measuring method, device and system based on machine vision | |
| CN107993227B (en) | Method and device for acquiring hand-eye matrix of 3D laparoscope |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201106 |