CN117301052A - Pose conversion method, device, equipment and storage medium - Google Patents
Pose conversion method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117301052A CN117301052A CN202311212463.XA CN202311212463A CN117301052A CN 117301052 A CN117301052 A CN 117301052A CN 202311212463 A CN202311212463 A CN 202311212463A CN 117301052 A CN117301052 A CN 117301052A
- Authority
- CN
- China
- Prior art keywords
- corrected
- hand
- pose
- mechanical arm
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 88
- 238000000034 method Methods 0.000 title claims abstract description 67
- 230000000007 visual effect Effects 0.000 claims abstract description 38
- 238000012937 correction Methods 0.000 claims abstract description 35
- 230000006870 function Effects 0.000 claims description 52
- 238000005457 optimization Methods 0.000 claims description 38
- 239000012636 effector Substances 0.000 claims description 33
- 239000011159 matrix material Substances 0.000 claims description 23
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004891 communication Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 7
- 230000009471 action Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 238000005299 abrasion Methods 0.000 description 3
- 238000003754 machining Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000010561 standard procedure Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The embodiment of the application provides a pose conversion method, a pose conversion device, pose conversion equipment and a storage medium, and relates to the technical field of robots, wherein the pose conversion device comprises the following specific implementation scheme: acquiring the pose of an object to be grabbed by the mechanical arm under a camera coordinate system of a target camera, and taking the pose as a visual positioning pose; acquiring a corrected hand-eye calibration result and corrected DH parameters of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; and performing pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameters and the initial DH parameters to obtain the actual grabbing pose of the object. Therefore, through the scheme, the grabbing precision of the mechanical arm can be improved.
Description
Technical Field
The present disclosure relates to the field of robots, and in particular, to a pose conversion method, apparatus, device, and storage medium.
Background
With the continuous improvement of the industrial automation degree, a scheme of performing object gripping by using a vision-guided mechanical arm has been receiving a great deal of attention. The so-called vision guiding mechanical arm grabbing, namely, acquiring a vision positioning pose of an object to be grabbed under a camera coordinate system by using a camera, converting the pose of the object to be grabbed under the base coordinate system from the camera coordinate system according to a hand-eye calibration result obtained through calibration, namely, a conversion relation between the camera coordinate system and the base coordinate system of the mechanical arm, and then controlling an end effector of the mechanical arm to move to the pose of the object under the base coordinate system so as to execute grabbing actions.
In practical application, due to machining errors of the mechanical arm itself or abrasion existing along with the increase of the using times of the mechanical arm, the actual DH parameters of the mechanical arm may be different from the initial DH parameters when leaving the factory; the hand-eye calibration result is generated based on the initial DH parameter, and finally, the hand-eye calibration result is utilized to convert the pose of the object, and the grabbing precision is not high. The DH parameter is a parameter in a DH (Denavit-Hartenberg) kinematic model, the DH kinematic model is a robot kinematic representation and coordinate system modeling method proposed by Denavit and Hartenberg, and the DH kinematic model is a standard method for modeling the robot kinematically.
Therefore, how to improve the grabbing precision of the mechanical arm is a technical problem to be solved.
Disclosure of Invention
An object of the embodiment of the application is to provide a pose conversion method, a pose conversion device, pose conversion equipment and a storage medium, so as to improve grabbing precision of a mechanical arm. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present application provides a pose conversion method, where the method includes:
acquiring the pose of an object to be grabbed by the mechanical arm under a camera coordinate system of a target camera, and taking the pose as a visual positioning pose;
acquiring a corrected hand-eye calibration result and corrected DH parameters of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; the target image is an image acquired by the target camera when the calibration plate is positioned on the end effector of the mechanical arm;
and performing pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameters and the initial DH parameters to obtain the actual grabbing pose of the object.
Optionally, the method for jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameter according to the target image of the calibration plate and the re-projection result of the image plane of the calibration plate projected onto the target image includes:
constructing a first expression of a reprojection coordinate of a datum point aiming at the datum point of the calibration plate; the first expression is a functional expression based on a hand-eye calibration result of the mechanical arm and DH parameters; the re-projection coordinates of the reference point are re-projection results of the reference point projected to an image plane of the target image;
based on the error between the image coordinates corresponding to the datum points and the re-projection coordinates calculated by the first expression, carrying out joint correction on a pre-calibrated hand-eye calibration result and an initial DH parameter; the image coordinates corresponding to the reference points are the image coordinates of the reference points in the target image.
Optionally, the performing joint correction on the pre-calibrated hand-eye calibration result and the initial DH parameter based on the error between the image coordinate corresponding to the reference point and the re-projection coordinate calculated by the first expression includes:
Constructing a joint optimization function based on an error between the image coordinates corresponding to the reference points and the re-projection coordinates calculated by the first expression; the joint optimization function is used for obtaining a hand-eye calibration result and DH parameters of the mechanical arm when the error is minimum;
the method comprises the steps of adjusting a hand-eye calibration result and DH parameters contained in a first expression in the joint optimization function, and solving the joint optimization function to obtain a corrected hand-eye calibration result and corrected DH parameters;
the initial value of the hand-eye calibration result contained in the first expression is the hand-eye calibration result calibrated in advance.
Optionally, the performing pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameter, and the initial DH parameter to obtain an actual grabbing pose of the object includes:
based on the corrected hand-eye calibration result, converting the visual positioning pose into a grabbing pose under a corrected base coordinate system; the corrected base coordinate system is a base coordinate system of the mechanical arm under the corrected DH parameters;
Based on the corrected DH parameters, performing kinematic inverse solution calculation on the grabbing pose under the corrected base coordinate system to obtain various joint variables;
and performing kinematic positive solution calculation based on the calculated joint variables and the initial DH parameters to obtain the actual grabbing pose of the object.
Optionally, the first expression is:
wherein K is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is the length of the armParameters, θ is zero bias of the mechanical arm, +.>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
Optionally, the joint optimization function is:
wherein, the reference point has a value of i= … n, n is a positive integer; uv is the image coordinates corresponding to the reference point; k is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate, A conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
In a second aspect, an embodiment of the present application provides a pose conversion device, where the pose conversion device includes:
the first acquisition module is used for acquiring the pose of an object to be grabbed by the mechanical arm under the camera coordinate system of the target camera, and taking the pose as a visual positioning pose;
the second acquisition module is used for acquiring the corrected hand-eye calibration result and the corrected DH parameters of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; the target image is an image acquired by the target camera when the calibration plate is positioned on the end effector of the mechanical arm;
And the pose conversion module is used for carrying out pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameter and the initial DH parameter to obtain the actual grabbing pose of the object.
Optionally, the method for jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameter according to the target image of the calibration plate and the re-projection result of the image plane of the calibration plate projected onto the target image includes:
constructing a first expression of a reprojection coordinate of a datum point aiming at the datum point of the calibration plate; the first expression is a functional expression based on a hand-eye calibration result of the mechanical arm and DH parameters; the re-projection coordinates of the reference point are re-projection results of the reference point projected to an image plane of the target image;
based on the error between the image coordinates corresponding to the datum points and the re-projection coordinates calculated by the first expression, carrying out joint correction on a pre-calibrated hand-eye calibration result and an initial DH parameter; the image coordinates corresponding to the reference points are the image coordinates of the reference points in the target image.
Optionally, the performing joint correction on the pre-calibrated hand-eye calibration result and the initial DH parameter based on the error between the image coordinate corresponding to the reference point and the re-projection coordinate calculated by the first expression includes:
constructing a joint optimization function based on an error between the image coordinates corresponding to the reference points and the re-projection coordinates calculated by the first expression; the joint optimization function is used for obtaining a hand-eye calibration result and DH parameters of the mechanical arm when the error is minimum;
the method comprises the steps of adjusting a hand-eye calibration result and DH parameters contained in a first expression in the joint optimization function, and solving the joint optimization function to obtain a corrected hand-eye calibration result and corrected DH parameters;
the initial value of the hand-eye calibration result contained in the first expression is the hand-eye calibration result calibrated in advance.
Optionally, the pose conversion module includes:
the conversion sub-module is used for converting the visual positioning pose into a grabbing pose under a corrected base coordinate system based on the corrected hand-eye calibration result; the corrected base coordinate system is a base coordinate system of the mechanical arm under the corrected DH parameters;
The inverse solution sub-module is used for carrying out kinematic inverse solution calculation on the grabbing pose under the corrected base coordinate system based on the corrected DH parameters to obtain various joint variables;
and the orthographic solution sub-module is used for carrying out kinematic orthographic solution calculation based on the calculated joint variables and the initial DH parameters to obtain the actual grabbing pose of the object.
Optionally, the first expression is:
wherein K is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is the motionA mathematical orthometric function, wherein L is a rod length parameter of the mechanical arm, theta is zero offset of the mechanical arm, and +.>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
Optionally, the joint optimization function is:
wherein, the reference point has a value of i= … n, n is a positive integer; uv is the image coordinates corresponding to the reference point; k is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any one of the steps of the pose conversion method when executing the program stored in the memory.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium having a computer program stored therein, the computer program implementing the steps of any one of the above-described pose conversion methods when executed by a processor.
The beneficial effects of the embodiment of the application are that:
according to the scheme provided by the embodiment of the application, as the corrected hand-eye calibration result and the corrected DH parameters are obtained by jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameters by utilizing the target image of the calibration plate and the re-projection result of the image plane of the target image projected by the calibration plate, the corrected DH parameters are more accurate DH parameters of the mechanical arm. According to DH parameters before and after correction and corrected hand-eye calibration results, the visual positioning pose of the object to be grabbed is transformed, so that the grabbing precision can be effectively improved on the basis of not modifying the current DH parameters of the mechanical arm under the condition that the absolute precision of the mechanical arm is insufficient. Therefore, through the scheme, the grabbing precision of the mechanical arm can be improved.
Of course, not all of the above-described advantages need be achieved simultaneously in practicing any one of the products or methods of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following description will briefly introduce the drawings that are required to be used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other embodiments may also be obtained according to these drawings to those skilled in the art.
Fig. 1 is a flowchart of a pose conversion method provided in an embodiment of the present application;
fig. 2 is a flowchart of step S103 in implementing the pose conversion method provided in the embodiment of the present application;
fig. 3 is a flowchart of step S102 in implementing the pose conversion method provided in the embodiment of the present application;
FIG. 4 is a flowchart of a specific example of a pose conversion method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a pose conversion device according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device implementing the pose conversion method provided in the embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. Based on the embodiments herein, a person of ordinary skill in the art would be able to obtain all other embodiments based on the disclosure herein, which are within the scope of the disclosure herein.
Next, a pose conversion method provided in the embodiments of the present application will be described first.
The pose conversion method provided by the embodiment of the application can be applied to various electronic devices, such as personal computers, servers and other devices with data processing capability. In addition, it can be understood that the pose conversion method provided in the embodiment of the present application may be implemented by software, hardware, or a combination of software and hardware.
The pose conversion method provided by the embodiment of the application can include the following steps:
acquiring the pose of an object to be grabbed by the mechanical arm under a camera coordinate system of a target camera, and taking the pose as a visual positioning pose;
acquiring a corrected hand-eye calibration result and corrected DH parameters of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; the target image is an image acquired by the target camera when the calibration plate is positioned on the end effector of the mechanical arm;
And performing pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameters and the initial DH parameters to obtain the actual grabbing pose of the object.
According to the scheme provided by the embodiment of the application, as the corrected hand-eye calibration result and the corrected DH parameters are obtained by jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameters by utilizing the target image of the calibration plate and the re-projection result of the image plane of the target image projected by the calibration plate, the corrected DH parameters are more accurate DH parameters of the mechanical arm. According to DH parameters before and after correction and corrected hand-eye calibration results, the visual positioning pose of the object to be grabbed is transformed, so that the grabbing precision can be effectively improved on the basis of not modifying the current DH parameters of the mechanical arm under the condition that the absolute precision of the mechanical arm is insufficient. Therefore, through the scheme, the grabbing precision of the mechanical arm can be improved.
The pose conversion method provided by the embodiment of the application is described below with reference to the accompanying drawings.
As shown in fig. 1, the pose conversion method provided in the embodiment of the present application may include steps S101 to S103:
S101, acquiring the pose of an object to be grabbed by the mechanical arm under a camera coordinate system of a target camera, and taking the pose as a visual positioning pose;
in this embodiment, the target camera is a camera associated with the robotic arm, and the robotic arm performs capturing of the object by combining the visual positioning provided by the target camera. Illustratively, the target camera may be mounted at the end of the arm or may be fixed to a fixed mount outside the arm, as is reasonable. It should be noted that, due to different installation modes of the target camera, different calculation formulas are corresponding to the subsequent pose conversion; therefore, in the present embodiment, for the sake of clarity of the solution, the pose conversion method provided in the present embodiment is described by way of example in a manner in which the target camera is mounted on the fixed base outside the robot arm.
In practical application, the target camera acquires the pose of the object to be grabbed under the camera coordinate system of the target camera by acquiring the image of the object to be grabbed. That is, the target camera is an intelligent camera embedded with a pose calculating function, which can process the collected image of the object, including graying, edge detection, contour extraction and the like, and extract the characteristics of the object to be grabbed, such as the characteristics of angular points, edges and the like, from the image; therefore, the position and the direction of the object to be grabbed under the camera coordinate system are calculated by using the extracted features, and the pose of the object to be grabbed under the camera coordinate system is obtained.
S102, acquiring a corrected hand-eye calibration result and corrected DH parameters of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; the target image is an image acquired by the target camera when the calibration plate is positioned on the end effector of the mechanical arm;
it can be appreciated that when the mechanical arm is used to grasp in combination with the visual positioning pose, the pose of the object to be grasped under the camera coordinate system is obtained by the target camera and cannot be directly used. The pose of the object to be grabbed under the camera coordinate system is converted into the pose of the object to be grabbed under the base coordinate system of the mechanical arm, so that the end effector can move to the pose of the object to be grabbed under the base coordinate system to grab the object.
In practical application, the pose of the object to be grabbed can be converted from the camera coordinate system to the base coordinate system by using the hand-eye calibration result obtained by carrying out hand-eye calibration on the mechanical arm, namely, the visual positioning pose of the object to be grabbed is converted into the actual grabbing pose. The actual DH parameter of the mechanical arm may be different from the initial DH parameter when leaving the factory due to the machining error of the mechanical arm itself or abrasion caused by the increase of the using times of the mechanical arm; the hand-eye calibration result is generated based on the initial DH parameter, and finally, the hand-eye calibration result is utilized to convert the pose of the object, and the grabbing precision is not high. Therefore, before pose conversion is carried out, the corrected hand-eye calibration result and corrected DH parameters of the mechanical arm can be obtained. The corrected DH parameter is the actual DH parameter of the mechanical arm. It should be noted that DH parameters of the mechanical arm may include rod length, joint axial direction, joint angle zero offset, and so on.
In this embodiment, the corrected hand-eye calibration result and the corrected DH parameter are obtained by performing joint correction on a pre-calibrated hand-eye calibration result and an initial DH parameter according to the target image related to the calibration plate and a re-projection result of the calibration plate projected onto an image plane of the target image.
In practical application, the end effector of the mechanical arm can be controlled to move to a space point by carrying the calibration plate, and then the target camera acquires the image of the calibration plate as a target image. It can be understood that if the actual DH parameter of the mechanical arm is the same as the initial DH parameter when shipped, the re-projection result of the calibration plate projected onto the image plane of the target image is the same as the calibration plate in the target image, so that the pre-calibrated hand-eye calibration result and the initial DH parameter can be jointly corrected under the constraint that the re-projection result of the calibration plate is the same as the calibration plate in the target image, to obtain the corrected hand-eye calibration result and the corrected DH parameter of the mechanical arm.
The initial DH parameter may be obtained from a factory parameter of the mechanical arm. The calibration mode of the pre-calibrated hand-eye calibration result can comprise the following steps of A1-A3:
A1, acquiring an image of the end effector of the mechanical arm moving to a target space point position in a working space with a calibration plate, and recording the pose of the end effector of the mechanical arm as an endPose and position; wherein the number of the target space points is a plurality of; the terminal pose is represented as a transformation matrix
A2, determining the position and the posture of the calibration plate in the camera coordinate system based on the acquired image of the calibration plate aiming at each target space point
A3, constructing a homogeneous equation of AX=XB according to the positions and the tail end positions of the calibration plate corresponding to any two images;
wherein, a is a conversion matrix between the positions of the calibration plates corresponding to the two target space points i and j, and can be expressed as follows:wherein (1)>For the pose of the calibration plate corresponding to the target space point position j under the camera coordinate system, +.>The pose of the camera under the coordinate system of the calibration plate corresponding to the target space point position i is set; b is a conversion matrix between the terminal poses corresponding to the two target space points i and j, and can be expressed as follows:wherein (1)>Is the pose of the end pose corresponding to the target space point position j under the basic coordinate system,is that the base of the mechanical arm is under the terminal coordinate system corresponding to the target space point position i Pose and position; x is a hand-eye calibration result, namely, the pose conversion relation from a base coordinate system to a camera coordinate system, and the hand-eye calibration result can be obtained by solving an equation set.
In practical application, each two images can be constructed to obtain a homogeneous equation with ax=xb, so that a homogeneous equation set formed by a plurality of homogeneous equations can be solved to obtain X.
In addition, for the sake of clarity of layout, the following description of the mode of the joint correction will be omitted here.
S103, performing pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameters and the initial DH parameters to obtain the actual grabbing pose of the object.
It can be understood that after the corrected hand-eye calibration result and the corrected DH parameter of the mechanical arm are obtained, in consideration of that the user often has no authority to adjust the DH parameter of the mechanical arm based on the corrected DH parameter, under the condition that the DH parameter of the mechanical arm cannot be corrected, the visual positioning pose may be converted based on the corrected hand-eye calibration result and the corrected DH parameter, and in combination with the initial DH parameter.
Optionally, in one implementation, as shown in fig. 2, based on the corrected hand-eye calibration result and the corrected DH parameter, and the initial DH parameter, performing pose conversion on the visual positioning pose to obtain an actual capturing pose of the object, which may include steps S1031-S1033:
S1031, converting the visual positioning pose into a grabbing pose under a corrected base coordinate system based on the corrected hand-eye calibration result; the corrected base coordinate system is the base coordinate system of the mechanical arm under the corrected DH parameter;
because the hand-eye calibration result represents the conversion relation from the camera coordinate system to the base coordinate system of the mechanical arm, the visual positioning pose is the pose of the object to be grabbed under the camera coordinate system, and therefore, the visual positioning pose can be converted into the grabbing pose under the corrected base coordinate system by multiplying the visual positioning pose and the corrected hand-eye calibration result.
S1032, based on the corrected DH parameter, performing kinematic inverse solution calculation on the grabbing pose under the corrected base coordinate system to obtain each joint variable;
it can be understood that, since the inverse kinematics solution is calculated as a process of calculating each joint variable according to the end pose of the end effector, the corrected grasping pose in the base coordinate system is: when the mechanical arm grabs an object under the corrected DH parameters, the end effector moves to the end pose; therefore, according to the corrected DH parameter, the motion inverse solution calculation can be carried out on the grabbing pose under the corrected base coordinate system, so as to obtain each joint variable. Wherein the joint variables include joint angles.
S1033, performing kinematic positive solution calculation based on the calculated joint variables and the initial DH parameters to obtain the actual grabbing pose of the object.
It can be understood that, because of the corrected hand-eye calibration result and the corrected DH parameter, the corrected hand-eye calibration result and the corrected DH parameter are obtained by optimizing the pre-calibrated hand-eye calibration result and the initial DH parameter based on the constraint that the re-projection result of the calibration plate is the same as that of the calibration plate in the target image; therefore, when the mechanical arm performs object grabbing, the end effector needs to move to the actual joint variable of each joint under the grabbing pose based on the corrected hand-eye calibration result and the corrected DH parameter. Therefore, compared with the method of directly performing pose conversion on the visual positioning pose by using the pre-calibrated hand-eye calibration result, the method of performing kinematic correction on the joint variables and the initial DH parameters in the embodiment has higher precision of the obtained actual grabbing pose of the object. Therefore, the end effector of the mechanical arm is controlled to grasp the object according to the actual grasping pose calculated by the kinematic positive solution, so that the grasping precision of the mechanical arm can be improved.
According to the scheme provided by the embodiment of the application, as the corrected hand-eye calibration result and the corrected DH parameters are obtained by jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameters by utilizing the target image of the calibration plate and the re-projection result of the image plane of the target image projected by the calibration plate, the corrected DH parameters are more accurate DH parameters of the mechanical arm. According to DH parameters before and after correction and corrected hand-eye calibration results, the visual positioning pose of the object to be grabbed is transformed, so that the grabbing precision can be effectively improved on the basis of not modifying the current DH parameters of the mechanical arm under the condition that the absolute precision of the mechanical arm is insufficient. Therefore, through the scheme, the grabbing precision of the mechanical arm can be improved.
Optionally, in another embodiment of the present application, as shown in fig. 3, in the step S102, the method for performing joint correction on the pre-calibrated hand-eye calibration result and the initial DH parameter according to the target image of the calibration plate and the re-projection result of the image plane of the calibration plate projected onto the target image may include steps S1021-S1022:
S1021, constructing a first expression of the re-projection coordinates of the datum point aiming at the datum point of the calibration plate; the first expression is a functional expression based on a hand-eye calibration result of the mechanical arm and DH parameters; the re-projection coordinates of the reference point are the re-projection results of the reference point projected to the image plane of the target image;
by way of example, the calibration plate may be a checkerboard calibration plate, a circular calibration plate, or the like. If the calibration plate is a chessboard calibration plate, the datum point can be an angular point in the calibration plate; if the calibration plate is a circular calibration plate, the reference point may be the center of a circle in the calibration plate.
Illustratively, in one implementation, the first expression may be:
wherein K is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board Is the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, +.>The joint angle of the mechanical arm;And calibrating a result for the hand and eye of the mechanical arm.
It can be understood that, since the hand-eye calibration result is a transformation matrix of the base coordinate system to the camera coordinate system, and byThe kinematic correct calculation of the manipulator can obtain the end pose of the manipulator, namely the pose of an end effector under a basic coordinate system>Thus, by the first expression, the physical coordinates of the reference point in the calibration plate coordinate system of the calibration plate can be converted to the coordinates in the image plane of the target image according to the transfer relationship of the calibration plate coordinate system, the end coordinate system, the base coordinate system, the camera coordinate system, and the image plane, thereby obtaining the re-projection coordinates of the reference point.
S1022, based on the error between the image coordinate corresponding to the datum point and the re-projection coordinate calculated by the first expression, carrying out joint correction on a pre-calibrated hand-eye calibration result and an initial DH parameter; the image coordinates corresponding to the reference point are the image coordinates of the reference point in the target image.
In this embodiment, the pre-calibrated hand-eye calibration result and the initial DH parameter may be jointly corrected by minimizing the error between the image coordinates corresponding to the reference point and the re-projection coordinates calculated by the first expression.
Optionally, in one implementation, performing joint correction on the pre-calibrated hand-eye calibration result and the initial DH parameter based on an error between the image coordinate corresponding to the reference point and the re-projection coordinate calculated by the first expression may include steps B1-B2:
b1, constructing a joint optimization function based on errors between the image coordinates corresponding to the datum points and the re-projection coordinates calculated by the first expression; the combined optimization function is used for obtaining a hand-eye calibration result and DH parameters of the mechanical arm when the error is minimum;
illustratively, in one specific implementation, the joint optimization function is:
wherein, the reference point has a value of i= … n, n is a positive integer; uv is the image coordinates corresponding to the reference point; k is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board Is the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, +.>The joint angle of the mechanical arm; / >And calibrating a result for the hand and eye of the mechanical arm.
B2, solving the joint optimization function by adjusting the hand-eye calibration result and DH parameters contained in the first expression in the joint optimization function to obtain a corrected hand-eye calibration result and corrected DH parameters;
the initial value of the hand-eye calibration result contained in the first expression is the hand-eye calibration result calibrated in advance.
In this embodiment, the pre-calibrated hand-eye calibration result may be used as an initial value of the hand-eye calibration result included in the first expression, and the hand-eye calibration result may be adjusted based on the initial value, so as to solve the numerical value of the hand-eye calibration result and the DH parameter corresponding to the time when the joint optimization function is minimized, and be used as the corrected hand-eye calibration result and the corrected DH parameter. For example, in practical application, an LM (Levenberg-Marquardt) method, a gradient descent method, a Newton method and other algorithms may be used to solve the joint optimization function, so as to obtain a corrected hand-eye calibration result and a corrected DH parameter.
Therefore, according to the scheme, the hand-eye calibration result and DH parameters can be jointly optimized.
For a better understanding of the pose conversion method provided in the present application, the following description will be presented with reference to a specific example.
In the field of high-precision workpiece grabbing application, grabbing precision within 0.5mm is required, and high requirements are required for visual positioning and absolute precision of an industrial robot, but in practical application, the absolute precision of the mechanical arm cannot meet the grabbing requirement of high precision due to machining errors of the mechanical arm or abrasion existing along with the increase of the using times of the mechanical arm.
In order to solve the problem of large grabbing error caused by insufficient absolute precision of the mechanical arm, the example provides a pose conversion method. Firstly, controlling the mechanical arm to rotate and translate in a grabbing space, acquiring images of a plurality of calibration plates, and recording joint variables of the mechanical arm; respectively calculating the pose of the calibration plate under a camera coordinate system and the pose of the end effector, and then calculating the initial value of the hand-eye calibration result; then, carrying out joint optimization on an initial value of the hand-eye calibration result and DH parameters of the mechanical arm to obtain a corrected hand-eye calibration result and DH parameters; finally, the corrected hand-eye calibration result, the mechanical arm parameters before correction and the corrected DH parameters are combined to realize pose conversion. The camera-to-mechanical arm pose conversion relation can be obtained in the hand-eye calibration process, DH parameters of the mechanical arm can be corrected, pose conversion is performed according to DH parameters of the mechanical arm before and after correction, grabbing precision can be effectively improved, and practical application practicability can be effectively improved compared with the existing scheme.
As shown in fig. 4, the specific steps of this example include S401 to S405:
s401, controlling the mechanical arm to rotate and translate, acquiring a calibration plate image and recording various joint variables of the mechanical arm;
the control mechanical arm carries the calibration plate to translate and rotate in the grabbing space, and the control camera (corresponding to the target camera) acquires the image of the calibration plate when moving to one space point position, and records various joint variables of the mechanical arm in the current pose, such as joint angles of various joints.
S402, calculating the pose of the calibration plate under a camera coordinate system and the end pose of an end effector of the mechanical arm;
extracting characteristic points (corresponding to the datum points) of the calibration plates in the image, calculating the pose of each calibration plate in a camera coordinate system according to the physical size of the calibration plate, and representing the pose in the camera coordinate system by using a transformation matrix as followsPerforming kinematic correction by using DH parameters of the mechanical arm to obtain the end pose of the end effector where the calibration plate is positioned, and representing the end pose as +.>
S403, constructing an AX=XB homogeneous matrix equation set, and calculating an initial value of a hand-eye calibration result;
constructing a homogeneous matrix equation set with AX=XB according to the positions of the calibration plates and the terminal positions corresponding to any two images, wherein i and j represent any two spatial points; x is the pose conversion relation from the mechanical arm to the camera, namely the conversion relation from the base coordinate system of the mechanical arm to the camera coordinate system, and is expressed as +.>And then solving AX=XB to obtain an initial value of the hand-eye calibration result.
S404, constructing a reprojection error function containing a hand-eye calibration result and DH parameters of the mechanical arm, and jointly optimizing the hand-eye calibration result and the DH parameters;
the hand-eye calibration result and DH parameter are used as variables to carry out joint optimization, the known constraint conditions in the calibration system are the space physical coordinates of the characteristic points in the calibration plate and the internal parameters of the camera, a reprojection error function of the characteristic points in the image of the calibration plate is constructed by taking the 'calibration plate- > -end effector- > -camera- > -image plane' as a transmission route, and the hand-eye calibration result and DH parameter are solved iteratively by a nonlinear optimization method, so that the corrected hand-eye calibration result and corrected DH parameter are obtained. Wherein the re-projection error function e is expressed as:
wherein uv is the image coordinates corresponding to the feature points in the calibration plate, K is the internal reference of the camera, and D is the distortion coefficient of the camera; p (P) board Is the physical coordinates of the feature point in the calibration plate coordinate system of the calibration plate, A conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector;The conversion matrix is from the terminal coordinate system to the base coordinate system of the mechanical arm;The conversion matrix from the base coordinate system to the camera coordinate system is the calibration result of the hand and eye of the mechanical arm. Wherein (1)>The DH parameters involved in the kinematic positive solution comprise the length L of a rod and zero offset theta, and the joint angle for erecting each shaft is +.>The kinematic formula is expressed as:
thus, the objective function of constructing the joint optimization (corresponding to the joint optimization function above) is:
wherein parameters to be optimized include camera internal parameters K and D and hand-eye calibration resultsDH parameters L and θ, transformation matrix of calibration plate coordinate system to terminal coordinate system +.>The known physical quantity comprises the physical coordinates P of the feature points in the calibration plate board Joint angle per pose +.>And calibrating the image coordinates uv corresponding to the feature points in the plate. The value of the characteristic point in each calibration plate is i= … n, and n is a positive integer. The objective function constructed by the method belongs to a typical nonlinear optimization problem, and can be solved by adopting an LM method, a gradient descent method, a Newton method and other algorithms. />
S405, performing grabbing pose conversion according to the corrected hand-eye calibration result, the DH parameters before correction and the DH parameters after correction.
After the hand-eye calibration is performed, a corrected hand-eye calibration result and corrected DH parameters are obtained, and the grabbing pose conversion is needed to be performed by combining the DH parameters before correction in order to achieve high-precision grabbing. Visual localization pose for objects to be graspedFirstly, converting the corrected hand-eye calibration result into a corrected base coordinate system +.>Then, performing kinematic inverse solution according to the corrected DH parameters to obtain various joint variables; finally, combining DH parameters before correction to perform kinematic correction to obtain the actual grabbing pose +.>
Therefore, through the scheme, the hand-eye calibration process can obtain the pose conversion relation from the camera to the mechanical arm and the DH parameters of the corrected mechanical arm; according to DH parameter and hand-eye calibration result before and after correction, carry out the pose conversion of waiting to snatch the object, can be on the condition that absolute accuracy of arm itself is not enough, on the basis of not modifying the current DH parameter of arm, effectively promote and snatch the precision to compare in the current scheme can effectively promote the ease of use in the application.
Based on the above method embodiment, the embodiment of the present application further provides a pose conversion device, as shown in fig. 5, where the device includes:
The first obtaining module 510 is configured to obtain a pose of an object to be grabbed by the mechanical arm under a camera coordinate system of the target camera, as a visual positioning pose;
the second obtaining module 520 is configured to obtain a corrected hand-eye calibration result and a corrected DH parameter of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; the target image is an image acquired by the target camera when the calibration plate is positioned on the end effector of the mechanical arm;
the pose conversion module 530 is configured to perform pose conversion on the visual positioning pose based on the corrected hand-eye calibration result and the corrected DH parameter, and the initial DH parameter, so as to obtain an actual grabbing pose of the object.
Optionally, the method for jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameter according to the target image of the calibration plate and the re-projection result of the image plane of the calibration plate projected onto the target image includes:
Constructing a first expression of a reprojection coordinate of a datum point aiming at the datum point of the calibration plate; the first expression is a functional expression based on a hand-eye calibration result of the mechanical arm and DH parameters; the re-projection coordinates of the reference point are re-projection results of the reference point projected to an image plane of the target image;
based on the error between the image coordinates corresponding to the datum points and the re-projection coordinates calculated by the first expression, carrying out joint correction on a pre-calibrated hand-eye calibration result and an initial DH parameter; the image coordinates corresponding to the reference points are the image coordinates of the reference points in the target image.
Optionally, the performing joint correction on the pre-calibrated hand-eye calibration result and the initial DH parameter based on the error between the image coordinate corresponding to the reference point and the re-projection coordinate calculated by the first expression includes:
constructing a joint optimization function based on an error between the image coordinates corresponding to the reference points and the re-projection coordinates calculated by the first expression; the joint optimization function is used for obtaining a hand-eye calibration result and DH parameters of the mechanical arm when the error is minimum;
The method comprises the steps of adjusting a hand-eye calibration result and DH parameters contained in a first expression in the joint optimization function, and solving the joint optimization function to obtain a corrected hand-eye calibration result and corrected DH parameters;
the initial value of the hand-eye calibration result contained in the first expression is the hand-eye calibration result calibrated in advance.
Optionally, the pose conversion module includes:
the conversion sub-module is used for converting the visual positioning pose into a grabbing pose under a corrected base coordinate system based on the corrected hand-eye calibration result; the corrected base coordinate system is a base coordinate system of the mechanical arm under the corrected DH parameters;
the inverse solution sub-module is used for carrying out kinematic inverse solution calculation on the grabbing pose under the corrected base coordinate system based on the corrected DH parameters to obtain various joint variables;
and the orthographic solution sub-module is used for carrying out kinematic orthographic solution calculation based on the calculated joint variables and the initial DH parameters to obtain the actual grabbing pose of the object.
Optionally, the first expression is:
wherein K is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
Optionally, the joint optimization function is:
wherein, the reference point has a value of i= … n, n is a positive integer; uv is the image coordinates corresponding to the reference point; k is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
In the technical scheme of the application, the related operations of acquiring, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user are all performed under the condition that the authorization of the user is obtained.
The embodiment of the application also provides an electronic device, as shown in fig. 6, including:
a memory 601 for storing a computer program;
the processor 602 is configured to implement any of the above-described pose conversion methods when executing the program stored in the memory 601.
And the electronic device may further comprise a communication bus and/or a communication interface, through which the processor 602, the communication interface, and the memory 601 communicate with each other.
The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment provided herein, there is also provided a computer readable storage medium having stored therein a computer program which when executed by a processor implements the steps of any of the above-described pose conversion methods.
In yet another embodiment provided herein, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the above-described pose conversion methods.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a Solid State Disk (SSD), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments in part.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. that are within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (14)
1. A method for pose conversion, the method comprising:
acquiring the pose of an object to be grabbed by the mechanical arm under a camera coordinate system of a target camera, and taking the pose as a visual positioning pose;
acquiring a corrected hand-eye calibration result and corrected DH parameters of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; the target image is an image acquired by the target camera when the calibration plate is positioned on the end effector of the mechanical arm;
and performing pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameters and the initial DH parameters to obtain the actual grabbing pose of the object.
2. The method of claim 1, wherein jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameters based on the target image for the calibration plate and the re-projection result of the calibration plate onto the image plane of the target image comprises:
constructing a first expression of a reprojection coordinate of a datum point aiming at the datum point of the calibration plate; the first expression is a functional expression based on a hand-eye calibration result of the mechanical arm and DH parameters; the re-projection coordinates of the reference point are re-projection results of the reference point projected to an image plane of the target image;
based on the error between the image coordinates corresponding to the datum points and the re-projection coordinates calculated by the first expression, carrying out joint correction on a pre-calibrated hand-eye calibration result and an initial DH parameter; the image coordinates corresponding to the reference points are the image coordinates of the reference points in the target image.
3. The method of claim 2, wherein the jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameters based on the error between the image coordinates corresponding to the fiducial point and the re-projection coordinates calculated by the first expression comprises:
Constructing a joint optimization function based on an error between the image coordinates corresponding to the reference points and the re-projection coordinates calculated by the first expression; the joint optimization function is used for obtaining a hand-eye calibration result and DH parameters of the mechanical arm when the error is minimum;
the method comprises the steps of adjusting a hand-eye calibration result and DH parameters contained in a first expression in the joint optimization function, and solving the joint optimization function to obtain a corrected hand-eye calibration result and corrected DH parameters;
the initial value of the hand-eye calibration result contained in the first expression is the hand-eye calibration result calibrated in advance.
4. The method of claim 1, wherein performing pose conversion on the visual positioning pose based on the corrected hand-eye calibration result and the corrected DH parameter, and the initial DH parameter, to obtain an actual gripping pose of the object, comprises:
based on the corrected hand-eye calibration result, converting the visual positioning pose into a grabbing pose under a corrected base coordinate system; the corrected base coordinate system is a base coordinate system of the mechanical arm under the corrected DH parameters;
Based on the corrected DH parameters, performing kinematic inverse solution calculation on the grabbing pose under the corrected base coordinate system to obtain various joint variables;
and performing kinematic positive solution calculation based on the calculated joint variables and the initial DH parameters to obtain the actual grabbing pose of the object.
5. The method of claim 2, wherein the first expression is:
wherein K is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
6. A method according to claim 3, wherein the joint optimization function is:
wherein, the reference point has a value of i= … n, n is a positive integer; uv is the image coordinates corresponding to the reference point; k is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
7. A pose conversion device, characterized in that the device comprises:
the first acquisition module is used for acquiring the pose of an object to be grabbed by the mechanical arm under the camera coordinate system of the target camera, and taking the pose as a visual positioning pose;
the second acquisition module is used for acquiring the corrected hand-eye calibration result and the corrected DH parameters of the mechanical arm; the corrected hand-eye calibration result and the corrected DH parameters are as follows: according to a target image of a calibration plate and a re-projection result of the calibration plate projected to an image plane of the target image, carrying out joint correction on a hand-eye calibration result which is calibrated in advance and an initial DH parameter, and obtaining the target image; the target image is an image acquired by the target camera when the calibration plate is positioned on the end effector of the mechanical arm;
And the pose conversion module is used for carrying out pose conversion on the visual positioning pose based on the corrected hand-eye calibration result, the corrected DH parameter and the initial DH parameter to obtain the actual grabbing pose of the object.
8. The apparatus of claim 7, wherein the means for jointly correcting the pre-calibrated hand-eye calibration result and the initial DH parameters based on the target image for the calibration plate and the re-projection result of the calibration plate onto the image plane of the target image comprises:
constructing a first expression of a reprojection coordinate of a datum point aiming at the datum point of the calibration plate; the first expression is a functional expression based on a hand-eye calibration result of the mechanical arm and DH parameters; the re-projection coordinates of the reference point are re-projection results of the reference point projected to an image plane of the target image;
based on the error between the image coordinates corresponding to the datum points and the re-projection coordinates calculated by the first expression, carrying out joint correction on a pre-calibrated hand-eye calibration result and an initial DH parameter; the image coordinates corresponding to the reference points are the image coordinates of the reference points in the target image.
9. The apparatus of claim 8, wherein the joint correction of the pre-calibrated hand-eye calibration result and the initial DH parameters based on the error between the image coordinates corresponding to the fiducial point and the re-projection coordinates calculated by the first expression comprises:
constructing a joint optimization function based on an error between the image coordinates corresponding to the reference points and the re-projection coordinates calculated by the first expression; the joint optimization function is used for obtaining a hand-eye calibration result and DH parameters of the mechanical arm when the error is minimum;
the method comprises the steps of adjusting a hand-eye calibration result and DH parameters contained in a first expression in the joint optimization function, and solving the joint optimization function to obtain a corrected hand-eye calibration result and corrected DH parameters;
the initial value of the hand-eye calibration result contained in the first expression is the hand-eye calibration result calibrated in advance.
10. The apparatus of claim 7, wherein the pose conversion module comprises:
the conversion sub-module is used for converting the visual positioning pose into a grabbing pose under a corrected base coordinate system based on the corrected hand-eye calibration result; the corrected base coordinate system is a base coordinate system of the mechanical arm under the corrected DH parameters;
The inverse solution sub-module is used for carrying out kinematic inverse solution calculation on the grabbing pose under the corrected base coordinate system based on the corrected DH parameters to obtain various joint variables;
and the orthographic solution sub-module is used for carrying out kinematic orthographic solution calculation based on the calculated joint variables and the initial DH parameters to obtain the actual grabbing pose of the object.
11. The apparatus of claim 8, wherein the first expression is:
wherein K is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the referenceThe physical coordinates of the point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
12. The apparatus of claim 9, wherein the joint optimization function is:
wherein, the reference point has a value of i= … n, n is a positive integer; uv is the image coordinates corresponding to the reference point; k is an internal reference of the target camera, and D is a distortion coefficient of the target camera; p (P) board For the physical coordinates of the datum point in the calibration plate coordinate system of the calibration plate,a conversion matrix for the calibration plate coordinate system to an end coordinate system of the end effector; g () is a kinematic positive solution function, L is a rod length parameter of the mechanical arm, θ is a zero bias of the mechanical arm, < >>The joint angle of the mechanical arm is;And calibrating a result for the hand and eye of the mechanical arm.
13. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the method of any of claims 1-6 when executing a program stored on a memory.
14. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311212463.XA CN117301052A (en) | 2023-09-19 | 2023-09-19 | Pose conversion method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311212463.XA CN117301052A (en) | 2023-09-19 | 2023-09-19 | Pose conversion method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117301052A true CN117301052A (en) | 2023-12-29 |
Family
ID=89249157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311212463.XA Pending CN117301052A (en) | 2023-09-19 | 2023-09-19 | Pose conversion method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117301052A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118342520A (en) * | 2024-06-17 | 2024-07-16 | 深圳思谋信息科技有限公司 | Mechanical arm calibration method and device, intelligent equipment and storage medium |
-
2023
- 2023-09-19 CN CN202311212463.XA patent/CN117301052A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118342520A (en) * | 2024-06-17 | 2024-07-16 | 深圳思谋信息科技有限公司 | Mechanical arm calibration method and device, intelligent equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI776113B (en) | Object pose estimation method, device and computer readable storage medium thereof | |
| CN109159114B (en) | High-precision method for hand-eye calibration of SCARA manipulator fixed camera vision system | |
| CN110640746B (en) | Method, system, equipment and medium for calibrating and positioning coordinate system of robot | |
| CN110640747B (en) | Hand-eye calibration method and system for robot, electronic equipment and storage medium | |
| CN110148187B (en) | High-precision Hand-Eye calibration method and system for SCARA manipulator Eye-in-Hand | |
| CN113715016B (en) | Robot grabbing method, system, device and medium based on 3D vision | |
| CN112621743B (en) | Robot, hand-eye calibration method for fixing camera at tail end of robot and storage medium | |
| CN109910014B (en) | Robot hand-eye calibration method based on neural network | |
| CN113379849A (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| CN117301052A (en) | Pose conversion method, device, equipment and storage medium | |
| CN109909999B (en) | Method and device for acquiring TCP (Transmission control protocol) coordinates of robot | |
| CN113211445B (en) | Robot parameter calibration method, device, equipment and storage medium | |
| CN112330752A (en) | Multi-camera combined calibration method and device, terminal equipment and readable storage medium | |
| CN111360821A (en) | Picking control method, device and equipment and computer scale storage medium | |
| US11577400B2 (en) | Method and apparatus for managing robot system | |
| CN111098306A (en) | Calibration method and device of robot, robot and storage medium | |
| CN113997295B (en) | Hand-eye calibration method and device for mechanical arm, electronic equipment and storage medium | |
| CN114474056A (en) | Grabbing operation-oriented monocular vision high-precision target positioning method | |
| CN114347013A (en) | Method for assembling printed circuit board and FPC flexible cable and related equipment | |
| CN117173254A (en) | Camera calibration method, system, device and electronic equipment | |
| CN115588054A (en) | Camera calibration method and device without angle constraint, electronic equipment and storage medium | |
| CN115439633A (en) | Calibration method and device and electronic equipment | |
| CN113172636B (en) | Automatic hand-eye calibration method and device and storage medium | |
| CN215701709U (en) | Configurable hand-eye calibration device | |
| WO2020010625A1 (en) | Method and system for optimizing kinematic model of robot, and storage device. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |