CN116652970A - Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium - Google Patents

Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium Download PDF

Info

Publication number
CN116652970A
CN116652970A CN202310941577.1A CN202310941577A CN116652970A CN 116652970 A CN116652970 A CN 116652970A CN 202310941577 A CN202310941577 A CN 202310941577A CN 116652970 A CN116652970 A CN 116652970A
Authority
CN
China
Prior art keywords
mechanical arm
hand
eye
steps
matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310941577.1A
Other languages
Chinese (zh)
Other versions
CN116652970B (en
Inventor
王益亮
赵越
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Xiangong Intelligent Technology Co ltd
Original Assignee
Shanghai Xiangong Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Xiangong Intelligent Technology Co ltd filed Critical Shanghai Xiangong Intelligent Technology Co ltd
Priority to CN202310941577.1A priority Critical patent/CN116652970B/en
Publication of CN116652970A publication Critical patent/CN116652970A/en
Application granted granted Critical
Publication of CN116652970B publication Critical patent/CN116652970B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • B25J9/1692—Calibration of manipulator
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/0095—Means or methods for testing manipulators
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Image Processing (AREA)
  • Manipulator (AREA)

Abstract

The application provides a four-axis mechanical arm 2D hand-eye calibration method and system and a storage medium, wherein the method comprises the following steps: the hand-eye matrix relationRespectively decomposed into rotation matrix relationsTranslation component relationThe method comprises the steps of carrying out a first treatment on the surface of the At an angular distanceDefining the distance between the poses of the two rotation matrix representations willAs a oneScalar, build optimized objective functionThe method comprises the steps of carrying out a first treatment on the surface of the Through minimization ofTo solve for rotational component by value of (2)Acquisition ofThree-direction gestures; decomposition based on SVD methodTo solve for the translational component inValues. Therefore, the hand-eye pose relation with five degrees of freedom is obtained, and the visual grabbing mode of the four-axis SCARA mechanical arm is not limited by the fixed installation angle of the camera and the fixed height of the grabbing plane.

Description

Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium
Technical Field
The application relates to a robot hand-eye calibration technology, in particular to a four-axis mechanical arm 2D hand-eye calibration method and system and a storage medium.
Background
At present, visual identification plays a vital role in a scene that a mechanical arm and an object have relative position movement. One of the key factors for the visual grabbing of the mechanical arm is the calibration of the 'hand-eye relation', namely the conversion relation from the camera to the tail end of the mechanical arm.
The mounting modes of the cameras in the hand-eye system can be divided into two cases: the eye is on the hand (eye in hand) and the eye is off the hand (eye on hand). In order to determine the hand-eye relationship, a common method is a "nine-point calibration method", that is, a conversion relationship between a camera and a mechanical arm is calculated by pixel coordinates of nine points on a calibration plate in an image and physical coordinates marked by a mechanical arm base. The nine-point method has the characteristic of dimensional fixation, and is only suitable for the condition that the grabbing plane and the camera plane are relatively fixed, in other words, the pose relation of 6 degrees of freedom in the hand-eye relationThe nine-point method can only calculate three +.>Therefore, the applicable scene is greatly limited.
To calibrate the complete 6 degree of freedom hand-eye relationship, it is generally considered necessary to rotate and translate into a secondary transformation matrix based on the base coordinates, camera, end-grip, and calibrationConversion relation structure between fixed platesAnd (3) a relational expression, namely acquiring a plurality of groups of data, and calculating a hand-eye matrix by using a linear or nonlinear algorithm. But this method requires that the end pose, which contains at least two non-parallel rotational movements, be solved (Tsai and Lenz 1989,A New Technique for Fully Autonomous and Efficient 3D Robotics Hand/Eye registration).
In the four-axis SCARA robot arm, the rotation axes are all parallel, so that the hand-eye relationship cannot be solved by the method.
For a four-axis SCARA mechanical arm, various hand-eye calibration schemes are proposed in the prior art, such as a high-precision method for hand-eye calibration of a visual system of a SCARA mechanical arm fixed camera (Chinese patent publication No. CN 109159114A), a hand-eye calibration method and device of the four-axis mechanical arm (Chinese patent publication No. CN 116038701A) and the like, which are all provided on the basis of a nine-point method, but still only the hand-eye relationship can be calculatedThe three degrees of freedom relation limits that the camera plane and the grabbing plane in the application scene are parallel, and the grabbing height is relatively fixed, so that the application scene is limited.
Disclosure of Invention
Therefore, the main purpose of the application is to provide a four-axis mechanical arm 2D hand-eye calibration method, a system and a storage medium, so as to realize that the hand-eye pose relation of five degrees of freedom can be calculated at least, and the visual grabbing mode of the four-axis SCARA mechanical arm is not limited by the fixed installation angle of a camera and the height of a grabbing plane.
In order to achieve the above object, according to a first aspect of the present application, there is provided a four-axis mechanical arm 2D hand-eye calibration method, comprising the steps of:
step S100, hand-eye matrix relationRespectively decomposed into rotation matrix relations>Translation component relation->；
Step S200 is performed at an angular distanceDefining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the By minimizing +.>To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures;
step S300 is based on SVD method decompositionTo solve for the translational component of +.>Values.
In a possible preferred embodiment, the four-axis mechanical arm 2D hand-eye calibration method, wherein the angular distance in step S200
。
wherein ,for the transpose of matrix Q, tr denotes the trace of the matrix,/->；
While whenWhen in use, then->。
In a possibly preferred embodiment, wherein in step S300The solving step of (1) comprises:
step S310 pairSingular value decomposition into->The method comprises the steps of carrying out a first treatment on the surface of the Combining the translation component relation, converting into +.>；
Step S320 orderAfter SVD decomposition, S is a diagonal matrix composed of singular values:thus->To obtain +.>Values.
In a possible preferred embodiment, the four-axis mechanical arm 2D hand-eye calibration method further includes:
step S400 defines error termsThe method comprises the steps of carrying out a first treatment on the surface of the Establishing an optimized objective function->Performing iterative optimization solution to optimize +.>Value of>Is a preset fixed value;
wherein W is an adjustment parameter of the influence caused by the difference of the rotation amount and the data dimension of the translation vector in the balance matrix E.
In a possible preferred embodiment, the four-axis mechanical arm 2D hand-eye calibration method further includes:
step S500, moving the tail end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration plate, and acquiring the point 1 corresponding to the point and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the end of the mechanical arm tool to the origin of the calibration plate, denoted as ose2, and obtaining +.>；
Step S600 of recording the pose changes of the phase 1 to the phase 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>Whereas phase 1 to phase 2, the z-axis in the camera is transformed to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Calculating +.>。
In a possibly preferred embodiment, in step S100, the hand-eye matrix relation is，, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>。
In a possibly preferred embodiment, whereinThe step of acquiring each set of data of (a) includes:
step S110, the mechanical arm is moved, so that the camera can clearly capture the picture of the calibration plate, and the position of the tail end of the mechanical arm tool at the moment is displayedThe gesture is recorded asCollecting corresponding calibration plate images;
step S120, repeatedly moving the mechanical arm, collecting n groups of data, and recording the pose of the tool tail end of the mechanical armThe method comprises the following steps:。
in a possibly preferred embodiment, whereinThe calculating step of each group of data of (a) comprises:
step S130 establishes a projection relationship from the world coordinate system to the pixel coordinate system:
wherein For pixel coordinate system coordinates +.>Is an internal reference matrix of a camera>Representing the pose of the calibration plate to the camera coordinate system, and uniformly matrix +.>Marked as->，/>Representing coordinates in the world coordinate system, +.>Representing the depth of the feature points in a camera coordinate system;
step S140 solves the optimal parameters by minimizing the re-projection error method：
Meanwhile, each calibration plate image acquired in the step S110 corresponds to a pose under a camera coordinate system, and is marked as:
。
in order to achieve the above object, corresponding to the above method, according to a second aspect of the present application, there is also provided a four-axis mechanical arm 2D hand-eye calibration system, comprising:
the storage unit is used for storing a program comprising the steps of the four-axis mechanical arm 2D hand-eye calibration method, so that the control unit and the processing unit can timely adjust and execute the program;
the control unit is used for controlling the mechanical arm to enable the camera to clearly capture the picture of the calibration plate, and recording the pose of the tail end of the mechanical arm at the momentAnd acquiring images of corresponding calibration plates, repeatedly moving the mechanical arm, acquiring n groups of data, and recording the terminal pose as；
The processing unit is used for calculating the pose of the calibration plate under the camera coordinate system according to the camera internal parametersSimultaneously, each acquired calibration plate image corresponds to a pose under a camera coordinate system and is recorded asThe method comprises the steps of carrying out a first treatment on the surface of the The hand-eye matrix relation is->Respectively decomposed into rotation matrix relations>Translation component relation->； wherein />，/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>The method comprises the steps of carrying out a first treatment on the surface of the At an angular distance +>Defining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the By minimizing +.>To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures; when->When in use, thenThe method comprises the steps of carrying out a first treatment on the surface of the Then decompose ∈10 based on SVD method>To solve for the translational component of +.>Values.
In order to achieve the above object, according to a third aspect of the present application, there is further provided a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the four-axis mechanical arm 2D hand-eye calibration method as described in any one of the above.
The four-axis mechanical arm 2D hand-eye calibration method, the system and the storage medium provided by the application can skillfully decompose the hand-eye matrix relation into the rotation matrix relation and the translation component relation so as to respectively calculate and take out the hand-eye matrix relation) (at->) Five degrees of freedom are overcome, so that the calibration scheme of the prior art for the four-axis SCARA mechanical arm can only calculate +.>Three degrees of freedom relation.
Furthermore, in the application of the four-axis SCARA mechanical arm, due toThe value of (2) is usually set to be a fixed value, so that in theory, the complete 6-degree-of-freedom hand-eye pose relationship can be obtained, but in order to adapt to some special visual application modes, the application also provides a calculation method>The proposal of the scheme ensures that the scheme can completely calculate the hand-eye pose relation with 6 degrees of freedomThe visual grabbing mode of the four-axis SCARA mechanical arm is not limited by scenes such as camera installation angles, fixed grabbing plane heights and the like. Thereby expanding the application range of the four-axis SCARA mechanical arm in industrial scenes.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is a schematic diagram of steps of a four-axis mechanical arm 2D hand-eye calibration method of the present application;
FIG. 2 is a schematic diagram of a four-axis SCARA robot in the prior art;
FIG. 3 is a diagram showing the data conversion relationship between the ROBOT base coordinate system ROBOT and the calibration plate coordinate system WORLD in different poses in the four-axis mechanical arm 2D hand-eye calibration method of the present application;
fig. 4 is a schematic structural diagram of a four-axis mechanical arm 2D hand-eye calibration system according to the present application.
Detailed Description
In order that those skilled in the art can better understand the technical solutions of the present application, the following description will clearly and completely describe the specific technical solutions of the present application in conjunction with the embodiments to help those skilled in the art to further understand the present application. It will be apparent that the embodiments described herein are merely some, but not all embodiments of the application. It should be noted that embodiments of the present application and features of embodiments may be combined with each other by those of ordinary skill in the art without departing from the spirit of the present application and without conflicting with each other. All other embodiments, which are derived from the embodiments herein without creative effort for a person skilled in the art, shall fall within the disclosure and the protection scope of the present application.
Furthermore, the terms "first," "second," "S100," "S200," and the like in the description and in the claims and drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those described herein. Also, the terms "comprising" and "having" and any variations thereof herein are intended to cover a non-exclusive inclusion. Unless specifically stated or limited otherwise, the terms "disposed," "configured," "mounted," "connected," "coupled" and "connected" are to be construed broadly, e.g., as being either permanently connected, removably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms in this case will be understood by those skilled in the art in view of the specific circumstances and in combination with the prior art.
According to the theory of robot kinematics and robot dynamics, the four-axis SCARA mechanical arm is generally designed into a structure shown in fig. 2, wherein an axis 1 is a vertical lifting axis, axes 2, 3 and 4 are axes of rotation around Z, wherein the axis 4 is an axis where a terminal flange is located, and a clamping tool is fixed on the axes. Due to the characteristics of the four-axis SCARA mechanical arm, the space pose which can be represented by the four-axis SCARA mechanical arm has only four degrees of freedom, namely x, y, z and Rz. That is, the gripping attitude of the four-axis SCARA robot arm can only rotate around the Z axis except for the spatial position coordinates (X, Y, Z), and the rotation around the X axis and the Y axis cannot be realized.
Further, the pose of the rigid body in the three-dimensional space can be represented by rotation R and translation t, and the rotation and translation of the rigid body motion are usually represented by a homogeneous transformation matrix H, and in the hand-eye calibration process of the six-axis articulated mechanical arm, if the eye is on hand (eye in hand), the calibration plate is in the position relationship under the calibration of the mechanical arm baseRepresented by the chained homogeneous coordinate transformation as:
if the eye is outside the hand (eye on hand), the calibration is fixed at the tail end of the mechanical arm, so the relationship between the calibration plate and the tail end is thatThe chain homogeneous coordinate transformation in the calibration process is kept unchanged and is as follows:
wherein For the pose of the mechanical arm tool end under the base standard system, +.>Representing the pose of the base coordinates in the robot arm end tool coordinate system, obviously +.>The method comprises the steps of carrying out a first treatment on the surface of the The pose data can be read directly from the robotic arm,representing the pose relation between the camera coordinate system and the tool tail end coordinate system, namely, a hand-eye matrix when the eye in hand needs to be solved; />The pose relation between the camera coordinate system and the base standard system is represented, and the pose relation is also the hand-eye relation when the eye on hand needs to be solved; />The pose of the calibration plate under the camera coordinate system is represented and can be obtained by a calibration plate identification algorithm.
Since there is in the above and />(or-> and />) Two unknown quantities, to be determined isOr->At the same time in view of->Or->Is fixed (because the position relation between the calibration plate and the mechanical arm base mark is fixed in the calibration process or the position relation between the calibration plate and the end tool is fixed), the calibration process needs to collect a plurality of groups of mechanical arm end data and calibration plate identification data, and eliminate->Or->Constructing an equation to solve +.>Or (b)。
For this reason, as shown in fig. 1, the application provides a four-axis mechanical arm 2D hand-eye calibration method, which comprises the following steps:
step S100
Assuming that the ROBOT base standard system is ROBOT, the calibration plate coordinate system is WORLD, and the hand-eye matrix relation can be established according to the data conversion relation of different poses as shown in figure 3
。
wherein ,，/>，/>mechanical arm tail end pose data respectively representing the first group data and the second group data>，/>Pose data of calibration plates respectively for first group data and second group data under camera coordinate system，/>For hand-eye relation to be solved->. A, B, X are homogeneous matrices of rigid body pose transformation:
specifically, according to the different mounting modes of the camera, the mechanical arm visual grabbing can be divided into eyes on the hand and eyes outside the hand. When the eyes are on the hands, the camera can be arranged at the tail end of the mechanical arm, so that the calibration plate and the base calibration position are ensured to be relatively fixed; when eyes are outside hands, the camera can be fixedly installed in the environment, and the calibration plate can be installed at the tail end of the mechanical arm so as to ensure that the positions of the camera and the tail end of the mechanical arm are relatively fixed. The manner of data acquisition is the same for both cases.
Before the subsequent calculation, if the camera internal parameters are not determined, the camera internal parameters can be calibrated first. Intrinsic properties of the camera are referred to as camera intrinsic properties, consisting ofFocal length +.>Origin offset->And a series of distortion coefficients, wherein the internal parameters are fixed after the focal length and the focusing ring of the camera are fixed. According to a camera aperture imaging model:
。
wherein ,is pixel coordinate point +.>Is a spatial coordinate point. The internal reference calibration process usually requires the aid of specific calibration objects, commonly used calibration objects are calibration plates of specific patterns, such as checkerboard calibrationPlates, circular calibration plates, churco, apriltag grid, and the like.
The calibration plate being a plane in three-dimensional spaceObtaining an imaging plane according to the projection imaging relationship shown in the above formula>The homography matrix between the two planes can be determined by the coordinates of the corresponding points of the two planes>. The specification and the size on the calibration plate are known, the corner points in the image can be extracted by a corner point detection algorithm, and the +.>, wherein />For the coordinates of points in the image, +.>For calibrating the point coordinates on the plate, +.>The homography matrix representing two planes is changed, the pose relation between the camera and the calibration plate is changed, a plurality of calibration plate images are collected, a plurality of groups of homography matrixes can be obtained, and the +_ is solved through least square>Will->The camera parameter K can be obtained by decomposition.
Further, whereinThe data acquisition step of each group of (a) includes:
step S110 shiftThe mechanical arm is moved, so that the camera can clearly capture the picture of the calibration plate, and the pose of the tail end of the mechanical arm tool at the moment is recorded asAnd acquiring the corresponding calibration plate image.
The calibration board in this example may be configured as a pattern with a specific specification, including but not limited to a checkerboard calibration board, a round calibration board, a churco, an Apriltag grid, and other forms, and as long as a scheme of obtaining the pose relationship from the calibration object to the camera can be within the scope of the disclosure of the calibration board.
Step S120, repeatedly moving the mechanical arm and collecting n groups of dataRecording the pose of the tool tail end of the mechanical arm>The method comprises the following steps:
i.e. n sets of data, by combining two by two to form one A, e.g。
Further, whereinThe calculating step of each group of data of (a) comprises:
step S130, in case the camera parameters are known, the calculation is exemplified using PnP (Persponsive-n-Point) algorithm. The solving modes of the PnP algorithm are various, and a direct linear transformation method (Direct Linear Transform, DLT), an EPnP method and a minimum reprojection error method are common, and the example takes the minimum reprojection error method as an example to solve the position and the pose of the calibration plate.
For example: according to the principle of pinhole imaging, a projection relation from a world coordinate system to a pixel coordinate system is established:
wherein For pixel coordinate system coordinates +.>Is an internal reference matrix of a camera>Representing the pose of the calibration plate to the camera coordinate system, and uniformly matrix +.>Marked as->，/>Representing coordinates in the world coordinate system, +.>Representing the depth of the feature points in a camera coordinate system;
step S140 solves the optimal parameters by minimizing the re-projection error method：
/>
Meanwhile, each calibration plate image acquired in the step S110 corresponds to a pose under a camera coordinate system, and is marked as:
i.e. n sets of data, generating a B, e.g. B, by combining two by two。
Further, either the eye in hand or the eye on hand may be represented by the formulaThe method comprises the following steps of:
/>
/>
as can be seen from (2) and (3), the rotation matrix R is independent of the translation component and can be calculated independently, and the translation vectorThen in relation to rotation, it is necessary to rely on +.>Is calculated. For the 6-joint mechanical arm, the mechanical arm can freely rotate around x, y and z axes in space, so that the hand-eye matrix can be directly calculated according to (2) and (3)>。
Step S200 calculates a rotation component。
Specifically, the rotation component according to expression (2) can be calculated alone,are all rotation matrices representing the rotational pose of a rigid body in space, wherein +.>Representing the rotational component of the hand-eye matrix X, comprising +.>Three-directional gestures. WhileThe rotation matrix representation on the left and right sides of the representation equation is consistent in terms of pose.
At this time, angular distanceRepresenting the distance between the poses of the two rotation matrix representations, let +.>Angular distance>The definition is as follows:
/>
wherein ,for the transpose of matrix Q, tr denotes the trace of the matrix (sum of matrix eigenvalues). />In the form of a scalar quantity,by minimizing +.>To calculate the rotation component +.>. The optimized objective function is:
/>
then, the iterative optimization solution is carried out by using a Levenberg-Marquardt algorithm in an example, and the iterative optimization solution can be calculated to obtainAs a result of (a), let->When in use, then->。
Step S300 calculation
Specifically, the following equation (3) is obtainedIn the case of (1) calculate->Due to->Rank of 2 (wherein +.>A constant in linear algebra, corresponding to 1 in real numbers, is the identity matrix, thus ∈1)>The solution of (1) is not unique and can not be directly solved, and the application provides a translation component solving scheme based on SVD (singular value decomposition)>Singular value decomposition is performed:
/>
the combination of formulas (3) (9) can be obtained:
/>
the right side of formula (10) is a known amount, let:
/>
after SVD decomposition, S is a diagonal matrix composed of singular values:
/>
by combining (10) (11) (12), it is possible to obtain:
/>
can be obtained according to the formula (13)For n sets of data acquired, each set of data can be evaluated as a set +.>Value, calculate n data sets +.>As a final result, the accuracy and robustness of the result may be increased.
Step S400 optimizationValue of
Since the present example is to solve for rotation and translation in two stepsThe calculation process is affected by the accumulated error of Rx, resulting in reduced accuracy. For this purpose, the present example proposes to use nonlinear optimization to fine tune the calculation result, further improve the precision of the calibration result, and the steps include:
defining an error term E as:
/>
the optimized objective function is:
/>
wherein The role of W is to balance the effect of the difference in the data dimensions of the rotation vector and the translation vector in matrix E (9 for the rotation matrix and 3 for the translation vector).
While all n sets of data are illustrated as being optimized by Levenberg-Marquardt, those skilled in the art will appreciate that all of the non-linear optimization algorithms referred to in this example may be substituted by, for example, gaussian-newton, steepest descent, etc.
In addition becauseCannot be determined, so that the whole optimization process needs to be kept +.>The preset fixed value is adopted.
Further, since in a four-axis SCARA robot application,the value of (2) is usually set to a fixed value, but for some special visual applications +.>The value of (2) must be accurate.
Thus, the present application also exemplifies a confirmation in order to obtain a complete external referenceThe values were as follows:
step S500, moving the tail end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration plate, and acquiring the point 1 corresponding to the point and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the end of the mechanical arm tool to the origin of the calibration plate, denoted as ose2, and obtaining +.>。
Step S600 of recording the pose changes of the phase 1 to the phase 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Whereas the z-axis in the camera is transformed to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Whereby the +.about.in the hand-eye relationship can be calculated>Thereby determining +.>Values.
In practical implementation, the operation difficulty of overlapping the tail end of the mechanical arm with the origin of the calibration plate is relatively high, and as an alternative method, the tail end of the mechanical arm can be moved to a point right above the origin of the calibration plate or right below the origin of the calibration plate to a known height, and then the known height is compensated in the calculation process.
Thus, through the above example steps, the six degrees of freedom hand-eye pose relationship of the complete four-axis SCARA mechanical arm can be calculated. The four-axis SCARA mechanical arm is free from the influence of factors such as a camera mounting mode, a clamping jaw mounting mode and the like in the visual grabbing application.
On the other hand, corresponding to the four-axis mechanical arm 2D hand-eye calibration method of the above example, as shown in fig. 4, the present application further provides a four-axis mechanical arm 2D hand-eye calibration system, which includes:
the storage unit is used for storing a program comprising the steps of the four-axis mechanical arm 2D hand-eye calibration method, so that the control unit and the processing unit can timely adjust and execute the program;
the control unit is used for controlling the mechanical arm to enable the camera to clearly capture the picture of the calibration plate, and recording the pose of the tail end of the mechanical arm at the momentAnd acquiring images of corresponding calibration plates, repeatedly moving the mechanical arm, acquiring n groups of data, and recording the terminal pose as；
The processing unit is used for calculating the pose of the calibration plate under the camera coordinate system according to the camera internal parametersSimultaneously, each acquired calibration plate image corresponds to a pose under a camera coordinate system and is recorded as。
The processing unit also uses the hand-eye matrix relationRespectively decomposed into rotation matrix relationsTranslation component relation->； wherein />，/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>The method comprises the steps of carrying out a first treatment on the surface of the And then>Defining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is established:
through minimization ofTo solve for the rotation component>, wherein />Is a rotation component of the hand-eye matrix X, and comprisesThree-directional gestures.
Wherein whenWhen in use, then->。
The processing unit then also decomposes based on SVDThe translational component of (a)To solve for->Values.
Further, when it is required to doWhen the value is optimized, the processing unit also passes through an optimized objective function:
optimization is performed on all n groups of data by using Levenberg-MarquardtOptimization of the values.
On the other hand, whenWhen the value of (2) must be determined, the control unit is further configured to move the end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration board, and obtain +_corresponding to point 1> and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the end of the mechanical arm tool to the origin of the calibration plate, denoted as ose2, and obtaining +.>。
The processing unit is also used for recording the pose changes of the pos 1 to the pos 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>Whereas phase 1 to phase 2, the z-axis in the camera is transformed to +.>Is denoted by>To calculate the +.>Thereby determining +.>Values.
On the other hand, the application also provides a computer readable storage medium, corresponding to the method, wherein the computer program is stored thereon, and when the computer program is executed by a processor, the steps of the four-axis mechanical arm 2D hand-eye calibration method are realized.
Finally, the four-axis mechanical arm 2D hand-eye calibration method, the system and the storage medium provided by the application skillfully decompose the hand-eye matrix relational expression into the rotation matrix relational expression and the translation component relational expression so as to respectively calculate and take out the hand-eye matrix relational expression and the translation component relational expression) (at->) Five degrees of freedom are adopted, so that the calibration scheme of the four-axis SCARA mechanical arm in the prior art is broken through, and only the calculation can be performedThree degrees of freedom relation.
Furthermore, in the application of the four-axis SCARA mechanical arm, due toThe value of (2) is usually set to be a fixed value, so that in theory, the scheme can obtain the complete hand-eye pose relationship with 6 degrees of freedom, but is thatThe application also provides a method for calculating +.>The proposal of the scheme ensures that the scheme can completely calculate the hand-eye pose relation with 6 degrees of freedomThe visual grabbing mode of the four-axis SCARA mechanical arm is not limited by scenes such as camera installation angles, fixed grabbing plane heights and the like. Thereby expanding the application range of the four-axis SCARA mechanical arm in industrial scenes.
The preferred embodiments of the application disclosed above are intended only to assist in the explanation of the application. The preferred embodiments are not exhaustive or to limit the application to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, to thereby enable others skilled in the art to best understand and utilize the application. The application is to be limited only by the following claims and their full scope and equivalents, and any modifications, equivalents, improvements, etc., which fall within the spirit and principles of the application are intended to be included within the scope of the application.
It will be appreciated by those skilled in the art that the system, apparatus and various modules/units thereof provided by the application may be implemented entirely by logic programming method steps, in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc., in addition to the system, apparatus and various modules/units thereof provided by the application being embodied in pure computer readable program code. Therefore, the system, the apparatus, and the respective modules/units thereof provided by the present application may be regarded as one hardware component, and the modules/units included therein for implementing various programs may also be regarded as structures within the hardware component; modules/units for implementing various functions may also be considered as either software programs for implementing the methods or structures within hardware components.
Furthermore, all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program, where the program is stored in a storage medium and includes several instructions for causing a single-chip microcomputer, chip or processor (processor) to execute all or part of the steps in the methods of the embodiments of the application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In addition, any combination of various embodiments of the present application may be performed, so long as the concept of the embodiments of the present application is not violated, and the disclosure of the embodiments of the present application should also be considered.

Claims (10)

1. A four-axis mechanical arm 2D hand-eye calibration method comprises the following steps:
step S100, hand-eye matrix relationRespectively decomposed into rotation matrix relations>Translation component relation->；
Step S200 is performed at an angular distanceDefining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the By minimizing +.>To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures;
step S300 is based on SVD method decompositionTo solve for the translational component of +.>Values.
2. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, wherein the angular distance in step S200
；
wherein ,for the transpose of matrix Q, tr denotes the trace of the matrix,/->；
While whenWhen in use, then->。
3. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, wherein in step S300The solving step of (1) comprises:
step S310 pairSingular value decomposition into->The method comprises the steps of carrying out a first treatment on the surface of the Combining the translation component relation to convert into；
Step S320 orderAfter SVD decomposition, S is a diagonal matrix composed of singular values:thus->To obtain +.>Values.
4. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, further comprising the steps of:
step S400 defines error termsThe method comprises the steps of carrying out a first treatment on the surface of the Establishing an optimized objective function->Performing iterative optimization solution to optimize +.>Value of>Is a preset fixed value;
；
wherein W is an adjustment parameter of the influence caused by the difference of the rotation amount and the data dimension of the translation vector in the balance matrix E.
5. The four-axis mechanical arm 2D hand-eye calibration method according to claim 1, further comprising the steps of:
step S500, moving the tail end of the mechanical arm tool to the point of point 1, so that the camera can clearly observe the calibration plate, and acquiring the point 1 corresponding to the point and />The method comprises the steps of carrying out a first treatment on the surface of the Moving the tail end of the mechanical arm tool to the origin of the calibration plate, namely, a point 2, and acquiring the point 2；
Step S600 of recording the pose changes of the phase 1 to the phase 2 asThe method comprises the steps of carrying out a first treatment on the surface of the Changing the actual z-axis of the robot tool tip from post 1 to post 2 to +.>Is denoted by>Whereas phase 1 to phase 2, the z-axis in the camera is transformed to +.>Is denoted by>The method comprises the steps of carrying out a first treatment on the surface of the Calculating +.>。
6. The method for calibrating a 2D hand and eye of a four-axis mechanical arm according to claim 1, wherein in step S100, the hand and eye matrix relation is，/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>。
7. The four-axis mechanical arm 2D hand-eye calibration method according to claim 6, whereinThe data acquisition step of each group of (a) includes:
step S110, the mechanical arm is moved, so that the camera can clearly capture the picture of the calibration plate, and the pose of the tail end of the mechanical arm tool at the moment is recorded asCollecting corresponding calibration plate images;
step S120, repeatedly moving the mechanical arm, collecting n groups of data, and recording the pose of the tool tail end of the mechanical armThe method comprises the following steps:。
8. the four-axis mechanical arm 2D hand-eye calibration method according to claim 6, whereinThe calculating step of each group of data of (a) comprises:
step S130 establishes a projection relationship from the world coordinate system to the pixel coordinate system:
；
wherein For pixel coordinate system coordinates +.>Is an internal reference matrix of a camera>Representing the pose of the calibration plate to the camera coordinate system, and uniformly matrix +.>Marked as->，/>Representing coordinates in the world coordinate system,representing the depth of the feature points in a camera coordinate system;
step S140 solves the optimal parameters by minimizing the re-projection error method：
；
Meanwhile, each calibration plate image acquired in the step S110 corresponds to a pose under a camera coordinate system, and is marked as:
。
9. a four-axis robotic arm 2D hand-eye calibration system, comprising:
a storage unit, configured to store a program including the steps of the four-axis mechanical arm 2D hand-eye calibration method according to any one of claims 1 to 8, for the control unit, and the processing unit to timely retrieve and execute the program;
the control unit is used for controlling the mechanical arm to enable the camera to clearly capture the picture of the calibration plate, and recording the pose of the tail end of the mechanical arm at the momentAnd acquiring images of corresponding calibration plates, repeatedly moving the mechanical arm, acquiring n groups of data, and recording the terminal pose as；
The processing unit is used for calculating the pose of the calibration plate under the camera coordinate system according to the camera internal parametersSimultaneously, each acquired calibration plate image corresponds to a pose under a camera coordinate system and is marked as +.>The method comprises the steps of carrying out a first treatment on the surface of the The hand-eye matrix relation is->Respectively decomposed into rotation matrix relations>Translation component relation； wherein />，/>, wherein />Respectively representing the pose data of the tail ends of two groups of mechanical arm tools +.>And->Respectively representing pose data of two groups of calibration plates under a camera coordinate system>The method comprises the steps of carrying out a first treatment on the surface of the At an angular distance +>Defining the distance between the poses of the two rotation matrix representations, will +.>As a scalar, an optimized objective function is created +.>The method comprises the steps of carrying out a first treatment on the surface of the By minimizing +.>To solve for the rotation component>, wherein />For the rotational component of the hand-eye matrix X, comprising +.>Three-direction gestures; when->When in use, thenThe method comprises the steps of carrying out a first treatment on the surface of the Then decompose ∈10 based on SVD method>To solve for the translational component of +.>Values.
10. A computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the steps of the four-axis robotic arm 2D hand-eye calibration method of any of claims 1 to 8.
CN202310941577.1A 2023-07-28 2023-07-28 Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium Active CN116652970B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310941577.1A CN116652970B (en) 2023-07-28 2023-07-28 Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310941577.1A CN116652970B (en) 2023-07-28 2023-07-28 Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium

Publications (2)

Publication Number Publication Date
CN116652970A true CN116652970A (en) 2023-08-29
CN116652970B CN116652970B (en) 2023-10-31

Family

ID=87720964

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310941577.1A Active CN116652970B (en) 2023-07-28 2023-07-28 Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium

Country Status (1)

Country Link
CN (1) CN116652970B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117182931A (en) * 2023-11-07 2023-12-08 上海仙工智能科技有限公司 Calibration method and system for calibration data of mechanical arm and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109483516A (en) * 2018-10-16 2019-03-19 浙江大学 A kind of mechanical arm hand and eye calibrating method based on space length and epipolar-line constraint
CN111775152A (en) * 2020-06-29 2020-10-16 深圳大学 Method and system for guiding mechanical arm to grab scattered stacked workpieces based on three-dimensional measurement
CN111823233A (en) * 2020-06-30 2020-10-27 浙江德尚韵兴医疗科技有限公司 Mechanical arm hand-eye calibration system and method based on high-precision three-dimensional optical positioning
EP4094897A1 (en) * 2021-05-26 2022-11-30 MVTec Software GmbH Hand-eye calibration of camera-guided devices
CN116038701A (en) * 2022-12-30 2023-05-02 北京中科原动力科技有限公司 Hand-eye calibration method and device for four-axis mechanical arm
CN116352710A (en) * 2023-03-24 2023-06-30 电子科技大学 Robot automatic calibration and three-dimensional measurement method for large aerospace component
WO2023134237A1 (en) * 2022-01-14 2023-07-20 北京京东乾石科技有限公司 Coordinate system calibration method, apparatus and system for robot, and medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109483516A (en) * 2018-10-16 2019-03-19 浙江大学 A kind of mechanical arm hand and eye calibrating method based on space length and epipolar-line constraint
CN111775152A (en) * 2020-06-29 2020-10-16 深圳大学 Method and system for guiding mechanical arm to grab scattered stacked workpieces based on three-dimensional measurement
CN111823233A (en) * 2020-06-30 2020-10-27 浙江德尚韵兴医疗科技有限公司 Mechanical arm hand-eye calibration system and method based on high-precision three-dimensional optical positioning
EP4094897A1 (en) * 2021-05-26 2022-11-30 MVTec Software GmbH Hand-eye calibration of camera-guided devices
WO2023134237A1 (en) * 2022-01-14 2023-07-20 北京京东乾石科技有限公司 Coordinate system calibration method, apparatus and system for robot, and medium
CN116038701A (en) * 2022-12-30 2023-05-02 北京中科原动力科技有限公司 Hand-eye calibration method and device for four-axis mechanical arm
CN116352710A (en) * 2023-03-24 2023-06-30 电子科技大学 Robot automatic calibration and three-dimensional measurement method for large aerospace component

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117182931A (en) * 2023-11-07 2023-12-08 上海仙工智能科技有限公司 Calibration method and system for calibration data of mechanical arm and storage medium
CN117182931B (en) * 2023-11-07 2024-01-23 上海仙工智能科技有限公司 Calibration method and system for calibration data of mechanical arm and storage medium

Also Published As

Publication number Publication date
CN116652970B (en) 2023-10-31

Similar Documents

Publication Publication Date Title
CN110276806B (en) Online hand-eye calibration and grabbing pose calculation method for four-degree-of-freedom parallel robot stereoscopic vision hand-eye system
CN109483516B (en) Mechanical arm hand-eye calibration method based on space distance and polar line constraint
CN109859275B (en) Monocular vision hand-eye calibration method of rehabilitation mechanical arm based on S-R-S structure
CN110717943A (en) Method and system for calibrating eyes of on-hand manipulator for two-dimensional plane
JP4021413B2 (en) Measuring device
CN111801198B (en) Hand-eye calibration method, system and computer storage medium
CN109658460A (en) A kind of mechanical arm tail end camera hand and eye calibrating method and system
CN111775146A (en) Visual alignment method under industrial mechanical arm multi-station operation
CN109940626B (en) Control method of eyebrow drawing robot system based on robot vision
CN110136204B (en) Sound film dome assembly system based on calibration of machine tool position of bilateral telecentric lens camera
CN116652970B (en) Four-axis mechanical arm 2D hand-eye calibration method and system and storage medium
CN113379849A (en) Robot autonomous recognition intelligent grabbing method and system based on depth camera
CN111862238B (en) Full-space monocular light pen type vision measurement method
Hu et al. Automatic calibration of hand–eye–workspace and camera using hand-mounted line laser
CN111612794A (en) Multi-2D vision-based high-precision three-dimensional pose estimation method and system for parts
CN114643578B (en) Calibration device and method for improving robot vision guiding precision
CN115861445B (en) Hand-eye calibration method based on three-dimensional point cloud of calibration plate
CN111360821A (en) Picking control method, device and equipment and computer scale storage medium
CN110136068B (en) Sound membrane dome assembly system based on position calibration between bilateral telecentric lens cameras
CN112229323B (en) Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method
CN113751981B (en) Space high-precision assembling method and system based on binocular vision servo
CN110465946B (en) Method for calibrating relation between pixel coordinate and robot coordinate
CN112958960A (en) Robot hand-eye calibration device based on optical target
Hvilshøj et al. Calibration techniques for industrial mobile manipulators: Theoretical configurations and best practices
CN114029982A (en) Hand-eye calibration device and calibration method of camera outside robot arm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant