CN115674277A - Double-mechanical-arm device with hand-eye camera and control method thereof - Google Patents

Double-mechanical-arm device with hand-eye camera and control method thereof Download PDF

Info

Publication number
CN115674277A
CN115674277A CN202211404747.4A CN202211404747A CN115674277A CN 115674277 A CN115674277 A CN 115674277A CN 202211404747 A CN202211404747 A CN 202211404747A CN 115674277 A CN115674277 A CN 115674277A
Authority
CN
China
Prior art keywords
camera
mechanical arm
arm
hand
obtaining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211404747.4A
Other languages
Chinese (zh)
Inventor
黄伟
吴洪成
徐拴锋
刘大勇
郭瑞科
李宜鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Center Of China Academy Of Space Technology
Original Assignee
Hangzhou Center Of China Academy Of Space Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Center Of China Academy Of Space Technology filed Critical Hangzhou Center Of China Academy Of Space Technology
Priority to CN202211404747.4A priority Critical patent/CN115674277A/en
Publication of CN115674277A publication Critical patent/CN115674277A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Manipulator (AREA)

Abstract

The application provides a duplex mechanical arm device with hand eye camera includes: a first robot arm, the end of which is provided with a first operating tool through an end flange; the tail end of the second mechanical arm is provided with a second operating tool through a tail end flange; and the first eye-mobile camera is arranged on the first operating tool of the first mechanical arm and used for guiding the motion of the second mechanical arm. The first eye-lens camera is a depth camera and is also used to guide the movement of the first robot arm.

Description

Double-mechanical-arm device with hand-eye camera and control method thereof
Technical Field
The invention relates to a depth camera calibration technology for cooperation of two mechanical arms.
Background
With the social progress and the continuous development and change of science and technology, the robot is continuously and widely applied in various industries, and the requirements of the operation environment and tasks are more personalized, customized, complicated and diversified. The traditional single mechanical arm working mode has the defects of insufficient task adaptability and increasingly remarkable limitation, and some application occasions often start to adopt two or even a plurality of mechanical arms to realize the function which is difficult to realize or even cannot be realized by the single mechanical arm, wherein the double-arm cooperation is relatively common. The camera is the 'eye' of the mechanical arm and has the functions of target recognition, positioning, measurement and the like.
Depth camera refers to a camera that can provide color and depth maps, such as realsense d435i. The hand-eye camera refers to a camera working in cooperation with a mechanical arm, and is generally divided into a case of being on the hand and a case of being outside the hand, wherein the case of being on the hand refers to a camera mounted on a tool flange at the tail end of the mechanical arm, and the case of being outside the hand refers to a camera not mounted on the mechanical arm but mounted in the surrounding environment.
In the case of the eye on the hand, a camera is mounted on the robot arm a for guiding the robot arm a to work. In some complex conditions, however, such eyes are not suitable for use with a mechanical arm on a cell phone.
Disclosure of Invention
In order to solve the problems of two mechanical arms in the prior art, the application provides a double-mechanical-arm device with a hand-eye camera and a control method thereof, wherein a camera C installed on a mechanical arm A is used for guiding a mechanical arm B to work, so that the working efficiency can be improved, and the double-mechanical-arm device can be applied to complex working conditions.
The application provides a duplex mechanical arm device with hand eye camera includes:
a first robot arm, the end of which is provided with a first operating tool through an end flange;
the tail end of the second mechanical arm is provided with a second operating tool through a tail end flange;
and the first mechanical arm is arranged on the first operating tool of the first mechanical arm and used for guiding the motion of the second mechanical arm.
In at least one embodiment, wherein the first eye-of-hand camera is a depth camera.
In at least one embodiment, among others, the first eye-of-hand camera is also used to guide the motion of the first robotic arm.
In at least one embodiment, there is further included a second eye camera mounted on the second manipulation tool of the second robotic arm for guiding the motion of the first robotic arm.
In at least one embodiment, wherein the second eye camera is a depth camera.
In at least one embodiment, wherein the second robotic arm is further configured to guide movement of the second robotic arm.
In at least one embodiment, the first and second robotic arms are six-axis robotic arms.
The application also provides a control method of the double-mechanical-arm device, which comprises the following steps:
s1, obtaining a homogeneous transformation matrix of a first operation tool relative to a tail end flange of a first mechanical arm
Figure BDA0003936515690000021
Obtaining a homogeneous transformation matrix of two operating tools with respect to an end flange of a second robot arm
Figure BDA0003936515690000022
S2, obtaining a homogeneous transformation matrix of the end flange of the first mechanical arm relative to a base coordinate system of the first mechanical arm
Figure BDA0003936515690000023
S3, acquiring a homogeneous transformation matrix from the first hand-eye camera to the end flange of the first mechanical arm
Figure BDA0003936515690000024
Obtaining an internal reference transformation matrix M of a first eye camera cama
Figure BDA0003936515690000025
Wherein u is 0 、v 0 Is the origin of the image coordinate system, f x ,f y Depending on the pixel size;
s4, obtaining a base coordinate system of the second mechanical arm relative to the base of the first mechanical armHomogeneous transformation matrix of base coordinate system
Figure BDA0003936515690000026
S5, measuring to obtain the position P of the target object in the image coordinate system obj (u, v) the depth information value measured by the first hand-eye camera at the position (u, v) is z;
s6, if the second mechanical arm needs to move to the target position, calculating according to the following method to obtain:
s61, utilizing P in step S5 obj (u, v) and a depth information value z, and obtaining a homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure BDA0003936515690000031
S62, obtaining
Figure BDA0003936515690000032
I.e., the position and posture of the second manipulation tool of the second robot arm when it is moved to the target point.
In at least one embodiment, if the first robot arm needs to move to the target position, the following calculation is performed:
a. using P in step S5 obj (u, v) and a depth information value z, and obtaining a homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure BDA0003936515690000033
b. To obtain
Figure BDA0003936515690000034
I.e. the position and attitude of the first manipulator of the first robot arm when it has moved to the target point.
The camera C arranged on the mechanical arm A is used for guiding the mechanical arm B to work, so that the working efficiency can be improved, and the method can be applied to complex working conditions.
Drawings
The above features, technical features, advantages and modes of realisation of the present application will be further described in the following detailed description of preferred embodiments in a clearly understandable manner, in conjunction with the accompanying drawings. The drawings are only for purposes of illustrating and explaining the present application and are not to be construed as limiting the scope of the present application. Wherein:
fig. 1 illustrates a dual robot arm device with a hand-eye camera, in accordance with one embodiment;
fig. 2 illustrates a dual robot arm device with a hand-eye camera according to another embodiment;
fig. 3 illustrates a dual robot arm device with a hand-eye camera according to yet another embodiment.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present application, embodiments of the present application will now be described with reference to the accompanying drawings.
The improvement and innovation of the invention are as follows: 1) A camera C arranged on the mechanical arm A is used for guiding the mechanical arm B to work; 2) The camera C moves along with the mechanical arm A, the position and the posture are changed, and relative to the mechanical arm B, the camera C is a global camera which can move rapidly, and after the camera C is calibrated once, the position can be changed randomly in the working range of the mechanical arm A, so that the mechanical arm B is guided to work; 3) The mechanical arm B can work while the camera C can detect, so that the working efficiency is improved; 4) Can be applied to complex working conditions.
A camera working in conjunction with a robotic arm is often referred to as a hand-eye camera. Hand-eye cameras are divided into eye-on-hand and eye-off-hand. The camera calibration is divided into an internal reference and an external reference. The results of the referencing are related to the camera process, such as pixel size, offset, distortion, etc. The result of the external reference calibration is related to the installation position. The eye-on-hand referencing results are rotation and translation of the camera relative to the end of the arm, and the eye-on-hand referencing results are rotation and translation of the camera relative to the base coordinate system of the arm.
The invention uses the calibrated internal and external parameters, the positive kinematics of the mechanical arm and the pose relationship of the two mechanical arms to convert the position of a detection target to a base coordinate system of the mechanical arm for work and guide the movement of the mechanical arm.
The invention takes the cooperation of two mechanical arms as an application scene, the two mechanical arms are respectively named as a mechanical arm A and a mechanical arm B, and the mounting mode and the function of eyes on hands are as follows: 1) Only one camera of the system is arranged on the mechanical arm A, and the camera is used for guiding the mechanical arm A and the mechanical arm B to work; 2) Only one camera of the system is arranged on the mechanical arm B, and the camera is used for guiding the mechanical arms A and B to work; 3) The system is provided with two cameras (named as a camera C and a camera D) which are respectively arranged on the mechanical arms A and B, wherein the camera C can be used for independently guiding the mechanical arm A, B to work and can also be used for guiding A, B to work simultaneously, and the camera D also has the same function. The invention applies to the three mounting modes and functions described above.
According to an embodiment of the present invention, there is provided a dual robot arm device with a hand-eye camera, the structure of which is shown in fig. 1, including:
the tail end of the first mechanical arm A is provided with a first operating tool 11 through a tail end flange, and the first operating tool is used for clamping, carrying and the like;
a second manipulator B having a second operation tool 21 mounted at its end via a distal flange, for performing operations such as clamping and carrying;
the hand-eye camera C, which is a depth camera, is mounted on the first operating tool 11 of the first robot arm a, and is used for guiding the movement of the second robot arm B, and optionally, may also be used for guiding the movement of the first robot arm a.
The method for controlling a double-robot arm device with a hand-eye camera provided by the embodiment comprises the following steps:
s1, obtaining a 4 x 4 homogeneous transformation matrix of a first operation tool 11 arranged at the tail end of a first mechanical arm A relative to a tail end flange of the first mechanical arm A
Figure BDA0003936515690000051
Obtaining a 4 x 4 homogeneous transformation matrix of the second handling tool 21 mounted on the tip of the second robot arm B with respect to the tip flange of the second robot arm B
Figure BDA0003936515690000052
S2, obtaining a 4 multiplied by 4 homogeneous transformation matrix of the end flange of the first mechanical arm A relative to a base coordinate system of the first mechanical arm through forward kinematics of the mechanical arm or directly from a mechanical arm demonstrator, and naming the matrix as the base coordinate system of the first mechanical arm
Figure BDA0003936515690000053
S3, acquiring a 4 multiplied by 4 homogeneous transformation matrix-external parameter from the hand-eye camera C to the end flange of the first mechanical arm A by combining the mechanical arm data in the step S2 through a two-dimensional camera calibration method (such as based on opencv or matlab tool box) and the like, wherein the external parameter is named as
Figure BDA0003936515690000054
Obtaining hand-eye camera C internal reference 3X 3 transformation matrix M cama The internal reference matrix is related to the camera manufacturing process, where u 0 、v 0 Is the origin of the image coordinate system, i.e. the location of the principal point in pixel coordinates, f x ,f y Related to the pixel size.
Figure BDA0003936515690000055
S4, obtaining a 4 multiplied by 4 homogeneous transformation matrix of the base coordinate system of the second mechanical arm B relative to the base coordinate system of the first mechanical arm A through methods such as measurement or calibration and the like, and naming the matrix as the transformation matrix
Figure BDA0003936515690000056
S5, setting the position of the target object in the pixel coordinate system measured by the detection algorithm as P obj (u, v), the depth information value measured by the depth camera at the (u, v) position is z.
And S6, if the first mechanical arm A needs to move to the target position, calculating according to a pressing method.
a. Using P in step S5 obj (u, v) and depth information z, and obtaining a 4 x 4 homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure BDA0003936515690000061
b. Using formulas
Figure BDA0003936515690000062
To obtain
Figure BDA0003936515690000063
I.e., the position and posture of the first operating tool 11 of the first robot arm a when it moves to the target point.
And S7, if the second mechanical arm B needs to move to the target position, calculating according to a pressing method.
a. Using P in step S5 obj (u, v) and depth information z, and obtaining a 4 x 4 homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure BDA0003936515690000064
b. Using formulas
Figure BDA0003936515690000065
To obtain
Figure BDA0003936515690000066
I.e., the position and posture of the second operating tool 21 of the second robot arm B when it is moved to the target point.
According to still another embodiment of the present invention, there is provided a dual robot arm device with a hand-eye camera, the structure of which is shown in fig. 2, including:
the tail end of the first mechanical arm A is provided with a first operating tool 11 through a tail end flange, and the first operating tool is used for clamping, carrying and the like;
a second manipulator B having a second operation tool 21 mounted at its end via a distal flange, for performing operations such as clamping and carrying;
the hand-eye camera D, which is a depth camera, is mounted on the second operating tool 21 of the second mechanical arm B, and is used for guiding the movement of the first mechanical arm a, and optionally, may also be used for guiding the movement of the second mechanical arm B.
The method for controlling a dual-robot apparatus with a hand-eye camera provided in this embodiment includes:
s1, obtaining a 4 multiplied by 4 homogeneous transformation matrix of a first operation tool 11 arranged at the tail end of a first mechanical arm A relative to a tail end flange of the first mechanical arm A
Figure BDA0003936515690000071
Obtaining a 4 x 4 homogeneous transformation matrix of the second operating tool 21 mounted on the tip end of the second robot arm B with respect to the tip end flange of the second robot arm B
Figure BDA0003936515690000072
S2, obtaining a 4 multiplied by 4 homogeneous transformation matrix of the end flange of the second mechanical arm B relative to a base coordinate system of the second mechanical arm through forward kinematics of the mechanical arm or directly from a mechanical arm demonstrator, and naming the matrix as the base coordinate system of the second mechanical arm
Figure BDA0003936515690000073
S3, acquiring a 4 multiplied by 4 homogeneous transformation matrix-external reference from the hand-eye camera D to the end flange of the second mechanical arm B by combining the mechanical arm data in the step S2 through a two-dimensional camera calibration method (such as based on opencv or matlab toolbox and the like), and naming the matrix as the external reference
Figure BDA0003936515690000074
Obtaining hand-eye camera D internal parameter 3X 3 transformation matrix M cama The internal reference matrix is related to the camera manufacturing process, where u 0 、v 0 Is the origin of the image coordinate system, i.e. the location of the principal point in pixel coordinates, f x ,f y Related to the pixel size.
Figure BDA0003936515690000075
S4, obtaining a 4 multiplied by 4 homogeneous transformation matrix of the base coordinate system of the second mechanical arm B relative to the base coordinate system of the first mechanical arm A through methods such as measurement or calibration and the like, and naming the matrix as the transformation matrix
Figure BDA0003936515690000076
S5, setting the position of the target object in the pixel coordinate system measured by the detection algorithm as P obj (u, v), the depth information value measured by the depth camera at the (u, v) position is z.
And S6, if the second mechanical arm B needs to move to the target position, calculating according to a pressing method.
a. Using P in step S5 obj (u, v) and depth information z, and obtaining a 4 x 4 homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure BDA0003936515690000081
b. Using formulas
Figure BDA0003936515690000082
To obtain
Figure BDA0003936515690000083
I.e., the position and posture at which the second operating tool 21 of the second robot arm B is moved to the target point.
And S7, if the first mechanical arm A needs to move to the target position, calculating according to a pressing method.
a. Using P in step S5 obj (u, v) and depth information z, and obtaining a 4 x 4 homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure BDA0003936515690000084
b. Using formulas
Figure BDA0003936515690000085
To obtain
Figure BDA0003936515690000086
I.e., the position and posture when the first manipulation tool 1+1 of the first robot arm a moves to the target point.
According to another embodiment of the present invention, there is provided a dual robot arm device with a hand-eye camera, the structure of which is shown in fig. 3, including:
the tail end of the first mechanical arm A is provided with a first operating tool 11 through a tail end flange, and the first operating tool is used for clamping, carrying and the like;
a second manipulator B having a second operation tool 21 mounted at its end via a distal flange, for performing operations such as clamping and carrying;
the first eye camera C is a depth camera, is mounted on the first operating tool 11 of the first mechanical arm a, and is used for guiding the movement of the second mechanical arm B, and optionally, may also be used for guiding the movement of the first mechanical arm a;
the second eye camera D, which is a depth camera, is mounted on the second operating tool 21 of the second mechanical arm B, and is used for guiding the movement of the first mechanical arm a, and optionally, may also be used for guiding the movement of the second mechanical arm B.
The method for controlling a dual-robot apparatus with a hand-eye camera provided in this embodiment includes:
s1, obtaining a 4 x 4 homogeneous transformation matrix of a first operation tool 11 arranged at the tail end of a first mechanical arm A relative to a tail end flange of the first mechanical arm A
Figure BDA0003936515690000091
Obtaining a 4 x 4 homogeneous transformation matrix of the second handling tool 21 mounted on the tip of the second robot arm B with respect to the tip flange of the second robot arm B
Figure BDA0003936515690000092
S2, obtaining a 4 multiplied by 4 homogeneous transformation matrix of the end flange of the first mechanical arm A relative to a base coordinate system of the first mechanical arm through forward kinematics of the mechanical arm or directly from a mechanical arm demonstrator, and naming the matrix as the base coordinate system of the first mechanical arm
Figure BDA0003936515690000093
Obtaining a 4 multiplied by 4 homogeneous transformation matrix of the end flange of the second mechanical arm B relative to a base coordinate system of the second mechanical arm through forward kinematics of the mechanical arm or directly from a mechanical arm demonstrator, and naming the matrix as the transformation matrix
Figure BDA0003936515690000094
S3, acquiring a 4 multiplied by 4 homogeneous transformation matrix-external parameter from the hand-eye camera C to the end flange of the first mechanical arm A by combining the mechanical arm data in the step S2 through a two-dimensional camera calibration method (such as based on opencv or matlab tool box) and the like, and naming the matrix as the external parameter
Figure BDA0003936515690000095
Obtaining hand-eye camera C internal reference 3X 3 transformation matrix M camaC The internal reference matrix is related to the camera manufacturing process, where u C0 、v C0 Is the origin of the image coordinate system, i.e. the location of the principal point in pixel coordinates, f Cx ,f Cy Related to the pixel size. Acquiring a 4 multiplied by 4 homogeneous transformation matrix-external parameter from the hand-eye camera D to the end flange of the second mechanical arm B, and naming the external parameter
Figure BDA0003936515690000096
Obtaining hand-eye camera D internal parameter 3X 3 transformation matrix M camaD The internal reference matrix is related to the camera manufacturing process, where u D0 、v D0 Is the origin of the image coordinate system, i.e. the location of the principal point in pixel coordinates, f Dx ,f Dy Related to pixel size
Figure BDA0003936515690000097
S4, obtaining a 4 multiplied by 4 homogeneous transformation matrix of the base coordinate system of the second mechanical arm B relative to the base coordinate system of the first mechanical arm A through methods such as measurement or calibration and the like, and naming the matrix as the transformation matrix
Figure BDA0003936515690000101
S5, setting the position of the target object measured by the detection algorithm in the pixel coordinate system of the camera C as P objC (u C ,v C ) Depth camera in (u) C ,v C ) Measured value at position z C The position of the target object in the pixel coordinate system of the camera D is measured by the detection algorithm to be P objD (u D ,v D ) Depth camera in (u) D ,v D ) Measured value under position z D
S6, if the first mechanical arm A needs to move to the target position T under the first eye-camera C camaC And then, the calculation is carried out according to the following method.
a. Using S5P in step S5 objC (u C ,v C ) And depth information z C Obtaining a 4 x 4 homogeneous matrix in the camera coordinate system, wherein x C =(u C -u C0 )×z C /f Cx
y C =(v C -v C0 )×z C /f Cy
Figure BDA0003936515690000102
b. Using formulas
Figure BDA0003936515690000103
To obtain
Figure BDA0003936515690000104
I.e. the position and posture of the first robot arm a with the tool when it moves to the target point under the first eye-camera C.
S7, if the first mechanical arm A needs to move to the target position T under the second eye-grasping camera D camaD And then, the calculation is carried out according to the following method.
a. Step 5P objD (u D ,v D ) And depth information z D Obtaining a 4 x 4 homogeneous matrix in the camera coordinate system, wherein x D =(u D -u D0 )×z D /f Dx
y D =(v D -v D0 )×z D /f Dy
Figure BDA0003936515690000111
b. Using formulas
Figure BDA0003936515690000112
To obtain
Figure BDA0003936515690000113
I.e. the position and posture when the first mechanical arm a with the tool moves to the target point under the second eye-grasping camera D.
S8, if the second mechanical arm B needs to move to the target position T under the second eye-grasping camera D camaD And then, the calculation is carried out according to the following method.
a. Step 5P objD (u D ,v D ) And depthInformation z D Obtaining a 4 x 4 homogeneous matrix in the camera coordinate system, wherein x D =(u D -u D0 )×z D /f Dx
y D =(v D -v D0 )×z D /f Dy
Figure BDA0003936515690000114
b. Using formulas
Figure BDA0003936515690000115
To obtain
Figure BDA0003936515690000116
Namely the position and the posture when the second mechanical arm B with the tool moves to the target point under the second eye-holding camera D.
S9, if the second mechanical arm B needs to move to the target position T below the first eye camera C camaC And then, the calculation is carried out according to the following method.
a. Step 5P objC (u C ,v C ) And depth information z C Obtaining a 4 x 4 homogeneous matrix in the camera coordinate system, wherein x C =(u C -u C0 )×z C /f Cx
y C =(v C -v C0 )×z C /f Cy
Figure BDA0003936515690000121
b. Using formulas
Figure BDA0003936515690000122
To obtain
Figure BDA0003936515690000123
Namely the position and the posture when the second mechanical arm B with the tool moves to the target point under the first eye-camera C.
It should be understood that although the present description has been described in terms of various embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and those skilled in the art will recognize that the embodiments described herein may be combined as suitable to form other embodiments, as will be appreciated by those skilled in the art.
The above description is only illustrative of the present invention and is not intended to limit the scope of the present invention. Any equivalent alterations, modifications and combinations that may be made by those skilled in the art without departing from the spirit and principles of this application shall fall within the scope of this application.

Claims (10)

1. A dual-robot arm device with a hand-eye camera, comprising:
a first robot arm, the end of which is provided with a first operating tool through an end flange;
the tail end of the second mechanical arm is provided with a second operating tool through a tail end flange;
and the first eye-mobile camera is arranged on the first operating tool of the first mechanical arm and used for guiding the motion of the second mechanical arm.
2. The apparatus of claim 1, wherein the first hand-eye camera is a depth camera.
3. The apparatus of claim 1, wherein the first hand-eye camera is further configured to guide the movement of the first robotic arm.
4. The apparatus of claim 1, further comprising a second hand-eye camera mounted on the second manipulation tool of the second robotic arm for guiding the movement of the first robotic arm.
5. The apparatus of claim 4, wherein the second eye camera is a depth camera.
6. The apparatus of claim 4, wherein the second eye-grasping camera is further for guiding the motion of the second robotic arm.
7. The apparatus of claim 1, the first and second robotic arms being six-axis robotic arms.
8. The control method of the double robot arm device with the hand-eye camera according to claim 1, comprising:
s1, obtaining a homogeneous transformation matrix of a first operation tool relative to a tail end flange of a first mechanical arm
Figure FDA0003936515680000011
Obtaining a homogeneous transformation matrix of two operating tools with respect to an end flange of a second robot arm
Figure FDA0003936515680000012
S2, obtaining a homogeneous transformation matrix of the end flange of the first mechanical arm relative to a first mechanical arm base coordinate system
Figure FDA0003936515680000021
S3, acquiring a homogeneous transformation matrix from the first hand-eye camera to the end flange of the first mechanical arm
Figure FDA0003936515680000022
Obtaining an internal reference transformation matrix M of a first eye camera cama
Figure FDA0003936515680000023
Wherein u is 0 、v 0 Is shown as a drawingLike origin of coordinate system, f x ,f y Depending on the pixel size;
s4, obtaining a homogeneous transformation matrix of the base coordinate system of the second mechanical arm relative to the base coordinate system of the first mechanical arm
Figure FDA0003936515680000024
S5, measuring to obtain the position P of the target object in the image coordinate system obj (u, v) the depth information value measured by the first hand-eye camera at the position (u, v) is z;
s6, if the second mechanical arm needs to move to the target position, calculating according to the following method to obtain:
s61, utilizing P in step S5 obj (u, v) and a depth information value z, and obtaining a homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure FDA0003936515680000025
S62, obtaining
Figure FDA0003936515680000026
I.e., the position and posture of the second manipulation tool of the second robot arm when it is moved to the target point.
9. The method of claim 8, wherein if the first robot arm needs to move to the target position, the following is calculated:
a. using P in step S5 obj (u, v) and a depth information value z, and obtaining a homogeneous matrix in a camera coordinate system, wherein x = (u-u) 0 )×z/f x
y=(v-v 0 )×z/f y
Figure FDA0003936515680000031
b. To obtain
Figure FDA0003936515680000032
I.e. the position and attitude of the first manipulator of the first robot arm when it has moved to the target point.
10. A computer-readable storage medium comprising a stored computer program, wherein the method of any of claims 8 to 9 is performed when the computer program is executed by a processor.
CN202211404747.4A 2022-11-10 2022-11-10 Double-mechanical-arm device with hand-eye camera and control method thereof Pending CN115674277A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211404747.4A CN115674277A (en) 2022-11-10 2022-11-10 Double-mechanical-arm device with hand-eye camera and control method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211404747.4A CN115674277A (en) 2022-11-10 2022-11-10 Double-mechanical-arm device with hand-eye camera and control method thereof

Publications (1)

Publication Number Publication Date
CN115674277A true CN115674277A (en) 2023-02-03

Family

ID=85051162

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211404747.4A Pending CN115674277A (en) 2022-11-10 2022-11-10 Double-mechanical-arm device with hand-eye camera and control method thereof

Country Status (1)

Country Link
CN (1) CN115674277A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114872050A (en) * 2022-06-01 2022-08-09 清华大学深圳国际研究生院 Control method and control system for double-arm crawler-type mobile operation robot

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114872050A (en) * 2022-06-01 2022-08-09 清华大学深圳国际研究生院 Control method and control system for double-arm crawler-type mobile operation robot

Similar Documents

Publication Publication Date Title
JP6966582B2 (en) Systems and methods for automatic hand-eye calibration of vision systems for robot motion
US11090814B2 (en) Robot control method
CN107443377B (en) Sensor-robot coordinate system conversion method and robot eye calibration method
US10569419B2 (en) Control device and robot system
CN113379849B (en) Robot autonomous recognition intelligent grabbing method and system based on depth camera
US20150273689A1 (en) Robot control device, robot, robotic system, teaching method, and program
CN116157837A (en) Calibration method and device for robot
CN113715016B (en) Robot grabbing method, system, device and medium based on 3D vision
JP6897396B2 (en) Control devices, robot systems and control methods
CN111360821A (en) Picking control method, device and equipment and computer scale storage medium
JP5223407B2 (en) Redundant robot teaching method
CN115674277A (en) Double-mechanical-arm device with hand-eye camera and control method thereof
CN112767479A (en) Position information detection method, device and system and computer readable storage medium
CN112958974A (en) Interactive automatic welding system based on three-dimensional vision
KR20230065881A (en) Robot teaching system
JP3940998B2 (en) Robot equipment
CN112264996A (en) Steel grabbing machine positioning control method and system
WO2022092168A1 (en) Robot control device, and robot system
JP7372161B2 (en) Manipulators, automation methods and programs
CN211890823U (en) Four-degree-of-freedom mechanical arm vision servo control system based on RealSense camera
CN109664273B (en) Industrial robot cursor dragging teaching method and system
JP2022163836A (en) Method for displaying robot image, computer program, and method for displaying robot image
CN112743546A (en) Robot hand-eye calibration pose selection method and device, robot system and medium
JPH09323280A (en) Control method and system of manupulator
JP2009274191A (en) Working robot and computer program applied to the working robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination