CN113804106A - Double-arm cooperative assembly method based on twice calibration of vision - Google Patents
Double-arm cooperative assembly method based on twice calibration of vision Download PDFInfo
- Publication number
- CN113804106A CN113804106A CN202110946161.XA CN202110946161A CN113804106A CN 113804106 A CN113804106 A CN 113804106A CN 202110946161 A CN202110946161 A CN 202110946161A CN 113804106 A CN113804106 A CN 113804106A
- Authority
- CN
- China
- Prior art keywords
- assembled
- target
- mechanical arm
- calibration
- double
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000000007 visual effect Effects 0.000 claims abstract description 21
- 239000011159 matrix material Substances 0.000 claims description 18
- 230000003287 optical effect Effects 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 7
- 208000003164 Diplopia Diseases 0.000 claims description 5
- 238000013459 approach Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000009776 industrial production Methods 0.000 description 2
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
Abstract
The invention discloses a double-arm cooperative assembly method based on twice calibration of vision, which comprises the steps of firstly, acquiring images by using a binocular camera, extracting characteristic points of a target to be assembled, determining the orientation of the target to be assembled, then adjusting the posture and the position of a main mechanical arm, then adjusting a slave mechanical arm, acquiring real-time image data stream, and acquiring gray scale information of the target to be assembled; selecting the slave mechanical arm pose with the minimum gray degree change degree in the target frame corresponding to the target to be assembled for image acquisition, and adjusting the clamping jaw of the master mechanical arm according to the visual angle to finish the calibration in the left and right directions; and (4) carrying out image acquisition on the side surface of the slave mechanical arm by taking the central point of the target object as the center of a circle to finish the calibration of the front-back direction and the vertical angle. The double-arm cooperative assembly method based on twice calibration of vision solves the problem of low assembly precision due to insufficient information acquired by a single camera in the existing scheme, improves the assembly precision of the mechanical arm, and has rapidness and robustness.
Description
Technical Field
The invention belongs to the field of robot vision, and particularly relates to a double-arm cooperative assembly method based on twice calibration of vision.
Background
In industrial production, the assembly cost accounts for more than 50% of the total cost. With the development of industrial robots, the robots are expected to replace manual work to complete assembly tasks. However, the industrial production has the problems of multiple types of parts, small volume and difficult positioning, and the application of the robot in the precision assembly work is a challenge. Taking the clamping jaw to clamp the screw precisely as an example, when the central axis of the clamping jaw is not aligned with the central axis of the bolt, the clamping jaw may fail to assemble when exerting force, and the assembly workpiece or the robot may be damaged.
At present, in a simple application scene, the initial pose and the target pose of a workpiece are specified in advance, and the mechanical arm is only fixed through a teaching system or according to a fixed program. However, in most practical application scenes, the initial pose or the target pose of the workpiece is not strictly fixed and unchanged. In this case, visual guided positioning is an ideal solution. The mechanical arm senses the working environment through the vision system and obtains the initial pose of the workpiece. The existing single-camera vision system has the problems of view blind areas or self mechanical structure shielding after approaching an object to be assembled, so that the assembly precision is low; the multi-camera vision system is difficult to find an observation position suitable for high-precision assembly.
Disclosure of Invention
The invention aims to provide a double-arm cooperative assembly method based on twice calibration of vision, which solves the problems of a single-camera vision system that a visual field blind area or a self mechanical structure is shielded and the assembly precision is low after the single-camera vision system approaches an object to be assembled; the multi-camera vision system is difficult to find the observation position suitable for high-precision assembly, and high-precision calibration of the assembly target is realized.
The technical solution for realizing the purpose of the invention is as follows: a double-arm cooperative assembling method based on vision double calibration comprises the following steps:
step 1, initializing and setting binocular cameras on mechanical arms of a double-arm robot respectively;
step 2, acquiring images by using a binocular camera, preprocessing the images and detecting angular points, extracting characteristic points of the target to be assembled, and determining the orientation of the target to be assembled;
step 3, adjusting the posture of the main mechanical arm to enable the central shaft of the clamping jaw to be parallel to the central shaft of the target to be assembled;
step 4, adjusting the main mechanical arm to enable the main mechanical arm to be close to the object to be assembled until the main mechanical arm enters a visual field blind area of the main camera or the information of the target object is obviously reduced;
step 5, adjusting the slave mechanical arm, scanning a half cycle in a counterclockwise or clockwise direction by taking the central point of the target to be assembled as a circle center, collecting a real-time image data stream, and acquiring gray scale information of the target to be assembled;
step 6, selecting the slave mechanical arm pose with the minimum gray degree change degree in the target frame corresponding to the target to be assembled for image acquisition, and adjusting the clamping jaw of the master mechanical arm according to the visual angle to finish the calibration in the left and right directions;
and 7, rotating the slave mechanical arm for 1/4 circles in a clockwise or anticlockwise direction by taking the center point of the target object as a circle center, observing on the side surface, and finishing the calibration of the front-back direction and the vertical angle.
Compared with the prior art, the invention has the following remarkable advantages:
(1) according to the invention, two binocular cameras are used, so that the pose of the object to be assembled with higher precision can be obtained under an initial condition before the first calibration;
(2) the twice calibration method can effectively help the main mechanical arm to align the central shaft of the clamping jaw and the central shaft of the object to be assembled;
(3) the invention is applied to the situation that when a target object is in a dangerous environment and a person is difficult to perform teaching operation, the pose of the slave mechanical arm is adjusted to perform two times of calibration, and the master mechanical arm is assisted to complete an assembly task;
(4) the invention is applied to complex environment, when the target object is shielded, deformed and changed in scale, the invention can still automatically adjust the pose from the mechanical arm to be suitable for observation to acquire the visual information, and has high precision, rapidity and robustness.
The invention is described in further detail below with reference to the figures and the detailed description.
Drawings
FIG. 1 is a flow chart of a vision-based double-calibration dual-arm cooperative assembly method of the present invention.
Detailed Description
A double-arm cooperative assembling method based on vision double calibration comprises the following steps:
step 1, carry out initialization setting respectively to the binocular camera on the arm of double-armed robot, specifically do:
setting initial working positions of the binocular cameras, enabling optical axes of the two binocular cameras to be parallel and perpendicular to a connecting line of optical centers of the two binocular cameras, and respectively calibrating a transfer matrix from the binocular cameras to the tail end of the mechanical arm provided with the binocular cameras, a transformation matrix from the tail end of the mechanical arm to the base, and respective internal reference matrix, external reference matrix and distortion coefficient of the two cameras.
Step 2, acquiring images by using a binocular camera, preprocessing the images and detecting angular points, extracting characteristic points of the target to be assembled, and determining the orientation of the target to be assembled, wherein the method specifically comprises the following steps:
step 2-1: acquiring images of a target to be assembled by using binocular cameras on the master mechanical arm and the slave mechanical arm;
step 2-2: correcting the image pair acquired by the master camera and the slave camera in the step 2-1 based on the co-polarity constraint in the epipolar geometry by using the internal reference matrix, the external reference matrix and the distortion coefficient obtained in the step 1;
step 2-3: based on the Harris operator, performing corner point detection on the image which is corrected in the step 2-2 and shot by the binocular camera of the robot arm, extracting 6 feature points of the object to be assembled, wherein the feature points are u1, u2, u3, u4, u5 and u6, and using the central coordinates of the 6 feature points as the central coordinates u0 of the object to be assembled:
step 2-4: calculating the parallax of the image pair corrected in the step 2-3 based on the NCC (normalized Cross correlation) matching algorithm on the basis of the alignment of the two view lines, and obtaining a parallax map;
step 2-5, finding the parallax value of the corresponding point in the parallax image obtained in the step 2-4 by using the image coordinates obtained in the step 2-3, determining depth information based on a triangulation distance measuring principle, and obtaining six feature points of the target to be assembled, world coordinates of the center point of the target to be assembled and a gray average value delta 0 in a target frame corresponding to the target to be assembled based on the depth value;
and 2-6, defining the plane of the base of the mechanical arm as a horizontal plane, selecting 3 feature points with the minimum horizontal coordinates to determine a normal vector of the target to be assembled, and determining the orientation of the target to be assembled.
Step 3, adjusting the posture of the main mechanical arm to enable the central shaft of the clamping jaw to be parallel to the central shaft of the target to be assembled, and specifically comprises the following steps:
and (3) determining a central shaft of the target to be assembled according to the normal vector of the target to be assembled and the central point of the target to be assembled obtained in the step (2), and adjusting the posture of the main mechanical arm to enable the central shaft of the clamping jaw to be parallel to the central shaft of the target to be assembled.
Step 4, adjusting the main mechanical arm to enable the main mechanical arm to be close to the object to be assembled until the main mechanical arm enters a visual field blind area of the main camera or the information of the target object is obviously reduced, and the method specifically comprises the following steps:
step 4-1: adjusting a main mechanical arm to approach to a target to be assembled, and scanning the target to be assembled by a binocular camera on the main mechanical arm;
step 4-2: when the number of the characteristic points in the image changes by delta n being more than or equal to 1 in the front and back two frames of pictures, the information of the target to be assembled is considered to be obviously reduced, and the target to be assembled enters a visual field blind area of the main camera;
and 4-3, obtaining the distance d between the binocular camera on the main mechanical arm and the central shaft of the target to be assembled.
Step 5, adjusting the slave mechanical arm, scanning a half cycle in a counterclockwise or clockwise direction (counterclockwise when the slave mechanical arm is a left arm and clockwise when the slave mechanical arm is a right arm) by taking a central point of the target to be assembled as a circle center, collecting a real-time image data stream, and acquiring gray scale information of the target to be assembled, wherein the specific steps are as follows:
step 5-1: adjusting the pose of the slave mechanical arm to enable the optical axis of the binocular camera on the slave mechanical arm to be superposed with the intersection line L of the plane where the clamping jaw is located and the plane where the target to be assembled is located, and enabling the binocular camera on the slave mechanical arm to face an operation area at the moment, specifically:
step 5-1-1: adjusting the position of the slave mechanical arm to enable the center point of a binocular camera on the slave mechanical arm to reach the intersection line L of the plane where the clamping jaw is located and the plane where the target to be assembled is located;
step 5-1-2: and adjusting the posture of the slave mechanical arm to ensure that the optical axis of the slave binocular camera on the slave mechanical arm is superposed with the intersection line L of the plane of the clamping jaw and the plane of the object to be assembled, and the slave camera faces to the operation area at the moment.
Step 5-2, moving the slave mechanical arm to be close to the central point of the target to be assembled along the intersecting line L until the distance between the binocular camera on the slave mechanical arm and the central point of the target to be assembled is 2 d;
and 5-3, scanning a half cycle from the mechanical arm by taking the central point of the target to be assembled as the center of the circle, simultaneously collecting real-time image data flow by using a binocular camera on the mechanical arm, preprocessing the image, namely, Gaussian smoothing, and extracting gray level information of the target object after eliminating noise points existing in a local area.
Step 6, selecting the slave mechanical arm pose with the minimum gray degree change degree in the target frame corresponding to the target to be assembled for image acquisition, and adjusting the clamping jaw of the master mechanical arm according to the visual angle to finish the calibration in the left and right directions, wherein the method specifically comprises the following steps:
step 6-1, calculating the gray average value delta i in the target frame corresponding to each pose according to the gray value extracted in the step 5, wherein i is 1, 2, 3 … … 10t, t is scanning time, the difference | delta i-delta 0| of the gray average value in the target frame corresponding to the initial pose is obtained, and the pose with the minimum absolute value of the difference is taken as the optimal pose B;
step 6-2, moving the slave mechanical arm to the optimal pose B recorded in the step 6-1;
and 6-3, acquiring images by using a binocular camera on the slave mechanical arm, acquiring two-dimensional information of the clamping jaw and the target to be assembled under the pose B, horizontally moving the clamping jaw until the projection of the central axis of the clamping jaw in the two-dimensional plane obtained from the visual angle is superposed with the central axis of the target to be assembled, and completing the calibration in the left and right directions.
And 7, rotating the slave mechanical arm for 1/4 circles in a clockwise or counterclockwise direction by taking the center point of the target object as the center of a circle to finish calibration of the front-back direction and the vertical angle, wherein the calibration specifically comprises the following steps:
step 7-1, rotating the slave mechanical arm in a clockwise or counterclockwise direction (clockwise when the slave mechanical arm is a left arm and counterclockwise when the slave mechanical arm is a right arm) by using a central point u0 of the object to be assembled as a circle center for 1/4 weeks, and collecting images from the side;
7-2, adjusting the posture of the clamping jaw to enable the projection of the central axis of the clamping jaw under the visual angle to be parallel to the central axis of the target object, and completing the calibration of the vertical angle;
and 7-3, horizontally moving the clamping jaw to enable the central axis of the clamping jaw to coincide with the central axis of the target object, and completing calibration in the front-back direction.
The invention is further described below with reference to the figures and examples.
Examples
As shown in fig. 1, a double-arm cooperative assembling method based on two-time calibration of vision includes the following steps:
step 1, carry out initialization setting respectively to the binocular camera on the arm of double-armed robot, specifically do:
setting initial working positions of the binocular cameras, enabling optical axes of the two binocular cameras to be parallel and perpendicular to a connecting line of optical centers of the two binocular cameras, and respectively calibrating a transfer matrix from the binocular cameras to the tail end of the mechanical arm provided with the binocular cameras, a transformation matrix from the tail end of the mechanical arm to the base, and respective internal reference matrix, external reference matrix and distortion coefficient of the two cameras.
Step 2, acquiring images by using a binocular camera, preprocessing the images and detecting angular points, extracting characteristic points of the bolts to be assembled, and determining the orientation of the bolts to be assembled, wherein the method specifically comprises the following steps:
step 2-1: acquiring images of bolts to be assembled by using binocular cameras on the master mechanical arm and the slave mechanical arm;
step 2-2: correcting the image pair acquired by the master camera and the slave camera in the step 2-1 based on the co-polarity constraint in the epipolar geometry by using the internal reference matrix, the external reference matrix and the distortion coefficient obtained in the step 1;
step 2-3: based on Harris operator, performing corner point detection on the image corrected in the step 2-2 and shot from the binocular camera of the robot arm, extracting 6 feature points of the bolt to be assembled, wherein the feature points are u1, u2, u3, u4, u5 and u6, and using the central coordinates of the 6 feature points as the central coordinates u0 of the bolt to be assembled:
step 2-4: calculating the parallax of the image pair corrected in the step 2-3 based on the NCC (normalized Cross correlation) matching algorithm on the basis of the alignment of the two view lines, and obtaining a parallax map;
step 2-5, finding a parallax value of a corresponding point in the parallax map obtained in the step 2-4 by using the image coordinates obtained in the step 2-3, determining depth information based on a triangulation distance measuring principle, and obtaining six feature points of the bolt to be assembled, world coordinates of the center point of the bolt to be assembled and a gray average value delta 0 in a target frame corresponding to the bolt to be assembled based on a depth value;
and 2-6, defining the plane of the base of the mechanical arm as a horizontal plane, selecting 3 feature points with the minimum horizontal coordinates to determine a normal vector of the bolt to be assembled, and determining the orientation of the bolt to be assembled.
Step 3, adjusting the posture of the main mechanical arm to enable the central shaft of the clamping jaw to be parallel to the central shaft of the bolt to be assembled, and specifically comprises the following steps:
and (4) determining a central shaft of the bolt to be assembled according to the normal vector of the bolt to be assembled and the central point of the bolt to be assembled obtained in the step (2), and adjusting the posture of the main mechanical arm to enable the central shaft of the clamping jaw to be parallel to the central shaft of the bolt to be assembled.
Step 4, adjusting the main mechanical arm to enable the main mechanical arm to be close to the object to be assembled until the main mechanical arm enters a main camera view blind area or the bolt object information is obviously reduced, and the method specifically comprises the following steps:
step 4-1: adjusting the main mechanical arm to approach the bolt to be assembled, and scanning the bolt to be assembled by a binocular camera on the main mechanical arm;
step 4-2: when the number of the characteristic points in the image changes by delta n being more than or equal to 1 in the front and back two frames of pictures, the information of the bolts to be assembled is considered to be remarkably reduced, and the bolts enter a visual field blind area of a main camera;
and 4-3, obtaining the distance d between the binocular camera on the main mechanical arm and the central shaft of the bolt to be assembled.
Step 5, adjusting the slave mechanical arm, scanning a half cycle in a counterclockwise or clockwise direction (counterclockwise when the slave mechanical arm is a left arm and clockwise when the slave mechanical arm is a right arm) by taking a central point of the bolt to be assembled as a circle center, collecting a real-time image data stream, and acquiring gray scale information of the bolt to be assembled, wherein the method specifically comprises the following steps:
step 5-1: adjusting the pose of the slave mechanical arm to enable the optical axis of the binocular camera on the slave mechanical arm to be superposed with the intersection line L of the plane where the clamping jaw is located and the plane where the bolt to be assembled is located, and enabling the binocular camera on the slave mechanical arm to face an operation area at the moment, specifically:
step 5-1-1: adjusting the position of the slave mechanical arm to enable the center point of a binocular camera on the mechanical arm to reach the intersection line L of the plane where the clamping jaw is located and the plane where the bolt to be assembled is located;
step 5-1-2: and adjusting the posture of the slave mechanical arm to ensure that the optical axis of the slave binocular camera on the slave mechanical arm is superposed with the intersection line L of the plane of the clamping jaw and the plane of the bolt to be assembled, and the slave camera faces to the operation area at the moment.
Step 5-2, moving the slave mechanical arm, and approaching the central point of the bolt to be assembled along the intersecting line L until the distance between the binocular camera on the slave mechanical arm and the central point of the bolt to be assembled is 2 d;
and 5-3, scanning a half cycle from the mechanical arm by taking the central point of the bolt to be assembled as a circle center, simultaneously acquiring real-time image data stream by using a binocular camera on the mechanical arm, preprocessing the image, namely, Gaussian smoothing, and extracting gray level information of the bolt object after eliminating noise points existing in a local area, wherein a Gaussian coefficient delta is 0.8.
Step 6, selecting the slave mechanical arm pose with the minimum gray degree change in the bolt frame corresponding to the bolt to be assembled for image acquisition, and adjusting the clamping jaw of the master mechanical arm according to the visual angle to finish the calibration in the left and right directions, wherein the method specifically comprises the following steps:
step 6-1, calculating the gray average value delta i in the target frame corresponding to each pose according to the gray value extracted in the step 5, wherein i is 1, 2, 3 … … 10t, t is scanning time, the difference | delta i-delta 0| of the gray average value in the target frame corresponding to the initial pose is obtained, and the pose with the minimum absolute value of the difference is taken as the optimal pose B;
step 6-2, moving the slave mechanical arm to the optimal pose B recorded in the step 6-1;
and 6-3, acquiring images by using a binocular camera on the slave mechanical arm, acquiring two-dimensional information of the clamping jaw and the bolt to be assembled under the pose B, horizontally moving the clamping jaw until the projection of the central axis of the clamping jaw in the two-dimensional plane obtained from the visual angle is superposed with the central axis of the bolt to be assembled, and completing the calibration in the left and right directions.
And 7, rotating the slave mechanical arm for 1/4 circles in a clockwise or anticlockwise direction by taking the center point of the bolt object as a circle center to finish calibration of the front-back direction and the vertical angle, wherein the calibration specifically comprises the following steps:
step 7-1, rotating the slave mechanical arm in a clockwise or counterclockwise direction (clockwise when the mechanical arm is a left arm and counterclockwise when the mechanical arm is a right arm) by using a center point u0 of the bolt to be assembled as a circle center for 1/4 weeks, and collecting images from the side;
7-2, adjusting the posture of the clamping jaw to enable the projection of the central axis of the clamping jaw under the visual angle to be parallel to the central axis of the bolt object, and completing the calibration of the vertical angle;
and 7-3, horizontally moving the clamping jaw to enable the central axis of the clamping jaw to coincide with the central axis of the bolt object, and completing calibration in the front-back direction.
The method can be applied to complex environments, can automatically adjust the pose from the mechanical arm to be suitable for observation to acquire visual information when a target object is shielded, deformed and changed in scale, and has high precision, rapidity and robustness.
Claims (9)
1. A double-arm cooperative assembling method based on vision double calibration is characterized by comprising the following steps:
step 1, initializing and setting binocular cameras on mechanical arms of a double-arm robot respectively;
step 2, acquiring images by using a binocular camera, preprocessing the images and detecting angular points, extracting characteristic points of the target to be assembled, and determining the orientation of the target to be assembled;
step 3, adjusting the posture of the main mechanical arm to enable the central shaft of the clamping jaw to be parallel to the central shaft of the target to be assembled;
step 4, adjusting the main mechanical arm to enable the main mechanical arm to be close to the object to be assembled until the main mechanical arm enters a visual field blind area of the main camera or the information of the target object is obviously reduced;
step 5, adjusting the slave mechanical arm, scanning a half cycle in a counterclockwise or clockwise direction by taking the central point of the target to be assembled as a circle center, collecting a real-time image data stream, and acquiring gray scale information of the target to be assembled;
step 6, selecting the slave mechanical arm pose with the minimum gray degree change degree in the target frame corresponding to the target to be assembled for image acquisition, and adjusting the clamping jaw of the master mechanical arm according to the visual angle to finish the calibration in the left and right directions;
and 7, rotating the slave mechanical arm for 1/4 circles in a clockwise or anticlockwise direction by taking the center point of the target object as a circle center, observing on the side surface, and finishing the calibration of the front-back direction and the vertical angle.
2. The vision-based double-calibration double-arm cooperative assembling method according to claim 1, wherein the initialization setting in the step 1 is specifically as follows:
setting initial working positions of the binocular cameras, enabling optical axes of the two binocular cameras to be parallel and perpendicular to a connecting line of optical centers of the two binocular cameras, and respectively calibrating a transfer matrix from the binocular cameras to the tail end of the mechanical arm provided with the binocular cameras, a transformation matrix from the tail end of the mechanical arm to the base, and respective internal reference matrix, external reference matrix and distortion coefficient of the two cameras.
3. The vision-based double-calibration double-arm cooperative assembling method according to claim 1, wherein the determining of the orientation of the target to be assembled in the step 2 specifically comprises the following steps:
step 2-1: acquiring images of a target to be assembled by using binocular cameras on the master mechanical arm and the slave mechanical arm;
step 2-2: correcting the image pair acquired by the master camera and the slave camera in the step 2-1 based on the co-polarity constraint in the epipolar geometry by using the internal reference matrix, the external reference matrix and the distortion coefficient obtained in the step 1;
step 2-3: performing corner point detection on the image which is corrected in the step 2-2 and shot by the binocular camera of the mechanical arm based on a Harris operator, extracting 6 feature points of the object to be assembled, wherein the feature points are u1, u2, u3, u4, u5 and u6 respectively, and using the central coordinates of the 6 feature points as the central coordinates u0 of the object to be assembled;
step 2-4: calculating the parallax of the image pair which is corrected in the step 2-3 based on the NCC matching algorithm on the basis of the alignment of the two view lines, and obtaining a parallax map;
step 2-5, finding the parallax value of the corresponding point in the parallax image obtained in the step 2-4 by using the image coordinates obtained in the step 2-3, determining depth information based on a triangulation distance measuring principle, and obtaining six feature points of the target to be assembled, world coordinates of the center point of the target to be assembled and a gray average value delta 0 in a target frame corresponding to the target to be assembled based on the depth value;
and 2-6, defining the plane of the base of the mechanical arm as a horizontal plane, selecting 3 feature points with the minimum horizontal coordinates to determine a normal vector of the target to be assembled, and determining the orientation of the target to be assembled.
4. The vision-based double-calibration double-arm cooperative assembling method according to claim 3, wherein the adjusting of the posture of the main mechanical arm in the step 3 is specifically as follows:
and (3) determining a central shaft of the target to be assembled according to the normal vector of the target to be assembled and the central point of the target to be assembled obtained in the step (2), and adjusting the posture of the main mechanical arm to enable the central shaft of the clamping jaw to be parallel to the central shaft of the target to be assembled.
5. The vision-based double-calibration double-arm cooperative assembling method according to claim 1, wherein the adjusting of the main robot arm in the step 4 specifically comprises the following steps:
step 4-1: adjusting a main mechanical arm to approach to a target to be assembled, and scanning the target to be assembled by a binocular camera on the main mechanical arm;
step 4-2: when the number of the characteristic points in the image changes by delta n being more than or equal to 1 in the front and back two frames of pictures, the information of the target to be assembled is considered to be obviously reduced, and the target to be assembled enters a visual field blind area of the main camera;
and 4-3, obtaining the distance d between the binocular camera on the main mechanical arm and the central shaft of the target to be assembled.
6. The vision-based double-calibration double-arm cooperative assembling method according to claim 1, wherein the adjusting in the step 5 is to obtain gray scale information of the target to be assembled from the mechanical arm, and specifically comprises the following steps:
step 5-1: adjusting the pose of the slave mechanical arm to ensure that the optical axis of the binocular camera on the slave mechanical arm is superposed with the intersection line L of the plane of the clamping jaw and the plane of the target to be assembled, and the binocular camera on the slave mechanical arm faces to an operation area;
step 5-2, moving the slave mechanical arm to be close to the central point of the target to be assembled along the intersecting line L until the distance between the binocular camera on the slave mechanical arm and the central point of the target to be assembled is 2 d;
and 5-3, scanning a half cycle from the mechanical arm by taking the central point of the target to be assembled as the center of the circle, simultaneously collecting real-time image data flow by using a binocular camera on the mechanical arm, preprocessing the image, namely, Gaussian smoothing, and extracting gray level information of the target object after eliminating noise points existing in a local area.
7. The vision-based double-calibration double-arm cooperative assembling method according to claim 6, wherein the adjusting in the step 5-1 is from a robot arm pose, specifically:
step 5-1-1: adjusting the position of the slave mechanical arm to enable the center point of a binocular camera on the slave mechanical arm to reach the intersection line L of the plane where the clamping jaw is located and the plane where the target to be assembled is located;
step 5-1-2: and adjusting the posture of the slave mechanical arm to ensure that the optical axis of the slave binocular camera on the slave mechanical arm is superposed with the intersection line L of the plane of the clamping jaw and the plane of the object to be assembled, and the slave camera faces to the operation area at the moment.
8. The double-arm cooperative assembling method based on vision double calibration as recited in claim 1, wherein the calibration in the left-right direction in the step 6 is completed by the following steps:
step 6-1, calculating the gray average value delta i in the target frame corresponding to each pose according to the gray value extracted in the step 5, wherein i is 1, 2, 3 … … 10t, t is scanning time, the difference | delta i-delta 0| of the gray average value in the target frame corresponding to the initial pose is obtained, and the pose with the minimum absolute value of the difference is taken as the optimal pose B;
step 6-2, moving the slave mechanical arm to the optimal pose B recorded in the step 6-1;
and 6-3, acquiring images by using a binocular camera on the slave mechanical arm, acquiring two-dimensional information of the clamping jaw and the target to be assembled under the pose B, horizontally moving the clamping jaw until the projection of the central axis of the clamping jaw in the two-dimensional plane obtained from the visual angle is superposed with the central axis of the target to be assembled, and completing the calibration in the left and right directions.
9. The double-arm cooperative assembling method based on vision double calibration as recited in claim 1, wherein the calibration of the front-back direction and the vertical angle in the step 7 is completed by the following steps:
7-1, rotating the slave mechanical arm for 1/4 circles clockwise or anticlockwise by taking a central point u0 of the target to be assembled as a circle center, and collecting images from the side;
7-2, adjusting the posture of the clamping jaw to enable the projection of the central axis of the clamping jaw under the visual angle to be parallel to the central axis of the target object, and completing the calibration of the vertical angle;
and 7-3, horizontally moving the clamping jaw to enable the central axis of the clamping jaw to coincide with the central axis of the target object, and completing calibration in the front-back direction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110946161.XA CN113804106A (en) | 2021-08-18 | 2021-08-18 | Double-arm cooperative assembly method based on twice calibration of vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110946161.XA CN113804106A (en) | 2021-08-18 | 2021-08-18 | Double-arm cooperative assembly method based on twice calibration of vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113804106A true CN113804106A (en) | 2021-12-17 |
Family
ID=78893790
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110946161.XA Pending CN113804106A (en) | 2021-08-18 | 2021-08-18 | Double-arm cooperative assembly method based on twice calibration of vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113804106A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007061979A (en) * | 2005-09-01 | 2007-03-15 | Sharp Corp | Visual sensor correction method for robot arm and computer program |
| CN107471218A (en) * | 2017-09-07 | 2017-12-15 | 南京理工大学 | A kind of tow-armed robot hand eye coordination method based on multi-vision visual |
| CN206840057U (en) * | 2016-12-09 | 2018-01-05 | 南京理工大学 | A kind of hot line robot control system based on double mechanical arms and sub-arm |
| CN107756417A (en) * | 2017-11-21 | 2018-03-06 | 北方民族大学 | The intelligent man-machine co-melting robot system of both arms security cooperation |
| CN109773798A (en) * | 2019-03-28 | 2019-05-21 | 大连理工大学 | A kind of double mechanical arms cooperation control method based on binocular vision |
| US20190176348A1 (en) * | 2017-12-12 | 2019-06-13 | X Development Llc | Sensorized Robotic Gripping Device |
-
2021
- 2021-08-18 CN CN202110946161.XA patent/CN113804106A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007061979A (en) * | 2005-09-01 | 2007-03-15 | Sharp Corp | Visual sensor correction method for robot arm and computer program |
| CN206840057U (en) * | 2016-12-09 | 2018-01-05 | 南京理工大学 | A kind of hot line robot control system based on double mechanical arms and sub-arm |
| CN107471218A (en) * | 2017-09-07 | 2017-12-15 | 南京理工大学 | A kind of tow-armed robot hand eye coordination method based on multi-vision visual |
| CN107756417A (en) * | 2017-11-21 | 2018-03-06 | 北方民族大学 | The intelligent man-machine co-melting robot system of both arms security cooperation |
| US20190176348A1 (en) * | 2017-12-12 | 2019-06-13 | X Development Llc | Sensorized Robotic Gripping Device |
| CN109773798A (en) * | 2019-03-28 | 2019-05-21 | 大连理工大学 | A kind of double mechanical arms cooperation control method based on binocular vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111775146B (en) | Visual alignment method under industrial mechanical arm multi-station operation | |
| CN107214703B (en) | Robot self-calibration method based on vision-assisted positioning | |
| CN109035200B (en) | Bolt positioning and pose detection method based on single-eye and double-eye vision cooperation | |
| CN111089569B (en) | Large box body measuring method based on monocular vision | |
| CN111415391B (en) | External azimuth parameter calibration method for multi-camera by adopting mutual shooting method | |
| CN102353340B (en) | Cylinder-cover blank machining-size identifying method and device | |
| Chen et al. | Acquisition of weld seam dimensional position information for arc welding robot based on vision computing | |
| CN112700501B (en) | Underwater monocular subpixel relative pose estimation method | |
| CN111897349A (en) | Underwater robot autonomous obstacle avoidance method based on binocular vision | |
| CN109470149B (en) | Method and device for measuring position and posture of pipeline | |
| CN112362034B (en) | Solid engine multi-cylinder section butt joint guiding measurement method based on binocular vision | |
| CN113379849A (en) | Robot autonomous recognition intelligent grabbing method and system based on depth camera | |
| CN113724337B (en) | Camera dynamic external parameter calibration method and device without depending on tripod head angle | |
| CN111879354A (en) | Unmanned aerial vehicle measurement system that becomes more meticulous | |
| CN114001651B (en) | Large-scale slender barrel type component pose in-situ measurement method based on binocular vision measurement and priori detection data | |
| Wan et al. | High-precision six-degree-of-freedom pose measurement and grasping system for large-size object based on binocular vision | |
| CN111145267B (en) | 360-degree panoramic view multi-camera calibration method based on IMU assistance | |
| CN111699445B (en) | Robot kinematics model optimization method and system and storage device | |
| Makovetskii et al. | An algorithm for rough alignment of point clouds in three-dimensional space | |
| CN113804106A (en) | Double-arm cooperative assembly method based on twice calibration of vision | |
| Zou et al. | Flexible Extrinsic Parameter Calibration for Multicameras With Nonoverlapping Field of View | |
| Horaud et al. | Euclidean reconstruction and affine camera calibration using controlled robot motions | |
| CN114565714B (en) | Monocular vision sensor hybrid high-precision three-dimensional structure recovery method | |
| CN115578465B (en) | Laser positioning and labeling method based on binocular vision, product and application | |
| CN114178832B (en) | Robot guide assembly robot method based on vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |