CN115283905B - Welding gun posture adjusting method of welding robot - Google Patents
Welding gun posture adjusting method of welding robot Download PDFInfo
- Publication number
- CN115283905B CN115283905B CN202211011654.5A CN202211011654A CN115283905B CN 115283905 B CN115283905 B CN 115283905B CN 202211011654 A CN202211011654 A CN 202211011654A CN 115283905 B CN115283905 B CN 115283905B
- Authority
- CN
- China
- Prior art keywords
- welding
- key
- groove
- image
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003466 welding Methods 0.000 title claims abstract description 299
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012545 processing Methods 0.000 claims description 38
- 239000011159 matrix material Substances 0.000 claims description 25
- 238000010586 diagram Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000005260 corrosion Methods 0.000 claims description 6
- 230000007797 corrosion Effects 0.000 claims description 6
- 230000036544 posture Effects 0.000 description 48
- 230000008569 process Effects 0.000 description 12
- 230000009466 transformation Effects 0.000 description 7
- 239000003758 nuclear fuel Substances 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000012805 post-processing Methods 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 238000012800 visualization Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000000877 morphologic effect Effects 0.000 description 3
- 230000008439 repair process Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 230000000149 penetrating effect Effects 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 241001270131 Agaricus moelleri Species 0.000 description 1
- 101100394003 Butyrivibrio fibrisolvens end1 gene Proteins 0.000 description 1
- 201000009310 astigmatism Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005429 filling process Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012958 reprocessing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
- B23K37/02—Carriages for supporting the welding or cutting element
- B23K37/0252—Steering means
Landscapes
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The embodiment of the application provides a welding gun posture adjusting method of a welding robot, which comprises the following steps: determining key information of the welding groove based on the welding groove image; determining gesture key point information corresponding to a welding robot, wherein the gesture key point information is information corresponding to gesture key points, and the gesture key points are pre-marked on a mechanical arm of the welding robot; and displaying the key information of the welding groove and the key point information of the welding gun posture of the welding robot in real time, so that a worker can adjust the welding gun posture of the welding robot. According to the embodiment of the application, the welding gun posture of the welding robot can be timely adjusted, so that the welding quality is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of welding equipment, in particular to a welding gun posture adjusting method of a welding robot.
Background
The welding robot is mechanical equipment for completing welding operation under the control of the automatic controller, and has reliable welding quality, high precision and good environmental adaptability, so that the application of robot welding is more widespread and common.
At present, a welding robot often replaces manual work to finish welding of a welding groove in processing equipment, but the internal circuit of the processing equipment is complex, so that in the actual welding process, the welding robot can only weld the welding groove through a preset algorithm, but the key points of the welding robot are inaccurate due to the diversification of the welding groove and the complex internal circuit of the processing equipment, and the welding quality is difficult to guarantee when the welding robot is singly adopted for welding.
Therefore, it is needed to provide a welding gun posture adjustment method of a welding robot, which can adjust the welding gun posture of the welding robot in time, so as to reduce the problems of poor welding seam strength uniformity, unqualified welding seam quality and the like caused by inaccurate posture as much as possible.
Disclosure of Invention
In order to solve the above problems, the embodiment of the application provides a welding gun posture adjustment method of a welding robot, which can timely adjust the welding gun posture of the welding robot, thereby improving welding quality.
In a first aspect, an embodiment of the present application provides a welding gun posture adjustment method of a welding robot, including:
determining key information of the welding groove based on the welding groove image;
determining gesture key point information corresponding to a welding robot, wherein the gesture key point information is information corresponding to gesture key points, and the gesture key points are pre-marked on a mechanical arm of the welding robot;
and displaying the key information of the welding groove and the key point information of the welding gun posture of the welding robot in real time, so that a worker can adjust the welding gun posture of the welding robot.
Optionally, the determining, based on the welding groove image, key information of the welding groove includes:
image interference removal is carried out on the welding groove image, and an initial key groove image is obtained;
performing operation treatment on the initial key groove image to obtain a key groove image;
and determining key information of the welding groove based on the key groove image.
Optionally, the performing image noise removal on the welding groove image to obtain an initial key groove image includes:
performing gray level conversion on the welding groove image to obtain a gray level welding groove image corresponding to the welding groove image;
performing Fourier transform processing on the gray welding groove image to obtain a groove frequency diagram corresponding to the welding groove image;
conducting guide filtering treatment on the groove frequency map to obtain a filtered groove image, wherein the guide filtering treatment is used for removing noise in the groove frequency map;
and performing binarization processing on the filtered groove image to obtain an initial key groove image, wherein the binarization processing is used for removing interference information in the filtered groove image and retaining key information in the filtered groove image.
Optionally, the calculating the initial key groove image to obtain a key groove image includes:
performing open operation processing on the initial key groove image to obtain a first processed key groove image, wherein the open operation processing comprises expansion processing and corrosion processing;
and carrying out image region filling treatment on the first treatment key groove, and determining a second treatment key groove image.
Optionally, the determining, based on the key groove image, key information of the welding groove includes:
converting the key region in the second processed key groove image into an initial key line by using a gauss operator;
determining contour lines in the initial key lines by using a xld operator;
determining the breaking point of the line segment in the contour line as a key point of the welding groove;
and determining key information of the welding groove based on the key points of the welding groove.
Optionally, the determining the key information of the welding groove based on the key point of the welding groove includes:
determining pixel coordinate information and depth information corresponding to key points of the welding groove;
and determining three-dimensional coordinate information of the welding groove in a world coordinate system based on the pixel coordinate information and the depth information, wherein the three-dimensional coordinate information of the welding groove is key information of the welding groove.
Optionally, before determining the pixel coordinate information and the depth information corresponding to the key point of the welding groove, the method includes:
calibrating the camera position of a camera, and determining a camera internal reference matrix corresponding to the camera, wherein the camera internal reference matrix comprises internal references of the camera;
calibrating a laser plane based on a structure light sensor, and acquiring a laser plane equation corresponding to the laser plane;
the determining pixel coordinate information and depth information corresponding to key points of the welding groove comprises the following steps:
and determining pixel coordinate information corresponding to the key points of the welding groove based on the camera internal reference matrix, and determining depth information corresponding to the key points of the welding groove based on the laser plane equation.
Optionally, the determining the gesture key point information corresponding to the welding robot includes:
and determining the gesture key point information corresponding to the welding robot based on the gesture key points of the welding robot.
Optionally, the determining the pose key point information corresponding to the welding robot based on the pose key points of the welding robot includes:
calibrating the tail end of at least one mechanical arm of the welding robot as a gesture key point;
determining an association relationship between the gesture key points and camera positions of the cameras;
and determining the posture key point information corresponding to the welding robot based on the association relation and the posture key points of the welding robot.
Optionally, the determining the association relationship between the gesture key point and the camera position of the camera includes:
and acquiring the association relation between the gesture key points and the camera positions of the camera by using a TSAI algorithm.
Compared with the prior art, the technical scheme of the embodiment of the application has the following advantages:
according to the welding gun posture adjusting method of the welding robot, key information of a welding groove is determined based on the welding groove image; determining gesture key point information corresponding to a welding robot, wherein the gesture key point information is information corresponding to gesture key points, and the gesture key points are pre-marked on a mechanical arm of the welding robot; and finally, displaying the key information of the welding groove and the key point information of the welding gun posture of the welding robot in real time, so that a worker can adjust the welding gun posture of the welding robot. According to the embodiment of the application, the key information of the welding groove and the key point information of the welding gun posture of the welding robot are displayed together, so that a worker can remotely control, and timely adjust the welding gun posture of the welding robot, the problems of poor welding seam strength uniformity, unqualified welding seam quality and the like caused by inaccurate posture are avoided, and the welding quality is improved.
In addition, the adjusting method of the welding robot in the embodiment of the application can be applied to a repairing scene of the nuclear fuel post-processing equipment, and because operation radiation exists in the nuclear fuel post-processing equipment frequently, the welding gun posture adjusting method of the welding robot can also avoid the personal safety problem during manual repairing and improve the operation efficiency in the nuclear fuel post-processing equipment.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings may be obtained according to the provided drawings without inventive effort to a person skilled in the art.
FIG. 1 is a schematic flow chart of a welding gun posture adjustment method of a welding robot provided by the invention;
FIG. 2 is an alternative flow chart of determining key information for a weld groove in an embodiment of the present application;
FIG. 3 is an alternative flowchart of obtaining an initial key groove image in an embodiment of the present application;
FIG. 4 is an alternative schematic diagram of a gray level histogram in an embodiment of the present application;
FIG. 5 is an alternative flowchart of obtaining a key groove image in an embodiment of the present application;
FIG. 6 is an alternative flow chart of obtaining key information for a weld groove in an embodiment of the present application;
FIG. 7 is a schematic illustration of a key portion of a substantially preserved line in an embodiment of the present application;
FIG. 8 is a schematic view of a partial calibration photograph in an embodiment of the present application;
FIG. 9 is a graph of calibration visualization results in an embodiment of the present application;
FIG. 10 is a diagram of the visual effects of the external parameters in an embodiment of the present application;
FIG. 11 is an alternative flowchart for determining pose keypoint information in an embodiment of the present application;
fig. 12 is a schematic diagram of steps of the TSAI method in the embodiment of the present application.
Detailed Description
Since nuclear fuel reprocessing equipment contains a large number of pipe structures, these pipes are reworked once a year, with hundreds of welds reworked at a time. The damage of the pipeline is half of a penetrating or non-penetrating defect caused by liquid corrosion, and the damaged part is mainly concentrated on the corrosion of the welding seams of the valve and the pipeline. Because nuclear radiation exists in the working environment, the manual repair mode is poor in safety, high in difficulty, large in working amount and low in efficiency.
Therefore, as described in the background art, the welding robot can replace the manual work to finish the welding of the welding groove in the nuclear fuel post-treatment device at present. However, the welding robot generally adopts a preset algorithm to weld the welding groove, so that the welding quality of the welding robot is difficult to ensure.
Based on this, the embodiment of the application discloses a welding gun posture adjusting method of a welding robot, which comprises the following steps: determining key information of the welding groove based on the welding groove image; determining gesture key point information corresponding to a welding robot, wherein the gesture key point information is information corresponding to gesture key points, and the gesture key points are pre-marked on a mechanical arm of the welding robot; and displaying the key information of the welding groove and the key point information of the welding gun posture of the welding robot in real time, so that a worker can adjust the welding gun posture of the welding robot.
In addition, key information of the welding groove and welding gun posture key point information of the welding robot are displayed together in the embodiment of the application, so that a worker can remotely control, and timely adjust the welding gun posture of the welding robot, the problems of poor welding seam strength uniformity, unqualified welding seam quality and the like caused by inaccurate posture are avoided, and welding quality is improved.
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Fig. 1 is a schematic flow chart of a welding gun posture adjustment method of a welding robot. In a specific embodiment, the welding gun posture adjustment method of the welding robot provided by the invention can be applied to a repair scene of nuclear fuel post-processing equipment, and the welding gun posture adjustment method of the welding robot comprises the following steps:
and S11, determining key information of the welding groove based on the welding groove image.
The welding groove image can be acquired by an industrial camera. In order to facilitate the real-time monitoring of the welding process of the staff, the image analysis module of the industrial camera can be designed in the same computer, so that the image analysis and control of the welding groove can be realized in the same computer.
In addition, the key information of the welding groove is used for representing information corresponding to key points of the welding groove.
Step S12, determining gesture key point information corresponding to the welding robot, wherein the gesture key point information is information corresponding to gesture key points, and the gesture key points are pre-marked on a mechanical arm of the welding robot.
In an alternative embodiment, the gesture key point is pre-marked at the end of the mechanical arm of the welding robot, typically, the end of the mechanical arm of the welding robot is connected with a welding gun, for this purpose, the end of the mechanical arm of the welding robot can be set as the gesture key point of the welding robot, and the welding gun gesture of the welding robot can be determined by measuring the coordinate information of the end of the mechanical arm of the welding robot.
And S13, displaying the key information of the welding groove and the key point information of the welding gun posture of the welding robot in real time, so that a worker can adjust the welding gun posture of the welding robot.
And displaying the key information of the welding groove and the key point information of the welding gun posture of the welding robot in real time, so that a worker can intuitively observe the welding gun posture of the welding robot, and the welding gun posture of the welding robot can be timely adjusted based on the information displayed in real time.
Therefore, the key information of the welding groove and the welding gun posture key point information of the welding robot are displayed together in the embodiment of the application, so that the remote control of a worker is facilitated, the worker can timely adjust the welding gun posture of the welding robot, the problems of poor welding seam strength uniformity, unqualified welding seam quality and the like caused by inaccurate welding gun posture are avoided, and the welding quality of the welding robot is improved.
In order to facilitate the operator to master the welding groove more accurately during welding, a camera is used for acquiring a welding groove image, the welding groove image can be used for representing two-dimensional information of the welding groove, extracting characteristic information of the welding groove by utilizing a line structure light sensor, and key information such as width, depth and the like of the welding groove is transmitted to a computer in real time through a socket protocol.
Because the welding groove image obtained by the camera may have the problems of serious reflection, loud noise and the like, which affect the quality of the picture, the welding groove image needs to be processed to obtain an image convenient for feature extraction. Taking a welding groove in a pipeline as an example, a process of acquiring key information of the welding groove is described.
Fig. 2 is an alternative flow chart of determining key information for a weld groove in an embodiment of the present application. Referring to fig. 2, in order to obtain the key information of the welding groove, determining the key information of the welding groove in the present application includes:
s21, performing image interference removal on the welding groove image to obtain an initial key groove image;
the initial groove image is an image after image interference is removed, compared with the welding groove image, the original information integrity (namely main characteristics) can be maintained as far as possible, and meanwhile useless information in the image can be removed, so that the calculated amount of the welding groove image is reduced.
Step S22, carrying out operation treatment on the initial key groove image to obtain a key groove image;
the operation processing is morphological open operation, wherein the morphological open operation processing is equivalent to performing corrosion operation and then expansion operation on the initial key groove image, so that discrete points and burrs are eliminated, the discrete points in the initial key groove image can be removed by adopting an operation processing mode, and the most critical information is reserved, namely the key groove image is reserved with the most critical information.
And S23, determining key information of the welding groove based on the key groove image.
Because the key groove image in the embodiment of the application has noise removed and part of discrete points are removed, the key groove image in the embodiment of the application only contains key information of the welding groove.
Fig. 3 is an alternative flowchart of obtaining an initial key groove image in an embodiment of the present application. Referring to fig. 3, the step S21 of performing image noise removal on the welding groove image to obtain an initial key groove image may specifically include:
step S211, carrying out gray level conversion on the welding groove image to obtain a gray level welding groove image corresponding to the welding groove image;
the gray level conversion is a method for changing the gray level value of each pixel in the welding groove image point by point according to a certain target condition and a certain transformation relation. The purpose is to improve the image quality, and the display effect of the welding groove image is clearer. Therefore, the gray-scale welding groove image subjected to gray-scale conversion in the embodiment of the application is displayed more clearly than the welding groove image.
And step S212, carrying out Fourier processing on the gray welding groove image to obtain a groove frequency image.
The fourier processing may specifically include fourier transformation and inverse fourier transformation. The fourier transform may be a Fast Fourier Transform (FFT), the inverse fourier transform is an inverse fast fourier transform, and the FFT algorithm is adopted to greatly reduce the number of multiplications required for computing the discrete fourier transform by a computer, and particularly, the more the number of sampling points N that are transformed, the more significant the saving of the computation amount of the FFT algorithm, and the inverse fast fourier transform is the same.
In one embodiment, when the welding groove produces strong reflection interference, the strong reflection area needs to be removed, and the gray scale welding groove image is processed by adopting Fourier transformation, and the groove frequency diagram is obtained by adopting Fourier transformation. Specifically, the fourier transform can transform the gray welding groove image in the spatial domain and the frequency domain, the frequency map in the frequency domain can just reflect the intensity of the gray change of the image in the spatial domain, and the gray transform is generally strong in the strong reflection part, the gray value of the strong reflection part can be set to 0 in the groove frequency map to remove the strong reflection region, and then the inverse fourier transform is adopted to obtain the groove frequency image,
in other embodiments, when the welding groove image does not encounter strong reflection interference, to avoid the strong reflection area that may be encountered in real-time video acquisition, the image may also be subjected to a fast fourier transform and inverse fast fourier transform process as described above to remove the strong reflection area that may occur.
And step S213, conducting guide filtering treatment on the groove frequency image to obtain a filtered groove image, wherein the guide filtering treatment is used for smoothing the image edge of the groove frequency image and removing noise outside the image edge.
The guiding filtering treatment enables gradients of the filtered groove image to be similar to the groove frequency image as far as possible, and enables gray scales (brightness) of the filtered groove image to be similar to the groove frequency image, so that image edges are reserved, and noise is filtered.
Specifically, the filtered groove image processed as described above still has small noise points, which may interfere with the image processing process, and thus requires a filtering process to remove noise. Because the key point in the welding groove is the detail of the edge of the groove, in the embodiment, a guiding filtering mode is adopted, the edge area only carries out proportional conversion on the image, the proportional relation of the gradient is kept unchanged, and the method has a good keeping effect on the edge of the image and can remove noise.
Step S214, performing binarization processing on the filtered groove image to obtain an initial key groove image, wherein the binarization processing is used for removing interference information in the filtered groove image and reserving key information in the filtered groove image.
The filtered groove image subjected to the guide filtering treatment still has astigmatism with a few small gray values near the edge of the image, the image can be subjected to binarization treatment to remove the interference information, and key information can be kept as much as possible by adopting the binarization treatment.
In an alternative embodiment, the threshold range corresponding to the binarization process is determined by looking at the gray level histogram. As shown in fig. 4, fig. 4 is an optional schematic diagram of a gray level histogram in the embodiment of the present application, where the minimum value in the channel 1 in the gray level histogram is 125 and the maximum value in the channel 1 is 242, and for this reason, the threshold range corresponding to the binarization processing may be determined to be 125-242 according to the gray level histogram, which is available through fig. 4.
After the initial key groove image is obtained, the initial key groove image can be further processed, and then the key groove image is obtained. Fig. 5 is an alternative flowchart of obtaining a key groove image in an embodiment of the present application. Step S22, performing an operation on the initial key groove image to obtain a key groove image, as shown in fig. 5, may specifically include:
step S221, performing open operation processing on the initial key groove image to obtain a first processed key groove image, wherein the open operation processing comprises expansion processing and corrosion processing.
Even after the foregoing step S214, some fine noise points remain after the binarization processing, which may affect the subsequent connected domain segmentation, and the image is continuously subjected to morphological open operation processing in the embodiment of the present application, so that the fine noise points can be effectively removed.
And step S222, performing image region filling treatment on the first treatment key groove, and determining a second treatment key groove image.
In order to keep the key information in the key groove image as much as possible, the image area filling processing is performed on the first processing key groove. After the binarization process, the details of the straight line region may be lost, and the straight line region may be divided into a plurality of connected domains, so that the contents in the straight line region may be filled and the small connected domains may be merged into the surrounding connected domains to determine the image after the image region filling process as the second process key groove image.
In a further embodiment, reference is made to fig. 6 for specific steps in order to obtain key information of the welding groove. Fig. 6 is an alternative flow chart of obtaining key information for a weld groove in an embodiment of the present application. Referring to fig. 6, in step S23, determining, based on the key groove image, key information of the welding groove may specifically include:
and step S231, converting the key region in the second processing key groove image into an initial key line by using a gauss operator.
It should be noted that, the gauss operator is used for extracting the main line of the key area in the second processed key groove image, so that an initial key line is obtained according to the direction of the main line, and the initial key line generally has a certain width.
Step S232, determining contour lines in the initial key lines by using xld operators;
since the initial key line generally has a certain width, the contour line corresponding to the initial key line is relatively easy to obtain. The interference of the tiny line segments in the initial key line can be removed through a xld operator.
And step S233, determining the disconnection point of the line segment in the contour line as a key point of the welding groove.
And step S234, determining key information of the welding groove based on the key points of the welding groove.
In one embodiment, the above process may be described as: in order to repair broken lines in the key groove image, a gauss operator can be used for extracting first, a key region in the key groove image is converted into lines, at the moment, small line segment interference remains in the picture, screening and removal are carried out according to a xld operator, and a final processing result is shown in fig. 7. Fig. 7 is a schematic diagram of a key portion of a line that is basically reserved in the embodiment of the present application, in fig. 7, three line segments exist, and a break point exists between any two line segments, where the break point of the line segment can be determined to be a key point of a welding groove.
After determining the key point of the welding groove, determining pixel coordinate information and depth information of the key point, thereby determining three-dimensional coordinate information of the welding groove in a world coordinate system, and determining the three-dimensional coordinate information as the key information of the welding groove.
In an alternative embodiment, the camera position for acquiring the image of the welding groove may be determined first, the camera plane corresponding to the camera position is determined, and the laser plane defined by the structural light sensor is determined, so that the three-dimensional coordinate information under the world coordinate system is obtained through the coordinate system formed by the camera plane and the laser plane.
Optionally, in order to obtain key information of the welding groove, a camera position of the camera needs to be calibrated, and a camera internal reference matrix corresponding to the camera is determined, wherein the camera internal reference matrix comprises parameters of the camera.
In an alternative embodiment, the camera position of the camera is calibrated in a preset checkerboard calibration plate, and a camera plane corresponding to the camera is determined, wherein the preset checkerboard calibration plate is parallel to the camera plane.
Alternatively, camera calibration may be based on Matlab. Specifically, 7*9 checkerboard calibration plates with side lengths of 3.9mm are adopted, and fig. 8 is a schematic diagram of part of calibration photos in the embodiment of the application. As shown in fig. 8, the effect diagram is shown by 20 photos with different postures, and of course, in practical application, there may be more photos with postures, which will not be described here.
Alternatively, the calibration visualization result of Matlab after camera calibration can be seen in fig. 9. FIG. 9 is a graph of calibration visualization results in an embodiment of the present application. In which, as shown in fig. 9, the horizontal axis Images is used to represent the number of pictures, and meannerrorinpixels is the pixel average error of a single image.
The embodiment of the application also provides an effect diagram of camera external parameter visualization. As shown in fig. 10, in three coordinate systems of the X-axis, the Y-axis and the Z-axis, the position of each shot of the photo with respect to the camera can be intuitively seen, and the left side of the figure is 20 photos with different postures, which can be numbered 1-20 respectively.
Further, table 1 shows the internal parameters of the camera in the embodiment of the present application.
TABLE 1
Wherein fx is the image distance of the camera in the X axis, fy is the image distance of the camera in the Y axis, cx is the coordinate of the camera in the pixel coordinate system, cy is the coordinate of the camera in the pixel coordinate system, K1 is the distortion parameter of the camera in the X axis, and K2 is the distortion parameter of the camera in the Y axis.
The camera internal reference matrix K corresponding to the camera internal reference may be as follows:
further, a laser plane is calibrated based on the structure light sensor, and a laser plane equation corresponding to the laser plane is obtained.
In one embodiment, three images based on laser illumination from a structured light sensor may be used for the determination. The specific implementation mode is as follows:
(1) The camera plane and the plane where the calibration plate is positioned are kept horizontal, three pictures with laser irradiation at different heights are respectively taken, and the heights from the camera plane corresponding to the three pictures to the plane of the calibration plate are different;
(2) Pixel values of one or more points irradiated by laser in any picture are read randomly and converted into a corresponding camera coordinate system.
(3) And fitting a laser plane equation by using a least square method.
After the camera reference matrix and the laser plane equation are obtained, pixel coordinate information corresponding to the key points of the welding groove can be determined based on the camera reference matrix, and depth information corresponding to the key points of the welding groove can be determined based on the laser plane equation.
Further, referring to fig. 11, the determining, based on the pose keypoints of the welding robot, pose keypoint information corresponding to the welding robot includes:
step S91, calibrating the tail end of at least one mechanical arm of the welding robot as a gesture key point;
the tail end of one mechanical arm of the welding robot can be marked as a gesture key point; in other embodiments, the ends of the two mechanical arms of the welding robot may be calibrated as the gesture key points, and the invention is not limited herein.
Step S92, determining the association relation between the gesture key points and the camera positions of the cameras.
And step S93, determining the posture key point information corresponding to the welding robot based on the association relation and the posture key points of the welding robot.
And keeping the calibration plate motionless, wherein the relation in the corresponding formula 1 exists for the first gesture and the second gesture in the moving process of the mechanical arm.
Wherein Robot1 is base information of the welding Robot in the first posture, end1 is End information of the mechanical arm in the first posture, and Camera1 is Camera position information in the first posture; robot2 is base information of the welding Robot in the second pose, end2 is End information of the mechanical arm in the second pose, and Camera2 is Camera position information in the second pose.
Furthermore, introducing a homogeneous transformation matrix of the gesture, determining an association relationship between the gesture key points and the camera positions of the camera, wherein in an alternative implementation, the gesture key points may be the tail ends of the mechanical arms, as shown in formula 2:
as shown in formula 2, T ejj A translation matrix for the relative posture of the tail end of the mechanical arm, R ejj R is a rotation matrix of relative gestures of the tail end of the mechanical arm ce R is a rotation matrix between the camera and the tail end of the mechanical arm cjj Rotational moment for relative pose of cameraArray, T cjj Is a translation matrix of the relative pose of the camera.
Based on the above, the association relationship between the gesture key points and the camera positions of the camera may also be obtained by using a TSAI algorithm.
Based on the above formula 2, the rotation matrix and the translation matrix in the formula are solved by adopting the classical TSAI method in Matlab.
Referring to fig. 12, the solution process of the TSAI method may be to convert a rotation matrix into a rotation vector using the rodgers formula; further carrying out vector normalization processing on the rotation axis to obtain a unit rotation axis and a rotation angle; representing the gesture transformation with the rodrich parameter; further, determining an initial rotation variable by adopting a forward kinematic formula of the mechanical arm and a machine vision method; and finally, calculating a rotation matrix and a translation matrix.
In an alternative embodiment of the invention, the conversion matrix X between the end of the mechanical arm and the camera is:
and when the gesture key points of the welding robot are obtained, the pose of the mechanical arm is read, and the gesture key point information corresponding to the welding robot is solved by using a formula 3.
Wherein X is w 、Y w 、Z w X is coordinate information of the mechanical arm in a world coordinate system c 、Y c 、Z c The method is characterized in that the method comprises the steps that coordinate information of a mechanical arm under a camera coordinate system is obtained by converting a mechanical arm rotation vector into a rotation matrix, D is a pose matrix, and X is a conversion matrix from a camera obtained by hand-eye calibration to the tail end of the mechanical arm.
Based on the welding gun posture adjustment method of the welding robot in the embodiment of the application, the measurement accuracy measured by the experiment is shown in table 2, in the experiment, the measurement results of the welding guns with welding groove numbers of 1, 2, 3, 4 and 5 are used for illustration, as shown in table 2, the measurement values of the five groups of welding guns have certain errors from the actual values, but compared with the mode of welding by only using the staff, the staff can adjust the forward/backward angle, the lateral angle and the spin angle of the welding gun, and the conventional welding gun posture adjustment requirement is met.
TABLE 2
The foregoing describes a number of embodiments provided by embodiments of the present application, and the various alternatives presented by the various embodiments may be combined, cross-referenced, with each other without conflict, extending beyond what is possible, all of which may be considered embodiments disclosed and disclosed by embodiments of the present application.
Although the embodiments of the present application are disclosed above, the present application is not limited thereto. Various changes and modifications may be made by one skilled in the art without departing from the spirit and scope of the invention, and the scope of the invention shall be defined by the appended claims.
Claims (4)
1. A welding gun posture adjusting method of a welding robot is characterized by comprising the following steps:
determining key information of the welding groove based on the welding groove image;
determining gesture key point information corresponding to a welding robot, wherein the gesture key point information is information corresponding to gesture key points, and the gesture key points are pre-marked on a mechanical arm of the welding robot;
displaying the key information of the welding groove and the key point information of the welding gun posture of the welding robot in real time so as to enable a worker to adjust the welding gun posture of the welding robot;
wherein, based on the welding groove image, determining key information of the welding groove comprises:
image interference removal is carried out on the welding groove image, and an initial key groove image is obtained;
performing operation treatment on the initial key groove image to obtain a key groove image;
determining key information of the welding groove based on the key groove image;
determining gesture key point information corresponding to the welding robot comprises the following steps:
determining gesture key point information corresponding to the welding robot based on gesture key points of the welding robot;
the step of performing operation processing on the initial key groove image to obtain a key groove image comprises the following steps:
performing open operation processing on the initial key groove image to obtain a first processed key groove image, wherein the open operation processing comprises expansion processing and corrosion processing; performing image region filling treatment on the first treatment key groove, and determining a second treatment key groove image;
the determining key information of the welding groove based on the key groove image comprises:
converting the key region in the second processed key groove image into an initial key line by using a gauss operator; determining contour lines in the initial key lines by using a xld operator; determining the breaking point of the line segment in the contour line as a key point of the welding groove; determining key information of the welding groove based on the key points of the welding groove;
the determining key information of the welding groove based on the key points of the welding groove comprises the following steps: determining pixel coordinate information and depth information corresponding to key points of the welding groove; determining three-dimensional coordinate information of the welding groove in a world coordinate system based on the pixel coordinate information and the depth information, wherein the three-dimensional coordinate information of the welding groove is key information of the welding groove;
the determining of the gesture key point information corresponding to the welding robot based on the gesture key point of the welding robot comprises the following steps: calibrating the tail end of at least one mechanical arm of the welding robot as a gesture key point; determining an association relationship between the gesture key points and camera positions of the cameras; and determining the posture key point information corresponding to the welding robot based on the association relation and the posture key points of the welding robot.
2. The welding gun posture adjustment method of a welding robot of claim 1, wherein the performing image disturbance removal on the welding groove image to obtain an initial key groove image comprises:
performing gray level conversion on the welding groove image to obtain a gray level welding groove image corresponding to the welding groove image;
performing Fourier transform processing on the gray welding groove image to obtain a groove frequency diagram corresponding to the welding groove image;
conducting guide filtering treatment on the groove frequency map to obtain a filtered groove image, wherein the guide filtering treatment is used for removing noise in the groove frequency map;
and performing binarization processing on the filtered groove image to obtain an initial key groove image, wherein the binarization processing is used for removing interference information in the filtered groove image and retaining key information in the filtered groove image.
3. The welding robot welding gun posture adjustment method of claim 1, wherein before determining the pixel coordinate information and the depth information corresponding to the key point of the welding groove, comprising:
calibrating the camera position of a camera, and determining a camera internal reference matrix corresponding to the camera, wherein the camera internal reference matrix comprises internal references of the camera;
calibrating a laser plane based on a structure light sensor, and acquiring a laser plane equation corresponding to the laser plane;
the determining pixel coordinate information and depth information corresponding to key points of the welding groove comprises the following steps:
and determining pixel coordinate information corresponding to the key points of the welding groove based on the camera internal reference matrix, and determining depth information corresponding to the key points of the welding groove based on the laser plane equation.
4. The welding gun pose adjustment method of a welding robot according to claim 1, wherein the determining an association relationship between the pose key point and a camera position of a camera comprises:
and acquiring the association relation between the gesture key points and the camera positions of the camera by using a TSAI algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211011654.5A CN115283905B (en) | 2022-08-23 | 2022-08-23 | Welding gun posture adjusting method of welding robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211011654.5A CN115283905B (en) | 2022-08-23 | 2022-08-23 | Welding gun posture adjusting method of welding robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115283905A CN115283905A (en) | 2022-11-04 |
| CN115283905B true CN115283905B (en) | 2024-03-26 |
Family
ID=83833056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211011654.5A Active CN115283905B (en) | 2022-08-23 | 2022-08-23 | Welding gun posture adjusting method of welding robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115283905B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115890053B (en) * | 2023-03-02 | 2023-08-18 | 成都熊谷加世电器有限公司 | Internal welding machine alignment method and device, internal welding machine and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102794763A (en) * | 2012-08-31 | 2012-11-28 | 江南大学 | Systematic calibration method of welding robot guided by line structured light vision sensor |
| CN107649802A (en) * | 2017-09-28 | 2018-02-02 | 中国东方电气集团有限公司 | A kind of laser vision welded seam tracing system and scaling method |
| CN109604777A (en) * | 2017-12-07 | 2019-04-12 | 淮安信息职业技术学院 | Welding seam traking system and method based on laser structure light |
| CN112508926A (en) * | 2020-12-16 | 2021-03-16 | 广州大学 | Method, system and device for detecting surface scratches of metal stamping part and storage medium |
| CN114473153A (en) * | 2022-04-01 | 2022-05-13 | 电王精密电器(北京)有限公司 | Oil-gas long-distance pipeline welding system and method |
| CN114792337A (en) * | 2022-03-04 | 2022-07-26 | 清华大学 | High-precision detection method and device for welding groove size and relative pose of welding gun |
-
2022
- 2022-08-23 CN CN202211011654.5A patent/CN115283905B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102794763A (en) * | 2012-08-31 | 2012-11-28 | 江南大学 | Systematic calibration method of welding robot guided by line structured light vision sensor |
| CN107649802A (en) * | 2017-09-28 | 2018-02-02 | 中国东方电气集团有限公司 | A kind of laser vision welded seam tracing system and scaling method |
| CN109604777A (en) * | 2017-12-07 | 2019-04-12 | 淮安信息职业技术学院 | Welding seam traking system and method based on laser structure light |
| CN112508926A (en) * | 2020-12-16 | 2021-03-16 | 广州大学 | Method, system and device for detecting surface scratches of metal stamping part and storage medium |
| CN114792337A (en) * | 2022-03-04 | 2022-07-26 | 清华大学 | High-precision detection method and device for welding groove size and relative pose of welding gun |
| CN114473153A (en) * | 2022-04-01 | 2022-05-13 | 电王精密电器(北京)有限公司 | Oil-gas long-distance pipeline welding system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115283905A (en) | 2022-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Shan et al. | A stereovision-based crack width detection approach for concrete surface assessment | |
| US9091528B2 (en) | System and method for object measurement | |
| He et al. | A critical review for machining positioning based on computer vision | |
| CN108257171A (en) | Car radar assembling aperture detection method based on light vision | |
| JP2021168143A (en) | System and method for efficiently scoring probe in image by vision system | |
| CN115283905B (en) | Welding gun posture adjusting method of welding robot | |
| CN112381791A (en) | Bolt looseness detection method based on 3D point cloud | |
| CN107388991B (en) | A kind of more fillet axial workpiece radius of corner measurement methods in end face | |
| CN111257329A (en) | Smartphone camera defect detection method and detection system | |
| CN107561736B (en) | LCD defect detection method based on Fourier transform and Hough transform | |
| CN110634128A (en) | Ball pin size detection method and device, computer equipment and storage medium | |
| CN113971669A (en) | Three-dimensional detection system applied to pipeline damage identification | |
| CN115855955A (en) | Mold surface structure defect detection device and method based on multi-beam laser | |
| CN113705564B (en) | Pointer type instrument identification reading method | |
| CN109015632A (en) | A kind of robot hand end localization method | |
| WO2021179400A1 (en) | Computer vision-based adaptive measurement system and method for geometric parameters in assembly process | |
| JP2008116206A (en) | Apparatus, method, and program for pattern size measurement | |
| CN112902869A (en) | Method and device for adjusting laser plane of rail profile measuring system | |
| CN115753791B (en) | Defect detection method, device and system based on machine vision | |
| CN116804359A (en) | Virtual rock core imaging equipment and method based on forward-looking panoramic drilling shooting | |
| CN116258838A (en) | Intelligent visual guiding method for duct piece mold clamping system | |
| CN115641326A (en) | Sub-pixel size detection method and system for ceramic antenna PIN needle image | |
| CN115147764A (en) | Pipe die bolt identification and positioning method based on multi-view vision | |
| CN115035071A (en) | Visual detection method for black spot defect of PAD light guide plate | |
| CN113052823A (en) | Oil level and oil color detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |