CN115358965A - Welding deformation adaptive linear weld grinding track generation method and device - Google Patents

Welding deformation adaptive linear weld grinding track generation method and device Download PDF

Info

Publication number
CN115358965A
CN115358965A CN202210805549.2A CN202210805549A CN115358965A CN 115358965 A CN115358965 A CN 115358965A CN 202210805549 A CN202210805549 A CN 202210805549A CN 115358965 A CN115358965 A CN 115358965A
Authority
CN
China
Prior art keywords
dimensional
point cloud
welding
line
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210805549.2A
Other languages
Chinese (zh)
Inventor
陈毅然
陈新度
吴磊
张宇
甘胜斯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202210805549.2A priority Critical patent/CN115358965A/en
Publication of CN115358965A publication Critical patent/CN115358965A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration by the use of local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/168—Segmentation; Edge detection involving transform domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30152—Solder

Abstract

The invention relates to a welding deformation adaptive linear weld grinding track generation method, which comprises the following steps: acquiring multi-frame two-dimensional images of a target welding plate by a color RGB camera in a kinect V2 camera, and acquiring multi-frame three-dimensional point cloud images of the target welding plate by an infrared ir camera; according to the obtained multiple frames of the two-dimensional images and the three-dimensional point cloud images, performing three-dimensional linear weld joint crude extraction to obtain three-dimensional weld joint information; carrying out noise reduction processing on the three-dimensional welding seam information, and eliminating outliers and unreal welding seam track points to obtain a three-dimensional welding seam point cloud pi; generating path points which can be executed by actual production of the robot according to the three-dimensional welding line point cloud pi; and controlling the robot to polish the welding line according to the path points. The whole process of the method is completed in a full-automatic mode, a series of problems caused by manual processing of the welding seam can be avoided, the method is high in accuracy when welding seam identification and polishing path planning are carried out, and intelligent polishing of the welding seam can be completed efficiently and stably.

Description

Welding deformation self-adaptive linear weld polishing track generation method and device
Technical Field
The invention relates to the technical field of robot visual positioning, in particular to a method and a device for generating a polishing track of a linear welding seam with self-adaptive welding deformation.
Background
Welding technology is widely applied to enterprise production, and welding seams after welding are polished to meet requirements of plates on size, flatness and attractiveness. Because the required control factor requirement is more in the welding seam process of polishing, and the scene is complicated changeable and changeable, utilizes the robot to polish and hardly realizes the required precision, so the welding seam process of polishing all basically solves through the manpower now.
Through the manpower solution on the one hand noise, dust, harmful gas, spark etc. that produce in the process of polishing can influence operation personnel's physical and mental health. The grinding process usually consumes physical strength, the used tools generally run at high speed, and if the tools are careless in the use process, physical damage to the body can be caused to a certain extent; on the other hand, the cost is relatively high, and the standardization degree is difficult to unify.
Disclosure of Invention
The invention aims to solve at least one of the defects of the prior art and provides a method and a device for generating a polishing track of a linear welding seam with adaptive welding deformation.
In order to achieve the above object, the present invention adopts the following technical means,
specifically, the method for generating the polishing track of the linear welding seam with the self-adaptive welding deformation comprises the following steps:
acquiring a multi-frame two-dimensional image of a target welding plate by a color RGB camera in a kinect V2 camera, and acquiring a multi-frame three-dimensional point cloud image of the target welding plate by an infrared ir camera;
according to the obtained multiple frames of the two-dimensional image and the three-dimensional point cloud image, performing three-dimensional linear weld joint crude extraction to obtain three-dimensional weld joint information;
denoising the three-dimensional welding seam information, and removing outliers and unreal welding seam track points to obtain a three-dimensional welding seam point cloud pi;
generating path points which can be executed by actual production of the robot according to the three-dimensional welding line point cloud pi;
and controlling the robot to polish the welding line according to the path points.
Further, specifically, the three-dimensional linear weld information is roughly extracted in the following way,
performing gray processing on the obtained two-dimensional image, performing Gaussian filtering, extracting line information by canny edge detection, extracting straight lines in the line information by using Hough straight line transformation to obtain a plurality of straight lines, extracting a target straight line based on a BDSCAN idea to obtain a welding line, and obtaining the position of the welding line in the two-dimensional image;
calibrating internal parameters and external parameters of a color RGB camera and an infrared ir camera, specifically comprising,
let P ir ＝[X ir Y ir Z ir ]Is the spatially non-homogeneous coordinate, p, of a point under the ir camera coordinate system ir ＝[u ir v ir 1]Is p on image plane of ir camera ir Projected point of (H) ir The method is characterized in that the method is an internal reference matrix of the ir depth camera calibrated in advance, and the following relation exists:
Z ir P ir ＝H ir P ir (1)
the corresponding relation of the RGB camera can be obtained in the same way:
Z rgb P rgb ＝H rgb P rgb (2)
is provided with
Figure BDA0003737122110000021
Is an external reference of the infrared camera,
Figure BDA0003737122110000022
is an external reference of a color camera,
Figure BDA0003737122110000023
for the spatial pose transformation of the color camera coordinate system relative to the infrared camera coordinate system, the following are provided:
Figure BDA0003737122110000024
Figure BDA0003737122110000025
Figure BDA0003737122110000026
substituting (1) and (2) into (5) can eliminate P rgb And P ir ：
Figure BDA0003737122110000027
And projecting the position of the welding seam in the two-dimensional image into the three-dimensional image according to the established corresponding relation of the pixel points between the color RGB camera and the infrared ir camera.
Further, the process of obtaining the three-dimensional weld point cloud pi specifically includes the following steps,
and simplifying the three-dimensional weld information through a DBSCAN algorithm to obtain a three-dimensional weld point cloud pi, and recording as a weld point cluster line _2.
Further, specifically, generating path points that can be actually executed by the robot according to the three-dimensional weld point cloud pi includes the following steps,
step 510, calculating the average value n of the unit point normal vectors of the track points in the line _2 1 And fitting a straight line based on line _2Line unit direction vector a 1 Cross over n 1 Calculating a three-dimensional unit vector m: m = n 1 ×a 1 ；
Step 520, based on the unit vector m, each point p in the point cloud cluster line _2 i Deviation is +/-1.5 times in the m vector direction, and a corresponding domain point p is calculated i1 ＝p i +1.5*m、p i2 ＝p i 1.5m, and obtaining a neighborhood point cluster line _2+ and line _2 of the line _ 2;
step 530, respectively calculate p i1 And p i2 Nearest neighbor point p on target weld plate point cloud i1t 、p i2t Then p is calculated i1t 、p i2t Point normal vector n i1t 、n i2t Then, the polishing position and the polishing attitude of the real polishing track point are respectively: p i ＝(p i1t +p i2t )/2；N i ＝(n i1t +n i2t ) At the moment, all points in the Pi are recorded as a point cloud cluster line _3;
step 540, sorting the polishing positions and postures of the polishing track points based on the point cloud cluster line _3 to obtain a sorted point cloud cluster line _4;
step 550, utilizing the B-spline curve to determine the position P in the point cloud cluster line _4 i And attitude N i And smoothing the information to obtain a point cloud cluster line _5.
Step 560, processing the point cloud cluster line _5 by using the B-spline curve again to obtain point cloud cluster line _6 with equal point intervals, and synchronously fitting normal vectors;
and 570, calculating a kinematic inverse solution according to the point cloud cluster line _6, and setting a sixth axis as a fixed value in an inverse solution result to obtain a path point which can be executed by the actual production of the robot.
The invention also provides a welding deformation self-adaptive linear weld grinding track generation device, which comprises:
the image acquisition module is used for acquiring multi-frame two-dimensional images of the target welding plate through a color RGB camera in a kinect V2 camera and acquiring multi-frame three-dimensional point cloud images of the target welding plate through an infrared ir camera;
the three-dimensional welding seam information acquisition module is used for crudely extracting a three-dimensional linear welding seam according to the acquired multi-frame two-dimensional image and three-dimensional point cloud image to acquire three-dimensional welding seam information;
the simplification module is used for carrying out noise reduction processing on the three-dimensional welding seam information and eliminating outliers and unreal welding seam track points to obtain a three-dimensional welding seam point cloud pi;
the path point generating module is used for generating path points which can be executed by actual production of the robot according to the three-dimensional welding line point cloud pi;
and the execution module is used for controlling the robot to polish the welding seam according to the path points.
The invention also provides a welding seam polishing platform, which applies any one of the above welding deformation self-adaptive linear welding seam polishing track generation methods, comprising,
a robot;
the kinect V2 camera is used for acquiring multi-frame two-dimensional images of the target welding plate through a color RGB camera and acquiring multi-frame three-dimensional point cloud images of the target welding plate through an infrared ir camera;
the platform is used for fixing the target welding plate;
and the industrial personal computer based on the ROS robot control system is used for receiving the multiframe two-dimensional images and the three-dimensional point cloud images transmitted by the kinect V2 camera and carrying out polishing track planning and execution based on the multiframe two-dimensional images and the three-dimensional point cloud images.
The invention also provides a computer-readable storage medium, which stores a computer program, and is characterized in that the computer program is executed by a processor to implement the steps of the welding deformation adaptive straight-line weld grinding track generation method.
The invention has the beneficial effects that:
according to the method for generating the polishing track of the linear welding seam with the self-adaptive welding deformation, the welding seam is identified in a two-dimensional space at first, then the welding seam in the two-dimensional space is projected to the three-dimensional space, the three-dimensional space welding seam is obtained, and finally the three-dimensional space welding seam is processed, so that the path point which can be executed by the actual production of the robot is generated, the whole process is completed in a full-automatic mode, a series of problems caused by manual processing of the welding seam can be avoided, the accuracy is high when the welding seam identification and the polishing path planning are carried out, and the intelligent polishing of the welding seam can be completed efficiently and stably.
Drawings
The foregoing and other features of the present disclosure will become more apparent from the detailed description of the embodiments shown in conjunction with the drawings in which like reference characters designate the same or similar elements throughout the several views, and it is apparent that the drawings in the following description are merely some examples of the present disclosure and that other drawings may be derived therefrom by those skilled in the art without the benefit of any inventive faculty, and in which:
FIG. 1 is a flow chart of a method for generating a polishing track of a straight weld according to the invention, wherein the method is adaptive to welding deformation;
FIG. 2 is a flowchart illustrating an implementation of the method for generating a polishing track of a linear weld according to the present invention;
FIG. 3 is a diagram illustrating an exemplary target weld plate in one embodiment of the method for generating a welding deformation adaptive linear weld grinding track according to the present invention;
FIG. 4 is a schematic diagram of a two-dimensional weld projected to a three-dimensional weld according to the welding deformation adaptive linear weld polishing track generation method of the present invention;
FIG. 5 is a Hough line transformation schematic diagram of the welding deformation adaptive linear weld grinding track generation method of the invention;
fig. 6 is a schematic diagram of a track calculated based on line _2 in an embodiment of the method for generating a linear weld polishing track adaptive to welding deformation according to the present invention;
fig. 7 is a diagram illustrating a rough processing effect of a three-dimensional weld polishing track in an embodiment of a method for generating a linear weld polishing track with adaptive welding deformation according to the present invention;
fig. 8 is a schematic diagram illustrating a grinding track point and a corresponding normal vector drawn based on line _5 in one embodiment of the welding deformation adaptive linear weld grinding track generation method of the present invention;
fig. 9 is a diagram illustrating an improved three-dimensional weld grinding track effect in an embodiment of a method for generating a linear weld grinding track with adaptive welding deformation according to the present invention.
Detailed Description
The conception, the specific structure and the technical effects produced by the present invention will be clearly and completely described in conjunction with the embodiments and the attached drawings, so as to fully understand the objects, the schemes and the effects of the present invention. It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict. The same reference numbers will be used throughout the drawings to refer to the same or like parts.
Referring to fig. 1, 2 and 3, in embodiment 1, the present invention provides a method for generating a welding deformation adaptive linear weld grinding track, including the following steps:
110, acquiring a plurality of frames of two-dimensional images of a target welding plate through a color RGB camera in a kinect V2 camera, and acquiring a plurality of frames of three-dimensional point cloud images of the target welding plate through an infrared ir camera;
120, according to the obtained multiple frames of the two-dimensional images and the three-dimensional point cloud images, performing three-dimensional linear weld joint crude extraction to obtain three-dimensional weld joint information;
step 130, carrying out noise reduction processing on the three-dimensional welding seam information, and eliminating outliers and unreal welding seam track points to obtain a three-dimensional welding seam point cloud pi;
140, generating path points which can be executed by actual production of the robot according to the three-dimensional welding line point cloud pi;
and 150, controlling the robot to polish the welding line according to the path point.
Referring to fig. 4 and 5, as a preferred embodiment of the present invention, specifically, the three-dimensional straight-line bead information is roughly extracted in the following manner,
performing gray processing on the obtained two-dimensional image, performing Gaussian filtering, extracting line information by canny edge detection, extracting straight lines in the line information by using Hough straight line transformation to obtain a plurality of straight lines, extracting a target straight line based on a BDSCAN idea to obtain a welding line, and obtaining the position of the welding line in the two-dimensional image;
it can be seen from the figure that a plurality of straight lines are obtained after hough transform, the straight lines are represented by a straight line cluster L1, the number of the straight lines is represented by size1, only one piece of weld information is needed, and the principle of extracting the straight lines corresponding to the welds is as follows:
in hough line transformation, the line is described by two parameters (as in fig. 5): theta, r. r is the distance from the original point to the straight line, and theta is the included angle between the line and the X axis of the coordinate system. At this time, we can sort the linear cluster L1 based on r, set a coefficient λ 1 (0 < λ 1 and 1), and discard the front (size 1 × λ 1/2) and the end (size 1 × λ 1/2) straight lines after sorting based on r, to obtain a linear cluster L2; carrying out second simplification after the first simplification, and sorting and simplifying the linear cluster L2 based on theta by using the same principle to obtain a linear cluster L3; and finally, averaging parameters theta and r in the linear cluster L3, and enabling a straight line represented by the obtained average value to correspond to the welding line.
In order to enable a robot to polish a welding seam in a real space, the welding seam point information in a three-dimensional space needs to be taken, and a two-dimensional space lacks dimension information relative to the three-dimensional space, so that a corresponding relation between each pixel of an RGB image and a point cloud needs to be carried out, and the following principle is realized:
the kinect V2 camera consists of two cameras, one is a color RBG camera, the other is an infrared ir camera, the ir camera can obtain a three-dimensional point cloud and an infrared gray scale image, and each point of the three-dimensional point cloud corresponds to each pixel point of the infrared gray scale image one by one. The infrared camera is mainly used for point cloud imaging, and the obtained ir gray image has very poor imaging quality, so that the corresponding pixels are projected on the ir image after the RGB camera is used for two-dimensional identification, and the three-dimensional space point location center of the corresponding pixels can be obtained. The internal and external parameters of the two cameras need to be calibrated before projection is performed.
Based on the method, the internal reference and the external reference of the color RGB camera and the infrared ir camera are calibrated, and the method specifically comprises the following steps,
let P ir ＝[X ir Y ir Z ir ]Is the spatial negation of a point under the ir camera coordinate systemHomogeneous coordinate, p ir ＝[u ir v ir 1]Is p on image plane of ir camera ir Projected point of (H) ir The method is characterized in that the method is an internal reference matrix of the ir depth camera calibrated in advance, and the following relation exists:
Z ir p ir ＝H ir P ir (1)
the corresponding relation of the RGB camera can be obtained in the same way:
Z rgb p rgb ＝H rgb P rgb (2)
is provided with
Figure BDA0003737122110000061
Is an external reference of an infrared camera,
Figure BDA0003737122110000062
is an external reference of a color camera,
Figure BDA0003737122110000063
for the spatial pose transformation of the color camera coordinate system relative to the infrared camera coordinate system, the following are provided:
Figure BDA0003737122110000064
Figure BDA0003737122110000065
Figure BDA0003737122110000066
substituting (1) and (2) into (5) can eliminate P rgb And P ir ：
Figure BDA0003737122110000067
According to the established corresponding relation of the pixel points between the color RGB camera and the infrared ir camera, welding is carried outProjecting the position of the seam in the two-dimensional image into the three-dimensional image, finding out the corresponding relation of pixel points of the color camera and the infrared camera, and inputting a certain pixel value p rgb That is to say, the corresponding p can be obtained ir And at the moment, point location information in the three-dimensional space can be obtained.
Referring to fig. 6, 7, 8 and 9, as a preferred embodiment of the present invention, specifically, after acquiring corresponding three-dimensional space weld information (herein, referred to as a weld point cluster line _ 1), the robot cannot perform polishing based on these points, and further processes the point cluster to calculate the information of the point method suitable for polishing. The process of obtaining the three-dimensional welding point cloud pi comprises the following steps,
and simplifying the three-dimensional welding line information through a DBSCAN algorithm to obtain a three-dimensional welding line point cloud pi which is recorded as a welding line point cluster line _2.
As a preferred embodiment of the present invention, specifically, path points which can be actually produced and executed by a robot are generated according to the three-dimensional weld point cloud pi, if a normal vector of a trajectory point is directly calculated by using line _2, the result is as shown in fig. 6, and the actual polishing path pose of the planned robot is as shown in the simulation effect of fig. 7, it can be seen that the robot has a non-uniform running speed and severe chatter, and a safety accident is easily caused in the actual polishing process, and the polishing effect cannot be satisfied, and the solution is as follows: comprises the following steps of (a) preparing a mixture,
step 510, calculating the unit point normal vector average value n of the trace points in the line _2 1 And a unit direction vector a of the straight line fitted based on line _2 1 Cross over n 1 Calculating a three-dimensional unit vector m: m = n 1 ×a 1 ；
Step 520, based on the unit vector m, each point p in the point cloud cluster line _2 is processed i Deviation is +/-1.5 times in the m vector direction, and a corresponding domain point p is calculated i1 ＝p i +1.5*m、p i2 ＝p i 1.5m, and obtaining a line _2+ and a line _ 2-of a neighborhood point cluster of the line _ 2;
step 530, respectively calculate p i1 And p i2 Nearest neighbor point p on target weld plate point cloud i1t 、p i2t Then p is calculated i1t 、p i2t Point normal vector n i1t 、n i2t Then, the polishing position and the polishing attitude of the real polishing track point are respectively: p is i ＝(p i1t +p i2t )/2；N i ＝(n i1t +n i2t ) At the moment, all points in the Pi are recorded as a point cloud cluster line _3;
step 540, sorting the polishing positions and postures of the polishing track points based on the point cloud cluster line _3 to obtain a sorted point cloud cluster line _4;
step 550, utilizing the B-spline curve to determine the position P in the point cloud cluster line _4 i And attitude N i And smoothing the information to obtain a point cloud cluster line _5. The grinding device aims to enable the speed and the acceleration of the robot to be continuous in the grinding process, and a better machining effect is obtained. The point and normal vector information in line _5 at this time is as shown in fig. 8, and is improved a lot compared to fig. 6.
Step 560, processing the point cloud cluster line _5 by using the B-spline curve again to obtain point cloud cluster line _6 with equal point intervals, and synchronously fitting normal vectors; in the graph 8, the density degrees of the linear point clouds are different, and when the ROS-I robot control system plans to move, the operation time and the distance of each section are not in direct proportion, so that the speed of the planning process is not constant and is suddenly fast or suddenly slow. At this time, the B-spline is required to obtain the point cloud cluster line _6 with the same point interval again, and the normal vectors are also fitted synchronously, as shown in fig. 9.
And 570, calculating a kinematic inverse solution according to the point cloud cluster line _6, and setting a sixth axis as a fixed value in an inverse solution result to obtain a path point which can be executed by the actual production of the robot.
The initial grinding track has information of a plurality of pairs of points and normal vectors, each point corresponds to a unique position, the normal vectors correspond to the tail end postures of an infinite number of robots, so that each pair of normal vectors of the points corresponds to countless inverse solution states of the robots, and an optimal inverse solution is uniquely determined to come out:
the robot end attitude information, namely the spatial rotation transformation relation of the tool coordinate system relative to the polar coordinate system, is represented in the form of an axis angle angel-aix for the convenience of solving. And respectively cross-multiplying the Z-axis unit vector (0, 1) and the normal vector in the track under the base coordinate system to obtain a corresponding rotating axis aix, wherein the Z-axis unit vector and each normal vector share an included angle, namely the rotating angle. The axis angle can uniquely determine the spatial rotation transformation relation, but the sixth axis of the robot is required to be fixed at a zero position in the grinding process, and the grinding track attitude information does not meet the application condition.
The polishing system is established based on ROS-I, the robot kinematics solution library trakIK can utilize DH parameter information of the robot in the ROS to solve the inverse solution of the robot, a group of current joint space postures of the robot can be set, the target posture in the optimal joint space is solved inversely based on the current joint space postures, the single solution speed is within 0.5ms, and the use condition is completely met.
And solving track information in the corresponding joint space based on the initial polishing track inverse solution, setting a sixth joint value as 0, and solving a corresponding final polishing track point in a Cartesian space by using a KDL positive kinematics solution library. And finally, based on the final track point, utilizing Moveit in ROS to carry out Cartesian space path planning and executing a polishing task.
One implementation of step 540 is as follows,
the Pi and the Ni corresponding to the Pi are also a cluster of disordered points, and the track planning cannot be carried out on the point cluster, so that a quick ordering method of the linear point cloud is provided: a straight Line is fitted based on the point cloud cluster Line _3, and is represented by a straight Line point Line _ p and a normal vector Line _ n = (Line _ nx, line _ ny, line _ nz). The following are the pseudo codes for point cloud sorting (theta _ X, theta _ Y represent included angles between Line _ n and the X-axis and Y-axis, respectively):
if(0°<＝theta_x<＝60°||120°<＝theta_x<＝180°)
{ obtaining a point Line _ p2 when x =100 on a straight Line, calculating the distance between all points Pi of a point cloud cluster Line _3 and the point Line _ p2, sorting the point cloud cluster Line _3 according to the distance, synchronously sorting corresponding normal vectors Ni, and marking the sorted point cloud cluster as Line _4;
}else if(0°<＝theta_y<＝60°||120°<＝theta_y<＝180°)
{ acquiring a point Line _ p3 when y =100 on a straight Line, calculating the distance between all points Pi of a point cloud cluster Line _3 and the point Line _ p3, sorting the point cloud cluster Line _3 according to the distance, synchronously sorting corresponding normal vectors Ni, and marking the sorted point cloud cluster as Line _4;
}else
{ obtaining a point Line _ p4 when z =100 on a straight Line, calculating the distance between all points Pi of a point cloud cluster Line _3 and the point Line _ p4, sorting the point cloud cluster Line _3 according to the distance, synchronously sorting corresponding normal vectors Ni, and marking the sorted point cloud cluster as Line _4;
}
note: the point in the point cloud is in meters, and the point on the straight line obtained when x, y or z takes 100 is away from the point cloud cluster line _3, so the method has very high robustness in sorting the point cloud.
The invention also provides a welding deformation self-adaptive linear weld grinding track generation device, which comprises:
the image acquisition module is used for acquiring multi-frame two-dimensional images of the target welding plate through a color RGB camera in a kinect V2 camera and acquiring multi-frame three-dimensional point cloud images of the target welding plate through an infrared ir camera;
the three-dimensional welding seam information acquisition module is used for crudely extracting a three-dimensional linear welding seam according to the acquired multi-frame two-dimensional image and three-dimensional point cloud image to acquire three-dimensional welding seam information;
the simplification module is used for carrying out noise reduction processing on the three-dimensional welding seam information and eliminating outliers and unreal welding seam track points to obtain a three-dimensional welding seam point cloud pi;
the path point generating module is used for generating path points which can be executed by actual production of the robot according to the three-dimensional welding line point cloud pi;
and the execution module is used for controlling the robot to polish the welding seam according to the path point.
The invention also provides a welding seam polishing platform, which applies any one of the above linear welding seam polishing track generation methods with self-adaptive welding deformation, and comprises,
a robot;
the kinect V2 camera is used for acquiring multi-frame two-dimensional images of the target welding plate through a color RGB camera and acquiring multi-frame three-dimensional point cloud images of the target welding plate through an infrared ir camera;
the platform is used for fixing the target welding plate;
and the industrial personal computer based on the ROS robot control system is used for receiving the multiframe two-dimensional images and the three-dimensional point cloud images transmitted by the kinect V2 camera and carrying out polishing track planning and execution based on the multiframe two-dimensional images and the three-dimensional point cloud images.
In the embodiment, the welding seam is firstly identified in a two-dimensional space, then the welding seam in the two-dimensional space is projected to the three-dimensional space, the welding seam in the three-dimensional space is obtained, and finally the welding seam in the three-dimensional space is processed, so that the path points which can be executed by the actual production of the robot are generated, the whole process is completed in a full-automatic mode, a series of problems caused by manual processing of the welding seam can be avoided, the accuracy is high when the welding seam identification and polishing path planning are carried out, and the intelligent polishing of the welding seam can be efficiently and stably completed.
The invention also provides a computer-readable storage medium, which stores a computer program, and is characterized in that the computer program is executed by a processor to implement the steps of the welding deformation adaptive straight-line weld grinding track generation method.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer-readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium and can implement the steps of the above-described method embodiments when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or system capable of carrying said computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium includes content that can be suitably increased or decreased according to the requirements of legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunication signals according to legislation and patent practice.
While the present invention has been described in considerable detail and with particular reference to several of these embodiments, it is not intended to be limited to any such details or embodiments or any particular embodiment, but rather it is to be construed as effectively covering the intended scope of the invention by providing a broad, potential interpretation of such claims in view of the prior art with reference to the appended claims. Furthermore, the foregoing describes the invention in terms of embodiments foreseen by the inventor for which an enabling description was available, notwithstanding that insubstantial modifications of the invention, not presently foreseen, may nonetheless represent equivalents thereto.
The above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment, and the present invention shall fall within the protection scope of the present invention as long as the technical effects of the present invention are achieved by the same means. The technical solution and/or the embodiments thereof may be variously modified and varied within the scope of the present invention.

Claims (7)

1. The welding deformation self-adaptive linear weld grinding track generation method is characterized by comprising the following steps of:
acquiring multi-frame two-dimensional images of a target welding plate by a color RGB camera in a kinect V2 camera, and acquiring multi-frame three-dimensional point cloud images of the target welding plate by an infrared ir camera;
according to the obtained multiple frames of the two-dimensional images and the three-dimensional point cloud images, performing three-dimensional linear weld joint crude extraction to obtain three-dimensional weld joint information;
denoising the three-dimensional welding seam information, and removing outliers and unreal welding seam track points to obtain a three-dimensional welding seam point cloud pi;
generating path points which can be executed by actual production of the robot according to the three-dimensional welding line point cloud pi;
and controlling the robot to polish the welding line according to the path points.
2. The welding deformation adaptive linear weld grinding track generation method according to claim 1, characterized in that specifically, the three-dimensional linear weld information is roughly extracted by the following method,
performing gray processing on the obtained two-dimensional image, performing Gaussian filtering, extracting line information by canny edge detection, extracting straight lines in the line information by using Hough straight line transformation to obtain a plurality of straight lines, extracting a target straight line based on a BDSCAN idea to obtain a welding line, and obtaining the position of the welding line in the two-dimensional image;
calibrating internal parameters and external parameters of a color RGB camera and an infrared ir camera, specifically comprising,
let P ir ＝[X ir Y ir Z ir ]Is the spatially non-homogeneous coordinate, p, of a point under the ir camera coordinate system ir ＝[u ir v ir 1]Is p on image plane of ir camera ir Projected point of (H) ir For the internal reference matrix of the ir depth camera calibrated in advance, the following relationship exists：
Z ir p ir ＝H ir P ir (1)
The corresponding relation of the RGB camera can be obtained in the same way:
Z rgb p rgb ＝H rgb P rgb (2)
is provided with
Figure FDA0003737122100000011
Is an external reference of the infrared camera,
Figure FDA0003737122100000012
is an external reference of a color camera,
Figure FDA0003737122100000013
for the spatial pose transformation of the color camera coordinate system relative to the infrared camera coordinate system, the following are provided:
Figure FDA0003737122100000014
Figure FDA0003737122100000015
Figure FDA0003737122100000021
substituting (1) and (2) into (5) can eliminate P rgb And P ir ：
Figure FDA0003737122100000022
And projecting the position of the welding seam in the two-dimensional image into the three-dimensional image according to the established corresponding relation of the pixel points between the color RGB camera and the infrared ir camera.
3. The welding deformation adaptive linear weld grinding track generation method according to claim 1, wherein the process of obtaining a three-dimensional weld point cloud pi specifically comprises the following steps,
and simplifying the three-dimensional weld information through a DBSCAN algorithm to obtain a three-dimensional weld point cloud pi, and recording as a weld point cluster line _2.
4. The welding deformation adaptive linear weld grinding track generation method according to claim 3, characterized in that, specifically, generating path points which can be actually executed by a robot according to the three-dimensional weld point cloud pi comprises the following steps,
step 510, calculating a unit point normal vector average value n1 of the trace points in line _2, and calculating a three-dimensional unit vector m based on cross multiplication of a unit direction vector a1 of a straight line fitted by line _2 and n 1: m = n 1 ×a 1 ；
Step 520, based on the unit vector m, each point p in the point cloud cluster line _2 i Deviation is +/-1.5 times in the m vector direction, and a corresponding domain point p is calculated i1 ＝p i +1.5*m、p i2 ＝p i Sm, obtaining a neighborhood point cluster line _2+ and line _2 of line _ 2;
step 530, respectively calculate p i1 And p i2 Nearest neighbor point p on target weld plate point cloud i1t 、p i2t Then p is calculated i1t 、p i2t Point normal vector n i1t 、n i2t Then, the polishing position and the polishing attitude of the real polishing track point are respectively: p i ＝(p i1t +p i2t )/2；N i ＝(n i1t +n i2t ) /2, when P is recorded i All the points in the point cloud cluster are line _3;
step 540, sorting the polishing positions and postures of the polishing track points based on the point cloud cluster line _3 to obtain a sorted point cloud cluster line _4;
step 550, utilizing the B-spline curve to align the position P in the point cloud cluster line _4 i And attitude N i Smoothing the information to obtain a point cloud cluster line _5;
step 560, processing the point cloud cluster line _5 by using the B-spline curve again to obtain point cloud cluster line _6 with equal point intervals, and synchronously fitting normal vectors;
and 570, calculating a kinematic inverse solution according to the point cloud cluster line _6, and setting a sixth axis as a fixed value in an inverse solution result to obtain a path point which can be actually produced and executed by the robot.
5. Welding deformation self-adaptation's straight line welding seam track generation device of polishing, its characterized in that includes:
the image acquisition module is used for acquiring multi-frame two-dimensional images of the target welding plate through a color RGB camera in a kinect V2 camera and acquiring multi-frame three-dimensional point cloud images of the target welding plate through an infrared ir camera;
the three-dimensional welding seam information acquisition module is used for crudely extracting a three-dimensional linear welding seam according to the acquired multi-frame two-dimensional image and three-dimensional point cloud image to acquire three-dimensional welding seam information;
the simplification module is used for carrying out noise reduction processing on the three-dimensional welding seam information and eliminating outliers and unreal welding seam track points to obtain a three-dimensional welding seam point cloud pi;
the path point generating module is used for generating path points which can be executed by actual production of the robot according to the three-dimensional welding line point cloud pi;
and the execution module is used for controlling the robot to polish the welding seam according to the path point.
6. A weld grinding platform is characterized in that the method for generating the linear weld grinding track with the adaptive welding deformation according to any one of the claims 1 to 5 is applied, and comprises the following steps,
a robot;
the kinect V2 camera is used for acquiring multi-frame two-dimensional images of the target welding plate through a color RGB camera and acquiring multi-frame three-dimensional point cloud images of the target welding plate through an infrared ir camera;
the platform is used for fixing the target welding plate;
and the industrial personal computer based on the ROS robot control system is used for receiving the multiframe two-dimensional images and the three-dimensional point cloud images transmitted by the kinect V2 camera and carrying out polishing track planning and execution based on the multiframe two-dimensional images and the three-dimensional point cloud images.
7. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.
CN202210805549.2A 2022-07-08 2022-07-08 Welding deformation adaptive linear weld grinding track generation method and device Pending CN115358965A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210805549.2A CN115358965A (en) 2022-07-08 2022-07-08 Welding deformation adaptive linear weld grinding track generation method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210805549.2A CN115358965A (en) 2022-07-08 2022-07-08 Welding deformation adaptive linear weld grinding track generation method and device

Publications (1)

Publication Number Publication Date
CN115358965A true CN115358965A (en) 2022-11-18

Family

ID=84031732

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210805549.2A Pending CN115358965A (en) 2022-07-08 2022-07-08 Welding deformation adaptive linear weld grinding track generation method and device

Country Status (1)

Country Link
CN (1) CN115358965A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117226855A (en) * 2023-11-15 2023-12-15 泉州华中科技大学智能制造研究院 Weld polishing track planning method based on three-dimensional point cloud
CN117576094A (en) * 2024-01-15 2024-02-20 中铁科工集团有限公司 3D point cloud intelligent sensing weld joint pose extraction method, system and equipment

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117226855A (en) * 2023-11-15 2023-12-15 泉州华中科技大学智能制造研究院 Weld polishing track planning method based on three-dimensional point cloud
CN117226855B (en) * 2023-11-15 2024-03-15 泉州华中科技大学智能制造研究院 Weld polishing track planning method based on three-dimensional point cloud
CN117576094A (en) * 2024-01-15 2024-02-20 中铁科工集团有限公司 3D point cloud intelligent sensing weld joint pose extraction method, system and equipment
CN117576094B (en) * 2024-01-15 2024-04-19 中铁科工集团有限公司 3D point cloud intelligent sensing weld joint pose extraction method, system and equipment

Similar Documents

Publication Publication Date Title
CN115358965A (en) Welding deformation adaptive linear weld grinding track generation method and device
CN110014426B (en) Method for grabbing symmetrically-shaped workpieces at high precision by using low-precision depth camera
CN110065068B (en) Robot assembly operation demonstration programming method and device based on reverse engineering
CN110553600B (en) Method for generating simulated laser line of structured light sensor for workpiece detection
JP6899189B2 (en) Systems and methods for efficiently scoring probes in images with a vision system
US20190139255A1 (en) Posture positioning system for machine and the method thereof
CN113781561B (en) Target pose estimation method based on self-adaptive Gaussian weight quick point feature histogram
CN110136211A (en) A kind of workpiece localization method and system based on active binocular vision technology
CN107300100A (en) A kind of tandem type mechanical arm vision guide approach method of Online CA D model-drivens
CN109766903B (en) Point cloud model curved surface matching method based on curved surface features
CN114011608A (en) Spraying process optimization system based on digital twinning and spraying optimization method thereof
CN113103235A (en) Method for vertically operating cabinet surface equipment based on RGB-D image
CN113269723A (en) Unordered grasping system for three-dimensional visual positioning and mechanical arm cooperative work parts
CN109255801A (en) The method, apparatus, equipment and storage medium of three-dimension object Edge Following in video
CN112109072A (en) Method for measuring and grabbing accurate 6D pose of large sparse feature tray
CN110956066B (en) Face part distance measurement method and device and vehicle-mounted terminal
CN110153582B (en) Welding scheme generation method and device and welding system
CN115018813A (en) Method for robot to autonomously identify and accurately position welding line
CN110517323A (en) 3 D positioning system and method based on manipulator one camera multi-vision visual
CN111275758B (en) Hybrid 3D visual positioning method, device, computer equipment and storage medium
CN115661023A (en) Cylindrical arc welding line polishing track generation method and device based on three-dimensional vision
CN114299039B (en) Robot and collision detection device and method thereof
CN108389204B (en) Degraded image fuzzy kernel double-parameter direct estimation method for high-speed online detection
Wu et al. A novel approach for porcupine crab identification and processing based on point cloud segmentation
Greggio et al. Real-time 3d stereo tracking and localizing of spherical objects with the icub robotic platform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination