CN116175035A - Intelligent welding method for steel structure high-altitude welding robot based on deep learning - Google Patents
Intelligent welding method for steel structure high-altitude welding robot based on deep learning Download PDFInfo
- Publication number
- CN116175035A CN116175035A CN202310282342.6A CN202310282342A CN116175035A CN 116175035 A CN116175035 A CN 116175035A CN 202310282342 A CN202310282342 A CN 202310282342A CN 116175035 A CN116175035 A CN 116175035A
- Authority
- CN
- China
- Prior art keywords
- welding
- robot
- depth
- deep learning
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003466 welding Methods 0.000 title claims abstract description 248
- 238000000034 method Methods 0.000 title claims abstract description 54
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 28
- 239000010959 steel Substances 0.000 title claims abstract description 28
- 238000013135 deep learning Methods 0.000 title claims abstract description 22
- 238000011156 evaluation Methods 0.000 claims description 34
- 230000006870 function Effects 0.000 claims description 32
- 238000001514 detection method Methods 0.000 claims description 29
- 230000008569 process Effects 0.000 claims description 16
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- QJVKUMXDEUEQLH-UHFFFAOYSA-N [B].[Fe].[Nd] Chemical compound [B].[Fe].[Nd] QJVKUMXDEUEQLH-UHFFFAOYSA-N 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 229910001172 neodymium magnet Inorganic materials 0.000 claims description 3
- 238000010791 quenching Methods 0.000 claims description 3
- 230000000171 quenching effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000005493 welding type Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
- B23K37/02—Carriages for supporting the welding or cutting element
- B23K37/0211—Carriages for supporting the welding or cutting element travelling on a guide member, e.g. rail, track
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
- B23K37/02—Carriages for supporting the welding or cutting element
- B23K37/0252—Steering means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
- B25J19/021—Optical sensing devices
- B25J19/023—Optical sensing devices including video camera means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P70/00—Climate change mitigation technologies in the production process for final industrial or consumer products
- Y02P70/10—Greenhouse gas [GHG] capture, material saving, heat recovery or other energy efficient measures, e.g. motor control, characterised by manufacturing processes, e.g. for rolling metal or metal working
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Multimedia (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
- Laser Beam Processing (AREA)
Abstract
The invention discloses an intelligent welding method of a steel structure high-altitude welding robot based on deep learning, and belongs to the technical field of welding. According to the welding robot disclosed by the invention, the image is shot by carrying the depth camera, the shot image is sent to the industrial computer for recognition and ranging, when a welding line is detected, the robot is controlled to move towards the position of the welding line, the laser scanning is used for accurately measuring and positioning the position, finally, the mechanical arm is controlled to drive the welding gun to move to the starting point of the designated position, the spatial coordinates corresponding to the point on the welding track are calculated by the industrial computer, and the mechanical arm is controlled to drive the welding gun to move to the corresponding position, so that the full-automatic image acquisition, the motion track planning and the automatic welding work are realized, the labor cost of manual welding investment is avoided, and the danger in manual operation is also reduced. The automatic welding method can be applied to steel beam cracks with different shapes, and is wide in application range.
Description
Technical Field
The invention relates to the technical field of welding, in particular to a welding seam detection and mechanical arm motion planning technology, and more particularly relates to an intelligent welding method of a steel structure high-altitude welding robot based on deep learning.
Background
The high-altitude steel beam maintenance and welding technology is an indispensable step in modern industrial production, and mainly relates to detection and welding of steel beam cracks. The traditional manual operation has a certain danger and cannot achieve complete standardized operation; the welding mechanical arm used in the industrial field mainly adopts a fixed design mode, can not carry out corresponding welding work aiming at different types of welding seams in the steel beam, has large volume, can not work above the steel beam with the welding position needing to be continuously replaced, and is difficult to be widely applied to high-altitude welding operation.
The patent application number is searched: 201610116804.7, the invention is named: a laser recognition welding line 8-axis robot space curve welding system and method; in the application, a robot enables a laser sensor to be in a position and a posture which are easy to scan a welding seam of a workpiece, the laser sensor scans to obtain welding seam characteristic points, a least square method is used for fitting to obtain welding seam edge characteristics, and a central point of the welding seam under a laser sensor coordinate system is obtained after geometric calculation; sequentially passing the welding line through a laser sensor scanning area from a starting point to a finishing point to obtain discrete center points of the whole welding line, fitting the discrete center points by using a non-uniform rational B spline curve to obtain a parameter equation for uniformly describing the whole welding line, and obtaining a welding speed equation of the whole welding process; discretizing a weld curve according to the recurrence relation of the speed and the parameters to obtain a series of interpolation points; according to the welding gesture of the interpolation point required by the welding process, solving the kinematic inverse solution of the tilting/rotating two-axis positioner to obtain a tilting axis rotation angle theta 7 and a rotating axis rotation angle theta 8 corresponding to the interpolation point; solving the kinematic positive solution of the tilting/rotating two-axis positioner to obtain the position and the gesture of the tail end of the welding gun corresponding to the interpolation point; solving inverse solution of robot kinematics to obtain rotation angles theta 1, theta 2, theta 3, theta 4, theta 5 and theta 6 of six axes of the robot; the tilting/rotating two-axis positioner and the welding robot coordinate synchronous motion to weld; in the welding process, the laser sensor scans the welding seam, performs real-time welding seam tracking, and compensates the position deviation of the welding seam caused by factors such as welding thermal deformation. However, the above application is directed to a welding application of an 8-axis robot, and the design focus is on determining the rotation angles θ1, θ2, θ3, θ4, θ5, θ6, θ7 and θ8 of eight axes of the robot, so that the scheme is complicated in implementation and has high cost in engineering application.
Disclosure of Invention
1. Technical problem to be solved by the invention
In order to solve the problems that the traditional manual welding method needs repeated operation when welding the steel beam cracks and errors exist in the welding process, and the existing welding technology cannot automatically detect and weld the nonstandard steel beam cracks, the invention provides an intelligent welding method of a steel structure high-altitude welding robot based on deep learning.
2. Technical proposal
In order to achieve the above purpose, the technical scheme provided by the invention is as follows:
the invention discloses an intelligent welding method of a steel structure high-altitude welding robot based on deep learning, which comprises the following steps:
s1, carrying a depth camera on a welding robot to shoot an image, and sending the shot image to an industrial computer for identification processing;
s2, when a welding line is detected, controlling the robot to move towards the position of the welding line;
s3, for the acquired depth image, calculating the distance z of the pixel point according to xy pixel coordinate values, and calculating corresponding space coordinates [ u, v, z ] according to the distance in the space and camera internal parameters, fitting the space point to obtain a directional line segment in the space, wherein the line segment is the welding seam position;
s4, determining the relative position of the line segment under a laser and chassis coordinate system according to the calculated line segment space position, identifying the depth of the welding seam according to laser scanning, mapping the depth of the welding seam from the welding starting end A to the welding stopping end B onto the welding seam nodes from A to B, calculating the stay time of the welding head at the nodes, and ensuring that the welding seam is accurately welded;
s5, stopping welding work after the robot recognizes that the tail end is at the point B, closing laser scanning, reducing the welding line detection frequency, and continuing to perform welding work of the next stage.
Furthermore, in step S1, the detection mode adopted for the recognition is deep learning, and the detection network is a convolutional neural network.
Further, in step S2, the position and distance information of the robot moving are coarsely located by the picture detected by the depth camera, and after the welding seam is determined to be within the weldable range of the welding robot, the fine location is started to detect the position of the welding seam.
Further, in step S2, the depth map obtained by the camera is sent to the convolutional neural network at a speed of 5 frames per second, and the result is output (x, y, θ, w), and when the standard deviation of xy position in the results detected by 30 consecutive frames is set to be less than 0.003, the detected weld position is accurate.
Further, the process of identifying the weld in step S3 is as follows:
(1) Cutting the acquired depth image to obtain a target area, and repairing pixel points with infinite depth by using high-pass filtering;
(2) Setting the restored depth map as an image consisting of 1 and 0, wherein the identified weld joint point is set as 1, other position areas are set as 0, and setting an evaluation function Q for the detection result T ；
(3) Identifying and detecting the welding seam by using a deep learning network, and performing evaluation function Q in the step (2) T For standard, use fast orderingSequencing all detection results by a method, wherein the values of each group are all according to [ u, v, theta, w ]]Arranging, detecting depth values of the detected weld position sequence by using a depth information extraction module, and calculating spatial coordinates [ x, y, z ] corresponding to pixels according to the depth values]；
(4) And (3) obtaining a series of coordinate position sets with directions according to the step (2) and the step (3), connecting the points, fitting the points and the directions by a nonlinear least square method, and obtaining a line segment with the directions in the space, namely the real-time position of the welding seam.
Further, the evaluation function Q T Comprising an angle evaluation function phi T And a width evaluation function W T Angle evaluation function Φ T For finding out the detection result with the smallest difference between the detection angle and the actual angle, and a width evaluation function W T The method is used for finding out the detection result with the smallest difference between the detected weld width and the actual width.
Further, the angle evaluation function Φ T And width evaluation function W T Multiplying the corresponding weights alpha and beta and adding to obtain an evaluation function Q T Evaluation function Q T The most accurate images of the angle and the weld width can be selected by a linear weighting sum method.
Further, the welding process of the butt seam in step S4 is as follows:
1) Converting the position of the welding line based on the camera coordinate system to the coordinate based on the welding robot coordinate system;
2) Vector of line segment l connected according to space coordinate calculation point AB and each node l of line segment l 1 ,l 2 …l n Calculating a node direction theta based on a robot coordinate system by using the AB vector;
3) Using a laser sensor to point at a starting point A, scanning the welding point A through B, projecting laser to the depth of the welding line, and sending the depth information of the welding line to the sensor so as to obtain the actual depth d of the welding line 1 ,d 2 …d n Mapped to each weld joint node l 1 ,l 2 …l n The welding time t is calculated according to the depth of the welding spot in a lambda d way x Lambda is determined by the welding speed;
4) Controlling the triaxial mechanical arm to move to the point A, starting to move along the direction theta by taking the point as a starting point, starting a welding machine to perform welding operation, and controlling the moving speed of the welding head according to the time required by stay of the node in the step 3) during welding;
5) And when the robot recognizes that the welding head is positioned to the point B, arc quenching stops welding.
Further, in step 4), the welding head speed is controlled during the movement of the welding head by using a proportional-integral method.
Still further, high altitude welding robot includes robot chassis, welder, arm, industrial computer, degree of depth camera, laser sensor, and degree of depth camera and laser sensor are fixed in the robot chassis top, and the welder fixture with the arm is fixed, and the robot communicates through the ethernet, handles and send the welding seam position of discernment through industrial computer, and the fixed neodymium iron boron strong magnet of every track below of robot chassis.
3. Advantageous effects
Compared with the prior art, the technical scheme provided by the invention has the following remarkable effects:
the invention automatically welds the high-altitude steel girder cracks in a mode of positioning, measuring and then welding. The welding seam can be fully automatically identified and welded under the condition of complete unmanned accompanying, so that the labor and time cost are greatly saved, and meanwhile, the error caused by manual operation is reduced. Compared with the existing automatic welding technology of the mechanical arm, the automatic welding device not only can carry out welding work on any welding line, but also can realize downward compatible design programmable welding; meanwhile, the invention combines the laser and image technologies, so that the welding positioning is more accurate and the welding effect is better.
Drawings
Fig. 1 is a schematic view of the whole structure of a welding robot according to the present invention.
FIG. 2 is a schematic view of a weld.
Reference numerals in the schematic drawings illustrate:
1. a Z-axis guide rail; 2. a welding gun clamping mechanism; 3. a robot chassis; 4. a Y-axis guide rail motor; 5. an X-axis guide rail motor; 6. a Z-axis guide rail motor; 7. a Z-axis guide rail platform; 8. a Y-axis guide rail platform; 9. a depth camera.
Detailed Description
In order that the manner in which the functions, features and advantages of the invention are obtained will be readily understood, a particular embodiment of the invention will be described, some but not all embodiments of which are illustrated in the accompanying drawings. All other embodiments, which can be made by one of ordinary skill in the art without undue burden on the person of ordinary skill in the art based on the embodiments of the present invention, are intended to be within the scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below. The three-axis coordinate direction of the invention is the X positive direction of the robot, the X positive direction is the Y positive direction of the robot moving plane, the Y positive direction of the robot rotating 90 degrees anticlockwise, and the Z positive direction of the robot moving plane.
Referring to fig. 1, the invention discloses a full-automatic high-altitude steel beam welding robot, which comprises a robot chassis 3, a welding gun, a three-degree-of-freedom gantry mechanical arm, an industrial computer, a depth camera and a laser sensor. The Z-axis guide rail 1 and the Z-axis guide rail motor 6 of the three-degree-of-freedom gantry mechanical arm drive the Z-axis guide rail platform 7 to move in the Z-axis direction, the welding gun clamping mechanism 2 is arranged on the Z-axis guide rail platform 7, the Y-axis guide rail and the Y-axis guide rail motor 4 drive the Y-axis guide rail platform 8 to move in the Y-axis direction, the Z-axis guide rail 1 is arranged on the Y-axis guide rail platform 8, and the X-axis guide rail motor 5 drive the Z-axis guide rail 1, the welding gun clamping mechanism 2 and the Y-axis guide rail to move in the X-axis direction. The motor adopts a servo motor to control the movement of each shaft joint, so that the welding gun can freely move in multiple directions, and the whole welding mechanism is driven by the robot chassis 3 to control the whole welding robot to realize movement. The depth camera 9 performs image acquisition and transmits acquisition information to the mechanical arm. The depth camera 9 and the laser sensor are fixed above the robot chassis 3 and are fixed with the welding gun clamping mechanism 2 of the mechanical arm, the depth measurement precision of the depth camera 9 is 10mm, the precision of the laser sensor is 0.01mm, and the welding precision of the mechanical arm can reach 1mm. The robot communicates through the TCP/IP Ethernet, processes and transmits the identified welding seam position through a computer in the robot, and controls the mechanical arm to drive the welding machine to carry out welding work. The welding robot can automatically make repeated motion on the steel beam, and a neodymium iron boron strong magnet with the thickness of 40mm and 20mm is fixed below each crawler of the welding robot chassis, so that the welding robot can be adsorbed on the steel beam.
The detection and welding method of the high-altitude steel beam welding seam comprises the steps of welding seam identification, positioning, path identification, mechanical arm path planning and the like, and the whole identification and welding process is full-automatic operation of a robot. The specific process is as follows:
taking the weld joint shown in fig. 2 as an example, one end of the weld joint is set as a point a, and the other end is set as a point B. The depth camera mounted on the welding robot is kept on, and the shot images are sent to the computer every 0.5s for identification processing, wherein the picture size is 640 x 480, and the frame rate is 30fps. Each frame of image is saved as a detection input, and the picture is saved by using a serial number starting from 1. The detection mode adopted by recognition is deep learning, the detection network is a convolutional neural network, the network is mainly synthesized by a convolutional layer, a pooling layer and a nonlinear activation layer, and when the detection is carried out by using a pre-trained model, the detection rate of the welding seam can reach 99.5%.
When the welding line is detected, the robot is controlled to move to the welding line position, the crawler-type chassis can be controlled to rotate at the moment through differential speed, and when the crawler-type chassis is required to shift leftwards, the left wheel motor is controlled to rotate forwards, and the right wheel motor is controlled to rotate backwards; when the welding line is detected to be on the right side, the robot needs to be controlled to move in the right front direction, at the moment, the left wheel motor is controlled to rotate backwards, and the right wheel motor rotates forwards; when the welding line is detected to be positioned right in front of the robot, the left motor and the right motor of the robot are controlled to rotate backwards simultaneously.
The control robot moves to the position of the welding line, which is 40cm away, and the position and distance information can be roughly positioned by the pictures detected by the depth camera. After the welding seam is determined to be within the welding range of the welding robot, starting the accurate positioning to detect the position of the welding seam, and stabilizing the chassis of the robot is needed at the moment. The depth map obtained by the camera is fed into the grabbing neural network at a speed of 5 frames per second, the result is output (x, y, theta, w), and the detected weld position is considered to be accurate if the standard deviation of xy positions of the results detected by 30 continuous frames is smaller than 0.003.
The specific spatial position of the detected sequence point value is calculated according to the principle of aperture imaging, the specific operation is that the distance z of the point is calculated according to xy pixel coordinate values, and the corresponding spatial coordinates [ u, v, z ] are calculated according to the distance in the space and the camera internal parameters. Fitting the spatial points according to a nonlinear optimization mode results in a directional line segment in space, which can be regarded as the weld position.
And determining the relative position of the line segment under the coordinate system of the laser and the chassis according to the calculated line segment space position and the coordinate relation of the known camera, the laser and the robot chassis. And selecting one end A of the detected welding line close to the robot as a welding starting end, and selecting one end B with a longer distance as a welding stopping end. Performing weld depth identification according to laser scanning, mapping the weld depths from A to B onto weld nodes from A to B, and determining a weld depth according to a formula lambdad x And calculating the stay time of the welding head at the node, and ensuring that the welding seams can be correctly welded.
When the mechanical arm recognizes that the tail end is at the point B, the welding work is stopped, at the moment, the computer controls the welding head to conduct arc extinguishing operation, meanwhile, the mechanical arm is lifted to the Z axis by 5cm, and the XY axis is reset. And meanwhile, the laser scanning is closed, the welding line detection frequency is reduced, and the welding work of the next stage is continued.
The image processing process for weld recognition according to the present invention will be specifically described with reference to example 1, and the welding process according to the present invention will be specifically described with reference to example 2.
Example 1
The processing procedure of the weld joint identification image in this embodiment is as follows:
step 1: and (3) performing image acquisition by using a depth camera, processing the acquired picture by using an upper computer in the welding robot, and cutting each frame of acquired image into a square target area with the size of 300 x 300. In order to ensure that the acquired sparse graph has distance testability, repairing pixel points with infinite depth by using high-pass filtering, and storing the repaired dense image.
Step 2: the restored depth map is set to an image consisting of 1 and 0, wherein the identified weld point is set to 1 and the other location areas are set to 0. Setting an evaluation function Q for the detection result T Dividing it into angle evaluation functions phi T Width evaluation function W T The angle evaluation function phi T And width evaluation function W T Multiplying the corresponding weights alpha and beta and adding to obtain a final evaluation function Q T . Angle evaluation function phi T For finding the detection result with the smallest difference between the detected angle and the actual angle, i.e. the evaluation is within [ -pi/2, pi/2]Within the range, detecting the difference between the angle result and the actual angle, and using the network learning distribution and the evaluation corresponding value, the result should be distributed in [ -pi/2, pi/2]And (3) upper part. The width evaluation function is used for finding out a detection result with the smallest difference between the detected weld width and the actual width, and mapping the result to a range of 0-1. The final evaluation function is Q T ＝αΦ T+ βW T And obtaining the most accurate weld width and angle according to the evaluation function of each image.
Step 3: and 2, carrying out recognition detection on the welding seam by using a deep learning network, obtaining differences between the width and angle of the welding seam of each image and actual values from the step 2, and sequencing all detection results by using a rapid sequencing method by taking the differences as a standard, wherein each group of values are data arranged according to [ u, v, theta and w ] and respectively represent the width and high pixel values, deflection angles and the welding seam width of the welding seam position in the image.
The depth information extraction module in the computer vision library is used for detecting the depth value of the detected weld position sequence, and the space coordinates [ x, y,z]the depth value z is added to the corresponding pixel value u, v]In which Eigen is used to define the camera internal parameters as a two-dimensional array K, and the specific parameters are [ [ f ] x ,0,c x ],[0,f y ,c y ],[0,0,1]]Calculating the value of x as ((u-c) according to the principle of small-hole imaging x )*z/f x ) Y has a value ((v-c) y )*z/f y ) The z value is the calculated depth value.
Step 4: according to step 2 and step 3 a series of sets of coordinate positions with directions can be obtained and each point has a certain width, i.e. corresponds to a point with a width. The points are connected and fitted with the direction by a nonlinear least square method of Ceres library, so that a line segment with the direction in space is obtained, and each point on the line segment has a certain width corresponding to the actual weld width. Therefore, any coordinate in the obtained coordinate position set can be detected, and the detected result can be basically output as an actual value, so that the real-time position of the welding line can be obtained.
Example 2
The process of automatically welding the weld seam by using the welding robot in the embodiment is as follows:
step 1: and selecting one end, close to the robot, of the detected welding line segments as a starting point A, and one end, far away from the robot, as an end point B, and converting the position of the welding line based on a camera coordinate system into coordinates based on the welding robot coordinate system. Measuring the relative position T of a camera and a robot base 1 The relative position of the welding line under the camera coordinate system is T 2 Obtaining the position T of the welding line based on the robot coordinate system according to the spatial coordinate conversion relation of the rigid body 3 ＝T 1 *T 2 Wherein T is 3 A 4*4 transformation matrix is formed by combining the 3*3 rotation matrix with the 3*1 translation matrix. In this way, the weld position is converted into coordinates based on the robot coordinate system.
Step 2: setting the starting point in the step 1 as A, the end point as B, and calculating the vector of a line segment l connected by the point AB according to the space coordinates, wherein each node of the line segment l is l 1 ,l 2 …l n And calculating a node direction theta= (r, p, y) based on a robot coordinate system by using the AB vector, representing the welding line as a directional line segment with A as a starting point, B as an end point and theta as a direction, and stopping welding by arc extinguishing operation when the welding line is welded to the point B.
Step 3: pointing to the starting point A calculated in the step 2 by using a laser sensor on a mechanical arm, scanning the starting point A from A to B once, projecting laser to the depth of a welding seam, and sending the depth information of the welding seam to the sensor so as to obtain the actual depth d of the welding seam 1 ,d 2 …d n Mapped to each weld joint node l 1 ,l 2 …l n The welding time t is calculated according to the depth of the welding spot in a lambda d way x Lambda is determined by the welding speed.
Step 4: controlling the triaxial mechanical arm to move to a position 8.5mm away from the point A, starting to move along the direction theta by taking the point as a starting point, starting a welding machine to perform welding operation, and controlling the moving speed of the welding head according to the time required by stay of the node in the step 3 during welding; meanwhile, in order to prevent the welding head from being damaged by continuous instantaneous acceleration and deceleration, a proportional integral method is used for controlling the welding head speed so as to ensure that the welding head moves in a smooth variable speed motion.
Step 5: when the robot recognizes that the position of the welding head is 8.5mm away from the position above the point B, arc quenching is stopped. And (5) raising the welding head position, and simultaneously moving the XY axis position of the mechanical arm to the central initial position to finish welding.
Step 6: and after the mechanical arm is welded, carrying out weld joint identification and positioning again so as to ensure that the robot can carry out automatic welding on the residual weld joint.
The invention can fully automatically identify and weld the welding line under the condition of no accompanying person, saves labor and time cost, and reduces errors caused by manual operation. Compared with the existing automatic welding technology of the mechanical arm, the invention combines the laser and image technologies, so that the welding positioning is more accurate and the welding effect is better.
Claims (10)
1. The intelligent welding method of the steel structure high-altitude welding robot based on deep learning is characterized by comprising the following steps of:
s1, carrying a depth camera on a welding robot to shoot an image, and sending the shot image to an industrial computer for identification processing;
s2, when a welding line is detected, controlling the robot to move towards the position of the welding line;
s3, for the acquired depth image, calculating the distance z of the pixel point according to xy pixel coordinate values, and calculating corresponding space coordinates [ u, v, z ] according to the distance in the space and camera internal parameters, fitting the space point to obtain a directional line segment in the space, wherein the line segment is the welding seam position;
s4, determining the relative position of the line segment under a laser and chassis coordinate system according to the calculated line segment space position, identifying the depth of the welding seam according to laser scanning, mapping the depth of the welding seam from the welding starting end A to the welding stopping end B onto the welding seam nodes from A to B, calculating the stay time of the welding head at the nodes, and ensuring that the welding seam is accurately welded;
s5, stopping welding work after the robot recognizes that the tail end is at the point B, closing laser scanning, reducing the welding line detection frequency, and continuing to perform welding work of the next stage.
2. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning according to claim 1, wherein the intelligent welding method is characterized by comprising the following steps of: and step S1, the detection mode adopted by the identification is deep learning, and the detection network is a convolutional neural network.
3. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning according to claim 2, wherein the intelligent welding method is characterized by comprising the following steps of: and S2, roughly positioning the position and distance information of the movement of the robot by the picture detected by the depth camera, and starting the fine positioning to detect the position of the welding seam after the welding seam is determined to be in the welding range of the welding robot.
4. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning according to claim 3, wherein the intelligent welding method comprises the following steps: in step S2, the depth map obtained by the camera is sent to the convolutional neural network at a speed of 5 frames per second, and the result is output (x, y, θ, w), and when the standard deviation of xy positions in the results detected by 30 consecutive frames is set to be less than 0.003, the detected weld position is accurate.
5. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning according to any one of claims 1 to 4, wherein the intelligent welding method comprises the following steps: the process of identifying the weld in step S3 is as follows:
(1) Cutting the acquired depth image to obtain a target area, and repairing pixel points with infinite depth by using high-pass filtering;
(2) Setting the restored depth map as an image consisting of 1 and 0, wherein the identified weld joint point is set as 1, other position areas are set as 0, and setting an evaluation function Q for the detection result T ；
(3) Identifying and detecting the welding seam by using a deep learning network, and performing evaluation function Q in the step (2) T As a standard, all detection results are ranked by a rapid ranking method, and the values of each group are all according to [ u, v, theta, w ]]Arranging, detecting depth values of the detected weld position sequence by using a depth information extraction module, and calculating spatial coordinates [ x, y, z ] corresponding to pixels according to the depth values]；
(4) And (3) obtaining a series of coordinate position sets with directions according to the step (2) and the step (3), connecting the points, fitting the points and the directions by a nonlinear least square method, and obtaining a line segment with the directions in the space, namely the real-time position of the welding seam.
6. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning of claim 5 is characterized by comprising the following steps: the evaluation function Q T Comprising an angle evaluation function phi T And a width evaluation function W T Angle evaluation function Φ T For finding out the detection result with the smallest difference between the detected angle and the actual angle, and the width evaluation functionNumber W T The method is used for finding out the detection result with the smallest difference between the detected weld width and the actual width.
7. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning of claim 6 is characterized by comprising the following steps: the angle evaluation function phi T And width evaluation function W T Multiplying the corresponding weights alpha and beta and adding to obtain an evaluation function Q T Evaluation function Q T The most accurate images of the angle and the weld width can be selected by a linear weighting sum method.
8. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning of claim 7 is characterized by comprising the following steps: the welding process of the butt seam in the step S4 is as follows:
1) Converting the position of the welding line based on the camera coordinate system to the coordinate based on the welding robot coordinate system;
2) Vector of line segment l connected according to space coordinate calculation point AB and each node l of line segment l 1 ,l 2 …l n Calculating a node direction theta based on a robot coordinate system by using the AB vector;
3) Using a laser sensor to point at a starting point A, scanning the welding point A through B, projecting laser to the depth of the welding line, and sending the depth information of the welding line to the sensor so as to obtain the actual depth d of the welding line 1 ,d 2 …d n Mapped to each weld joint node l 1 ,l 2 …l n The welding time t is calculated according to the depth of the welding spot in a lambda d way x Lambda is determined by the welding speed;
4) Controlling the triaxial mechanical arm to move to the point A, starting to move along the direction theta by taking the point as a starting point, starting a welding machine to perform welding operation, and controlling the moving speed of the welding head according to the time required by stay of the node in the step 3) during welding;
5) And when the robot recognizes that the welding head is positioned to the point B, arc quenching stops welding.
9. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning of claim 8 is characterized by comprising the following steps: and 4) controlling the welding head speed by using a proportional integral method in the welding head moving process.
10. The intelligent welding method for the steel structure high-altitude welding robot based on deep learning of claim 9 is characterized by comprising the following steps: the high-altitude welding robot comprises a robot chassis, a welding gun, a mechanical arm, an industrial computer, a depth camera and a laser sensor, wherein the depth camera and the laser sensor are fixed above the robot chassis and are fixed with a welding gun clamping mechanism of the mechanical arm, the robot communicates through an Ethernet, the industrial computer processes and sends the identified welding seam position, and a neodymium iron boron strong magnet is fixed below each crawler of the robot chassis.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310282342.6A CN116175035B (en) | 2023-03-20 | 2023-03-20 | Intelligent welding method for steel structure high-altitude welding robot based on deep learning |
| PCT/CN2023/133211 WO2024193077A1 (en) | 2023-03-20 | 2023-11-22 | Deep-learning-based intelligent welding method for high-altitude steel structure welding robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310282342.6A CN116175035B (en) | 2023-03-20 | 2023-03-20 | Intelligent welding method for steel structure high-altitude welding robot based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116175035A true CN116175035A (en) | 2023-05-30 |
| CN116175035B CN116175035B (en) | 2023-08-04 |
Family
ID=86438552
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310282342.6A Active CN116175035B (en) | 2023-03-20 | 2023-03-20 | Intelligent welding method for steel structure high-altitude welding robot based on deep learning |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN116175035B (en) |
| WO (1) | WO2024193077A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024193077A1 (en) * | 2023-03-20 | 2024-09-26 | 中国十七冶集团有限公司 | Deep-learning-based intelligent welding method for high-altitude steel structure welding robot |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106425181A (en) * | 2016-10-24 | 2017-02-22 | 南京工业大学 | Curve weld joint welding technology based on line structured light |
| CN107020449A (en) * | 2017-04-26 | 2017-08-08 | 苏州睿牛机器人技术有限公司 | Weld seam tracking sensor and its welding seam tracking method |
| DE102019209100A1 (en) * | 2019-06-24 | 2020-12-24 | Robert Bosch Gmbh | Quality monitoring for a laser beam welding process |
| CN113070612A (en) * | 2021-04-01 | 2021-07-06 | 温州科技职业学院 | Welding robot for automatic automobile assembly line |
| CN113666028A (en) * | 2021-07-27 | 2021-11-19 | 南京航空航天大学 | Garbage can detecting and grabbing method based on fusion of laser radar and camera |
| CN113954085A (en) * | 2021-09-08 | 2022-01-21 | 重庆大学 | Intelligent positioning and control method of welding robot based on binocular vision and linear laser sensing data fusion |
| CN114140439A (en) * | 2021-12-03 | 2022-03-04 | 厦门大学 | Laser welding seam feature point identification method and device based on deep learning |
| CN114266974A (en) * | 2021-12-23 | 2022-04-01 | 福州大学 | Automatic positioning welding method based on deep learning |
| CN115018813A (en) * | 2022-06-30 | 2022-09-06 | 哈尔滨工业大学（威海） | Method for robot to autonomously identify and accurately position welding line |
| CN115213563A (en) * | 2022-09-20 | 2022-10-21 | 扬州耐施工程机械有限公司 | Laser intelligent welding method and system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109226967B (en) * | 2018-07-25 | 2021-03-09 | 同高先进制造科技（太仓）有限公司 | Active laser vision steady weld joint tracking system for laser-arc hybrid welding |
| CN109175608B (en) * | 2018-09-30 | 2023-06-20 | 华南理工大学 | Weld joint characteristic point position online measurement method and weld joint track automatic measurement system |
| CN110524580B (en) * | 2019-09-16 | 2023-06-02 | 西安中科光电精密工程有限公司 | Welding robot vision assembly and measuring method thereof |
| CN113920060A (en) * | 2021-09-09 | 2022-01-11 | 中国科学院自动化研究所 | Autonomous operation method and device for welding robot, electronic device, and storage medium |
| CN115041887A (en) * | 2022-04-24 | 2022-09-13 | 重庆创御智能装备有限公司 | Contour measuring and recognizing system for robot welding workpiece |
| CN116175035B (en) * | 2023-03-20 | 2023-08-04 | 中国十七冶集团有限公司 | Intelligent welding method for steel structure high-altitude welding robot based on deep learning |
-
2023
- 2023-03-20 CN CN202310282342.6A patent/CN116175035B/en active Active
- 2023-11-22 WO PCT/CN2023/133211 patent/WO2024193077A1/en unknown
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106425181A (en) * | 2016-10-24 | 2017-02-22 | 南京工业大学 | Curve weld joint welding technology based on line structured light |
| CN107020449A (en) * | 2017-04-26 | 2017-08-08 | 苏州睿牛机器人技术有限公司 | Weld seam tracking sensor and its welding seam tracking method |
| DE102019209100A1 (en) * | 2019-06-24 | 2020-12-24 | Robert Bosch Gmbh | Quality monitoring for a laser beam welding process |
| CN113070612A (en) * | 2021-04-01 | 2021-07-06 | 温州科技职业学院 | Welding robot for automatic automobile assembly line |
| CN113666028A (en) * | 2021-07-27 | 2021-11-19 | 南京航空航天大学 | Garbage can detecting and grabbing method based on fusion of laser radar and camera |
| CN113954085A (en) * | 2021-09-08 | 2022-01-21 | 重庆大学 | Intelligent positioning and control method of welding robot based on binocular vision and linear laser sensing data fusion |
| CN114140439A (en) * | 2021-12-03 | 2022-03-04 | 厦门大学 | Laser welding seam feature point identification method and device based on deep learning |
| CN114266974A (en) * | 2021-12-23 | 2022-04-01 | 福州大学 | Automatic positioning welding method based on deep learning |
| CN115018813A (en) * | 2022-06-30 | 2022-09-06 | 哈尔滨工业大学（威海） | Method for robot to autonomously identify and accurately position welding line |
| CN115213563A (en) * | 2022-09-20 | 2022-10-21 | 扬州耐施工程机械有限公司 | Laser intelligent welding method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024193077A1 (en) * | 2023-03-20 | 2024-09-26 | 中国十七冶集团有限公司 | Deep-learning-based intelligent welding method for high-altitude steel structure welding robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116175035B (en) | 2023-08-04 |
| WO2024193077A1 (en) | 2024-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112659123B (en) | Two-robot curve welding collaborative planning method based on line structured light vision | |
| CN113001535B (en) | Automatic correction system and method for robot workpiece coordinate system | |
| CN112518072B (en) | Spatial intersecting curve weld joint structure modeling method based on line structure light vision | |
| CN111745267A (en) | System and method for tracking groove weld in real time based on laser displacement sensor | |
| EP1486283B1 (en) | Method of controlling the welding of a three-dimensional construction by taking a two-dimensional image of the construction and adjusting in real time in the third dimension | |
| US4907169A (en) | Adaptive tracking vision and guidance system | |
| CN110102855A (en) | A kind of robot automtion welding system, device and method | |
| JP7153085B2 (en) | ROBOT CALIBRATION SYSTEM AND ROBOT CALIBRATION METHOD | |
| CN112958959A (en) | Automatic welding and detection method based on three-dimensional vision | |
| CN114474041B (en) | Welding automation intelligent guiding method and system based on cooperative robot | |
| CN106271081A (en) | Three coordinate rectangular robot line laser seam tracking system and trackings thereof | |
| CN111745266A (en) | Corrugated board welding track generation method and system based on 3D vision position finding | |
| JPH08505091A (en) | System and method for tracking features on an object using redundant axes | |
| CN117047237B (en) | Intelligent flexible welding system and method for special-shaped parts | |
| Zhou et al. | Autonomous acquisition of seam coordinates for arc welding robot based on visual servoing | |
| CN113334018A (en) | Visual guide robot automatic welding pipeline intersecting line device and method | |
| CN116175035B (en) | Intelligent welding method for steel structure high-altitude welding robot based on deep learning | |
| CN110450150B (en) | Trajectory tracking control method and trajectory tracking system | |
| CN105983802B (en) | A kind of welding robot control system and method | |
| CN104889529B (en) | A kind of welding method of large scale solid curve weld seam Intelligent welding equipment | |
| CN114378827B (en) | Dynamic target tracking and grabbing method based on overall control of mobile mechanical arm | |
| JP2023133088A (en) | Mobile flexible measurement system and measurement method for large complex component | |
| CN115026480B (en) | Full-automatic assembling and welding device and method for ocean platform jacket | |
| Lakhal et al. | Control of a hyper-redundant robot for quality inspection in additive manufacturing for construction | |
| CN116329824A (en) | Hoisting type intelligent welding robot and welding method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20230530 Assignee: Shanghai Jiading MCC Construction Co.,Ltd. Assignor: CHINA MCC17 GROUP Co.,Ltd. Contract record no.: X2023980053967 Denomination of invention: An Intelligent Welding Method for Steel Structure High Altitude Welding Robot Based on Deep Learning Granted publication date: 20230804 License type: Common License Record date: 20231229 Application publication date: 20230530 Assignee: Wuhu Jinjiang Ruijing Real Estate Co.,Ltd. Assignor: CHINA MCC17 GROUP Co.,Ltd. Contract record no.: X2023980053894 Denomination of invention: An Intelligent Welding Method for Steel Structure High Altitude Welding Robot Based on Deep Learning Granted publication date: 20230804 License type: Common License Record date: 20231229 |