US11951575B2 - Automatic welding system and method for large structural parts based on hybrid robots and 3D vision - Google Patents
Automatic welding system and method for large structural parts based on hybrid robots and 3D vision Download PDFInfo
- Publication number
- US11951575B2 US11951575B2 US18/041,129 US202218041129A US11951575B2 US 11951575 B2 US11951575 B2 US 11951575B2 US 202218041129 A US202218041129 A US 202218041129A US 11951575 B2 US11951575 B2 US 11951575B2
- Authority
- US
- United States
- Prior art keywords
- robot
- mdof
- coordinate system
- welding
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003466 welding Methods 0.000 title claims abstract description 135
- 238000000034 method Methods 0.000 title claims abstract description 57
- 230000009466 transformation Effects 0.000 claims description 33
- 230000033001 locomotion Effects 0.000 claims description 23
- 238000009616 inductively coupled plasma Methods 0.000 claims description 13
- 235000010384 tocopherol Nutrition 0.000 claims description 13
- 235000019731 tricalcium phosphate Nutrition 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 8
- 101100459256 Cyprinus carpio myca gene Proteins 0.000 claims description 6
- 101100459261 Cyprinus carpio mycb gene Proteins 0.000 claims description 6
- 230000003287 optical effect Effects 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 230000001174 ascending effect Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 claims description 2
- 238000003754 machining Methods 0.000 abstract description 2
- 238000004519 manufacturing process Methods 0.000 abstract description 2
- 230000007246 mechanism Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- FVJPPEWHZCSTAC-UHFFFAOYSA-N meta-O-Dealkylated flecainide Chemical compound OC1=CC=C(OCC(F)(F)F)C(C(=O)NCC2NCCCC2)=C1 FVJPPEWHZCSTAC-UHFFFAOYSA-N 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1615—Programme controls characterised by special kind of manipulator, e.g. planar, scara, gantry, cantilever, space, closed chain, passive/active joints and tendon driven manipulators
- B25J9/162—Mobile manipulator, movable base with manipulator arm mounted on it
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K9/00—Arc welding or cutting
- B23K9/12—Automatic feeding or moving of electrodes or work for spot or seam welding or cutting
- B23K9/126—Controlling the spatial relationship between the work and the gas torch
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
- B23K37/02—Carriages for supporting the welding or cutting element
- B23K37/0247—Driving means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K37/00—Auxiliary devices or processes, not specially adapted to a procedure covered by only one of the preceding main groups
- B23K37/02—Carriages for supporting the welding or cutting element
- B23K37/0294—Transport carriages or vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K9/00—Arc welding or cutting
- B23K9/0026—Arc welding or cutting specially adapted for particular articles or work
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K9/00—Arc welding or cutting
- B23K9/095—Monitoring or automatic control of welding parameters
- B23K9/0953—Monitoring or automatic control of welding parameters using computing means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/005—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators using batteries, e.g. as a back-up power source
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J5/00—Manipulators mounted on wheels or on carriages
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J5/00—Manipulators mounted on wheels or on carriages
- B25J5/005—Manipulators mounted on wheels or on carriages mounted on endless tracks or belts
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/39—Robotics, robotics to robotics hand
- G05B2219/39022—Transform between measuring and manipulator coordinate system
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/39—Robotics, robotics to robotics hand
- G05B2219/39045—Camera on end effector detects reference pattern
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/39—Robotics, robotics to robotics hand
- G05B2219/39398—Convert hand to tool coordinates, derive transform matrix
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40298—Manipulator on vehicle, wheels, mobile
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40584—Camera, non-contact sensor mounted on wrist, indep from gripper
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40613—Camera, laser scanner on end effector, hand eye manipulator, local
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/45—Nc applications
- G05B2219/45104—Lasrobot, welding robot
Definitions
- the invention relates to the technical field of industrial automation and machine vision for automatic welding of large structural parts, in particular to an automatic welding system and method for large structural parts based on hybrid robots and 3D vision.
- This scheme has the following drawbacks: (1) the precision requirement for the movement mechanism is high; however, when the moving distance of the movement mechanism is over several meters, accumulative errors will be caused, which have a great impact on the global positioning precision of the robots, so in order to guarantee the global positioning precision, precise guide rails or even grating scales have to be used for feedback to improve the precision, and the cost of the system is sharply increased and will grow in equal proportion with the increase of the moving distance of the movement mechanism; (2) a driver for controlling the extra external shaft and a corresponding external shaft motor have to be added to the robot system, which also increases the cost of the system. For these reasons, the accessories of the external shaft account for a large proportion of the cost of the robot welding system for large structural parts.
- the objective of the invention is to provide an automatic welding system and method for large structural parts based on hybrid robots and 3D vision, which expand the working range of industrial robots through a mobile robot and accurately recognize and position the tail end through the 3D vision technique to automatically complete welding tasks, so the working range of the system is expanded, the flexibility of the system is improved, and the cost of the system is reduced.
- An automatic welding system for large structural parts based on hybrid robots and 3D vision comprises a hybrid robot system composed of a mobile robot and an MDOF robot installed above the mobile robot, a welding system installed at a tail end of the MDOF robot and used for welding a target workpiece, and a 3D vision system installed at the tail end of the MDOF robot or on the welding system, and used for global calibration and positioning of the hybrid robot system, the welding system and the target workpiece.
- the mobile robot comprises a mobile robot chassis; a shell is fixedly disposed on the mobile robot chassis; a rechargeable battery pack and a power supply used for providing energy for the whole system, a controller of the MDOF robot, and a welding machine of the welding system are disposed in the shell; the rechargeable battery pack is connected to an external power supply through a power port on the shell;
- the MDOF robot comprises an MDOF robot body, and the MDOF robot body and a teach pendant are in a signal connection with the controller in the shell through cables;
- the welding system comprises the welding machine located in the shell, and a welding gun connected to the welding machine and disposed at the tail end of the MDOF robot body;
- the 3D vision system comprises a 3D camera, the 3D camera is installed at the tail end of the MDOF robot body or on the welding gun, and is connected to an industrial personal computer on the shell through a cable, and the industrial personal computer is connected to the controller through a cable;
- the measurement accuracy of the 3D camera is not lower than 0.5 mm, and a deep frame rate of the 3D camera is greater than one frame per second;
- the MDOF robot has a robot arm with more than six degrees of freedom, and an arm reach of the robot arm is 0.5 m-2 m.
- a welding method based on the automatic welding system for large structural parts based on hybrid robots and 3D vision comprises the following steps:
- FIG. 1 is a structural diagram of a system according to the invention.
- the invention provides an automatic welding system and method for large structural parts based on hybrid robots and 3D vision, which expand the working range of industrial robots by means of a mobile robot and then accurately recognize and position a tail end through the 3D vision technique, so the working range of the system is expanded, the flexibility of the system is improved, and the cost of the system is reduced.
- the automatic welding system for large structural parts based on hybrid robots and 3D vision comprises a hybrid robot system composed of a mobile robot 3 and an MDOF robot 2 installed above the mobile robot 3 , a welding system installed at a tail end of the MDOF robot 2 and used for welding a target workpiece 1 , and a 3D vision system installed at the tail end of the MDOF robot 2 or on the welding system, and used for global calibration and positioning of the hybrid robot system, the welding system and the target workpiece.
- the mobile robot 3 comprises a mobile robot chassis 13 , wherein the mobile robot chassis 13 comprises a movement module, a control module, a navigation sensor and structural members.
- a shell 15 is fixedly disposed on the mobile robot chassis 13 , and a rechargeable battery pack 11 and a power supply 12 used for providing energy for the whole system, a controller 10 of the MDOF robot 2 , and a welding machine 9 of the welding system are disposed in the shell 15 , the rechargeable battery pack 11 is connected to an external power supply through a power port 14 on the shell 15 , and when the whole system works, power is supplied by the rechargeable battery pack 11 or the power supply 12 .
- the mobile robot 3 is a mobile robot platform with a function of global navigational positioning, the spatial positioning accuracy of the mobile robot 3 is superior to 20 mm, the global navigational positioning is rough positioning realized by navigation through one or more of an electromagnetic method, a two-dimensional code method, visual SLAM, visual tracking, inertial navigation; when the electromagnetic method or two-dimensional code method is used for navigational positioning, corresponding navigation lines 4 are laid on the ground.
- an electromagnetic method a two-dimensional code method
- visual SLAM visual tracking
- inertial navigation when the electromagnetic method or two-dimensional code method is used for navigational positioning, corresponding navigation lines 4 are laid on the ground.
- the vision tracking is used for positioning the mobile robot, one or more tracking targets 16 are printed on the shell 15 of the mobile robot 3 , and cameras photograph the target to determine the position of the mobile robot.
- the target may be a specially designed light emitting or reflecting structure, which is directly installed on the mobile robot.
- the mobile robot 3 is preferably a mobile robot platform with a lifting function.
- the controller 10 comprises: a controller used for controlling the movement of a robot motor and a driver used for driving the robot motor, which are collectively referred to as the controller.
- the MDOF robot 2 comprises an MDOF robot body 6 rigidly installed on the mobile robot 3 , the MDOF robot body 6 and a teach pendant are in a signal connection with the controller 10 in the shell 15 through cables, and the MODF robot 2 is carried by the mobile robot 3 to move, such that the working range of the MDOF robot is expanded.
- the MDOF robot 2 preferably has a robot arm with over six degrees of freedom, the robot arm is an industrial robot or a collaborative robot, and an arm reach of the robot arm is preferably 0.5 m-2 m.
- the welding system comprises the welding machine 9 located in the shell 15 , a welding gun connected to the welding machine 9 , and other necessary components, wherein the welding gun 7 is disposed at the tail end of the MDOF robot body 6 , and the other components comprise a wire feeder, a welding wire, a water tank, a protective gas, a storage device, and an air compressor, and are used for completing a whole welding process.
- the 3D vision system comprises a 3D camera 5 , wherein the 3D camera 5 is installed at the tail end of the MDOF robot body 6 , or on the welding gun 7 , or at other positions where the camera can photograph the target workpiece.
- the 3D camera 5 is connected to an industrial personal computer 8 on the shell 15 through a cable, and the industrial personal computer 8 can be installed at any suitable position of the shell 15 and is connected to the controller 10 through a cable.
- the 3D camera 5 acquires 3D feature information of a workpiece to be welded, the measurement accuracy of the 3D camera 5 is not lower than 0.5 mm, the deep frame rate of the 3D camera 5 is greater than one frame per second; the 3D camera 5 is a low-power, small-size and light-weight 3D camera; and the 3D camera 5 preferably uses laser light as a light source to improve the light resistance.
- the 3D camera 5 is preferably a 3D camera based on MEMS structural light to meet the above feature requirements.
- the 3D camera 5 is provided with a protection device, which is used for protecting the camera against high temperature, splash and dust, to ensure normal work of the camera.
- the target workpiece 1 is a large metal structural part suitable for welding, which means that at least one dimension of the workpiece is within 5 m-500 m; the target workpiece 1 is placed on a basic plane, which is a flat plane without ups and downs, preferably a horizontal plane; the mobile robot 3 carries the MDOF robot 2 , the 3D vision system and the welding system, and moves around the target workpiece in the basic plane, is roughly positioned by navigation, and expands the working range in a direction perpendicular to the basic plane by ascending or descending.
- a welding method based on the automatic welding system for large structural parts based on hybrid robots and 3D vision comprises the following steps:
- Step (1) specifically comprises:
- Step (2) comprises the following three sub-steps:
- the equation is solved through multiple times of photographing to obtain a coordinate transformation relation cam T robot between the 3D camera 5 and the MDOF robot 2 ;
- cam T tool cam T robot ⁇ robot T base ⁇ base
- cam T tool cam T robot ⁇ robot T base ⁇ tool T ⁇ 1 base ;
- Step (3) an origin of a workpiece coordinate system is set at a position where working features are obvious, preferably the intersection of multiple planes, or an angular point, which is beneficial to alignment.
- An X-direction, a Y-direction and a Z-direction of the workpiece coordinate system should be consistent with main structural feature directions; preferably, a longest dimension direction is selected as the X-direction, which is beneficial to placement.
- two or more limit mechanisms may be arranged in the X-direction to make the workpiece aligned with the X-axis.
- a support structure is adjusted to enable the X-direction, Y-direction and Z-direction of the workpiece coordinate system to be basically overlapped with the X-direction, Y-direction and Z-direction of the basic coordinate system, which means that the angle error is within 2°.
- the origin of the workpiece coordinate system is photographed by the 3D camera 5 to extract origin features the workpiece coordinate system, then coordinates of the origin of the workpiece coordinate system are transformed to the basic coordinate system of the robot system to obtain a transformation relation between the workpiece coordinate system and the basic coordinate system, and then, the position of the workpiece in the basic coordinate system is obtained.
- Step (3) specifically comprises the following sub-steps:
- transformation values of the coordinates of the workpiece to the basic coordinate system in three directions are ( ⁇ X T , ⁇ Y T , ⁇ Z T ).
- the sharp end of the welding gun of the MDOF robot touches the origin of the workpiece coordinate system, then the coordinates of the origin of the workpiece coordinate system are transformed to the basic coordinate system of the robot system to obtain the transformation relation between the workpiece coordinate system and the basic coordinate system, and then, the position of the workpiece in the basic coordinate system is obtained.
- the specific implementation has been described above.
- Step (4) the motion path of the robot, and the photographing position and pose of the camera are planned, wherein one photographing position and pose of the camera may correspond to one or more welding features, or multiple photographing positions and poses correspond to one welding feature, and the welding feature is a target welding position, which is a point, a line or a curve; the photographing position and pose of the camera should be a position and pose where the camera can easily photograph a target feature; when the camera is at this position and pose, the camera is located within an effective working range, the target feature is within the field range of the camera, and a principal normal direction of an area where the target feature is located should be parallel to a Z-direction of the camera to the maximum extent to realize an optimal photographing effect.
- the motion path of the robot is a shortest safety path for the robot to move to a target position, and at the target position, the arm reach and degree of freedom of the robot should allow the camera to reach a target photographing position and pose;
- the motion path of the robot, and the photographing position and pose of the camera are planned by means of off-line programming software through a parameterization method and is implemented through a teaching method.
- the motion control and welding programs comprise: a motion control program of the mobile robot 3 , and a control program and welding program of the MDOF robot 2 .
- the MDOF robot 2 is used as main control to communicate with and control the mobile robot 3 , the 3D camera and the welding system.
- the industrial personal computer 8 is used as main control to control the mobile robot 3 , the MDOF robot 2 and the welding system.
- an external PLC is used as main control to control the mobile robot 3 , the MDOF robot 2 and the welding system.
- the control program of the MDOF robot 2 comprises a motion control program of the robot arm, a communication program between the MDOF robot 2 and the camera, and a communication program between the MDOF robot 2 , the welding system and the mobile robot.
- the programs comprise: template programs manually written off-line, programs generated by off-line programming software using digital-analog drive, and a teach program generated by the teach pendant.
- the robot programs meet grammatical rules and data formats of robots of corresponding brands, and are able to run directly on the robots of the corresponding brands. If the programs are the template programs manually written off-line or the programs generated by off-line programming software using digital-analog drive, the robot programs are issued to controllers of the robots before running, which is realized through wired or wireless transmission or through copying by means of a memory device. In another implementation of the invention, an upper computer is used to control the robots to run the programs on-line, so the programs do not need to be issued to the controllers of the robots.
- the features of the method described above and below may be implemented through software, and can be implemented on a data processing system or other processing tools by executing a computer-executable instruction.
- the instruction may be a program code and is loaded into a memory (such as RAM) from a storage medium, or from another computer through the computer network.
- the features may be realized through a hard-wired circuit instead of software, or be realized through both the hard-wired circuit and the software.
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Plasma & Fusion (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Orthopedic Medicine & Surgery (AREA)
- Manipulator (AREA)
- Numerical Control (AREA)
Abstract
Description
-
- (1) Establishing a coordinate system of the hybrid robot system formed by the mobile robot and the MDOF robot;
- (2) Calibrating a relation between the welding system and the coordinate system of the hybrid robot system and a relation between the 3D vision system and the coordinate system of the hybrid robot system;
- (3) Placing the target workpiece in a working area, and aligning coordinates of the workpiece to a global coordinate system;
- (4) Planning a motion path of the mobile robot, and a photographing position and pose of the 3D camera in the 3D vision system;
- (5) Generating motion control and welding programs; and
- (6) Performing welding by the mobile robot.
-
- (1) Aiming at the features of large dimensions and poor consistency of large structural parts, hybrid robots and 3D vision are used to effectively improve the flexibility of the system and reduce the cost of the system;
- (2) The hybrid robots and the 3D vision technique are combined, that is, the rough positioning technique based on a mobile platform and the accurate recognition and positioning technique based on high-accuracy 3D vision are combined, such that the working range of the MDOF robot in the XYZ directions is expanded, and flexible welding of large structural parts is realized. Compared with traditional a solution for expanding the working range of a robot through an external shaft, this solution is low in cost; and compared with the solution adopting the external shaft, this solution can expand the working range of the robot in the XY directions to a greater extent, and can expand the working range in the XY directions freely as long as the condition permits. This solution has a great cost advantage in welding of large and ultra-large workpieces, and also has the advantages of occupying a small area and having no interference to hoisting of workpieces.
- (3) Compared with traditional robot welding systems based on touch positioning and laser positioning, the invention adopts 3D vision, thus having better error tolerance and lower requirements for the machining and assembling accuracy of workpieces, positioning accuracy of mobile robots and placement accuracy of the workpieces; and the cost is reduced, the flexibility is improved, and the working range is expanded.
- (4) The automatic welding system and method of the invention realize automatic welding of large structural parts, and compared with manual welding methods commonly used at present, labor is saved, production efficiency is improved, and welding quality is improved.
-
- (1) A coordinate system of the hybrid robot system formed by the
mobile robot 3 and the MDOFrobot 2 is established; - (2) A relation between the welding system and the coordinate system of the hybrid robot system and a relation between the 3D vision system and the coordinate system of the hybrid robot system are calibrated;
- (3) The
target workpiece 1 is placed in a working area, and coordinates of the workpiece are aligned to a global coordinate system; - (4) A motion path of the
mobile robot 3, and a photographing position and pose of the3D camera 5 in the 3D vision system are planned; - (5) Motion control and welding programs are generated; and
- (6) Welding is performed by the
mobile robot 3.
- (1) A coordinate system of the hybrid robot system formed by the
-
- (1.1) The
mobile robot 3 carrying the MDOFrobot 2, the 3D vision system and the welding system moves in a basic plane around thetarget workpiece 1, and is coarsely positioned by navigation, and expands the working range in a direction perpendicular to the basic plane by ascending or descending; - The mobile robot is roughly positioned by navigation through one or more of an electromagnetic method, a two-dimensional code method, visual SLAM, visual tracking and inertial navigation; when the electromagnetic method or the two-dimensional code method is used, corresponding
navigation lines 4 are laid on the ground. When the visual tracking is used for positioning themobile robot 3, one or more tracking targets 16 are printed on theshell 15 of themobile robot 3, and two or more extra cameras photograph the target to determine the position of the mobile robot. As an alternative solution, the target may be a specially designed light emitting or reflecting structure which is directly installed on the mobile robot, such that cameras at different angels can observe the targets from any angles; - (1.2) The coordinate system of the hybrid robot system is established: a coordinate system of the
mobile robot 3 is established, wherein an origin of the coordinate system is a fixed point in the basic plane, an XY plane is overlapped with the basic plane, a Z-direction is perpendicular to the basic plane, and the coordinate system is a global basic coordinate system; a coordinate system of theMDOF robot 2 is established, wherein for the convenience of operation, an origin of the coordinate system is located on a base of the robot arm, and an X-direction, a Y-direction and a Z-direction are overlapped with those of the coordinate system of themobile robot 3; a coordinate system of the 3D vision system is established, wherein an original of the coordinate system is an optical center of a lens of the3D camera 5, an X-direction of the coordinate system is an optical center of a projection system to which the optical center of the lens of the3D camera 5 points, a Y-direction is perpendicular to the X-direction and is parallel to an imaging chip of the camera, and a Z-direction is perpendicular to the XY plane and points right ahead the camera.
- (1.1) The
-
- (2.1) A coordinate relation between the
MDOF robot 2 and the3D camera 5 is calibrated to obtain a transformation relation between the coordinate system of the vision system of the camera and the coordinate system of theMDOF robot 2; - (2.11) First, it should be ensured that the
3D camera 5 is calibrated, and intrinsic parameters of the camera are obtained, wherein the intrinsic parameters comprise: focal length, position of a principal point, pixel size, resolution, and distortion parameters; - (2.12) Then, the
3D camera 5 and theMDOF robot 2 are calibrated, a homogeneous transformation matrix from the tail end of theMDOF robot 2 to the base of the robot is defined as robotTbase, and similarly, a transformation matrix from the3D camera 5 to a target object is defined as camTobj; the3D camera 5 is mounted on theMDOF robot 2 and photographs a calibration plate, coordinates of which are known, and the position and pose of theMDOF robot 2 are recorded; the calibration plate is kept still, the position and pose of theMDOF robot 2 are changed multiple times to photograph the calibration plate, wherein two different times of photographing may be expressed as: robot1Tbase·cam1Trobot1·objTcam1=robot2Tbase·cam2Trobot2·objTcam2;
- (2.1) A coordinate relation between the
(robot2 T −1 base·robot1 T base)·cam T robot=cam T robot·(obj T cam2·obj T −1 cam1)
-
- The energy equation is: P=|P′1(x,y,z)P″1 (x,y,z)|+|P′2(x,y,z)P″2(x,y,z)|+ . . . .
- Where, |P′1(x,y,z)P″1(x,y,z)| represents an Euclidean distance from P′1(x,y,z) to P″1(x,y,z), and the subscript represents multiple points;
- (2.2) A TCP coordinate system of the robot is calibrated to obtain a position transformation relation of a sharp end of the
welding gun 7 in the coordinate system of theMDOF robot 2; - A TCP calibration method of the robot is a direct input method, a four-point method or a six-point method. In this embodiment, the four-point method is used for TCP calibration;
- (2.21) A new TCP coordinate system of the robot is established;
- (2.22) A fixed point is placed in the working space of the
MDOF robot 2, wherein the fixed point is generally a conical sharp point; - (2.23) The pose of the
MDOF robot 2 is controlled to make a TCP be overlapped with the fixed point in the working space; - (2.24) The above step is repeated three times to enable TCPs to move to the same point by changing the pose of the
MDOF robot 2; - (2.25) Under the condition where the coordinates of the four TCPs in a world coordinate system are identical, an equation set is established and is solved to realize position calibration of the tool coordinate system, such that a pose transformation relation of the coordinates of the tail end of the
welding gun 7 of the welding system in the coordinate system of theMDOF robot 2 is obtained; - (2.3) The coordinate system of the MDOF robot and the coordinate system of the mobile robot are calibrated to obtain a matrix for transforming the coordinate system of the MDOF robot to the coordinate system of the mobile robot;
- The 3D camera is mounted on the MDOF robot to photograph the calibration plate, the coordinates of which are known, and the position and pose of the
mobile robot 3 are recorded; the calibration plate is kept still, the position of themobile robot 3 is changed multiple times as significantly as possible, and then the robot is adjusted to photograph the calibration plate again according to the transformation relation camTrobot between the camera and the robot, wherein two different times of photographing may be expressed as: baseTBASE robot1Tbase·camTrobot·objTcam1=baseTBASE robot2Tbase·camTrobot·objTcam2; - The equation is solved through multiple times of photographing to obtain a transformation relation baseTBASE between a base coordinate system of the
mobile robot 3 and the basic coordinate system of the system.
-
- (3.1) The
mobile robot 3 is controlled to move close to a target point, and theMDOF robot 2 is controlled to ensure that the3D camera 5 can photograph the target point; - (3.2) A point cloud is photographed and uploaded to the industrial
personal computer 8 to be processed; - (3.3) Data processing is performed according to the originally selected origin of coordinates of the workpiece, and in this embodiment, the origin of coordinates of the workpiece is the intersection of three planes; plane fitting is performed on the three-dimensional point cloud to find three planes, and then, the intersection O(x, y, z) of the three planes is solved, wherein O(x, y, z) is coordinates in the coordinate system of the camera;
- (3.4) O(x, y, z) is transformed to the basic coordinate system: Obase(x,y,z)=camTrobot□′ . . . □( ).
- (3.1) The
-
- (6.1) First, the
MDOF robot 2 and themobile robot 3 are controlled to return to the origin of coordinates; - (6.2) Next, the
mobile robot 3 is controlled to reach a first target position P1; - (6.3) Then, at P1, the
MDOF robot 2 is controlled to reach a first photographing position, then photographing is performed, and data is sent to the industrialpersonal computer 8 to be processed; if there is another photographing position at P1, theMDOF robot 2 continues to move to the next photographing position, then photographing is performed, and data is uploaded to the industrialpersonal computer 8; this step is repeated until all photographing tasks at P1 are completed; - (6.4) All point cloud information at P1 is fused by the industrial
personal computer 8, and key information of welding features is recognized, wherein the key information of the welding features is key point information of the welding features which are recognized through a geometric algorithm or a neural network method, with 3D point clouds as inputs; the key point information comprises an initial point, end point and direction of a segment, spline curve parameters, arc parameters, and coordinates of multiple discrete points; - As an alternative solution, the key feature information is recognized first, then features are fused to obtain new key feature information, wherein the new key feature information comprises an initial point, end point and direction of a segment, spline curve parameters, arc parameters, and coordinates of multiple discrete points;
- Point locations and direction information have been transformed to the basic coordinate system of the system;
- (6.5) A track of the welding gun is calculated according to the key point information, wherein the track of the welding gun comprises position and pose information of the welding gun;
- (6.6) The track of the
welding gun 7 is sent to thecontroller 10, thecontroller 10 guides themobile robot 3 to move after receiving the track of thewelding gun 7, and the corresponding welding program is called to perform welding; - (6.7) The
mobile robot 3 moves to a position P2 for another cycle, and Steps (6.3)-(6.6) are repeated until welding is completed or stopped.
- (6.1) First, the
Claims (7)
robot1 T base·cam1 T robot1·obj T cam1=robot2 T base·cam2 T robot2·obj T cam2 (1)
cam1 T robot1=cam2 T robot2=cam T robot (2)
(robot2 T −1 base·robot1 T base)·cam T robot=cam T robot·(obj T cam2·obj T −1 cam1) (3)
cam T tool=cam T robot·robot T base·base T tool=cam T robot·robot T base·tool T −1 base (4)
P=|P′ 1(x,y,z)P″ 1(x,y,z)|+|P′ 2(x,y,z)P″ 2(x,y,z)|+ . . . .
base T BASE robot1 T base·cam T robot·obj T cam1=base T BASE robot2 T base·cam T robot·obj T cam2;
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210362511.2A CN114434059B (en) | 2022-04-08 | 2022-04-08 | Automatic welding system and method for large structural part with combined robot and three-dimensional vision |
CN202210362511.2 | 2022-04-08 | ||
PCT/CN2022/106029 WO2023193362A1 (en) | 2022-04-08 | 2022-07-15 | Hybrid robot and three-dimensional vision based large-scale structural part automatic welding system and method |
Publications (2)
Publication Number | Publication Date |
---|---|
US20230390853A1 US20230390853A1 (en) | 2023-12-07 |
US11951575B2 true US11951575B2 (en) | 2024-04-09 |
Family
ID=81358983
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/041,129 Active US11951575B2 (en) | 2022-04-08 | 2022-07-15 | Automatic welding system and method for large structural parts based on hybrid robots and 3D vision |
Country Status (4)
Country | Link |
---|---|
US (1) | US11951575B2 (en) |
EP (1) | EP4279211A4 (en) |
CN (1) | CN114434059B (en) |
WO (1) | WO2023193362A1 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114434059B (en) | 2022-04-08 | 2022-07-01 | 西安知象光电科技有限公司 | Automatic welding system and method for large structural part with combined robot and three-dimensional vision |
CN114749848A (en) * | 2022-05-31 | 2022-07-15 | 深圳了然视觉科技有限公司 | Steel bar welding automatic system based on 3D vision guide |
CN117102725B (en) * | 2023-10-25 | 2024-01-09 | 湖南大学 | Welding method and system for steel-concrete combined structure connecting piece |
CN117484508B (en) * | 2023-12-11 | 2024-06-18 | 嘉兴布鲁艾诺机器人有限公司 | Intelligent control system and method for multi-joint robot for loading and unloading |
CN118023786B (en) * | 2024-04-09 | 2024-07-19 | 河南威猛振动设备股份有限公司 | Multi-station intelligent somatosensory welding equipment and welding method |
CN118023799B (en) * | 2024-04-11 | 2024-07-16 | 华南理工大学 | Autonomous welding operation and planning method of humanoid welding robot based on camera |
CN118061198B (en) * | 2024-04-18 | 2024-07-23 | 中国长江电力股份有限公司 | Automatic programming method of composite mobile robot for cladding processing of top cover of water turbine |
CN118099503B (en) * | 2024-04-26 | 2024-07-02 | 无锡黎曼机器人科技有限公司 | Production system capable of realizing automatic stacking of multi-level battery modules and production method thereof |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112060103A (en) | 2020-08-07 | 2020-12-11 | 北京卫星制造厂有限公司 | Movable ultrafast laser processing robot equipment and processing method |
CN112958959A (en) | 2021-02-08 | 2021-06-15 | 西安知象光电科技有限公司 | Automatic welding and detection method based on three-dimensional vision |
CN113634958A (en) | 2021-09-27 | 2021-11-12 | 西安知象光电科技有限公司 | Three-dimensional vision-based automatic welding system and method for large structural part |
US20220016776A1 (en) * | 2020-07-17 | 2022-01-20 | Path Robotics, Inc. | Real time feedback and dynamic adjustment for welding robots |
CN113954085A (en) | 2021-09-08 | 2022-01-21 | 重庆大学 | Intelligent positioning and control method of welding robot based on binocular vision and linear laser sensing data fusion |
CN114434059A (en) | 2022-04-08 | 2022-05-06 | 西安知象光电科技有限公司 | Automatic welding system and method for large structural part with combined robot and three-dimensional vision |
US20220258267A1 (en) * | 2021-02-15 | 2022-08-18 | Illinois Tool Works Inc. | Helmet based weld tracking systems |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2369845A1 (en) * | 2002-01-31 | 2003-07-31 | Braintech, Inc. | Method and apparatus for single camera 3d vision guided robotics |
SE1050763A1 (en) * | 2010-07-08 | 2010-07-12 | Abb Research Ltd | A method for calibration of a mobile robot |
JP2014128840A (en) * | 2012-12-28 | 2014-07-10 | Kanto Seiki Kk | Robot control system |
CN105904107A (en) * | 2016-04-21 | 2016-08-31 | 大族激光科技产业集团股份有限公司 | Mobile robot laser marking system and laser marking method |
CN112958974A (en) * | 2021-02-08 | 2021-06-15 | 西安知象光电科技有限公司 | Interactive automatic welding system based on three-dimensional vision |
-
2022
- 2022-04-08 CN CN202210362511.2A patent/CN114434059B/en active Active
- 2022-07-15 US US18/041,129 patent/US11951575B2/en active Active
- 2022-07-15 EP EP22871144.6A patent/EP4279211A4/en active Pending
- 2022-07-15 WO PCT/CN2022/106029 patent/WO2023193362A1/en unknown
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220016776A1 (en) * | 2020-07-17 | 2022-01-20 | Path Robotics, Inc. | Real time feedback and dynamic adjustment for welding robots |
CN112060103A (en) | 2020-08-07 | 2020-12-11 | 北京卫星制造厂有限公司 | Movable ultrafast laser processing robot equipment and processing method |
CN112958959A (en) | 2021-02-08 | 2021-06-15 | 西安知象光电科技有限公司 | Automatic welding and detection method based on three-dimensional vision |
US20220258267A1 (en) * | 2021-02-15 | 2022-08-18 | Illinois Tool Works Inc. | Helmet based weld tracking systems |
CN113954085A (en) | 2021-09-08 | 2022-01-21 | 重庆大学 | Intelligent positioning and control method of welding robot based on binocular vision and linear laser sensing data fusion |
CN113634958A (en) | 2021-09-27 | 2021-11-12 | 西安知象光电科技有限公司 | Three-dimensional vision-based automatic welding system and method for large structural part |
CN114434059A (en) | 2022-04-08 | 2022-05-06 | 西安知象光电科技有限公司 | Automatic welding system and method for large structural part with combined robot and three-dimensional vision |
Also Published As
Publication number | Publication date |
---|---|
CN114434059A (en) | 2022-05-06 |
EP4279211A1 (en) | 2023-11-22 |
US20230390853A1 (en) | 2023-12-07 |
CN114434059B (en) | 2022-07-01 |
WO2023193362A1 (en) | 2023-10-12 |
EP4279211A4 (en) | 2024-05-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11951575B2 (en) | Automatic welding system and method for large structural parts based on hybrid robots and 3D vision | |
CN101623867B (en) | Device and method for making robot track given route at high accuracy | |
CN113001535B (en) | Automatic correction system and method for robot workpiece coordinate system | |
EP1592536B1 (en) | A method and a system for programming an industrial robot to move relative to defined positions on an object, including generation of a surface scanning program | |
CN201253852Y (en) | Device for enabling robot to track designated path with high precision | |
CN110355788B (en) | Large-scale space high-precision online calibration system of mobile operation robot | |
US20220331970A1 (en) | Robot-mounted moving device, system, and machine tool | |
KR20040106251A (en) | Method of controlling the welding of a three-dimensional construction | |
CN114043087B (en) | Three-dimensional trajectory laser welding seam tracking attitude planning method | |
CN114289934B (en) | Automatic welding system and method for large structural part based on three-dimensional vision | |
CN110009685B (en) | Laser camera hand-eye calibration method applied to electric arc material increase | |
CN114001653B (en) | Center point calibration method for robot tool | |
CN110039520B (en) | Teaching and processing system based on image contrast | |
CN114800551B (en) | Robot-powered grinding and polishing device for complex curved surface and control method | |
CN108656120B (en) | Teaching and processing method based on image contrast | |
CN110695494B (en) | Corrugated plate external shaft tracking system and method thereof | |
CN113664831B (en) | Welding robot system and method for acquiring weld point cloud information | |
US20220134577A1 (en) | Image processing method, image processing apparatus, robot-mounted transfer device, and system | |
CN111283676B (en) | Tool coordinate system calibration method and calibration device of three-axis mechanical arm | |
CN111283323B (en) | Welding method, welding device, terminal equipment and storage medium | |
CN114571199A (en) | Screw locking machine and screw positioning method | |
JP2006334731A (en) | Product transport carriage, robot position measuring system and its measuring method | |
CN116175035B (en) | Intelligent welding method for steel structure high-altitude welding robot based on deep learning | |
CN108673514B (en) | Target teaching and processing method based on image contrast | |
CN116276877A (en) | Mobile parallel resource allocation docking robot based on laser navigation and visual positioning and workpiece docking method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: XI'AN CHISHINE OPTOELECTRONICS TECHNOLOGY CO., LTD, CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YANG, TAO;LI, HUANHUAN;PENG, LEI;AND OTHERS;REEL/FRAME:062642/0720 Effective date: 20221221 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: MICROENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO MICRO (ORIGINAL EVENT CODE: MICR); ENTITY STATUS OF PATENT OWNER: MICROENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |