WO2022188482A1 - Active and passive vision combination-based welding deviation detection system and detection method - Google Patents

Abstract

An active and passive vision combination-based welding deviation detection system and detection method. First, a wavelet transform method is used to analyze the type of image noise, and according to a characteristic that an energy distribution of a welding image is a Gaussian distribution, a LoG operator is specifically used to separately perform filtering processing on ROIs at different positions of the welding image; then, a proper image processing algorithm is designed to separately extract a welding wire center line and a welding seam groove center line; and finally, deviation values of the welding wire center line and the welding seam center line in the X-axis direction are calculated, and the obtained results are applied to subsequent automatic welding seam tracking control.

Description

Welding deviation detection system and detection method based on combination of active and passive vision technical field

The invention relates to a welding deviation detection system and a detection method based on the combination of active and passive vision, and belongs to the technical field of intelligent welding application.

Background technique

With the increasing shortage of manual welders and the rapid development of robot technology, robot automatic welding has become the development trend of pipeline welding construction. Weld seam tracking control is one of the key technologies to realize robot welding automation, and its core technology is welding deviation information detection.

At present, the sensing technologies used for welding deviation detection include contact sensing, arc sensing, ultrasonic sensing, electromagnetic sensing, infrared sensing and visual sensing. Among them, the welding deviation detection technology based on visual sensing is the most Promising sensing technology. Depending on whether the visual sensing system uses auxiliary light sources, visual sensing can be divided into passive vision with arc light and natural light as the light source and active vision with auxiliary lighting such as lasers.

Passive vision uses cameras to directly monitor the molten pool and welding torch at the arc part. Since the detection target and the welding torch are in the same position, there is no error problem caused by active vision advanced detection, but it is seriously interfered by arc light, which makes subsequent image processing and feature information extraction. There is great difficulty. Compared with passive vision, due to high laser brightness and good coherence, active vision mostly uses lasers as auxiliary light sources. This method can also obtain weld size and joint information. The technology is relatively mature, and mature products have been formed. , but due to the distance between the laser projection position and the welding torch, there is an advance detection error.

The combination of active and passive vision can make full use of the advantages of the two and make up for each other, and more welding information can be obtained. Moreover, how to use the visual sensing technology to obtain more welding information to improve the accuracy of welding deviation detection is a technical problem that those skilled in the art urgently need to solve.

SUMMARY OF THE INVENTION

Purpose: In order to overcome the deficiencies in the prior art, the present invention provides a welding deviation detection system and detection method based on the combination of active and passive vision. The method of combining active vision and passive vision is adopted to obtain the same frame of welding image simultaneously. The image information includes laser stripes in the weld zone and welding wire in the arc zone, realizing the complementary advantages of the two methods.

Technical scheme: in order to solve the above-mentioned technical problems, the technical scheme adopted in the present invention is:

The welding deviation detection system based on the combination of active and passive vision includes: a pipeline welding robot, an image acquisition system, and an image processing system. The pipeline welding robot includes a magnetic crawling welding robot and a welding power source; the image acquisition system includes an industrial camera, Laser and filter device, the installation direction of the laser is parallel to the welding gun, the angle between the optical axis of the industrial camera and the central axis of the laser is 30°, the industrial camera is inclined to shoot the arc area and the welding seam area, and the vertical distance between the laser and the welding gun is 30mm. The in-line laser stripes projected by the laser cross perpendicularly to the weld. A filter device is arranged on the lens of the industrial camera, and the filter device sequentially includes a filter, a polarizer, and a dimming glass. Filters, polarizers, and dimming glass are all arranged coaxially with the lens. The image processing system calculates the distance Δx between the center line of the workpiece groove and the center line of the welding wire along the x-axis direction according to the image acquired by the image acquisition system.

As a preferred solution, the central wavelength of the filter is 660 nm.

As a preferred solution, the dimming glass is circular and is composed of two semi-circular glasses with different light transmittances. In the light sheet, the dividing line between the left and right semicircles is projected between the end of the welding wire and the laser stripe.

The welding deviation detection method based on the combination of active and passive vision includes the following steps:

Step 1: Select the ROI area in the image;

Step 2: Denoise the ROI area;

Step 3: Obtain the center point of the end of the welding wire in the ROI area;

Step 4: Obtain the weld feature points in the ROI area;

Step 5: Calculate the welding deviation according to the center point of the end of the welding wire and the characteristic point of the welding seam.

As a preferred solution, the specific steps of step 1 are as follows: according to the position of the laser stripe and the end of the welding wire in the welding image, two rectangular frames are respectively set, and the laser stripe and the end of the welding wire are respectively set in the rectangular frame.

As a preferred solution, the specific steps of step 2 are as follows: use the LoG operator to filter the image in the rectangular frame to obtain the bright spot area at the end of the welding wire and the black line segment at the position of the laser stripe.

As a preferred solution, the specific steps of step 3 are as follows: obtain the center point c for the bright spot area.

As a preferred solution, the specific steps of step 4 are as follows: using the Gaussian fitting method to extract the center point of the black line segment, and using the weighted least squares method to fit the horizontal line L ₁ on the left side of the groove, the oblique line on the left side of the groove L ₂ , For the inclined line L3 _on the right side of the groove and the horizontal line L4 on the right side _of the groove, the intersection of L1 and L2 is used as the left weld feature point a, and the intersection of L3 and L4 is used as the right weld feature point b.

As a preferred solution, the specific steps of step 5 are as follows: connect the left welding seam feature point a and the right welding seam feature point b in the image, and draw a vertical line x ₁ of the straight line ab, and the center point c in the image is From the starting point, a vertical line x ₂ is drawn, and the vertical line distance Δx between the vertical line x ₁ and the vertical line x ₂ is obtained as the welding deviation amount.

Beneficial effects: The welding deviation detection system and detection method based on the combination of active and passive vision provided by the present invention design a dimming glass, and the content of the welding image collected by the vision system includes the tip of the welding wire surrounded by the arc and the shape of the welding seam. The laser stripes of the appearance are very large, and a large amount of welding information is obtained.

The image processing method of the invention first uses the wavelet transform method to analyze the image noise type, and according to the characteristic that the energy distribution of the welding image is Gaussian distribution, the LoG operator is used to filter and process the ROI regions at different positions of the welding image respectively; Next, a suitable image processing algorithm is designed to extract the centerline of the welding wire and the centerline of the weld groove respectively. Automatic seam tracking control.

It has been verified that the method of obtaining the welding deviation by extracting the center position of the welding image welding wire and the welding seam in the present invention has high tracking accuracy and strong real-time performance, and can meet the requirements of the welding seam tracking accuracy.

Description of drawings

Figure 1 is a schematic diagram of the structure of the welding deviation detection test system.

Figure 2 is a schematic structural diagram of a specially designed dimming glass.

FIG. 3 is a schematic diagram of the structure of the target imaging model.

FIG. 4 is a schematic diagram of a welding image.

Figure 5 is a schematic diagram of welding deviation.

FIG. 6 is a schematic diagram of the welding deviation measurement process.

FIG. 7 is a schematic diagram of the ROI selected from the welding image.

Figure 8 is a histogram of energy distribution after wavelet transform.

Figure 9 is a schematic diagram of the denoising result of ROI1.

Figure 10 is a schematic diagram of the ROI2 denoising result.

Figure 11 is a schematic diagram of the detection result of the position of the tip of the welding wire.

Figure 12 is a schematic diagram of the center point of the laser stripe.

FIG. 13 is a schematic diagram of a straight line fitting result.

Figure 14 is a schematic diagram of the welding deviation test device on site.

Figure 15 is a schematic diagram of part of the image processing results.

Fig. 16 is a schematic diagram of the deviation correction value curve.

Figure 17 is a schematic diagram of the welding effect.

Detailed ways

The present invention will be further described below in conjunction with specific embodiments.

Example 1

As shown in Figure 1, the welding deviation detection system based on the combination of active and passive vision in this example is applied to all-position welding of pipeline welding robots, including: pipeline welding robots, image acquisition systems, and image processing systems. The pipeline welding robot includes magnetic suction 3. Welding power source. Among them, this example is tested, the magnetic crawling welding robot adopts GMAW method for welding, the welding test uses 45# carbon steel pipe 1, the base metal specification is φ400mm×20mm, the groove 4 is V-shaped, and the groove angle is 60 °. The welding power source is Panasonic YD-500GS5 welding machine, the shielding gas is 80% Ar+CO ₂ 20%, the welding material is the Atlantic brand ER50-6 solid welding wire, and the diameter of the welding wire is 1.2mm. The welding process parameters of the pipeline welding robot test are shown in Table 1.

Table 1 Main welding test parameters

parameter name	parameter value
Welding current I/A	138
Welding voltage U/V	17
Welding speed v/(mm.s -1 )	2.8
Welding torch swing speed v/(mm.s -1 )	30
Welding torch swing amplitude w/mm	7
Welding torch left and right delay t/ms	300
Gas flow q/(L.min -1 )	25

The image acquisition system includes an industrial camera 5, a laser 10, and a filter device.

As shown in Figure 2-Figure 3, the industrial camera is used to take welding images during the welding process. The welding workpiece is a horizontal fixed pipe, and a magnetic welding robot crawls around the circumference of the pipe. The industrial camera uses a Basler acA1300-60gm CMOS camera with a maximum capture frame rate of 60fps. The laser center wavelength is 660nm and the power is 16mW. The installation direction of the laser 10 is parallel to the welding torch 2, the angle between the optical axis of the industrial camera and the central axis of the laser is 30°, the industrial camera is inclined to shoot the arc area and the welding seam area, the vertical distance between the laser and the welding torch is 30mm, and a The word-line laser stripes intersect perpendicularly to the weld.

The lens 6 of the industrial camera is provided with a filter device, and the filter device sequentially includes a filter 7 , a polarizer 8 , and a dimming glass 9 . The filter, polarizer and dimming glass are all arranged coaxially with the lens. One end of the filter is installed on the lens of the industrial camera, one end of the polarizer is installed on the other end of the filter, and the dimming glass is installed on the polarizer. on the other end of the sheet. Among them, the central wavelength of the filter is 660nm, the arc light intensity is weak in this band, and the polarizer is used to reduce the interference of reflected light on the surface of the welding workpiece. The dimming glass is circular and is composed of two semi-circular glass with different light transmittances. , The dividing line of the right two semicircles is projected between the end of the welding wire and the laser stripes. The 4.4% light reduction film is conducive to the collection of arc light, and the 100% transparent glass is conducive to the collection of laser stripes.

As shown in FIG. 4 , the welding image collected by the detection system of the present invention has the following characteristics: (1) the same frame of image includes the arc, the end of the welding wire, and the laser stripe, and the end of the welding wire is surrounded by the arc; (2) the image is contaminated by various types, and the pollution source includes the arc ③ The brightness of the laser line is high, and it can reflect the information of the groove section; ④ The area with the largest gray value in the upper half of the image is concentrated in the arc area.

The image processing system includes industrial computer and computer image processing software.

The image acquisition system acquires images during the welding process, and then transmits the image data to the industrial computer through a gigabit network cable. The computer image processing software is installed and operated in the industrial computer to process and analyze the welding images, and finally calculate the welding deviation.

As shown in Figure 5, the image processing system calculates the distance Δx between the center line of the groove and the center line of the welding wire along the x-axis direction as the correction amount of the welding torch.

Example 2

As shown in FIG. 6 , this embodiment discloses a welding deviation detection method based on the combination of active and passive vision, which is applied to the welding deviation detection system based on the combination of active and passive vision in Embodiment 1, and includes the following steps:

Step 1: Select ROI: ROI (Region of interset), that is, the region of interest. In the process of image processing, processing the image ROI will not only reduce the amount of image processing data and improve the processing speed, but also remove the interference other than the ROI and improve the accuracy. Since the relative positions of the CMOS camera, laser, and welding gun are fixed, and they perform yaw motion together, the laser line and welding wire always appear in a fixed position in the welding image, and the arc position is also relative in the image when the welding wire burns. stable. According to the position of the laser stripe and the end of the welding wire in the welding image, two rectangular frames are set respectively, and the laser stripe and the end of the welding wire are respectively set in the rectangular frame, as shown in Figure 7. The size and position of the ROI box may vary and can be preset on the visual sensing system software interface.

Step 2: Image de-noising: The image becomes blurred due to the large amount of spatter, arc light and smoke and other disturbances accompanying the welding process. The purpose of image denoising is to design appropriate filters to reduce image noise interference. According to the existing research, after the blurred image is wavelet transformed, the high frequency sub-band coefficients will change with the different types of noise. In addition, the image noise generated in the welding process is mainly divided into Gaussian noise and salt and pepper noise. Therefore, the type of noise can be determined by observing the high frequency subband coefficient histogram of the welding image. By analyzing Fig. 8, it can be determined that the noise type in the welding image is Gaussian noise.

The LoG (Laplance-of-Gauss) operator is the Laplacian-Gaussian operator. Its processing of images includes two steps: first, use Gaussian function to low-pass filter the image to suppress noise interference, and then perform Laplacian The second order differential operation of the Si operator. Since the grayscale image is a two-dimensional function, the expression for this process is as follows:

where I(x,y) is the filtered image, f(x,y) is the input image, σ represents the standard deviation of the Gaussian distribution,

is the LoG filter, and its calculation expression is:

In addition to smoothing the image and suppressing Gaussian noise, LoG operator filtering also has a feature that when the scale is equal to the scale of the Gaussian target, the response value is the largest, so LoG filtering is suitable for image filtering with a constant target width. It can be seen from the image of the ROI1 area that the end of the molten welding wire is surrounded by the arc and the shape changes little. Using this feature, the LoG operator is used to filter the image of the ROI1 area, and a bright spot area appears at the end of the welding wire, which is the response. The maximum position, Figure 9 shows the effect of filtering processing of the image in the ROI1 area when σ=3. Taking advantage of the basically consistent width of the laser stripes, the LoG operator is also used to filter the image in the ROI2 area, and a black line appears at the position of the laser stripe. Figure 10 shows the effect of filtering the image in the ROI2 area when σ=10. It can be clearly seen that both the wire end position and the laser stripe are highlighted.

Step 3: Detection of the position of the end of the welding wire: In order to determine the position of the center line of the welding wire, the center point c is obtained from the bright spot area in the filtered ROI1 area image. Figure 11 shows the detection result of the position of the tip of the welding wire, and the red "X" mark in Figure 11 is the center point of the tip of the welding wire.

Step 4: Weld seam feature point extraction: add black line segments to the filtered ROI2 area image, use Gaussian characteristics, and use Gaussian fitting method to extract the center points of the black line segments, as shown in Figure 12 for the extraction results of laser stripe center points. Show. Next, use the weighted least squares method to fit the horizontal line L ₁ on the left side of the groove, the diagonal line L ₂ on the left side of the groove, the diagonal line L ₃ on the right side of the groove, and the horizontal line L ₄ on the right side of the groove. Take the intersection of L1 and L2 as the left weld feature point a, and the intersection of L3 and L4 as the right weld feature point b. The straight line fitting results are shown in Figure 13.

Step 5: Calculate the welding deviation: connect the feature point a of the left welding seam and the feature point b of the right welding seam in the image, and make a vertical line x ₁ of the straight line ab, and use the center point c in the image as the starting point to make a vertical line. Line x ₂ , the vertical distance Δx between the vertical line x ₁ and the vertical line x ₂ is obtained as the welding deviation amount.

Example 3:

As shown in Figure 14, in order to verify the accuracy of the proposed deviation detection method, the test was carried out on the groove processed by the lathe, and the groove width processed by the lathe was uniform, simulating the groove after bevel machining. During the welding process, the welding deviation detected by the vision system is sent to the welding control system, which in turn controls and adjusts the swing center of the welding torch. Figure 15 shows the processing effect of a typical welding image, the real-time deviation correction value is shown in the curve of Figure 16, and the overall welding result is shown in Figure 17. Through a large number of welding tests, the welding error is basically within 0.2mm, which meets the requirements of automatic tracking of welding seams in all positions of the pipeline.

The above is only the preferred embodiment of the present invention, it should be pointed out that: for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made, and these improvements and modifications are also It should be regarded as the protection scope of the present invention.

Claims (10)

1. A welding deviation detection system based on the combination of active and passive vision, including: a pipeline welding robot, characterized in that: it also includes an image acquisition system and an image processing system, and the image acquisition system includes an industrial camera, a laser, and a filter device. The installation direction is parallel to the welding torch of the pipeline welding robot. The industrial camera tilts to shoot the arc area and the welding seam area. The linear laser stripe projected by the laser crosses the welding seam vertically. The lens of the industrial camera is provided with a filter device. It includes a filter, a polarizer, and a dimming glass in sequence. The image processing system calculates the distance Δx between the center line of the workpiece groove and the center line of the welding wire along the x-axis direction according to the image obtained by the image acquisition system.
2. The welding deviation detection system based on the combination of active and passive vision according to claim 1, wherein the angle between the optical axis of the industrial camera and the central axis of the laser is 30°, the vertical distance between the laser and the welding torch is 30mm, and the filter The light plate, polarizer, and dimming glass are all arranged coaxially with the lens.
3. The welding deviation detection system based on the combination of active and passive vision according to claim 1, wherein the center wavelength of the filter is 660 nm.
4. The welding deviation detection system based on the combination of active and passive vision according to claim 1, characterized in that: the dimming glass is circular, composed of two semi-circular glasses with different light transmittances, and the left half of the glass is made of transparent glass. The light rate is 100% transparent glass, the right half is made of 4.4% attenuation film, and the dividing line between the left and right semicircles is projected between the end of the welding wire and the laser stripe.
5. The welding deviation detection method based on the combination of active and passive vision is characterized in that it includes the following steps:

   Step 1: Select the ROI area in the image;

   Step 2: Denoise the ROI area;

   Step 3: Obtain the center point of the wire end in the ROI area;

   Step 4: Obtain the weld feature points in the ROI area;

   Step 5: Calculate the welding deviation according to the center point of the end of the welding wire and the characteristic point of the welding seam.
6. The welding deviation detection system method based on the combination of active and passive vision according to claim 1, characterized in that: the specific steps of step 1 are as follows: according to the position of the laser stripe and the end of the welding wire in the welding image, two rectangles are respectively set frame, place the laser stripe and the wire end in a rectangular frame, respectively.
7. The welding deviation detection method based on the combination of active and passive vision according to claim 1, wherein: the specific steps of step 2 are as follows: use LoG operator to filter the image in the rectangular frame to obtain the bright spot at the end of the welding wire area, the blackened line segment at the position of the laser stripe.
8. The welding deviation detection method based on the combination of active and passive vision according to claim 1, is characterized in that: the concrete steps of described step 3 are as follows: obtain the center point c to the bright spot area.
9. The welding deviation detection method based on the combination of active and passive vision according to claim 1, characterized in that: the specific steps of step 4 are as follows: using Gaussian fitting method to extract the center point of the black line segment, and using the weighted least squares method to respectively fit The horizontal line L ₁ on the left side of the groove, the diagonal line on the left side of the groove L ₂ , the diagonal line on the right side of the groove L ₃ , and the horizontal line on the right side of the groove L ₄ . The intersection of L3 and L4 serves as the right weld feature point b.
10. The welding deviation detection method based on the combination of active and passive vision according to claim 1 is characterized in that: the specific steps of step 5 are as follows: connect the left welding seam feature point a and the right welding seam feature point b in the image, and do Draw the vertical line x ₁ of the line ab, draw the vertical line x ₂ with the center point c in the image as the starting point, and obtain the vertical distance Δx between the vertical line x ₁ and the vertical line x ₂ as the welding deviation.

Images