Disclosure of Invention
In order to solve the above problems in the prior art, the present invention provides a weld feature extraction device and method.
In order to achieve the above object, the present invention adopts the following technical solutions.
In a first aspect, the present invention provides a weld feature extraction device, including: the main control unit is connected with the robot controller and the image acquisition card, and also comprises a sensor module which mainly consists of a first laser, a second laser and a CCD camera and is arranged at the front end of the welding gun; the first laser and the second laser are used for irradiating laser signals to the workpiece; the CCD camera is used for acquiring an image signal containing double laser stripes on the workpiece, and the image signal is converted into a digital signal by an image acquisition card and then is sent to the main control unit; the main control unit is used for obtaining the height of the welding gun and the tracking deviation of the welding seam through welding seam characteristic extraction, sending a deviation signal to the robot controller and controlling the welding gun on the welding robot to be aligned to the welding seam.
Further, the weld joint feature extraction comprises image preprocessing, double-laser stripe segmentation and curve fitting, and tracking deviation calculation.
Furthermore, the sensor module further comprises a first reflective mirror and a second reflective mirror which are obliquely arranged under the first laser and the second laser respectively, and the first reflective mirror and the second reflective mirror are used for reflecting laser signals emitted by the first laser and the second laser and irradiating the laser signals onto a workpiece.
Further, the sensor module further includes a filter and a polarizer mounted in front of the lens of the CCD camera.
Still further, the sensor module further includes a cover glass plate mounted adjacent to the side of the welding gun for preventing dust and spatter from entering the sensor module.
Still further, the method for calculating the height of the welding gun comprises the following steps:
calculating the actual physical length of each pixel point
:
In the formula (I), the compound is shown in the specification,
the focal height of the CCD camera, i.e. the vertical distance from the workpiece is
The actual length corresponding to the width of the image,
is the number of pixel points of the image width,
the angle of view of the CCD camera;
distance between two laser lines on workpiece
Comprises the following steps:
in the formula (I), the compound is shown in the specification,
the distance between two laser stripes on the image is represented by the number of pixel points;
by height difference
Expressed as:
in the formula (I), the compound is shown in the specification,
、
respectively are included angles between the first reflective mirror and the second reflective mirror and the vertical direction,
=
,
,
the distance between the intersection of the laser and the focus of the CCD camera is reflected by a first reflector and a second reflector, and the first reflector and the second reflector are installed at the same height and are symmetrical about the symmetry axis of the CCD camera;
obtained by the formulas (2) and (3):
height of welding gunHComprises the following steps:
in the formula (I), the compound is shown in the specification,
is the height difference between the welding gun and the focus of the CCD camera.
Still further, the method for calculating the weld joint tracking deviation comprises the following steps:
calculating the ordinate of the welding gun in the image:
in the formula (I), the compound is shown in the specification,
the horizontal distance of the welding gun from the center of the sensor module,
calculated by equation (1) and having an image size of
Pixel points;
fitting equation according to weld line straight line
Calculating the abscissa of the welding gun
:
Calculating the deviation between the welding gun and the welding seamΔ:
In the formula (I), the compound is shown in the specification,
the abscissa of the weld.
Further, the CCD camera is a WAT-704R type industrial camera.
Further, the image acquisition card is a DH-CG410 type acquisition card.
In a second aspect, the invention provides a method for extracting weld features by using the device, which comprises the following steps:
the first laser and the second laser irradiate laser signals to the surface of the workpiece;
the CCD camera shoots an image containing double laser stripes on the surface of the workpiece in real time;
the main control unit acquires the image in real time through an image acquisition card, and the height of the welding gun and the tracking deviation of the welding seam are obtained through welding seam characteristic extraction;
and the robot controller receives the deviation signal sent by the main control unit and controls the welding robot to enable the welding gun to be aligned with the welding seam.
Compared with the prior art, the invention has the following beneficial effects.
The invention is provided with a main control unit, a robot controller, an image acquisition card and a sensor module which mainly comprises a first laser, a second laser and a CCD camera and is arranged at the front end of a welding gun, wherein the first laser and the second laser are used for irradiating laser signals to a workpiece, the CCD camera is used for acquiring image signals containing double laser stripes on the workpiece, and the main control unit is used for obtaining the height of the welding gun and the tracking deviation of the welding seam through welding seam characteristic extraction and sending the deviation signals to the robot controller to control the welding gun on the welding robot to align the welding seam. According to the invention, the two lasers are arranged on the sensor module, so that the height of the welding gun and the tracking deviation of the welding seam can be accurately obtained, and the tracking precision of the welding seam is improved.
Detailed Description
In order to make the technical solution and advantages of the present invention more clear and obvious, the present invention is further described with reference to the accompanying drawings and the detailed description. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a block diagram of a weld feature extraction apparatus according to an embodiment of the present invention, including: the device comprises a main control unit 2, a robot controller 4 and an image acquisition card 3 which are connected with the main control unit 2, and a sensor module 1 which mainly comprises a first laser 11, a second laser 12 and a CCD camera 13 and is arranged at the front end of a welding gun; the first laser 11 and the second laser 12 are used for irradiating laser signals to the workpiece 5; the CCD camera 13 is used for acquiring an image signal containing double laser stripes on the workpiece 5, and the image signal is converted into a digital signal by the image acquisition card 3 and then is sent to the main control unit 2; the main control unit 2 is used for obtaining the height of the welding gun and the tracking deviation of the welding seam through welding seam characteristic extraction, sending a deviation signal to the robot controller 4 and controlling the welding gun on the welding robot to be aligned with the welding seam.
The device mainly comprises a main control unit 2, a robot controller 4, an image acquisition card 3 and a sensor module 1, wherein the sensor module 1 mainly comprises a first laser 11, a second laser 12 and a CCD camera 13. The connection relationship of the modules is shown in fig. 1, the robot controller 4 and the image acquisition card 3 are both electrically connected with the main control unit 2, and the image acquisition card 3 is electrically connected with the CCD camera 13. The embodiment realizes the welding operation to the workpiece 5 by controlling the position of the welding gun by the welding robot, and the sensor module 1 and the welding gun are both installed on the welding robot. The functional principle of each component will be described separately below.
The sensor module 1 belongs to a structured light vision sensor and mainly comprises a CCD camera 13 and a laser, wherein the laser emits laser signals to the surface of a workpiece 5, and the CCD camera 13 shoots an image containing laser stripes on the surface of the workpiece 5. The biggest difference from the existing structured light vision sensor is that the sensor module 1 of the present embodiment is modified from an existing single laser to provide two lasers, i.e., the first laser 11 and the second laser 12. Two lasers simultaneously emit laser signals to the surface of the workpiece 5, and the CCD camera 13 can shoot an image containing double laser stripes. The sensor module 1 is arranged at the front end of the welding gun in the welding direction, and a certain height difference exists between the sensor module 1 and the welding gun; the two lasers are located at the same height on both sides of the CCD camera 13. The double laser stripes comprise welding gun height information and welding line deviation information, and feature extraction is carried out based on the double laser stripes, so that high-precision welding gun height and welding line deviation can be obtained. Two lasers, optionally semiconductor lasers, are the most important components of the sensor module 1. Practice shows that the line width of the laser emitted by the laser is less than 1mm when the distance between the laser and an outlet is 120mm, and the effect of the shot image is optimal; when the wavelength is 600-700 nm, the arc light is less, so the wavelength of the laser is 650 nm.
The main control unit 2 is a control and data processing center of the device, and is mainly used for completing data processing tasks and coordinating the work of other modules by outputting various control signals. The data processing tasks completed by the main control unit 2 mainly include: weld joint feature extraction is performed by processing an image shot by the CCD camera 13, and the height of the welding gun and the weld joint deviation are calculated. The main control unit 2 outputs a deviation signal to the robot controller 4 to control the welding robot, thereby controlling the position of the welding gun. Since field control is required, the main control unit 2 generally employs an industrial control computer in which image processing software is installed. The image processing software is written by VC + +6.0 in combination with an open source computer vision library OpenCV. VC + +6.0 is a very powerful software development tool, and is widely used in the fields of industrial production, image processing, robots, etc. because it has the advantages of flexible operation, high efficiency, etc.
The image acquisition card 3 is connected with the CCD camera 13 through a video cable and is mainly used for acquiring images shot by the CCD camera 13 to the main control unit 2. The analog image signal is converted into a digital signal and then stored in the buffer memory of the main control unit 2 for the CPU to call.
And a robot controller 4 for controlling the welding robot. The robot controller 4 is a controller for controlling the robot, which is provided by a manufacturer, and is used in cooperation with the welding robot. The robot controller 4 is connected with the main control unit 2 through an external communication interface, and controls the welding robot by receiving and analyzing a control instruction sent by the main control unit 2.
As an optional embodiment, the weld joint feature extraction comprises image preprocessing, double-laser stripe segmentation and curve fitting, and tracking deviation calculation.
In this embodiment, the weld feature extraction includes several steps, such as image preprocessing, stripe segmentation, fitting, and deviation calculation. The image preprocessing method mainly comprises the following steps: ROI extraction, filtering processing, image threshold segmentation, edge extraction and deburring. When the welding seam is tracked, strict requirements are required on image processing time due to real-time requirements. And the characteristic information in the acquired welding seam image only occupies a small part of the whole image information, so that the processing speed and the welding seam tracking efficiency are improved, ROI extraction needs to be carried out on the acquired welding seam image, and the image processing time is reduced. Common ROI extraction methods include a human-computer interaction method, a method based on a point of regard, a method based on a visual attention model, a method based on specific object segmentation, and the like. Interference such as arc light, smoke, splash and the like can be generated in the welding process, so that a large amount of noise exists in the acquired image, and the noise can influence subsequent feature extraction and feature identification. Therefore, filtering and noise reduction processing is required for the image. The windowed image may be denoised using a fourier transform in the frequency domain. The image threshold segmentation is to segment the image by binarizing the image. The edge extraction and deburring are realized by adopting a Canny operator to obtain the boundary of the image. The Canny operator has the function of smoothing the image boundary and can play a role in filtering and noise reduction. The double laser stripe segmentation is realized by averaging the extracted image boundaries to obtain the center line of the weld, and the schematic diagram of the obtained upper and lower stripes is shown in fig. 2. Windowing the left straight line, the left oblique straight line, the right oblique straight line and the right straight line of the stripe, linearly fitting the windowed image by adopting a least square method, and then solving the characteristic points by intersecting the fitted straight lines. The tracking deviation calculation comprises height identification of the welding gun and deviation calculation of the welding seam, and is realized by calculation based on the obtained geometric characteristics of the upper and lower stripes and the installation parameters of the sensor module 1. The following examples will give specific calculation methods.
As an alternative embodiment, the sensor module 1 further includes a first reflective mirror 14 and a second reflective mirror 15, which are respectively installed under the first laser 11 and the second laser 12 and are obliquely placed, and are respectively used for reflecting the laser signals emitted by the first laser 11 and the second laser 12 and then irradiating the laser signals onto the workpiece 5.
The sensor module 1 of the present embodiment is further provided with a first reflecting mirror 14 and a second reflecting mirror 15 installed obliquely just below the first laser 11 and the second laser 12. The laser generated by the first laser 11 and the laser generated by the second laser 12 are vertically irradiated downwards, irradiate the first reflective mirror 14 and the second reflective mirror 15, reflect and irradiate the surface of the workpiece 5, and then are reflected and scattered by the surface of the workpiece 5 to enter the field range of the CCD camera 13. The angle of inclination of the two mirrors can be adjusted, for example to 15 ° each, as shown in figure 3.
As an alternative embodiment, the sensor module 1 further includes a filter and a polarizer mounted in front of the lens of the CCD camera 13.
Because a lot of smoke and splashes exist in the welding process, in order to prevent the smoke and splashes from entering the field range of the CCD camera 13 of the sensor module 1 and further polluting the acquired image, an optical filter and a polarizing film are arranged in front of the lens of the CCD camera 13.
As an alternative embodiment, the sensor module 1 further comprises a cover glass plate 16 mounted adjacent the side of the torch to prevent dust and spatter from entering the sensor module 1.
In order to prevent the dust and spatter generated during welding from entering the sensor module 1, the present embodiment is further provided with a spatter-proof partition plate, i.e., a cover glass plate 16, at a side close to the welding gun. The mounting position of the cover glass plate 16 is shown in fig. 1.
As an alternative embodiment, the method for calculating the height of the welding gun comprises the following steps:
calculating the actual physical length of each pixel point
:
In the formula (I), the compound is shown in the specification,
the focal point height of the
CCD camera 13, namely the vertical distance from the workpiece 5 is
The actual length corresponding to the width of the image,
is the number of pixel points of the image width,
the angle of view of the
CCD camera 13;
distance between two laser lines on workpiece 5
Comprises the following steps:
in the formula (I), the compound is shown in the specification,
the distance between two laser stripes on the image is represented by the number of pixel points;
by height difference
Expressed as:
in the formula (I), the compound is shown in the specification,
、
the angles between the first
reflective mirror 14 and the second
reflective mirror 15 and the vertical direction are respectively,
=
,
,
the
first reflector 14 and the
second reflector 15 are installed at the same height and are symmetrical about the symmetry axis of the
CCD camera 13 for the distance between the intersection point of the laser reflected by the
first reflector 14 and the
second reflector 15 and the focus of the
CCD camera 13;
obtained by the formulas (2) and (3):
height of welding gunHComprises the following steps:
in the formula (I), the compound is shown in the specification,
is the height difference between the welding gun and the focal point of the
CCD camera 13.
The embodiment provides a technical scheme for calculating the height of the welding gun. Since the welding torch and the
sensor module 1 are mountedThe waiting time differs by a fixed distance
The calculation of the height of the welding gun can therefore be converted into a calculation of the height of the
sensor module 1. The present embodiment calculates the height of the
sensor module 1 based on the geometric positional relationship of the first laser 11, the
second laser 12, the
CCD camera 13, the first reflecting
mirror 14, and the second reflecting
mirror 15, as shown in fig. 3. The specific calculation model is shown in the above formulas (1) - (6), and the detailed explanation is not provided here.
As an alternative embodiment, the method for calculating the weld joint tracking deviation includes:
calculating the ordinate of the welding gun in the image:
in the formula (I), the compound is shown in the specification,
the horizontal distance of the welding gun from the center of the
sensor module 1,
calculated by equation (1), the image size is
Pixel points;
fitting equation according to weld line straight line
Calculating the abscissa of the welding gun
:
Calculating the deviation between the welding gun and the welding seamΔ:
In the formula (I), the compound is shown in the specification,
the abscissa of the weld.
The embodiment provides a technical scheme for calculating the weld joint tracking deviation. In order to automate the welding process and improve the quality of the weld, the welding torch must be controlled in real time to be positioned directly above the weld, which requires that the deviation between the welding torch and the weld be acquired in real time. In the device according to the present embodiment, the
sensor module 1 is mounted in front of the welding gun (in the welding direction) at a fixed value
Therefore, the position of the welding gun can be predicted according to the connecting line of the two laser stripe groove vertexes. Compared with the existing weld
joint sensor module 1 with a single laser, the dual laser sensing mode of the embodiment can reduce the lead error. The deviation model between the welding gun and the welding seam is shown in figure 4, the position of the welding gun is shown as a point C in the figure, and the deviation model is calibrated before welding
X C . The image is sized to be N × M pixels, the welding torch should be theoretically aligned with the weld, if a single laser is used, the theoretical positions of the welding torches are B, D points in the figure, and if the dual laser of the present embodiment is used, the theoretical positions of the welding torches are a point in the figure. It is clear that the accuracy of the dual laser scheme is better than the accuracy of the single laser scheme, which can effectively reduce the pre-error of the
sensor module 1. The specific calculation model of the weld joint deviation is shown in the above formulas (7) - (9), and detailed explanation is not provided here.
As an alternative embodiment, the CCD camera 13 is a WAT-704R type industrial camera.
This embodiment gives a specific model of the CCD camera 13. In this embodiment, a CCD camera13 industrial camera with model WAT-704R is selected. The camera can meet the requirements of miniaturization and light weight. The main technical parameters are as follows: the focal length is 3.8mm and is adjustable; viewing angle range of (V x H) 51 0 *40 0 (ii) a The effective pixel is (V × H) 768 × 554.
As an alternative embodiment, the image acquisition card 3 is a DH-CG410 type acquisition card.
This embodiment shows a specific model of the image capture card 3. In this embodiment, the image capture card 3 is a DH-CG410 capture card. The main performance characteristics are as follows:
standard PAL, NTSC color/black and white video signals;
maximum resolution of the image: 768 × 576 × 32bit (pal), 640 × 480 × 32bit (ntsc);
flexibly acquiring images: the method can be carried out in various modes such as single field, single frame, continuous field, continuous frame, several fields or several frames at intervals and the like;
the system supports a plurality of image display and storage formats of YUV4:2, RGB32, RGB24, RGB15, RGB16 and Y8 Bit;
the brightness, hue, color saturation and contrast of the image can be set programmably.
Fig. 5 is a flowchart of a method for extracting weld features by using the device according to an embodiment of the present invention, where the method includes the following steps:
101, irradiating laser signals to the surface of a workpiece 5 by a first laser 11 and a second laser 12;
step 102, shooting an image containing double laser stripes on the surface of the workpiece 5 by the CCD camera 13 in real time;
103, acquiring the image in real time by the main control unit 2 through the image acquisition card 3, and extracting the height of the welding gun and the tracking deviation of the welding seam through the characteristic of the welding seam;
and 104, receiving the deviation signal sent by the main control unit 2 by the robot controller 4, and controlling the welding robot to enable the welding gun to align to the welding seam.
Compared with the technical solution of the embodiment of the apparatus shown in fig. 1, the method of this embodiment has similar implementation principle and technical effect, and is not described herein again.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.