CN104400265A - Feature extraction method applicable to corner weld of laser vision guided welding robot - Google Patents

Abstract

The invention provides a feature extraction method applicable to a corner weld of a laser vision guided welding robot. The method can obtain the corner angle and the weld point coordinates of the corner weld. The method comprises the following steps: in the part of determination of the corner angle, collecting the corner images with feature light strips of the welding robot in two poses, and then fitting the equations of two corner surfaces to determine the corner angle; and in the part of extraction of weld points, first cutting out ROI zones, then conducting median filtering, binaryzation and thinning, and then extracting light strip straight lines by virtue of Hough transformation and recognizing the weld points. The determination of the corner angle is beneficial to effectively coordinating the welding robot to determine the poses at the beginning of welding; an effective reorganization method of the weld points ensures real-time and accurate extraction of the welding points; and the feature extraction of the corner weld is complete, is of great significance to the practical welding link process, and has favorable practicability.

Description

The extracting method of the welding robot corner connection characteristics of weld seam that a kind of laser vision guides

Technical field

The present invention relates to the welding robot of a kind of laser vision sensor guiding and the extracting method of diametrical connection characteristics of weld seam thereof, specifically refer to that the welding robot soldering angle in diametrical connection weld seam welding process guided based on line laser structured light vision sensor determines the recognition methods with weld bead feature points.

Technical background

In recent years, along with the develop rapidly of national economy, country constantly expands the demand of metal material, quality requirement improves constantly, and weld the important process link used as metal material, the quality of welding quality is directly connected to the use of metal material.Existing welding method labour intensity is large, poor working environment, complex process, large to injury of human, and welding quality mainly ensures by the human factor such as individual skill, experience of welder, so realize Automation of Welding, intellectuality can effectively liberate labour, improve welding quality, significant to China's modernization, industrialized development.

The key issue realizing Automation of Welding be weld seam from motion tracking, welding seam is combined with Robotics by the welding robot (schematic diagram is as shown in Figure 2) that laser vision guides, and can effectively solve a soldering joint automatic tracking difficult problem.Laser structure light is adopted to guide welding robot to become the main flow of current Intelligent welding robot system as active light vision sensor.

In welding procedure, corner connection weld seam is a kind of important welds types.Corner connection characteristics of weld seam mainly comprises the determination of the corner connection angle of corner connection weld seam and spot welds accurately identifies.The identification of corner connection characteristics of weld seam is the key link realizing fillet welding seam welding, and it identifies the precision extracted, and directly has influence in industrial processes and follows the tracks of and the precision of welding.

In corner connection characteristics of weld seam, Chinese scholars biases toward the recognition methods of research weld bead feature points, thinks and identifies that weld bead feature points can realize welding, but only have spot welds three-dimensional information, cannot automatically determine robot initial welding pose, be difficult to real realize Automation of Welding.Herein in conjunction with practical application request, use for reference the achievement in research of forefathers in spot welds identification, provide comprehensive corner connection weld seam feature extracting method, the corner connection angle comprising corner connection weld seam is determined accurately to identify with spot welds, makes welding robot complete weld task efficiently.

Summary of the invention

For determining the corner connection angle of corner connection weld seam, and the Real time identification realizing corner connection weld seam is extracted, and the invention provides the corner connection weld seam characteristic detection method of the welding robot platform based on laser structure light vision guide.

The key technology scheme that patent of the present invention adopts is: on the welding robot system demarcated, and by the corner connection image determination corner connection angle under two poses, and adopts image processing algorithm identification corner connection weld bead feature points.

Described in technical scheme provided by the invention, laser structure light vision sensor guides welding robot corner connection weld seam characteristic recognition method as follows:

The first step, determines the corner connection angle of corner connection weld seam; Given welding robot two pose T ₁, T ₂, at T ₁pose place acquisition angle welding seam image, extracts striation straight line and spot welds P ₁(identify spot welds method as second step), and on the two sections of striation straight lines of two planes belonging to corner connection respectively any selected point P ₁₀, P ₁₁, at T ₂pose place acquisition angle welding seam image, extracts striation straight line and spot welds P ₂, and on the two sections of striation straight lines of two planes belonging to corner connection respectively any selected point P ₂₀, P ₂₁, due to P ₁, P ₁₀, P ₂, P ₂₀and P ₁, P ₁₁, P ₂, P ₂₁belong to two corner connection faces of corner connection weld seam respectively, by the equation in four somes matching, two corner connection faces, and then the angle in two corner connection faces can be obtained.

Second step, accurately identifies spot welds; The welding robot system acquisition angle welding seam image guided by laser vision, grid windowing process is carried out to the image gathered and intercepts out ROI region, medium filtering, binaryzation, refinement are carried out to the ROI region intercepted out, image after refinement adopts Hough transform extract striation straight line, and then obtain spot welds coordinate.

Beneficial effect of the present invention: provide a kind of feature extracting method being applicable to the welding robot corner connection weld seam that laser vision guides, the method can obtain corner connection angle and the spot welds coordinate of corner connection weld seam.The determination of corner connection angle is of value at the beginning of starting in welding, effective coordination welding robot determination pose, more efficiently complete weld task, at the Extraction parts of spot welds, take on the basis intercepting ROI region, adopt and be verified all secure binaryzation of robustness, real-time, thinning algorithm, ensure that spot welds is extracted accurate in real time.This corner connection weld seam feature extraction is comprehensive, and to actual welding, also termination process is significant, has good practicality.

Accompanying drawing explanation

Fig. 1 overview flow chart of the present invention.

Fig. 2 laser vision guides welding robot schematic diagram.

Fig. 3 corner connection angle determination schematic diagram.

Detailed description of the invention

For making the object, technical solutions and advantages of the present invention clearly understand, here is specific implementation process, and with reference to accompanying drawing, is described in further detail the present invention.

Basic ideas of the present invention are: according to the actual application background of the welding robot that laser vision guides, in conjunction with the actual welding feature of corner connection weld seam, extract corner connection characteristics of weld seam.Corner connection characteristics of weld seam mainly comprises two information content, corner connection angle and spot welds.In corner connection angle determining section, gather the corner connection image of the band feature striation under two poses, solve the equation of two planes and then determine the angle of two planes, schematic diagram as shown in Figure 3; At the Extraction parts of spot welds, on the basis intercepting ROI region, adopt and be verified all secure binaryzation of robustness, real-time, thinning algorithm, accurately identify spot welds in real time.This corner connection weld seam feature extraction is comprehensive, and to actual welding, also termination process is significant, has good practicality.

Fig. 1 is integral calibrating flow chart of the present invention.Figure (a) is for determining corner connection angle; Because two planes determine that its angle is determined immediately, by solving the equation of two planes respectively and then determining the angle of two planes.Two plane equations are by the some matching of four in plane; The identification division that figure (b) is spot welds; Medium filtering is carried out on the basis intercepting ROI region, then carries out binaryzation, refinement, and then carry out Hough transform identification spot welds.

Described first step concrete grammar is as follows:

1.1, vision guide welding robot parameter and transformation relation

As shown in Figure 2, the welding robot Mathematical Modeling schematic diagram of laser structure light vision guide, its major parameter comprises camera intrinsic parameter K, trick relational matrix T, optic plane equations its Mathematical Modeling is:

(1), during system works, vision sensor obtains characteristic point n=(u, v, l) on structured light ^t, at the three-dimensional coordinate X of camera coordinate system _c:

$\left\{\begin{array}{c}{X}_C=z_c{K}^{-1}n\\ \left[\begin{array}{cc}\stackrel{\→}{w}& 1\end{array}\right]\left[\begin{array}{c}{X}_C\\ 1\end{array}\right]=0\end{array}\right.$

Wherein X _c=(x _c, y _c, z _c) ^tthat weld bead feature points is at camera coordinate system three-dimensional coordinate; $K=\left[\begin{array}{ccc}{k}_x& s& u_0\\ 0& k_y& v_0\\ 0& 0& 1\end{array}\right]$ Video camera internal reference matrix, wherein k _x, k _ythe proportionality coefficient of reflection image u axle, v axle, s reflects the angle situation of two axles, u ₀, v ₀for the intersection point of optical axis straight line and CCD plane; N=(u, v, l) ^timage homogeneous coordinates corresponding to weld bead feature points; the optic plane equations parameter of structured light plane under camera coordinate system.

(2) characteristic point camera coordinate system coordinate transform to basis coordinates system of robot as shown in the formula:

X _W＝R _HWRX _C+R _HWt+t _HW

Wherein X _wthe coordinate of weld bead feature points in basis coordinates system of robot, R _hW, t _hWbe the rotation and translation relation of basis coordinates system of robot and ending coordinates system, R, t are the relations of robot end's coordinate system and camera coordinate system, i.e. trick matrix $T=\left[\begin{array}{cc}R& t\\ 0& 1\end{array}\right].$

1.2, the some P on image ₁, P ₂, P ₁₀, P ₁₁, P ₂₀and P ₂₁each point transforms to welding robot basis coordinates system

If laser instrument is at pose T ₁time project laser stripe in two planes of object as shown in Figure 3, laser stripe is a broken line, if P ₁for the broken line flex point (being weld bead feature points during welding) on the intersection of two planes, at P ₁both sides laser stripe on get respectively a bit, be set to P ₁₀and P ₁₁, then P ₁₀, P ₁₁also the point in two planes is respectively.Detect through laser vision sensor and obtain P ₁point three-dimensional coordinate under camera coordinate system is X _c1(x _c1, y _c1, z _c1), its coordinate X under basis coordinates system of robot can be tried to achieve _w1(x _w1, y _w1, z _w1):

[x _W1，y _W1，z _W1，1] ^T＝T ₁T[x _C1，y _C1，z _C1，1] ^T

Wherein $T_1=\left[\begin{array}{cc}{R}_{\mathrm{HW}}& t_{\mathrm{HW}}\\ 0& 1\end{array}\right]$ For robot posture information, T is trick matrix.

Change robot pose and reach T ₂, in like manner can solve P ₂, P ₂₀, P ₂₁.

1.3, plane included angle solves

By non-colinear coplanar 4 P ₁, P ₂, P ₁₀, P ₂₀, matching obtains the equation of plane 1:

a ₁x+b ₁y+c ₁z+1＝0

By non-colinear coplanar 4 P ₁, P ₂, P ₁₁, P ₂₁matching obtains the equation of plane 2:

a ₂x+b ₂y+c ₂z+1＝0

Namely the normal vector of two planes is respectively with

$\cos \left(\mathrm{\θ}\right)=\frac{\stackrel{\→}{n_1}\·\stackrel{\→}{n_2}}{\left|\stackrel{\→}{n_1}\right|\·\left|\stackrel{\→}{n_2}\right|}$

Corresponding corner connection angle θ can be solved according to cos θ value.

Described second step concrete grammar is as follows:

2.1, ROI is intercepted

To the weld seam piece image gathered, intercept ROI region method as follows, sample at certain intervals entire image in level and vertical direction, carry out gray value to each pixel of sampling and add up, the average of sampling result is set to M gray value as a setting.

$M=\frac{1}{N_{1}W+N_{2}H}(\underset{i=1}{\overset{W}{\mathrm{\Σ}}}\underset{j=1}{\overset{N_1}{\mathrm{\Σ}}}I(i,10j)+\underset{i=1}{\overset{N_2}{\mathrm{\Σ}}}\underset{j=1}{\overset{H}{\mathrm{\Σ}}}I(10i,j))$

Wherein, W and H is respectively figure image width and height, $\left|\stackrel{\→}{n_1}\right|=(a_1,b_1,c_1){,N}_1=\mathrm{Int}(H/10),N_1=\mathrm{Int}(H/10),$ $N_2=\mathrm{Int}(W/10),$ I (x, y) is image intensity value.

Laser stripe is generally bright than background, gets M+M ₀as threshold value, intercept ROI region as follows:

$\left\{\begin{array}{c}{X}_{\mathrm{Max}}=\mathrm{Max}\{i:(I(i,10j_1)>M+M_0)\∪(I(10i,j_1)>M+M_0)\}\\ X_{\mathrm{Min}}=\mathrm{Min}\{i:(I(i,10j_1)>M+M_0)\∪(I(10i,j_1)>M+M_0)\}\\ Y_{\mathrm{Max}}=\mathrm{Max}\{j:(I(i,10j_1)>M+M_0)\∪(I(10i,j_1)>M+M_0)\}\\ Y_{\mathrm{Min}}=\mathrm{Min}\{j:(I(i10,j_1)>M+M_0)\∪(I(10i,j_1)>M+M_0)\}\end{array}\right.$

2.2, Da-Jin algorithm binaryzation

Threshold segmentation Binarization methods highlights background and characteristic area optimum method.Adaptive threshold along with the impact of enchancement factor various in welding surroundings, can carry out self-adaptative adjustment, is partitioned into striation region better.

The basic thought of Da-Jin algorithm is the image inter-class variance of selecting threshold value to make to split and maximum, and threshold value deterministic process is as follows:

(1) gradation of image statistical information is obtained:

$N=\underset{i=1}{\overset{L}{\mathrm{\Σ}}}{n}_i,P\left(i\right)=\frac{n_i}{N},w_0=\underset{i=1}{\overset{L}{\mathrm{\Σ}}}P\left(i\right)$

Wherein, n ₁for gray value is the sum of all pixels of i; N is area pixel sum; P (i) is gray value probability; w ₀for the probability of target.

(2) choose initial threshold K, calculate average, the variance of target and background:

$\left\{\begin{array}{c}{\mathrm{\μ}}_0=\underset{i=1}{\overset{k}{\mathrm{\Σ}}}\frac{\mathrm{iP}\left(i\right)}{w_0},{\mathrm{\μ}}_1=\underset{i=k+1}{\overset{L}{\mathrm{\Σ}}}\frac{\mathrm{iP}\left(i\right)}{w_1}\\ {\mathrm{\σ}}_0=\underset{i=1}{\overset{k}{\mathrm{\Σ}}}\frac{{(i-{\mathrm{\μ}}_0)}^{2}P\left(i\right)}{{\mathrm{\ω}}_0},{\mathrm{\σ}}_1=\underset{i=1}{\overset{k}{\mathrm{\Σ}}}\frac{{(i-{\mathrm{\μ}}_1)}^{2}P\left(i\right)}{{\mathrm{\ω}}_1}\end{array}\right.$

Wherein $w_0=\underset{i=0}{\overset{k}{\mathrm{\Σ}}}P\left(i\right),w_1=\underset{i=K+1}{\overset{L}{\mathrm{\Σ}}}P\left(i\right)=1-w_0.$

W ₀, w ₁for the probability of target, background, μ ₀, μ ₁for the gray average of target, background, σ ₀, σ ₁target, the variance of background.

(3) calculate inter-class variance, K corresponding to maximum is selected threshold.

${\mathrm{\σ}}_w^2\left(K\right)=w_1\left(K\right){\mathrm{\σ}}_1^2\left(K\right)+w_2\left(K\right){\mathrm{\σ}}_2^2\left(K\right)$

2.3, based on the refinement of Medial-Axis Transformation principle

Medial-Axis Transformation basic thought is iteration, the target area obtained is carried out to the template matches of iteration, removes non-central point, until be only left the point on center line, concrete grammar is: note central point is p ₁, its 8 field point is designated as p in the direction of the clock ₂, p ₃..., p ₉, wherein p ₂at p ₁top.

First the point simultaneously meeting following 4 conditions is marked:

I：2≤N(p ₁)≤6；

II：S(p ₁)＝1；

III：p ₂*p ₄*p ₆＝0；

IV：p ₄*p ₆*p ₈＝0；

Wherein, N (p ₁)=p ₂+ p ₃+ ...+p ₈+ p ₉; S (p ₁) be with p ₂, p ₃..., p ₉, p ₂for sequence wheel turn time, be worth the change total degree from 0 → 1.To traveling through a little on one side in region, delete flag point.

Then mark: I and II condition is constant, and III becomes p ₂* p ₄* p ₈=0; IV becomes p ₂* p ₆* p ₈the point of=0, delete flag point; Repeatedly perform above process and can realize striation refinement.

2.4, Hough transform

ρ＝xcosθ+ysinθ(0≤θ≤π)

Wherein ρ be the origin of coordinates to air line distance, θ is that the origin of coordinates arrives the vertical line of straight line and the angle of X-axis positive direction.

2.5, recognition feature point

According to the straight line that Hough transform obtains, get the intersection point of adjacent straight line as three feature angle points.Determine welding wire loading in welding according to the V-type recess region area that three feature angle points are formed, get groove triangle core as weld bead feature points.Coordinate is:

$\left\{\begin{array}{c}x=(u_{\mathrm{xl}}+u_{\mathrm{yl}}+u_{\mathrm{zl}})/3\\ y=(v_{\mathrm{xl}}+v_{\mathrm{yl}}+v_{\mathrm{zl}})/3\end{array}\right.$

Claims (3)

1. the invention provides a kind of feature extracting method being applicable to the welding robot corner connection weld seam that laser vision guides, the method can obtain corner connection angle and the spot welds coordinate of corner connection weld seam.The determination of corner connection angle is of value at the beginning of starting in welding, effective coordination welding robot determination pose, more efficiently complete weld task, at the Extraction parts of spot welds, take on the basis intercepting ROI region, adopt and be verified all secure binaryzation of robustness, real-time, thinning algorithm, ensure that spot welds is extracted accurate in real time.This corner connection weld seam feature extraction is comprehensive, and to actual welding, also termination process is significant, has good practicality.Whole algorithm comprises following module:

Determine the corner connection angle of corner connection weld seam; Given welding robot two pose T ₁, T ₂, at T ₁pose place acquisition angle welding seam image, extracts striation straight line and spot welds P ₁(identify spot welds method as second step), and on the two sections of striation straight lines of two planes belonging to corner connection respectively any selected point P ₁₀, P ₁₁, at T ₂pose place acquisition angle welding seam image, extracts striation straight line and spot welds P ₂, and on the two sections of striation straight lines of two planes belonging to corner connection respectively any selected point P ₂₀, P ₂₁, due to P ₁, P ₁₀, P ₂, P ₂₀and P ₁, P ₁₁, P ₂, P ₂₁belong to two corner connection faces of corner connection weld seam respectively, by the equation in four somes matching, two corner connection faces, and then the angle in two corner connection faces can be obtained.

Spot welds is extracted in accurate identification; The welding robot system acquisition angle welding seam image guided by laser vision, grid windowing process is carried out to the image gathered and intercepts out ROI region, medium filtering, binaryzation, refinement are carried out to the ROI region intercepted out, image after refinement adopts Hough transform extract striation straight line, and then obtain spot welds coordinate.

2. determine that the corner connection angle of weld seam has following feature as claimed in claim 1:

2.1, the some P on image ₁, P ₂, P ₁₀, P ₁₁, P ₂₀and P ₂₁each point transforms to welding robot basis coordinates system

If laser instrument is at pose T ₁time project laser stripe in two planes of object as shown in Figure 3, laser stripe is a broken line, if P ₁for the broken line flex point (being weld bead feature points during welding) on the intersection of two planes, at P ₁both sides laser stripe on get respectively a bit, be set to P ₁₀and P ₁₁, then P ₁₀p ₁₁also the point in two planes is respectively.Detect through laser vision sensor and obtain P ₁point three-dimensional coordinate under camera coordinate system is X _c1(x _c1, y _c1, z _c1), its coordinate X under basis coordinates system of robot can be tried to achieve _w1(x _w1, y _w1, z _w1):

[x _W1，y _W1，z _W1，1] ^T＝T ₁T[x _C1，y _C1，z _C1，1] ^T

Change robot pose and reach T ₂, in like manner can solve P ₂, P ₂₀, P ₂₁.

2.2, plane included angle solves

By non-colinear coplanar 4 P ₁, P ₂, P ₁₀, P ₂₀, matching obtains the equation of plane 1:

a ₁x+b ₁y+c ₁z+1＝0

By non-colinear coplanar 4 P ₁, P ₂, P ₁₁, P ₂₁matching obtains the equation of plane 2:

a ₂x+b ₂y+c ₂z+1＝0

Namely the normal vector of two planes is respectively $\left|\stackrel{\→}{n_1}\right|=(a_1,b_1,c_1)$ With $\left|\stackrel{\→}{n_2}\right|=(a_2,b_2,c_2).$

$\cos \mathrm{\θ}=\frac{\stackrel{\→}{n_1}\·\stackrel{\→}{n_2}}{\left|\stackrel{\→}{n_1}\right|\·\left|\stackrel{\→}{n_2}\right|}$

Corresponding corner connection angle θ can be solved according to cos θ value.

3. extract as the identification of claim 1 weld bead feature points and there is following feature: grid windowing process is carried out to the image gathered and intercepts out ROI region, medium filtering is carried out to the ROI region intercepted out, Da-Jin algorithm is used to carry out binaryzation to filtered image, then carry out striation refinement based on based on Medial-Axis Transformation principle, image after refinement adopt Hough transform extract striation straight line, the intersection point getting adjacent straight line as three feature angle points, then using the center of gravity of three characteristic points as weld bead feature points.