CN102794763A - Systematic calibration method of welding robot guided by line structured light vision sensor - Google Patents

Abstract

The invention relates to a systematic calibration method of a welding robot guided by a line structured light vision sensor, which comprises the following steps: firstly, controlling a mechanical arm to change pose, obtaining a round target image through a camera, accomplishing the matching of the round target image and a world coordinate, and then obtaining an internal parameter matrix and an external parameter matrix RT of the camera; secondly, solving a line equation of a line laser bar by Hough transformation, and using the external parameter matrix RT obtained in the first step to obtain a plane equation of the plane of the line laser bar under a coordinate system of the camera; thirdly, calculating to obtain a transformation matrix of a tail end coordinate system of the mechanical arm and a base coordinate system of the mechanical arm by utilizing a quaternion method; and fourthly, calculating a coordinate value of a tail end point of a welding workpiece under the coordinate of the mechanical arm, and then calculating an offset value of the workpiece in the pose combined with the pose of the mechanical arm. The systematic calibration method of the welding robot guided by the line structured light vision sensor is flexible, simple and fast, and is high in precision and generality, good in stability and timeliness and small in calculation amount.

Description

Welding robot system scaling method based on the line structured light vision sensor guiding

Technical field

The present invention relates to a kind of vision sensor of robot and scaling method thereof; Especially a kind of welding robot system scaling method based on the line structured light vision sensor guiding, specifically robot is based on the trick relational matrix of line structured light vision sensor and the quick calibrating method of sensor parameters.

Background technology

The characteristics of welding are that technological factor is complicated, labour intensity is big; Production cycle is long, work situation is poor; Its quality relies on operator's technical ability, technology and experience; Also relevant with operator's mood and health, therefore, welding automatization technology is for improving joint quality, guaranteeing that stability has very important meaning.The key issue that realizes Automation of Welding is a seam tracking, and the intelligent welding robot of laser vision guiding combines weld image identification with robot motion's control technology, can effectively solve a soldering joint automatic tracking difficult problem.

Demarcating (comprising that camera parameter is demarcated, the line laser optic plane equations is demarcated and the trick transformation matrix is demarcated) is very crucial and important link in vision measurement system; The precision of its calibration result and the stability of algorithm and real-time directly have influence on and measure in the industrial processes and the precision of following the tracks of.The parameter of line structured light vision sensor comprises the inner parameter (focal length, principal point and distortion factor etc.) of camera and the structural parameters of line structured light vision sensor (being the optic plane equations of line laser under camera coordinates system); Pass through Tsai; Zhang Zhengyou; The contribution of people such as Hu Zhanyi in vision calibration, we can obtain the inner parameter of camera easily.The structural parameters calibration method of line structured light vision sensor has many methods, like the wire drawing scaling method of R.Dewar proposition; D.Q.Huynh has proposed double ratio consistency scaling method; The line laser optic plane equations based on intersection that Bi Dexue proposes is demarcated has stronger robustness, and implementation method is simple, and characteristic is prone to extract, and satisfies the field calibration requirement.According to measurement model, the measurement data of being returned by sensor is recovered the three-dimensional coordinate information of tested curved surface must confirm that the terminal coordinate of mechanical arm is tied to the trick transformation matrix H of camera coordinates system.Be called hand and eye calibrating in this problem of field of machine vision.Robot Hand-eye scaling method commonly used is to utilize known calibration reference substance (calibrating block); The control manipulator is at known scaling reference of different azimuth observation space; Thereby derivation R and t and the relation of observed result repeatedly; Wherein R representes the rotating part of trick transformation matrix H, and t representes the translating sections of trick transformation matrix H.

Summary of the invention

The objective of the invention is to overcome the deficiency that exists in the prior art, a kind of welding robot system scaling method based on the line structured light vision sensor guiding is provided, it is flexible, precision is high, speed is fast, good stability; Real-time; Method is simple, and amount of calculation is little, highly versatile.

According to technical scheme provided by the invention, said welding robot system scaling method based on the line structured light vision sensor guiding, said welding robot system scaling method comprises the steps:

The first step, control mechanical arm conversion pose; Make camera take the round target of any placement and invariant position at a plurality of poses; Round dot on all circle targets that the pose of selecting must make and guarantees to be fixed on striation part that laser line generator produced on the camera line laser optical plane produces also in field range on the circle target in the viewing field of camera scope; After obtaining round target image through camera, extract the central coordinate of circle of circular spot on the circle target image and discern the ranks value of angle point,, obtain the inner parameter matrix of camera then according to the Zhang Zhengyou calibration algorithm to accomplish the coupling of round target image and world coordinates $A=\left[\begin{array}{ccc}\mathrm{\&alpha;}& \mathrm{\&gamma;}& {\mathrm{<figure-callout\; id="0"\; label="u"\; filenames="HDA00002085565200042.png"\; state="\{\{state\}\}">u</figure-callout>}}_0\\ 0& \mathrm{\&beta;}& v_0\\ 0& 0& 1\end{array}\right],$ Outer parameter matrix RT;

Wherein, α=f/dx, β=f/dy, f are camera focus, and dx, dy are single CCD photo-sensitive cell length, width in the camera; The physical quantity that γ arranges inclined degree for CCD photo-sensitive cell in the reflection camera, u ₀, v ₀Be the camera lens optical axis of camera and the intersection point pixel coordinate of CCD photo-sensitive cell;

Second step, according to the round target image of the band line laser striation that obtains in the first step, extract the line laser striation, the line laser striation of refinement extraction is obtained the line equation of line laser striation through the Hough conversion; The outer parameter matrix RT that utilizes the first step to obtain obtains the plane equation of line laser striation plane under camera coordinates system;

The 3rd step, the mechanical arm attitude corresponding according to each pose in the first step utilize the hypercomplex number method to calculate terminal coordinate system of mechanical arm and the transformation matrix that the mechanical arm basis coordinates is, promptly obtain trick transformation matrix H;

The 4th the step, welding work pieces is positioned over the mechanical arm end; The control welding work pieces under the terminal pose of solid mechanical arm to the circle target on a bit accurately point touch; Calculate the coordinate figure of welding work pieces distal point under the mechanical arm coordinate, and combine the mechanical arm pose to calculate the deviant of workpiece under said pose.

The said first step comprises the steps:

1.1, utilize the image of camera online real time collecting circle target, adopt big Tianjin method adaptive threshold to image binaryzation, make outstanding demonstration of circular spot on the circle target image;

1.2, utilize the closed operation operator that circle target image is carried out closed operation, to remove noise; All target round dots in the round target image after the above-mentioned binaryzation are carried out mark; The number of pixels that the area of target round dot comprises for the target round dot; Add up the number of pixels of the target round dot in the above-mentioned round target image; It is NP_average that the target-marking round dot comprises the number of pixels average, removes in the target-marking to comprise number of pixels less than comprising the target of number of pixels greater than 1.5*NP_average in 0.5*NP_average and the target-marking;

1.3, calculate the girth C and the area S of target round dot behind the mark, utilize circularity expression formula e=C ²/ (2* π * S) distinguishes target round dot and interference;

1.4, adopt gravity model appoach to ask for the central coordinate of circle (x of each target round dot ₀, y ₀),

${\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA00002085565200042.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=\frac{\underset{i=0}{\overset{m}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n}{\mathrm{\&Sigma;}}}\mathrm{Hit}\left[i\right]\left[j\right]*y}{\underset{i=0}{\overset{m}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n}{\mathrm{\&Sigma;}}}\mathrm{Hit}\left[i\right]\left[j\right]};$

Wherein, m, n is row, the train value of circle target image, Hit [i] [j] expression (i, j) location point whether on the target round dot, (i, j) location point Hit [i] [j]=1 on the target round dot, otherwise Hit [i] [j]=0;

1.5, circle has the round dot that several rows, row are arranged on the target; Line-spacing between the adjacent two row round dots is 15mm; Distance with 2 adjacent in delegation round dot centers of circle is 10mm; There is circle of reference at the 5mm place on first line direction of circle target first row, obtains target round dot pixel coordinate based on step (1.4), utilizes circle of reference that the centre point on the circle target image is sorted;

1.6, according to above-mentioned round target image orbicular spot ordering, with given round target information, accomplish the coupling of centre point pixel coordinate and world coordinates in the circle target image.

Said second step comprises the steps:

2.1, laser line generator is fixed on the camera; It is only to need to guarantee that all target round dots and striation part get final product in the viewing field of camera scope that camera is gathered picture; Utilize predetermined threshold value that the line laser optical strip image that collects is carried out binaryzation, binary image is carried out the image closed operation to remove the edge singular point;

2.2, line laser striation regional center is carried out 8-neighborhood mark; The central point in note line laser striation zone is p1, and 8 points of line laser striation regional center point p1 neighborhood are respectively p2 around central point, p3 clockwise; P9, wherein p2 dot center's point satisfies the boundary point of following condition when line laser striation regional center point p1 is carried out 8-neighborhood mark above p1:

（ⅰ）、2≤N（p1）≤6；

（ⅱ）、S（p1）=1；

（ⅲ）、p2*p4*p6=0；

（ⅳ）、p4*p6*p8=0；

Wherein, N (p1) is the number of the non-zero adjoint point of central point p1; S (p1) is with p2, p3 ..., the value of these points was from 0 → 1 change frequency when p9 was preface; When all boundary points are all checked finish after, all gauge points are removed, iterate and do not satisfy flag condition up to point not, accomplish the striation refinement;

2.3, the point that obtains after the refinement is utilized the Hough conversion to extract to obtain the line equation a of laser striation of reaching the standard grade on circle target plane _kX+b _kY+c _k=0, wherein, a _k, b _k, c _kThe parameter of representing the line equation respectively.

Said the 3rd step comprises the steps:

3.1, the circle plane equation of target under world coordinate system can be expressed as

In the formula: π ₁=[0,0,1,0] ^T, the outer parameter matrix RT that utilizes the first step to obtain, the plane equation that calculates target plane under camera coordinates system does Wherein

With

Be respectively the planar process of target plane under world coordinate system and camera coordinates system to coordinate vector;

3.2, in camera coordinates system, establish

Represent the normal direction coordinate vector of laser plane normal direction coordinate vector, an i circle target, the normal direction coordinate vector of a j circle target, the coordinate vector of the laser intersection image in the i circle target, the coordinate vector of the laser intersection image in the j plane target drone respectively; Note the equation λ of i the laser intersection image on the plane target drone image _iWith

Can directly calculate:

${\stackrel{\&RightArrow;}{\mathrm{\&lambda;}}}_i=A^T{\mathrm{\&lambda;}}_i;$

(3.3), get, according to the duality relation on two laser intersections in the projector space and the plane that is associated thereof $\stackrel{\&RightArrow;}{w}=\frac{1}{2}(\stackrel{\&RightArrow;}{w_1}+\stackrel{\&RightArrow;}{w_2});$ $\left\{\begin{array}{c}\stackrel{\&RightArrow;}{w_1}=\stackrel{\&RightArrow;}{w_i}+{\mathrm{\&alpha;}}_i\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j}\\ \stackrel{\&RightArrow;}{w_2}=\stackrel{\&RightArrow;}{w_j}+{\mathrm{\&alpha;}}_j\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j}\end{array}\right.,$

Wherein $\left[\begin{array}{c}{\mathrm{\&alpha;}}_i\\ {\mathrm{\&alpha;}}_j\end{array}\right]=D^{-1}Q,$ D is one 2 * 2 matrix, and Q is a bivector, defines as follows: $D=\left[\begin{array}{cc}(\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_i},\stackrel{\&RightArrow;}{w_i})& -(\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j},\stackrel{\&RightArrow;}{w_i})\\ (\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_i},\stackrel{\&RightArrow;}{w_j})& -(\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j},\stackrel{\&RightArrow;}{w_j})\end{array}\right],$ $Q=\left[\begin{array}{c}(\stackrel{\&RightArrow;}{w_i},\stackrel{\&RightArrow;}{w_j}-\stackrel{\&RightArrow;}{w_i})\\ (\stackrel{\&RightArrow;}{w_j},\stackrel{\&RightArrow;}{w_i}-\stackrel{\&RightArrow;}{w_j})\end{array}\right];$

It is the laser plane equation of being asked.

, said the 4th step comprises the steps:

4.1, get any 2 poses in the first step;

be the transition matrix of the terminal coordinate system of mechanical arm under 2 poses of

expression wherein; Make the transition matrix of camera coordinates system under 2 poses of

expression, make

then can be abbreviated as Φ H=H Θ;

4.2, Φ H=H Θ; Expand into

R _ΦR _H=R _HR _Θ ①

R _Φt _H+t _Φ=R _Ht _Θ+t _H ②

Wherein: R _Φ, R _Θ, R _HExpression Φ, Θ, the rotating part of correspondence among the H, t _Φ, t _Θ, t _HThe expression Φ, Θ, among the H for translating sections;

4.3, the order $q_{\mathrm{\&Phi;}}={[a_0,{\stackrel{\&RightArrow;}{a}}^T]}^T,$ $q_H={[x_0,{\stackrel{\&RightArrow;}{x}}^T]}^T,$ $q_{\mathrm{\&Theta;}}={[{\mathrm{<figure-callout\; id="0"\; label="b"\; filenames="HDA00002085565200042.png"\; state="\{\{state\}\}">b</figure-callout>}}_0,{\stackrel{\&RightArrow;}{b}}^T]}^T$ Be R _Φ, R _H, R _ΘCorresponding hypercomplex number, q _ΦCalculate q by homogeneous matrix Φ _ΘCalculate order by homogeneous matrix Θ

Through

Try to achieve q _H, again with q _HConvert the rotating part of trick matrix into, Ω () the title matrix function that makes difficulties wherein,

Expression

Antisymmetric matrix; x ₀, x is q _HIn unknown quantity to be calculated;

4.4,2. the spin matrix data are brought into just can obtain t again _H, obtain trick transformation matrix H.

Calculating the welding work pieces shift value in said the 4th step comprises the steps:

5.1, the control welding work pieces under the terminal pose of solid mechanical arm to the circle target on the preset accurately point touch, read current mechanical arm pose, obtain first coordinate figure of the terminal coordinate of mechanical arm;

5.2, the definition coordinate of preset under world coordinate system; According to the outer parameter matrix RT that obtains in the first step; Obtain the coordinate of preset under camera coordinates system; Mechanical arm pose during according to trick transformation matrix H and shooting circle target image calculates second coordinate figure of preset under mechanical arm basis coordinates system, and said second coordinate figure and the comparison of first coordinate figure are obtained welding work pieces shift value under the current pose.

When utilizing circle of reference that the centre point on the circle target image is sorted, the distance of center circle of finding out effective round dot on the round target image wherein leaves 0 point that is defined as of all centre point average centers circle far away from two minimum target round dots; Another is defined as 1 point; Be provided with then at 0,1 and be match point, in match point not, look for defining 2 points with 1 nearest point; Be provided with at 2 and be match point; In match point not, look for and define 3 points, be provided with at 3 and be match point with 2 nearest points ... Until have a few all look for complete, so accomplish the ordering of centre point.

Advantage of the present invention: the present invention is through analyzing image-forming principle, structural light measurement principle and the hand-eye system operation principle of camera; Designed a kind of scaling method that guides mechanical arm, three-D to follow the tracks of based on mechanism's light of simple and flexible; Demarcate comprising the camera confidential reference items; The line laser optic plane equations is demarcated, and the trick transformation matrix is demarcated and the workpiece skew is demarcated.This scaling method has overcome the conventional laser optic plane equations and the trick matrix is demarcated the shortcoming that condition is harsh, demarcating steps is loaded down with trivial details.This algorithm only need be controlled the pose of mechanical arm voltuntary movement more than 3 and take fixedly target and just can accomplish the integral calibrating task, and utilizes the structured light of this method method to guide the mechanical arm tracking system that higher tracking accuracy is arranged.This algorithm makes line laser, the Adjustable structure of camera, and the field calibration of system structure parameter becomes possibility.Increased the flexibility of structure light guide system greatly, significant to the vision measurement of reality with tracking, have good practicality.

Description of drawings

Fig. 1 is the flow chart that the present invention demarcates.

Fig. 2 is the pinhole imaging system illustraton of model of camera of the present invention.

Fig. 3 is the sketch map of round dot on the present invention's circle target.

Fig. 4 is the flow chart that line-structured light guiding mechanical arm of the present invention is followed the tracks of.

Fig. 5 is the dual representations of two laser intersections of the present invention at projector space.

Fig. 6 is that camera fixing of the present invention is the trick graph of a relation in the terminal relation of mechanical arm.

The specific embodiment

It is harsh to the objective of the invention is to overcome traditional line laser optic plane equations and hand and eye calibrating condition; Shortcomings such as demarcating steps is loaded down with trivial details, propose a kind of flexibly, precision is high, speed is fast, good stability, real-time, method is simple, amount of calculation is little, the scaling method of highly versatile.

Below in conjunction with accompanying drawing and embodiment the present invention is described further.

The present invention utilizes round target as shown in Figure 3, can realize the quick location of centre point, matees the world coordinates and the pixel coordinate of center of circle characteristic point automatically, helps the automation of calibration of camera internal parameters; The Mathematical Modeling of analytical line structured light guiding mechanical arm tracking system; Introduce the hypercomplex number method and find the solution the trick matrix; And simplify the integral calibrating flow process; Only need the control mechanical arm to take fixing circle target, can realize that camera calibration, line laser optic plane equations are demarcated and the demarcation of trick matrix, for accurate tracking provides adequate condition with any pose more than 3; The accurate extraction at line laser striation center can online in real time, and is stable, and Detection and Extraction image characteristic point accurately guarantees to demarcate stability and reliability with tracking system; According to demarcating the structural parameters of accomplishing, realize following the tracks of by three characteristic points of line-structured light guiding mechanical arm are real-time, accurate.

The sub-pix central coordinate of circle of extraction circular spot according to the invention and the automatic ranks value of identification round dot, and the matching technique of accomplishing automatically with world coordinates is to utilize image binaryzation, and image corrodes and carries out preliminary treatment; According to area size with whether be in the edge and remove non-target part; Utilize the circularity equation to remove noise spot, circular spot is carried out mark, extract marked region; Behind gaussian filtering, adopt gravity model appoach, extract the sub-pix coordinate at round dot center.The centre point of some ranks is set on the circle target; Line-spacing between the adjacent two row centers of circle is 15mm; 2 centre points distance with adjacent in the delegation is 10mm, and there is a circle of reference at the 5mm place on first line direction of first row, and the distance of center circle of at first finding out all effective round dots is from two minimum points; Arrange and can know through the centre point on the above-mentioned round target, the center of circle of circle of reference and said circle of reference below round dot is nearest; Wherein leave 0 point that is defined as of all centre point average centers circle far away, another is defined as 1 point, promptly is that circle of reference is defined as 0 here, and the round dot under the circle of reference is defined as 1; Right be provided with at 0,1 and be match point, in match point not, look for defining 2 points with 1 nearest point; Being provided with at 2 looks for 2 nearest points in match point not for match point and to define 3 points; Be provided with at 3 and be match point ... Until have a few and all look for entirely, so accomplish the ordering of centre point, sort according to round dot; With given round target information, the coupling of centre point pixel coordinate and world coordinates in the completion circle target image.

Scaling method based on duality relation according to the invention is to utilize two laser intersections in the projector space and the duality relation Calibration of Laser plane equation on the plane that is associated.Demarcate to accomplish and obtain the plane equation of laser light plane under camera coordinates system.

The technology of utilizing the hypercomplex number method to find the solution the trick transformation matrix according to the invention is to utilize to demarcate to accomplish transformation matrix and the mechanical arm pose matrix that the world coordinate system obtain and camera coordinates are; Through the structure constraint equation; Utilize the corresponding relation of unit quaternion and spin matrix to find the solution constrained procedure, obtain trick transformation matrix H.Utilize the image that field real-time acquisition arrives to confirm characteristic point fast, the laser plane equation that obtains by said method and the accurate tracking of trick transformation matrix realization three-dimensional feature point.

As shown in Figure 1, demarcation idiographic flow of the present invention is following:

Said demarcation flow process is made up of camera calibration, the demarcation of line laser optic plane equations, the demarcation of trick transformation matrix and workpiece skew demarcation four parts, utilizes bandpass filter and CCD to gather the laser striation and scans weld image.Adopt the Medial-Axis Transformation principle that the laser striation is carried out thinning processing, obtain striation refinement center.The control mechanical arm is taken fixedly target with any pose more than 3; The round target center of circle that utilization obtains and its matched data ginseng inside and outside the row camera is calculated; The line laser line equation that outer ginseng that utilization obtains and target picture extract calibrates the laser light plane equation; Utilize the outer ginseng of per twice mechanical arm in moving to change and mechanical arm pose varied configurations Φ H=H Θ equation, and utilize the hypercomplex number method to find the solution trick transformation matrix H.

The first step; Control mechanical arm conversion pose; Make camera take the round target of any placement and invariant position at a plurality of poses; Round dot on all circle targets that the pose of selecting must make and guarantees to be fixed on striation part that laser line generator produced on the camera line laser optical plane produces also in field range on the circle target in the viewing field of camera scope; After obtaining round target image through camera, extract the central coordinate of circle of circular spot on the circle target image and discern the ranks value of angle point,, obtain the inner parameter matrix of camera then according to the Zhang Zhengyou calibration algorithm to accomplish the coupling of round target image and world coordinates $A=\left[\begin{array}{ccc}\mathrm{\&alpha;}& \mathrm{\&gamma;}& {\mathrm{<figure-callout\; id="0"\; label="u"\; filenames="HDA00002085565200042.png"\; state="\{\{state\}\}">u</figure-callout>}}_0\\ 0& \mathrm{\&beta;}& {\mathrm{<figure-callout\; id="0"\; label="v"\; filenames="HDA00002085565200042.png"\; state="\{\{state\}\}">v</figure-callout>}}_0\\ 0& 0& 1\end{array}\right],$ Outer parameter matrix RT;

Wherein, α=f/dx, β=f/dy, f are camera focus, and dx, dy are single CCD photo-sensitive cell length, width in the camera; The physical quantity that γ arranges inclined degree for CCD photo-sensitive cell in the reflection camera, u ₀, v ₀Be the camera lens optical axis of camera and the intersection point pixel coordinate of CCD photo-sensitive cell;

1.1, utilize the image of camera online real time collecting circle target, adopt big Tianjin method adaptive threshold to image binaryzation, make outstanding demonstration of circular spot on the circle target image;

1.2, utilize the closed operation operator that circle target image is carried out closed operation, to remove noise; All target round dots in the round target image after the above-mentioned binaryzation are carried out mark; The number of pixels that the area of target round dot comprises for the target round dot; Add up the number of pixels of the target round dot in the above-mentioned round target image; It is NP_average that the target-marking round dot comprises the number of pixels average, removes in the target-marking to comprise number of pixels less than comprising the target of number of pixels greater than 1.5*NP_average in 0.5*NP_average and the target-marking;

1.3, calculate the girth C and the area S of target-marking round dot, utilize circularity expression formula e=C ²/ (2* π * S) distinguishes target round dot and interference;

1.4, adopt gravity model appoach to ask for the central coordinate of circle of each target round dot

$x_0=\frac{\underset{i=0}{\overset{m}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n}{\mathrm{\&Sigma;}}}\mathrm{Hit}\left[i\right]\left[j\right]*x}{\underset{i=0}{\overset{m}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n}{\mathrm{\&Sigma;}}}\mathrm{Hit}\left[i\right]\left[j\right]},$ ${\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="HDA00002085565200042.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=\frac{\underset{i=0}{\overset{m}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n}{\mathrm{\&Sigma;}}}\mathrm{Hit}\left[i\right]\left[j\right]*y}{\underset{i=0}{\overset{m}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{n}{\mathrm{\&Sigma;}}}\mathrm{Hit}\left[i\right]\left[j\right]};$

Wherein, m, n is row, the train value of circle target image, Hit [i] [j] expression (i, j) location point whether on the target round dot, (i, j) location point Hit [i] [j]=1 on the target round dot, otherwise Hit [i] [j]=0;

1.5, circle has some round dots of arranging with ranks on the target; Line-spacing between the adjacent two row round dots is 15mm; From being 10mm, there is a circle of reference at the 5mm place on first line direction of circle target first row, obtains target round dot pixel coordinate based on step (1.4) with 2 adjacent in delegation round dot distance of center circle; The distance of center circle of finding out effective round dot on the round target image sorts to the initial point on the circle target image according to aforementioned sortord from two minimum target round dots then;

1.6, according to above-mentioned round target image orbicular spot ordering, with given round target information, accomplish the coupling of centre point pixel coordinate and world coordinates in the circle target image.

Second step, to the target picture of the top band line laser striation that obtains, extract striation, the refinement striation is obtained striation line equation through the Hough conversion.The outer ginseng matrix that obtains in conjunction with the first step just can obtain the plane equation of line laser optical plane under camera coordinates system;

2.1, laser line generator is fixed on the camera; It is only to need to guarantee that all target round dots and striation part get final product in the viewing field of camera scope that camera is gathered picture; Utilize predetermined threshold value that the line laser optical strip image that collects is carried out binaryzation, binary image is carried out the image closed operation to remove the edge singular point;

2.2, line laser striation regional center is carried out 8-neighborhood mark; The central point in note line laser striation zone is p1, and 8 points of line laser striation regional center point p1 neighborhood are respectively p2 around central point, p3 clockwise; P9, wherein p2 dot center's point satisfies the boundary point of following condition when line laser striation regional center point p1 is carried out 8-neighborhood mark above p1:

（ⅰ）、2≤N（p1）≤6；

（ⅱ）、S（p1）=1；

（ⅲ）、p2*p4*p6=0；

（ⅳ）、p4*p6*p8=0；

Wherein, N (p1) is the number of the non-zero adjoint point of central point p1; S (p1) is with p2, p3 ..., the value of these points was from 0 → 1 change frequency when p9 was preface; When all boundary points are all checked finish after, all gauge points are removed, iterate and do not satisfy flag condition up to point not, accomplish the striation refinement;

2.3, the point that obtains after the refinement is utilized the Hough conversion to extract to obtain the line equation a of laser striation of reaching the standard grade on circle target plane _kX+b _kY+c _k=0, wherein, a _k, b _k, c _kThe parameter of representing the line equation respectively.

The 3rd step, according to the corresponding mechanical arm attitude of each pose in the first step, utilize the hypercomplex number method to calculate the transformation matrix of terminal coordinate system of mechanical arm and mechanical arm basis coordinates system, promptly obtain trick transformation matrix H;

Obtain the transformation matrix that the terminal coordinate system of mechanical arm and mechanical arm basis coordinates are according to corresponding six Attitude Calculation of mechanical arm of each pose in the first step, through hypercomplex number method solving equation Φ H=H Θ.Here Φ, H and Θ are the 4x4 matrixes; Wherein Φ and Θ represent the transformation matrix of terminal coordinate system of mechanical arm and the camera coordinates system position from first position to the second respectively; H representes that the terminal coordinate of mechanical arm is tied to the transformation matrix of camera coordinates system, i.e. trick relational matrix;

3.1, the circle plane equation of target under world coordinate system can be expressed as In the formula: π ₁=[0,0,1,0] ^T, the outer parameter matrix RT that utilizes the first step to obtain, the plane equation that calculates target plane under camera coordinates system does

Wherein

With

Be respectively the planar process of target plane under world coordinate system and camera coordinates system to coordinate vector;

3.2, in camera coordinates system, establish

Represent the normal direction coordinate vector of laser plane normal direction coordinate vector, an i circle target, the normal direction coordinate vector of a j circle target, the coordinate vector of the laser intersection image in the i circle target, the coordinate vector of the laser intersection image in the j plane target drone respectively; Note the equation λ of i the laser intersection image on the plane target drone image _iWith

Can directly calculate:

${\stackrel{\&RightArrow;}{\mathrm{\&lambda;}}}_i=A^T{\mathrm{\&lambda;}}_i;$

3.3, get according to the duality relation on two laser intersections in the projector space and the plane that is associated thereof, $\stackrel{\&RightArrow;}{w}=\frac{1}{2}(\stackrel{\&RightArrow;}{w_1}+\stackrel{\&RightArrow;}{w_2});$ $\left\{\begin{array}{c}\stackrel{\&RightArrow;}{w_1}=\stackrel{\&RightArrow;}{w_i}+{\mathrm{\&alpha;}}_i\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j}\\ \stackrel{\&RightArrow;}{w_2}=\stackrel{\&RightArrow;}{w_j}+{\mathrm{\&alpha;}}_j\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j}\end{array}\right.,$

Wherein $\left[\begin{array}{c}{\mathrm{\&alpha;}}_i\\ {\mathrm{\&alpha;}}_j\end{array}\right]=D^{-1}Q,$ D is one 2 * 2 matrix, and Q is a bivector, defines as follows: $D=\left[\begin{array}{cc}(\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_i},\stackrel{\&RightArrow;}{w_i})& -(\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j},\stackrel{\&RightArrow;}{w_i})\\ (\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_i},\stackrel{\&RightArrow;}{w_j})& -(\stackrel{\&RightArrow;}{{\mathrm{\&lambda;}}_j},\stackrel{\&RightArrow;}{w_j})\end{array}\right],$ $Q=\left[\begin{array}{c}(\stackrel{\&RightArrow;}{w_i},\stackrel{\&RightArrow;}{w_j}-\stackrel{\&RightArrow;}{w_i})\\ (\stackrel{\&RightArrow;}{w_j},\stackrel{\&RightArrow;}{w_i}-\stackrel{\&RightArrow;}{w_j})\end{array}\right];$

It is the laser plane equation of being asked.

In the 4th step, the control welding work pieces a bit touches through the row point target under the fixation ends pose, calculates the coordinate figure of workpiece end point under the mechanical arm coordinate, and combines the mechanical arm pose to calculate the deviant of workpiece under this pose.

4.1, get any 2 poses in the first step;

be the transition matrix of the terminal coordinate system of mechanical arm under 2 poses of expression wherein; Make the transition matrix of camera coordinates system under 2 poses of

expression, make

then can be abbreviated as Φ H=H Θ;

4.2, Φ H=H Θ; Expand into

R _ΦR _H=R _HR _Θ ①

R _Φt _H+t _Φ=R _Ht _Θ+t _H ②

Wherein: R _Φ, R _Θ, R _HExpression Φ, Θ, the rotating part of correspondence among the H, t _Φ, t _Θ, t _HThe expression Φ, Θ, among the H for translating sections;

4.3, the order $q_{\mathrm{\&Phi;}}={[a_0,{\stackrel{\&RightArrow;}{a}}^T]}^T,$ $q_H={[x_0,{\stackrel{\&RightArrow;}{x}}^T]}^T,$ $q_{\mathrm{\&Theta;}}={[{\mathrm{<figure-callout\; id="0"\; label="b"\; filenames="HDA00002085565200042.png"\; state="\{\{state\}\}">b</figure-callout>}}_0,{\stackrel{\&RightArrow;}{b}}^T]}^T$ Be R _Φ, R _H, R _ΘCorresponding hypercomplex number, q _ΦCalculate q by homogeneous matrix Φ _ΘCalculate order by homogeneous matrix Θ

Through

Try to achieve q _H, again with q _HConvert the rotating part of trick matrix into, Ω () the title matrix function that makes difficulties wherein,

Expression

Antisymmetric matrix; x ₀, x is q _HIn unknown quantity to be calculated;

4.4,2. the spin matrix data are brought into just can obtain t again _H, obtain trick transformation matrix H

Calculating the welding work pieces shift value in said the 4th step comprises the steps:

5.1, the control welding work pieces under the terminal pose of solid mechanical arm to the circle target on the preset accurately point touch, read current mechanical arm pose, obtain first coordinate figure of the terminal coordinate of mechanical arm;

5.2, the definition coordinate of preset under world coordinate system; According to the outer parameter matrix RT that obtains in the first step; Obtain the coordinate of preset under camera coordinates system; Mechanical arm pose during according to trick transformation matrix H and shooting circle target image calculates second coordinate figure of preset under mechanical arm basis coordinates system, and said second coordinate figure and the comparison of first coordinate figure are obtained welding work pieces shift value under the current pose.

In the embodiment of the invention, through above-mentioned steps, realized the demarcation of camera inner parameter respectively, the line laser optic plane equations is demarcated, and the trick transformation matrix is demarcated and the workpiece skew is demarcated, and has promptly reached welding robot system calibrating process.

The present invention is through analyzing image-forming principle, structural light measurement principle and the hand-eye system operation principle of camera; Designed a kind of scaling method that guides mechanical arm, three-D to follow the tracks of based on mechanism's light of simple and flexible; Demarcate comprising the camera inner parameter; The line laser optic plane equations is demarcated, and the trick transformation matrix is demarcated and the workpiece skew is demarcated.This scaling method has overcome the conventional laser optic plane equations and the trick matrix is demarcated the shortcoming that condition is harsh, demarcating steps is loaded down with trivial details.This algorithm only need be controlled the pose of mechanical arm voltuntary movement more than 3 and take fixedly target and just can accomplish the integral calibrating task, and utilizes the structured light of this method method to guide the mechanical arm tracking system that higher tracking accuracy is arranged.This algorithm makes line laser, the Adjustable structure of camera, and the field calibration of system structure parameter becomes possibility.Increased the flexibility of structure light guide system greatly, significant to the vision measurement of reality with tracking, have good practicality.

Claims (7)

1. the welding robot system scaling method based on the line structured light vision sensor guiding is characterized in that said welding robot system scaling method comprises the steps:

The first step, control mechanical arm conversion pose; Make camera take the round target of any placement and invariant position at a plurality of poses; Round dot on all circle targets that the pose of selecting must make and guarantees to be fixed on striation part that laser line generator produced on the camera line laser optical plane produces also in field range on the circle target in the viewing field of camera scope; After obtaining round target image through camera; Extract the central coordinate of circle of justifying circular spot on the target image and the ranks value of discerning angle point; To accomplish the coupling of round target image and world coordinates, obtain the outer parameter matrix RT of inner parameter matrix

of camera then according to the Zhang Zhengyou calibration algorithm;

Wherein, α=f/dx, β=f/dy, f are camera focus, and dx, dy are single CCD photo-sensitive cell length, width in the camera; The physical quantity that γ arranges inclined degree for CCD photo-sensitive cell in the reflection camera, u ₀, v ₀Be the camera lens optical axis of camera and the intersection point pixel coordinate of CCD photo-sensitive cell;

Second step, according to the round target image of the band line laser striation that obtains in the first step, extract the line laser striation, the line laser striation of refinement extraction is obtained the line equation of line laser striation through the Hough conversion; The outer parameter matrix RT that utilizes the first step to obtain obtains the plane equation of line laser striation plane under camera coordinates system;

The 3rd step, the mechanical arm attitude corresponding according to each pose in the first step utilize the hypercomplex number method to calculate terminal coordinate system of mechanical arm and the transformation matrix that the mechanical arm basis coordinates is, promptly obtain trick transformation matrix H;

The 4th the step, welding work pieces is positioned over the mechanical arm end; The control welding work pieces under the terminal pose of solid mechanical arm to the circle target on a bit accurately point touch; Calculate the coordinate figure of welding work pieces distal point under the mechanical arm coordinate, and combine the mechanical arm pose to calculate the deviant of workpiece under said pose.

2. the welding robot system scaling method based on the line structured light vision sensor guiding according to claim 1 is characterized in that the said first step comprises the steps:

(1.1), utilize the image of camera online real time collecting circle target, adopt big Tianjin method adaptive threshold to image binaryzation, make outstanding demonstration of circular spot on the circle target image;

(1.2), utilize the closed operation operator that circle target image is carried out closed operation, to remove noise; All target round dots in the round target image after the above-mentioned binaryzation are carried out mark; The number of pixels that the area of target round dot comprises for the target round dot; Add up the number of pixels of the target round dot in the above-mentioned round target image; It is NP_average that the target-marking round dot comprises the number of pixels average, removes in the target-marking to comprise number of pixels less than comprising the target of number of pixels greater than 1.5*NP_average in 0.5*NP_average and the target-marking;

(1.3), calculate the girth C and the area S of target round dot behind the mark, utilize circularity expression formula e=C ²/ (2* π * S) distinguishes target round dot and interference;

(1.4), adopt gravity model appoach to ask for the central coordinate of circle (x of each target round dot ₀, y ₀),

Wherein, m, n is row, the train value of circle target image, Hit [i] [j] expression (i, j) location point whether on the target round dot, (i, j) location point Hit [i] [j]=1 on the target round dot, otherwise Hit [i] [j]=0;

(1.5), justify have several rows on the target, the row round dot of arranging; Line-spacing between the adjacent two row round dots is 15mm; Distance with 2 adjacent in delegation round dot centers of circle is 10mm; There is circle of reference at the 5mm place on first line direction of circle target first row, obtains target round dot pixel coordinate based on step (1.4), utilizes circle of reference that the centre point on the circle target image is sorted;

(1.6), according to above-mentioned round target image orbicular spot ordering, with given round target information, accomplish the coupling of centre point pixel coordinate and world coordinates in the circle target image.

3. the welding robot system scaling method based on the line structured light vision sensor guiding according to claim 1 is characterized in that, said second step comprises the steps:

(2.1), laser line generator is fixed on the camera; It is only to need to guarantee that all target round dots and striation part get final product in the viewing field of camera scope that camera is gathered picture; Utilize predetermined threshold value that the line laser optical strip image that collects is carried out binaryzation, binary image is carried out the image closed operation to remove the edge singular point;

(2.2), line laser striation regional center is carried out 8-neighborhood mark; The central point in note line laser striation zone is p1, and 8 points of line laser striation regional center point p1 neighborhood are respectively p2 around central point, p3 clockwise; P9, wherein p2 dot center's point satisfies the boundary point of following condition when line laser striation regional center point p1 is carried out 8-neighborhood mark above p1:

（ⅰ）、2≤N（p1）≤6；

（ⅱ）、S（p1）=1；

（ⅲ）、p2*p4*p6=0；

（ⅳ）、p4*p6*p8=0；

Wherein, N (p1) is the number of the non-zero adjoint point of central point p1; S (p1) is with p2, p3 ..., the value of these points was from 0 → 1 change frequency when p9 was preface; When all boundary points are all checked finish after, all gauge points are removed, iterate and do not satisfy flag condition up to point not, accomplish the striation refinement;

(2.3), the point that obtains after the refinement is utilized the Hough conversion to extract to obtain the line equation a of laser striation of reaching the standard grade on circle target plane _kX+b _kY+c _k=0, wherein, a _k, b _k, c _kThe parameter of representing the line equation respectively.

4. the welding robot system scaling method based on the line structured light vision sensor guiding according to claim 1 is characterized in that, said the 3rd step comprises the steps:

(3.1), justifying the plane equation of target under world coordinate system can be expressed as

In the formula: π ₁=[0,0,1,0] ^T, the outer parameter matrix RT that utilizes the first step to obtain, the plane equation that calculates target plane under camera coordinates system does Wherein

With Be respectively the planar process of target plane under world coordinate system and camera coordinates system to coordinate vector;

(3.2), in camera coordinates system, establish Represent the normal direction coordinate vector of laser plane normal direction coordinate vector, an i circle target, the normal direction coordinate vector of a j circle target, the coordinate vector of the laser intersection image in the i circle target, the coordinate vector of the laser intersection image in the j plane target drone respectively; Note the equation λ of i the laser intersection image on the plane target drone image _iWith

Can directly calculate:

(3.3), the duality relation according to two laser intersections in the projector space and the plane that is associated thereof gets

Wherein

D is one 2 * 2 matrix; Q is a bivector, and define as follows:

is the laser plane equation of being asked.

5. the welding robot system scaling method based on the line structured light vision sensor guiding according to claim 1 is characterized in that, said the 4th step comprises the steps:

(4.1), get any 2 poses in the first step;

be the transition matrix of the terminal coordinate system of mechanical arm under 2 poses of

expression wherein; Make the transition matrix of camera coordinates system under 2 poses of

expression, make

then can be abbreviated as Φ H=H Θ;

(4.2), Φ H=H Θ; Expand into

R _ΦR _H=R _HR _Θ ①

R _Φt _H+t _Φ=R _Ht _Θ+t _H ②

Wherein: R _Φ, R _Θ, R _HExpression Φ, Θ, the rotating part of correspondence among the H, t _Φ, t _Θ, t _HThe expression Φ, Θ, among the H for translating sections;

(4.3), order

Be R _Φ, R _H, R _ΘCorresponding hypercomplex number, q _ΦCalculate q by homogeneous matrix Φ _ΘCalculate order by homogeneous matrix Θ

Through

Try to achieve q _H, again with q _HConvert the rotating part of trick matrix into, Ω () the title matrix function that makes difficulties wherein,

Expression

Antisymmetric matrix; x ₀, x is q _HIn unknown quantity to be calculated;

(4.4), 2. the spin matrix data are brought into just can obtain t again _H, obtain trick transformation matrix H.

6. the welding robot system scaling method based on the line structured light vision sensor guiding according to claim 1 is characterized in that, calculates the welding work pieces shift value in said the 4th step and comprises the steps:

(5.1), the control welding work pieces under the terminal pose of solid mechanical arm to the circle target on the preset accurately point touch, read current mechanical arm pose, obtain first coordinate figure of the terminal coordinate of mechanical arm;

(5.2), the coordinate of definition preset under world coordinate system; According to the outer parameter matrix RT that obtains in the first step; Obtain the coordinate of preset under camera coordinates system; Mechanical arm pose during according to trick transformation matrix H and shooting circle target image calculates second coordinate figure of preset under mechanical arm basis coordinates system, and said second coordinate figure and the comparison of first coordinate figure are obtained welding work pieces shift value under the current pose.

7. the welding robot system scaling method based on the line structured light vision sensor guiding according to claim 2; It is characterized in that: when utilizing circle of reference that the centre point on the circle target image is sorted, the distance of center circle of finding out effective round dot on the round target image wherein leaves 0 point that is defined as of all centre point average centers circle far away from two minimum target round dots; Another is defined as 1 point; Be provided with then at 0,1 and be match point, in match point not, look for defining 2 points with 1 nearest point; Be provided with at 2 and be match point; In match point not, look for and define 3 points, be provided with at 3 and be match point with 2 nearest points ... Until have a few all look for complete, so accomplish the ordering of centre point.

Images