CN111508022B - Line laser stripe positioning method based on random sampling consistency - Google Patents

Abstract

The invention discloses a line laser stripe positioning method based on random sampling consistency, belonging to industrial machine visionThe field of vision, comprising the steps of: firstly, carrying out binarization processing on a line laser picture acquired by an industrial camera, and carrying out skeleton extraction on the picture by using a Zhang skeleton extraction method to obtain data of a line laser straight line; constructing a distance binary tree for data, finding a certain number of points near a pixel point at random to construct a subdata set, carrying out random sampling consistency fitting on the subdata set, and carrying out an internal point threshold Th on a fitting result₁Judging, and performing interior point threshold Th on the obtained straight line in the global data point₂And judging, and performing random sampling consistency fitting on the preliminarily obtained straight line classification to obtain a final straight line classification result so as to realize the positioning of the linear laser. The method considers the abnormal conditions of laser stripe data loss, breakage, overexposure, stray light interference and the like in the actually acquired image, is suitable for line laser positioning detection in general industrial environment, and reduces the conservatism.

Description

Line laser stripe positioning method based on random sampling consistency

Technical Field

The invention belongs to the field of industrial machine vision, and relates to a line laser stripe positioning method based on random sampling consistency.

Background

In recent years, machine vision technology is gradually applied to industrial production, and great economic benefits are brought. In the industries of welding, grabbing, spraying and the like, the realization of three-dimensional measurement and positioning of an object through a vision technology is a research hotspot at present. The line structured light three-dimensional measurement has been widely used in three-dimensional visual measurement and detection due to its characteristics of low cost, small volume, light weight, convenience, flexibility, etc. The line structured light three-dimensional measurement technology takes line laser as a structured light source, and needed three-dimensional positioning information is solved by analyzing a deformation image irradiated on an object to be measured by laser stripes.

The stripe image of the line laser in the welding seam positioning generally consists of a plurality of straight lines, and the straight line parameters in the laser stripe image are accurately and quickly extracted, so that the significance of the accuracy and the efficiency of the welding seam positioning is great. The laser stripe has a certain width, and various interference factors such as stray light, diffuse reflection and the like exist in the actual industrial production environment, so that the imaging quality of a stripe image is influenced, and the accurate extraction of stripe linear parameters is difficult.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a line laser stripe positioning method based on random sampling consistency. In consideration of abnormal conditions such as possible deletion, breakage, overexposure and veiling glare interference of a fringe image acquired under an industrial production environment, a laser fringe straight line classification extraction algorithm based on Random Sample Consensus (RANSAC) is designed, so that extracted straight line data can accurately restore fringe straight line information in the image, a good inhibition effect on various interferences is achieved, and positioning of laser fringes is achieved.

The technical scheme of the invention is as follows:

a line laser stripe positioning method based on random sampling consistency comprises the following steps:

performing preprocessing operations such as morphological filtering, binary segmentation and the like on the laser stripe image, performing skeleton extraction on the preprocessed image by using a Zhang skeleton extraction method, acquiring pixel point coordinates of all skeletons, establishing a DATA set DATA1, constructing a distance binary tree for the DATA set DATA1, copying the DATA set DATA1 to obtain a DATA set DATA1_copy；

Wherein: the DATA set DATA1 is a two-dimensional DATA set representation of all pixel point coordinates extracted from the skeleton.

Randomly selecting a pixel point in a DATA set DATA1, searching n (n >100) DATA points nearest to the pixel point, and establishing a DATA set DATA 2;

wherein: DATA set DATA2 is a two-dimensional DATA set representation of all pixels in the neighborhood of random pixels.

Step (3) randomly sampling i (i) in the DATA set DATA2 according to the principle of random sampling consistency>100) Second, each sample k (k)>2) Point, fitting straight line l_iCalculating the point of each coordinate in the DATA set DATA2 to the straight line l_iWill be smaller than the threshold Th_dIs judged as a straight line l_iThe number of the inner points is counted as the number N of the inner points_i；

Wherein: linear equation l fitted by ith sampling_iIn a general form of (a) Ax + By + C is 0, a, B and C are fitted straight line equation parameters; x is the abscissa of the pixel and y is the ordinate of the pixel. For any point (x) of the DATA set DATA2₀,y₀) The distance d from the straight line is expressed as

Threshold Th_dFor characterizing all points in the DATA set DATA2 with respect to the straight line l_iThe deviation degree of the laser is determined by the laser line width, and the parameter setting can be generally 1-2 pixel widths; n is a radical of_iIs the ith sample fitting process, all of the DATA sets DATA2 satisfy the distance threshold Th_dA set of all points of (a).

Step (4) selecting the number of inner points N_iMaximum straight line l_mIf its inner point number N_mLess than threshold Th₁Returning to the operation of the step (2); if the number of inner points is N_mGreater than threshold Th₁Then calculate the straight line l_mIn the DATA set DATA1_copyInner point number N in_{m_copy}If the number of inner points is N_{m_copy}Greater than threshold Th₂Then count the number of inner points N_{m_copy}Storing new DATA sets DATA_mAnd a straight line l_mThe inner point of (3) is deleted from the DATA set DATA1, updating the DATA set DATA 1; otherwise, directly switching to the operation of the step (2);

wherein: threshold Th₁For measuring the straight line l screened_mWhether a reasonable straight line exists in the neighborhood point is the screened straight line l_mNumber of inner points N_mA constraint of (3) which is set to be half of the number of nearest points searched in the step (2); n is a radical of_{m_copy}Is a straight line l_mAt the global set of pixel points DATA1_copyThe inner point set in (1); threshold Th₂For measuring the straight line l screened_mIn the DATA set DATA1_copyWhether the inner part is a reasonable straight line or not, the value set by the experiment is a DATA set DATA1_copyHalf the number of points.

Step (5) number of points in the DATA set DATA1 updated in step (4)Less than threshold Th_{_Num}Inner point DATA indicating that all skeleton points at this time have been classified into respective straight lines_mAnd invalid point DATA1, stopping the loop. According to interior point DATA DATA_mAfter each straight line is fitted by using least square, parameters of each straight line are optimally fitted by using an IGGIII weighted least square regression method, so that the positioning extraction of laser stripe parameters is realized;

wherein: th_{_Num}Is a termination threshold, i.e. when the number of points in the DATA set DATA1 is less than a certain number of points, all possible linear classifications are considered to have been found, considering noise interference, a threshold Th_{_Num}Can be set to 20 points; DATA_mThe stored interior points corresponding to all the straight line classifications; IGG III is weight distribution function, and weight w of each point is distributed_iThe weight of the inner point of each type of straight line is specifically distributed as follows:

in the formula, k₀And k₁To harmonic coefficients, k₀Usually 1.0 to 1.5, k₁Usually 2.5-3.0; u. of_iIs the distance ratio coefficient, u_i＝d_i/σ，d_iIs a point (x)_i,y_i) The distance to the straight line is such that,

A_ls，B_lsand C_lsIs a linear equation parameter fitted by least square, sigma is a distance root mean square,

in the invention, in consideration of the condition that a line laser stripe image collected in an industrial environment may be subjected to various interferences, skeleton pixel points on the laser stripe image with a width are extracted by a Zhang skeleton extraction method, the extracted pixel points are classified into pixel point data and invalid noise point data on each straight line according to a random sampling consistency principle, and finally parameters of each straight line are fitted through random sampling consistency, so that the accurate positioning of the laser stripe is realized. Compared with the traditional method, the method has higher efficiency and better noise tolerance, and can definitely identify the data and the noise data of different straight lines on the laser stripe image.

Drawings

FIG. 1 is a flow chart of a line laser stripe positioning method based on random sampling;

FIG. 2 shows a target pixel P₁(i, j) a schematic diagram of 8 neighborhood pixel points around;

FIG. 3 is a raw graph containing fracture data;

FIG. 4 is a skeleton extracted binary image containing fracture data;

FIG. 5 is a graph of the results of a straight line classification with fracture data;

FIG. 6 is an original graph containing overexposure data and veiling glare;

FIG. 7 is a skeleton extraction binary image containing overexposure data and veiling glare interference;

FIG. 8 is a graph of linear classification results including overexposure data and veiling glare interference;

FIG. 9 is an original graph containing missing data;

FIG. 10 is a skeleton extracted binary image with missing data;

fig. 11 is a graph showing the results of straight-line classification with missing data.

Detailed Description

The following further describes the embodiments of the present invention with reference to the drawings.

As shown in fig. 1, a flow chart of a line laser stripe positioning method based on random sampling includes the following steps:

performing preprocessing operations such as morphological filtering, binary segmentation and the like on the laser stripe image, performing skeleton extraction on the preprocessed image by using a Zhang skeleton extraction method, acquiring pixel point coordinates of all skeletons, establishing a DATA set DATA1, constructing a distance binary tree for the DATA set DATA1, copying the DATA set DATA1 to obtain a DATA set DATA1_copy；

Randomly selecting a pixel point in a DATA set DATA1, searching n (n >100) DATA points nearest to the pixel point, and establishing a DATA set DATA 2;

step (3) randomly sampling i (i) in the DATA set DATA2 according to the principle of random sampling consistency>100) Second, each sample k (k)>2) Point, fitting straight line l_iCalculating the point of each coordinate in the DATA set DATA2 to the straight line l_iWill be smaller than the threshold Th_dIs judged as a straight line l_iThe number of the inner points is counted as the number N of the inner points_i；

The fitting straight line l_iCalculating the point of each coordinate in the DATA set DATA2 to the straight line l_iWill be smaller than the threshold Th_dIs judged as l_iInner point of (2), count the number of inner points N_iThe method specifically comprises the following steps: linear equation l fitted by ith sampling_iIs of the general form: ax + By + C is 0, wherein A, B and C are fitted linear equation parameters; x is the abscissa of the pixel point, and y is the ordinate of the pixel point; for any point (x) of the DATA set DATA2₀,y₀) The distance d from the straight line is expressed as

Threshold Th_dFor characterizing all points in the DATA set DATA2 with respect to the straight line l_iThe degree of deviation of (d); n is a radical of_iIs the ith sample fitting process, all of the DATA sets DATA2 satisfy the distance threshold Th_dA set of all points of (a).

Step (4) selecting the number of inner points N_iMaximum straight line l_mIf its inner point number N_mLess than threshold Th₁Returning to the operation of the step (2); if the number of inner points is N_mGreater than threshold Th₁Then calculate the straight line l_mIn the DATA set DATA1_copyInner point number N in_{m_copy}If the number of inner points is N_{m_copy}Greater than threshold Th₂Then count the number of inner points N_{m_copy}Storing new DATA sets DATA_mAnd a straight line l_mThe inner point of (3) is deleted from the DATA set DATA1, updating the DATA set DATA 1; otherwise, directly switching to the operation of the step (2);

the threshold Th₁For measuring the straight line l screened_mWhether a reasonable straight line exists in the neighborhood point is the screened straight line l_mNumber of inner points N_mA constraint of (3) which is set to be half of the number of nearest points searched in the step (2); number of inner points N_{m_copy}Is a straight line l_mAt the global pixel DATA set DATA1_copyThe inner point set in (1); threshold Th₂For measuring the straight line l screened_mIn the DATA set DATA1_copyWhether the inner part is a reasonable straight line or not, the value set by the experiment is a DATA set DATA1_copyHalf the number of points.

Step (5) when the number of points in the DATA set DATA1 updated in step 4) is less than the threshold Th_{_Num}Inner point DATA indicating that all skeleton points at this time have been classified into respective straight lines_mAnd invalid point DATA1, stopping the loop. According to interior point DATA DATA_mAnd after each straight line is fitted by using least square, the parameters of each straight line are optimally fitted by using an IGGIII weighted least square regression method, so that the positioning extraction of the laser stripe parameters is realized.

The threshold Th_{_Num}Is a threshold value of termination, i.e. threshold value Th, when the number of points in the DATA set DATA1 is less than a certain number of points, assuming that all possibilities for straight line classification have been found_{_Num}Can be set to 20 points; the inner dot DATA DATA_mThe stored interior points corresponding to all the straight line classifications; the IGG III is a weight distribution function, and distributes the weight w of each point_iThe weight of the inner point of each type of straight line is specifically assigned as follows:

wherein u is_iIs the distance ratio coefficient, u_i＝d_i/σ；k₀And k₁To harmonic coefficients, k₀Usually 1.0 to 1.5, k₁Usually 2.5 to 3.0, d_iIs a point (x)_i,y_i) Distance to the straight line, disclosed as:

wherein A is_ls，B_lsAnd C_lsThe linear equation parameters are obtained by least square fitting, sigma is a distance root mean square, and the formula is as follows:

as a further illustration of the invention:

the method comprises the following steps of (1) carrying out binarization on a preprocessed picture, specifically, determining whether gray level thresholding is required to be carried out on the picture according to the picture type collected by a laser camera, and converting the picture into a binary image, wherein all pixel points are not 0, namely 1, black is 0, and white is 1. Fig. 3 shows an original graph of data including fracture acquired by a laser camera, fig. 6 shows an original graph including overexposure data and veiling glare interference, and fig. 9 shows an original graph including missing data.

Skeleton extraction, wherein the acquisition of target processing data specifically comprises the following steps: performing skeleton extraction on the binary image by using a Zhang skeleton extraction method, and performing skeleton extraction on the pixel point P of the ith row and the jth column of the binary image₁(i, j) searching 8 neighborhood pixels around the target pixel, as shown in FIG. 2, which is the target pixel P₁(i, j) surrounding 8 neighborhood pixel point schematic diagrams, and the corresponding serial numbers thereof are respectively marked as P₂(i-1,j)，P₃(i-1,j+1)，P₄(i,j+1)，P₅(i+1,j+1)，P₆(i+1,j)，P₇(i+1,j-1)，P₈(i,j-1)，P₉(i-1, j-1) 8 pixels in total, and judging whether the following 4 conditions are met:

①2≤B(P₁)≤6

②A(P₁)＝1

③P₂*P₄*P₆＝0

④P₄*P₆*P₈＝0

wherein the condition (r) is the center pixel P₁Surrounding target pixel (binary)The sum of the number of 1) s is between 2 and 6; the condition II is that in 8 adjacent pixels around the central pixel, two adjacent pixels have the frequency of 0-1 change in the clockwise direction; the conditions (c) and (c) are such that the product of the pixel values of the pixels corresponding to the sequence numbers is 0, the condition (c) corresponds to the even-order iteration number, and the condition (c) corresponds to the odd-order iteration number. There is also another equivalent pattern for (c) and (d) as follows:

⑤P₂*P₄*P₈＝0

⑥P₂*P₆*P₈＝0

and if the current pixel point meets the four conditions, deleting the current pixel point, traversing all the points until no point can be deleted, finishing traversal, and finishing skeleton extraction. FIG. 4 shows the result of skeleton extraction with fracture data; FIG. 7 shows the result of skeleton extraction with overexposure data and veiling glare interference; fig. 10 shows the result of skeleton extraction for data containing a missing part.

The steps (2) - (4) are primary linear classification, which is specifically as follows:

constructing KD tree for extracted object processing DATA DATA1, and obtaining DATA1 by copying DATA set DATA1 only once_copyFor the subsequent calculation of the number of inner points of the straight line under the global data.

Randomly generating a point P in DATA1, searching for a certain number of points closest to the point P: (>100) Performing random sampling straight line fitting on the searched closest point, selecting 3 points for each sampling point, setting the maximum iteration number to be 200, counting the inner points of the straight line obtained by each fitting, and judging whether the distance d from the point to the straight line is smaller than a distance threshold Th or not according to the inner points_dIf the number of the inner points is less than the number of the outer points, the point is considered as an inner point, otherwise, the point is considered as an outer point, and a linear equation l with the most inner points is extracted_mThe number of corresponding inner points is N_mJudging the number of the inner points, if the number of the inner points is N_mLess than threshold Th₁Returning to randomly generating a point again and repeating the operation, otherwise, aligning the straight line l_mIn the DATA set DATA1_copyCalculating the number N of inner points_{m_copy}And go on again insideJudging if N is a point threshold_{m_copy}Less than threshold Th₂Then a random point is regenerated and the start step is repeated, otherwise the DATA set DATA1 is deleted containing the straight line/_mUpdate DATA1, and determine whether the number of DATA1 is less than Th_NumIf yes, the straight line division in the DATA set is finished, otherwise, the KD tree is built for the DATA1 again, and the operation is repeated until the number of points meeting the DATA1 is smaller than Th_NumAnd stopping searching to obtain the result of the primary classification straight line.

Step (5) is the final linear classification, which is specifically as follows:

and (3) further fitting the preliminarily divided straight lines, and optimizing and fitting parameters of each straight line by using a least square fitting method and then using an IGGIII weighted least square regression method to obtain a final straight line classification result, thereby realizing the positioning of the linear laser stripes. FIG. 5 shows the results of straight line classification with fracture data; FIG. 8 shows the results of linear classification including overexposure data and veiling glare; fig. 11 shows the results of straight-line classification with missing data.

Claims (4)

1. A line laser stripe positioning method based on random sampling consistency is characterized by comprising the following steps:

1) performing morphological filtering and binarization segmentation preprocessing on the laser stripe image, performing skeleton extraction on the preprocessed image by using a Zhang skeleton extraction method, acquiring pixel point coordinates of all skeletons, establishing a DATA set DATA1, constructing a distance binary tree for the DATA set DATA1, copying the DATA set DATA1 to obtain a DATA set DATA1_copy；

2) Randomly selecting a pixel point in a DATA set DATA1, searching n (n is more than 100) DATA points nearest to the pixel point, and establishing a DATA set DATA 2;

3) according to the principle of random sampling consistency, i (i >100) times are randomly sampled in a DATA set DATA2, k (k > 2) points are sampled each time, and a straight line l is fitted_iCalculating the point of each coordinate in the DATA set DATA2 to the straight line l_iWill be smaller than the threshold Th_dIs judged as a straight line l_iThe number of the inner points is counted as the number N of the inner points_i；

4) Selecting the number of interior points N_iMaximum straight line l_mIf its inner point number N_mLess than threshold Th₁Returning to the operation of the step 2); if the number of inner points is N_mGreater than threshold Th₁Then calculate the straight line l_mIn the DATA set DATA1_copyInner point number N in_{m_copy}If the number of inner points is N_{m_copy}Greater than threshold Th₂Then count the number of inner points N_{m_copy}Storing new DATA sets DATA_mAnd a straight line l_mThe inner point of (3) is deleted from the DATA set DATA1, updating the DATA set DATA 1; otherwise, directly switching to the operation of the step 2);

5) when the number of points in the updated DATA set DATA1 is less than the threshold Th in step 4)_{_Num}Inner point DATA indicating that all skeleton points at this time have been classified into respective straight lines_mAnd invalid point DATA1, stopping the loop; according to interior point DATA DATA_mAnd after each straight line is fitted by using least square, the parameters of each straight line are optimally fitted by using an IGGIII weighted least square regression method, so that the positioning extraction of the laser stripe parameters is realized.

2. A line laser stripe positioning method based on random sampling consistency is characterized in that the fitted straight line l in the step 3)_iCalculating the point of each coordinate in the DATA set DATA2 to the straight line l_iWill be smaller than the threshold Th_dIs judged as l_iInner point of (2), count the number of inner points N_iThe method specifically comprises the following steps: linear equation l fitted by ith sampling_iIs of the general form: ax + By + C is 0, wherein A, B and C are fitted linear equation parameters; x is the abscissa of the pixel point, and y is the ordinate of the pixel point; for any point (x) of the DATA set DATA2₀,y₀) The distance d from the straight line is expressed as

Threshold Th_dFor characterizing all points in the DATA set DATA2 with respect to the straight line l_iThe degree of deviation of (d); n is a radical of_iIs the ith sample fitting process, all of the DATA sets DATA2 satisfy the distance threshold Th_dA set of all points of (a).

3. A line laser stripe positioning method based on random sampling consistency is characterized in that the threshold Th in the step 4)₁For measuring the straight line l screened_mWhether a reasonable straight line exists in the neighborhood point is the screened straight line l_mNumber of inner points N_mThe constraint of (2), which is set to be half of the number of the nearest points searched in the step 2); number of inner points N_{m_copy}Is a straight line l_mAt the global pixel DATA set DATA1_copyThe inner point set in (1); threshold Th₂For measuring the straight line l screened_mIn the DATA set DATA1_copyWhether the inner part is a reasonable straight line or not, the value set by the experiment is a DATA set DATA1_copyHalf the number of points.

4. A line laser stripe positioning method based on random sampling consistency is characterized in that the threshold Th in step 5) is_{_Num}Is a threshold value of termination, i.e. threshold value Th, when the number of points in the DATA set DATA1 is less than a certain number of points, assuming that all possibilities for straight line classification have been found_{_Num}Can be set to 20 points; the inner dot DATA DATA_mThe stored interior points corresponding to all the straight line classifications; the IGG III is a weight distribution function, and distributes the weight w of each point_iThe weight of the inner point of each type of straight line is specifically assigned as follows:

wherein u is_iIs the distance ratio coefficient, u_i＝d_i/σ；k₀And k₁To harmonic coefficients, k₀Usually 1.0 to 1.5, k₁Usually 2.5 to 3.0, d_iIs a point (x)_i,y_i) Distance to the straight line, disclosed as:

wherein A is_ls，B_lsAnd C_lsThe linear equation parameters are obtained by least square fitting, sigma is a distance root mean square, and the formula is as follows:

Images