CN112508984B - Registration-based image laser center line extraction method - Google Patents
Registration-based image laser center line extraction method Download PDFInfo
- Publication number
- CN112508984B CN112508984B CN202011506666.6A CN202011506666A CN112508984B CN 112508984 B CN112508984 B CN 112508984B CN 202011506666 A CN202011506666 A CN 202011506666A CN 112508984 B CN112508984 B CN 112508984B
- Authority
- CN
- China
- Prior art keywords
- skeleton
- line
- point
- laser
- center line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
Abstract
The invention discloses a registration-based image laser center line extraction method, which comprises the steps of progressively sorting extracted incomplete streak skeleton points according to column coordinate values; performing piecewise linear fitting on the ordered skeleton points, and taking the obtained first piecewise point as a division point of the left part and the right part of the skeleton; calculating an intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton, and integrally moving the simulation laser line to the intersection point by taking the simulation weld joint point as a reference point; the simulated weld joint points divide the simulated laser line into a left part and a right part, compare the straightness of the two parts, select the part with high straightness to coincide with the corresponding actual skeleton segmentation line, and finish coarse registration; and matching the rough registered simulation laser line to the framework through an improved iterative closest point algorithm to obtain a laser stripe center line. The invention greatly improves the anti-interference capability, is not only suitable for extracting the common laser center line stripes, but also suitable for images with interference.
Description
Technical Field
The invention relates to the field of machine vision, in particular to an image laser center line extraction method based on registration.
Background
With the development of intelligent manufacturing at present, the non-contact structured light vision sensor is widely applied in industrial application, and the structured light vision sensor is widely applied in the fields of curved surface modeling processing detection, workpiece quality detection, weld seam tracking and the like. The vision sensor adopting the line structured light mode meets the laser triangulation measurement model, and is a non-contact measurement mode with high measurement speed and high precision. The laser line irradiates the surface of the measured object to form light stripes, the light stripes are affected by the geometric shape of the surface of the measured object to generate discontinuous and distorted phenomena, and the change contains depth information of the surface of the measured object. By analyzing the collected laser stripe image, the central line of the laser stripe is extracted, and the spatial position of a point on the central line of the laser can be calculated according to the geometric model formed by the camera and the laser, so that the structural information of the surface of the measured object is obtained.
At present, the laser center line extraction is more studied, and a common method is to acquire edge characteristics by detecting gray level changes in an image in an edge detection mode. This method is susceptible to noise in the image, and is not applicable to the use of edge detection, especially for specular reflection light interference. Meanwhile, for the laser center line extraction method adopting a straight line detection mode, such as Hough straight line transformation, the laser lines in the image are not straight lines any more but curves with different shapes due to the shape change of the surface of the detected object, and the straight line detection mode is not applicable any more. Meanwhile, other laser center line extraction methods adopting an ideal Gaussian distribution model are adopted, and the reflected laser stripes do not meet the ideal Gaussian distribution due to the change of the surface shape of the measured object, so that the extracted center line precision is low finally.
The above mentioned method is sensitive to environmental noise, has limited extraction accuracy, and cannot accurately process the structured light stripe image containing interference.
Disclosure of Invention
In order to overcome the defects and shortcomings in the prior art, the invention provides a registration-based image laser center line extraction method, which can accurately extract the center line of an effective laser stripe under the condition that images are interfered, and improves the application range of a structured light vision sensor.
The invention adopts the following technical scheme:
an image laser center line extraction method based on registration comprises the following steps:
the extracted incomplete streak skeleton points are sorted according to the increment of column coordinate values;
performing piecewise linear fitting on the ordered skeleton points, and taking the obtained first piecewise point as a division point of the left part and the right part of the skeleton;
calculating an intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton, and integrally moving the simulation laser line to the intersection point by taking the simulation weld joint point as a reference point;
the simulated weld joint points divide the simulated laser line into a left part and a right part, compare the straightness of the two parts, select the part with high straightness to coincide with the corresponding actual skeleton segmentation line, and finish coarse registration;
and matching the rough registered simulation laser line to the framework through an improved iterative closest point algorithm to obtain a laser stripe center line.
Further, the calculating the intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton, and moving the simulation laser line to the intersection point integrally by taking the simulation weld joint point as a reference point, specifically comprises: and executing translation operation on the simulated laser line, wherein the translation starting point is a simulated weld joint point, and the end point is the intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton.
Further, a piece-wise linear fitting is performed on the ordered skeleton points by using a Fabry-Perot algorithm.
Further, the improved iterative closest point algorithm is specifically solved by adopting an affine transformation matrix solving method in the calculation process of each round of iterative optimization.
Further, each iteration process carries out affine transformation matrix solving once, finally obtains an optimal registration transformation matrix of the simulated laser stripes and the skeleton, and finally transforms the simulated laser stripes according to the matrix.
Further, the affine transformation matrix is:
wherein a is 11 、a 12 、a 21 、a 22 Is a matrix element controlling a linear transformation part in affine transformation, t x 、t y Is a matrix element that controls the translational transformation portion.
Further, the left and right skeletons were fitted using RANSAC, and the intersection points were found.
The invention has the beneficial effects that:
the method introduces simulation laser stripe information given by visual simulation, adopts a reasonable registration strategy, greatly improves the anti-interference capability, is not only suitable for extracting common laser center line stripes, but also suitable for images with interference, has strong adaptability, and expands the application working range of the structured light visual sensor.
Drawings
FIG. 1 is a broken stripe skeleton of an embodiment of the present invention;
2 (a) -2 (d) simulate the process of laser line rough matching;
FIG. 3 is a schematic flow diagram of an improved iterative closest point algorithm of the present invention;
fig. 4 is a finely registered laser centerline.
Fig. 5 simulates the overall flow of laser line registration.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but embodiments of the present invention are not limited thereto.
Examples
As shown in fig. 5, a registration-based image laser centerline extraction method includes:
as shown in fig. 1, the extracted incomplete streak skeleton in the embodiment may be interfered by noise such as welding arc, spatter, etc. the extracted streak skeleton may be incomplete, so that the geometric features of the weld seam may not be correctly reflected.
S1, sorting the extracted incomplete streak skeleton points according to incremental column coordinate values;
s2, performing piecewise linear fitting on the ordered skeleton points by using a Fabry-Perot algorithm, and taking the obtained first piecewise point as a division point of the left part and the right part of the skeleton;
s3, calculating an intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton, and integrally moving the simulation laser line to the intersection point by taking the simulation weld joint point as a reference point, wherein the intersection point is specifically as follows: and executing translation operation on the simulated laser line, wherein the translation starting point is a simulated weld joint point, and the end point is the intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton.
S4, the simulated weld joint points divide the simulated laser line into a left part and a right part, the straightness of the two parts is compared, and the part with high straightness is selected to coincide with the corresponding skeleton segmentation line, so that rough registration is completed; the simulated weld joint points are obtained directly through a simulation program.
If the straightness of the left side of the simulation laser line is higher, the corresponding skeleton segmentation line is the rightmost segmentation line of the left skeleton; if the straightness of the right side of the simulation laser line is higher, the corresponding skeleton segmentation line is the leftmost segmentation line of the right skeleton.
S5, matching the rough registered simulation laser line to the framework through an improved iterative closest point algorithm to obtain a laser stripe center line.
As described in fig. 2 (a) -2 (d), S2, S3 and S4 constitute a process of coarse registration.
Fig. 2 (a) is a piecewise linear fit of a real skeleton point and a douglas, points P1-P10 are actual laser stripe skeleton points extracted by an image algorithm, pg is a point with the largest connecting line distance to P1P10 in P1-P10, and the point is used as a division point of a skeleton left part and a skeleton right part.
FIG. 2 (b) shows the maximum distance d between the point on the laser line and the line for the two simulated laser lines Lmax And d Rmax . If d Lmax <d Rmax The left laser line is "straighter" or vice versa.
FIG. 2 (c) calculates the intersection point P of the rightmost segmentation line of the left skeleton and the leftmost segmentation line of the right skeleton real As the approximate actual weld spot location.
FIG. 2 (d) rotates the simulated laser line around the simulated laser spot such that the "more straight" side is approximately parallel to the side corresponding to the actual skeleton, resulting in a rotation matrix R; then integrally translating and rotating the simulated laser line to enable the simulated laser point to be matched with P real And (5) coinciding to obtain a translation vector t.
As shown in fig. 3, an ICP strategy modified by the present method registers simulated laser stripes to the extracted skeleton to yield the final complete laser centerline. Fig. 3 shows a schematic diagram of an iterative closest point algorithm (ICP) improvement flow in the present invention. Different from common ICP registration, in the calculation process of each round of iterative optimization of the ICP algorithm, an affine transformation matrix solving mode is adopted to replace a SVD solving mode, so that the ICP algorithm can adapt to the difference between a real fringe included angle and a simulation fringe included angle. Affine transformations between two-dimensional point sets are shown in the following formula:
transformation points (x) through affine transformation matrix M i ,y i ) (i=1, 2,3, … n) can be transformed into a point (x i ’,y i ') (i=1, 2,3, … n). In each iteration process of the ICP algorithm, carrying out solution on the affine transformation matrix M once, finally obtaining an optimal registration transformation matrix of the simulated laser stripes and the skeleton, transforming the simulated laser stripes by using the matrix to obtain a complete laser center line shown in FIG. 4, and obtaining the positions ('X' positions) of welding points.
The invention utilizes the simulated laser stripe information given by the visual simulation system, greatly improves the anti-interference capability, is not only suitable for extracting the common laser center line stripes, but also suitable for the images with interference, has strong adaptability, and expands the application working range of the structured light visual sensor.
The embodiments described above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the embodiments described above, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principles of the present invention should be made in the equivalent manner, and are included in the scope of the present invention.
Claims (5)
1. The image laser center line extraction method based on registration is characterized by comprising the following steps of:
the extracted incomplete streak skeleton points are sorted according to the increment of column coordinate values;
performing piecewise linear fitting on the ordered skeleton points, and taking the obtained first piecewise point as a division point of the left part and the right part of the skeleton;
calculating an intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton, and integrally moving the simulation laser line to the intersection point by taking the simulation weld joint point as a reference point;
the simulated weld joint points divide the simulated laser line into a left part and a right part, compare the straightness of the two parts, select the part with high straightness to coincide with the corresponding actual skeleton segmentation line, and finish coarse registration;
matching the rough registered simulation laser line to a framework through an improved iterative nearest point algorithm to obtain a laser stripe center line;
the improved iterative closest point algorithm is characterized in that in the calculation process of each round of iterative optimization, an affine transformation matrix solving method is adopted for solving;
and carrying out affine transformation matrix solving once in each iteration process, finally obtaining an optimal registration transformation matrix of the simulated laser stripes and the skeleton, and finally transforming the simulated laser stripes according to the matrix.
2. The method for extracting an image laser center line according to claim 1, wherein the calculating the intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton, and moving the simulation laser line integrally to the intersection point with the simulation weld point as a reference point, specifically comprises: and executing translation operation on the simulated laser line, wherein the translation starting point is a simulated weld joint point, and the end point is the intersection point of the rightmost segment line of the left skeleton and the leftmost segment line of the right skeleton.
3. The method of claim 1, wherein the sorted skeleton points are piecewise linearly fitted using a douglas-purk algorithm.
4. The image laser center line extraction method according to claim 1, wherein the affine transformation matrix is:
wherein a is 11 、a 12 、a 21 、a 22 Is a matrix element controlling a linear transformation part in affine transformation, t x 、t y Is a matrix element that controls the translational transformation portion.
5. The method for extracting an image laser center line according to claim 1, wherein a RANSAC is used to fit the left and right skeletons and the intersection point is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011506666.6A CN112508984B (en) | 2020-12-18 | 2020-12-18 | Registration-based image laser center line extraction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011506666.6A CN112508984B (en) | 2020-12-18 | 2020-12-18 | Registration-based image laser center line extraction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112508984A CN112508984A (en) | 2021-03-16 |
| CN112508984B true CN112508984B (en) | 2023-08-18 |
Family
ID=74922507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011506666.6A Active CN112508984B (en) | 2020-12-18 | 2020-12-18 | Registration-based image laser center line extraction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112508984B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108132017A (en) * | 2018-01-12 | 2018-06-08 | 中国计量大学 | A kind of plane welded seam Feature Points Extraction based on laser vision system |
| CN108921818A (en) * | 2018-05-30 | 2018-11-30 | 华南理工大学 | A kind of weld joint tracking laser center line drawing method |
| CN108986160A (en) * | 2018-06-11 | 2018-12-11 | 华南理工大学 | A kind of image laser center line extraction method containing specular light interference |
-
2020
- 2020-12-18 CN CN202011506666.6A patent/CN112508984B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108132017A (en) * | 2018-01-12 | 2018-06-08 | 中国计量大学 | A kind of plane welded seam Feature Points Extraction based on laser vision system |
| CN108921818A (en) * | 2018-05-30 | 2018-11-30 | 华南理工大学 | A kind of weld joint tracking laser center line drawing method |
| CN108986160A (en) * | 2018-06-11 | 2018-12-11 | 华南理工大学 | A kind of image laser center line extraction method containing specular light interference |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112508984A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109900706B (en) | Weld joint based on deep learning and weld joint defect detection method | |
| CN111274671B (en) | Precise repair method for complex product assembly process based on digital twin and operation system thereof | |
| CN107876970B (en) | Robot multilayer multi-pass welding seam three-dimensional detection and welding seam inflection point identification method | |
| CN110227876A (en) | Robot welding autonomous path planning method based on 3D point cloud data | |
| CN107341802B (en) | Corner sub-pixel positioning method based on curvature and gray scale compounding | |
| CN109612390B (en) | Large-size workpiece automatic measuring system based on machine vision | |
| CN103425988B (en) | Real-time positioning and matching method with arc geometric primitives | |
| CN110031829B (en) | Target accurate distance measurement method based on monocular vision | |
| CN110069041B (en) | Workpiece machining method and system based on-machine measurement | |
| CN108007388A (en) | A kind of turntable angle high precision online measuring method based on machine vision | |
| CN112614098B (en) | Blank positioning and machining allowance analysis method based on augmented reality | |
| CN109272544A (en) | The structural light three-dimensional measurement model and image processing method of pipeline all-position weld seam | |
| CN115096206B (en) | High-precision part size measurement method based on machine vision | |
| CN112365439B (en) | Method for synchronously detecting forming characteristics of GMAW welding seam of galvanized steel and direction of welding gun in real time | |
| CN102279190A (en) | Image detection method for weld seam surface defects of laser welded plates of unequal thickness | |
| CN111311618A (en) | Circular arc workpiece matching and positioning method based on high-precision geometric primitive extraction | |
| Ma et al. | Efficient and accurate start point guiding and seam tracking method for curve weld based on structure light | |
| CN114140439A (en) | Laser welding seam feature point identification method and device based on deep learning | |
| JPH08136220A (en) | Method and device for detecting position of article | |
| CN114473309A (en) | Welding position identification method for automatic welding system and automatic welding system | |
| CN115546125A (en) | Method for error detection and track deviation correction of additive manufacturing cladding layer based on point cloud information | |
| Bauer et al. | Registration strategy of point clouds based on region-specific projections and virtual structures for robot-based inspection systems | |
| CN112508971B (en) | Weld joint positioning method, system and storage medium based on simulation structure light stripe | |
| CN112508984B (en) | Registration-based image laser center line extraction method | |
| Bauer et al. | Free-form surface analysis and linking strategies for high registration accuracy in quality assurance applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |