CN107248170B - Weld joint tracking method based on image matching - Google Patents
Weld joint tracking method based on image matching Download PDFInfo
- Publication number
- CN107248170B CN107248170B CN201710272076.3A CN201710272076A CN107248170B CN 107248170 B CN107248170 B CN 107248170B CN 201710272076 A CN201710272076 A CN 201710272076A CN 107248170 B CN107248170 B CN 107248170B
- Authority
- CN
- China
- Prior art keywords
- filter
- image
- weld
- welding seam
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K9/00—Arc welding or cutting
- B23K9/12—Automatic feeding or moving of electrodes or work for spot or seam welding or cutting
- B23K9/127—Means for tracking lines during arc welding or cutting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0014—Image feed-back for automatic industrial control, e.g. robot with camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Robotics (AREA)
- Plasma & Fusion (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a weld joint tracking method based on image matching, which comprehensively considers the weld joint area, a background image and the characteristic difference between the weld joint area and the background image, is not limited by the depth and the thickness of a weld joint and the like, is not influenced by environmental conditions such as illumination and the like, solves the problems of weld joint tracking loss and the like caused by gradual change of visual characteristics of the weld joint along with imaging conditions such as illumination and the like, and has high universality and higher robustness and accuracy.
Description
Technical Field
The invention relates to a welding seam tracking method, in particular to a welding seam tracking method based on image matching, which is not limited by welding seams and environmental conditions and has higher robustness and accuracy.
Background
At present, the weld joint tracking method based on a visual sensor mainly comprises the following two methods: one method is based on active light, namely, light bars generated by an auxiliary light source are projected onto a welding seam to generate deformation, and the center of the welding seam is found by using an image processing technology to track. The light strip is not easy to deform when the welding line is small and shallow, so that the method has poor tracking effect on fine welding lines, is unstable in tracking when the outside interferes with the auxiliary light source, and is complex to implement and high in cost; the other method is based on passive light, namely the diffuse reflection imaging detection weld seam tracking of the surface of a workpiece is caused by utilizing ambient natural light or the weld seam tracking is directly carried out by utilizing a welding pool image, and specifically comprises a template matching tracking method and an edge detection tracking method. The template matching tracking method mainly uses the gray value characteristics of the template to find the region with the minimum difference with the gray value of the template in the weld image for weld matching tracking, has the advantages of simple operation, low cost and the like, but is extremely sensitive to illumination, slow in tracking speed and poor in anti-interference capability; the edge detection tracking method is to utilize the edge of the welding seam for tracking the welding seam, and the method is not easily affected by illumination and has higher tracking speed, but has poor tracking effect when the edge of the welding seam is not obvious or the edges around the welding seam are more.
Disclosure of Invention
The invention aims to solve the technical problems in the prior art and provides the weld joint tracking method based on image matching, which is not limited by weld joints and environmental conditions and has higher robustness and accuracy.
The technical solution of the invention is as follows: a welding seam tracking method based on image matching is characterized in that the method comprises an initial welding seam filter training stage and an on-line tracking stage,
the initial weld filter training phase is carried out according to the following steps:
A1. getkA sample image;
A2. manually marking and extracting a welding seam area in the sample image;
A3. automatically intercepting areas on two sides of a welding seam area as background images;
A4. respectively extracting the characteristics of a welding seam area and a background image;
A5. calculating an initial weld filter;
A6. storing the obtained initial weld filter;
the on-line tracking stage is carried out according to the following steps:
B1. manually setting coordinates of initial position of weld center in imageAnd order the initial search center;
B2. Acquiring an initial weld filter;
B3. collecting a detected image in real time;
B4. extracting coordinates from the detected imageA rectangular area as a center is used as a search area;
B5. extracting the characteristics of the search area;
B6. calculating the maximum response value of the filter and the feature of the search areaAndposition coordinates in an image;
B7.1 and B7.2 were performed simultaneously:
b7.1.2 pixel biasing into stack;
b7.1.3 judging whether the stack is full; calculating deviation control quantity, outputting the deviation control quantity to a motion control module, releasing a stack, and updating(ii) a If not, update;
B7.1.4 judging whether the tracking is finished; NO, return to B3; if yes, the tracking is finished;
b7.2.1 in coordinatesCollecting background images of a welding seam area and two sides of the welding seam area for the center;
b7.2.2 extracting the characteristics of the weld region and the background image respectively;
B7.2.4 judgment(ii) a If yes, updating the filter; if not, it is determined whether or not tracking is completed, and if yes, tracking is completed, otherwise, the routine returns to B3.
The method comprehensively considers the weld joint characteristics, the background characteristics and the characteristic difference between the weld joint characteristics and the background characteristics, is not limited by the depth, the thickness and the like of the weld joint, is not influenced by environmental conditions such as illumination and the like, solves the problems of weld joint tracking loss and the like caused by gradual change of the visual characteristics of the weld joint along with imaging conditions such as illumination and the like, and has high universality and higher robustness and accuracy.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention.
FIG. 2 is a schematic diagram of the effect of seam tracking according to an embodiment of the present invention.
Detailed Description
The hardware part of the embodiment of the invention is the same as the prior art, namely comprises an image acquisition module (a CCD camera, an image acquisition card and the like), a motion control module, a power transmission module, a welding module and the like. The weld tracking method based on image matching of the embodiment of the invention is composed of an initial weld filter training stage and an online tracking stage as shown in fig. 1.
The initial weld filter training phase is carried out according to the following steps:
A2. and (3) manually marking and extracting a weld joint region in the sample image: selecting a part capable of showing weld joint characteristics from each image as a weld joint area, wherein the weld joint area selection requires that: taking the shape of a rectangle with a length ofWide isGround rectangular region, the side of the rectangle perpendicular to the direction of the weldShorter, parallel to the edges of the weldThe length is longer; the center of the welding seam is positioned in the center of the rectangular frame, and then the welding seam area sample set;
A3. Automatically intercepting areas on two sides of a welding seam area as background images;
in the region of the weld seamTwo sides of the image are respectively and automatically picked up to be used as background images, wherein the image is the same as the size of a welding line areaRepresenting the identity of the weld region, then the background image sample set;
A4. Respectively extracting the characteristics of a welding seam area and a background image;
FHOG and gray features of the weld region and the background image sample are extracted respectively, wherein the FHOG features are extracted by the method disclosed in the document "Object detection with discrete transformed part-based models, PAMI, 32(9): 1627-. And sequentially combining 27 characteristic maps formed by FHOG characteristics of each sample image and one gray characteristic map as the characteristics of the sample. By usingIs shown asFirst of a weld zoneThe characteristic diagram, in the same way,is shown asSecond of background imageA feature map;
A5. calculating an initial weld filter:
Wherein,,;Representing a cross-correlation;representing the response of the welding seam filter and the welding seam, and being a Gaussian distribution window with the same size as the image of the welding seam area;representing the response of the welding seam filter and the background, and being a matrix composed of small numbers, wherein the matrix is an all-0 matrix;、、is a weight factor representing three terms;
the specific calculation process is as follows:
selecting a Gaussian distribution window with the same size as the image of the welding seam area as the output of the filterRequireThe peak value of the template image is just positioned at the central position of the template image, and the position is the preset central position of the welding seam;
in the frequency domain, the filter is applied to the formula (1) relative to the weld seamCalculating the deviation, inputting the image characteristics of the welding seamAnd its expected outputFeatures of the background input imageAnd its expected outputAnd a filterAfter FFT respectively represented as、、、Andthe frequency domain representation of the initial weld filter is obtained as follows:
wherein the content of the first and second substances,representing the filter in the frequency domainA layer of a material selected from the group consisting of,represents a complex conjugate;
The on-line tracking stage is carried out according to the following steps:
B1. manually setting coordinates of initial position of weld center in imageAnd order the initial search centerManually adjusting the motion control module to enable a welding gun (the welding gun is bound with a camera) to be aligned with the center of a welding seam; meanwhile, marking the initial position of the center of the welding seam in the image shot by the camera, and recording the coordinate of the initial position;
B2. Obtaining an initial weld filter: obtaining an initial filter calculated through an initial weld filter training phase;
B3. Collecting a detected image in real time;
B4. extracting coordinates from the detected imageThe rectangular area at the center serves as a search area:
for an image acquired in real time, in coordinatesIs central, width and height are respectivelyAndas a search area for the center of the weld in the image, saidAndthe value is the same as the value of the initial welding seam filter in the training stage;
B5. extracting the characteristics of the search area:
extracting FHOG characteristics of the search area, and forming the characteristics of the image by sequentially forming the 27-dimensional FHOG characteristics and the gray scale mapAnd performing discrete Fourier transform to obtain;
B6. Calculating the maximum response value of the filter and the feature of the search areaAndposition coordinates in an image;
The obtained characteristicsCorrelation with the filter (equation 2) to obtain the responseThe formula is as follows:
wherein the content of the first and second substances,is the response output of the spatial domain, thenMaximum value ofThe position in the image coordinate is the central position of the welding seam, and the coordinate of the position is recorded as;
Then B7.1 and B7.2 were performed simultaneously:
b7.1.2 pixel offset into the stack: the stack layer is set according to the system output interval, and the embodiment of the invention is set to be 7 layers;
b7.1.3 judging whether the stack is full; removing the maximum value and the minimum value in the deviation values, averaging the rest deviation values to be used as motion deviation control quantity, and outputting the motion deviation control quantity to a motion control module so as to control the motor to start to move and further control the welding gun and the welding gunThe camera moves, then the stack is released, and the center coordinates of the search area are updated(ii) a If not, update;
B7.1.4 judging whether the tracking is finished; NO, return to B3; if yes, the tracking is finished;
b7.2.1 in coordinatesAcquiring background images of the welding seam area and two sides of the welding seam area for the center: the specific method is the same as the initial welding seam filter training stage;
b7.2.2 extracting the features of the weld region and the background image respectively: the specific method is the same as the initial welding seam filter training stage;
b7.2.3 calculating the maximum response value of the background image feature and the filter: the calculation formula is the same as formula (3), and the maximum response value of the background image characteristic and the filter is obtained;
if it is notWhen the maximum response value of the weld joint area is greatly different from the maximum response value of the background image, the weld joint is clear and obvious, and the filter is updated at the moment, namely the filter is updated, namelyThe filter is modified to the weighted sum of the currently calculated filter and the previous filter, i.e.:
wherein the content of the first and second substances,is shown asFirst of frame time filterThe number of the main components is one,is shown asFirst of frame time filterAnIs an indication function, ifIf true, thenEqual to 1; otherwise, it equals 0.
If it is notIf the difference between the maximum response value of the welding seam region and the maximum response value of the background image is not very large, the welding seam is not obvious, and the filter does not need to be updated; it is judged whether or not tracking is finished, yes, tracking is finished, no, return to B3.
The experimental results are as follows:
taking two weld experiments as an example, as shown in fig. 2, the error value between the weld tracking result of the experiment and the standard result is shown by a dashed line (zero error) as a reference, and the average error obtained is 0.553 and 0.347 pixels respectively, and the experimental result proves the robustness and the accuracy of the invention.
Claims (3)
1. A welding seam tracking method based on image matching is characterized in that the method comprises an initial welding seam filter training stage and an on-line tracking stage,
the initial weld filter training phase is carried out according to the following steps:
A1. taking k sample images;
A2. manually marking and extracting a welding seam area in the sample image;
A3. automatically intercepting areas on two sides of a welding seam area as background images;
A4. respectively extracting FHOG characteristics and gray characteristics of a welding seam area and a background image, sequentially combining 27 characteristic graphs and a gray characteristic graph formed by the FHOG characteristics of each sample image as the characteristics of the sample, and using the characteristicsThe ith feature map showing the ith weld zone,an ith feature map representing an ith background image;
A5. calculating h-h of the weld filter according to a formula1,h2,…,h28}
Wherein Representing a cross-correlation; g is a Gaussian distribution window with the same size as the image of the welding seam area; b0α, β and gamma are weight factors representing three items;
the specific calculation process is as follows:
the peak value of the g is required to be just positioned at the central position of the template image, and the position is the preset central position of the welding seam;
in the frequency domain, the partial derivative of the formula (1) relative to the weld filter h is calculated,g、and b0And filter h is represented as after FFTG、B0And H, the available initial weld filter frequency domain representation is:
wherein HlL ∈ {1, 2.., 28} denotes the l-th layer of the filter in the frequency domain, { denotes the complex conjugate;
A6. storing the obtained initial weld filter;
the on-line tracking stage is carried out according to the following steps:
B1. manually setting the coordinate p of the initial position of the weld center in the image0And let the initial search center ps=p0;
B2. Acquiring an initial weld filter;
B3. collecting a detected image in real time;
B4. extracting the coordinate p in the detected imagesA rectangular area as a center is used as a search area;
B5. extracting the characteristics of the search area;
B6. calculating α the maximum response value of the filter and the feature of the search area1And α1Position coordinates p in the image1;
Simultaneously carrying out B7.1.1-B7.1.4 steps and B7.2.1-B7.2.4 steps:
b7.1.1 calculating p1And p0Pixel deviation of (2);
b7.1.2 pixel biasing into stack;
b7.1.3 determine if the stack is full? Calculating deviation control quantity, outputting the deviation control quantity to a motion control module, releasing a stack, and updating ps=p0(ii) a If not, update ps=p1;
B7.1.4 determine whether tracking is finished? NO, return to B3; if yes, the tracking is finished;
b7.2.1 in the coordinate p1Collecting background images of a welding seam area and two sides of the welding seam area for the center;
b7.2.2 extracting the characteristics of the weld region and the background image respectively;
b7.2.3 calculating the maximum response value α of the background image feature and the filter2;
B7.2.4 decision α1≥k×α2(ii) a If yes, updating the filter; no, determine if tracking is complete? Yes, tracking ends, no, return to B3.
2. The method for tracking a weld according to claim 1, wherein the step B7.1.3 of calculating deviation control values includes removing the maximum value and the minimum value of the deviation values, and averaging the remaining deviation values to obtain the motion deviation control values.
3. The method for seam tracking based on image matching according to claim 2, wherein said updating filter of step B7.2.4 is to modify the filter to be a weighted sum of the current calculated filter and the previous filter, namely:
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710272076.3A CN107248170B (en) | 2017-04-24 | 2017-04-24 | Weld joint tracking method based on image matching |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710272076.3A CN107248170B (en) | 2017-04-24 | 2017-04-24 | Weld joint tracking method based on image matching |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107248170A CN107248170A (en) | 2017-10-13 |
CN107248170B true CN107248170B (en) | 2020-06-26 |
Family
ID=60016616
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710272076.3A Expired - Fee Related CN107248170B (en) | 2017-04-24 | 2017-04-24 | Weld joint tracking method based on image matching |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107248170B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108053419B (en) * | 2017-12-27 | 2020-04-24 | 武汉蛋玩科技有限公司 | Multi-scale target tracking method based on background suppression and foreground anti-interference |
CN110823090B (en) * | 2018-08-14 | 2021-07-20 | 中车唐山机车车辆有限公司 | Welding groove detection method |
CN109465582A (en) * | 2018-10-24 | 2019-03-15 | 成都埃森普特科技股份有限公司 | A kind of robot welding method based on computer vision |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2001081037A1 (en) * | 2000-04-26 | 2001-11-01 | Connell Dennis James O | Aid for welding or cutting |
KR101717582B1 (en) * | 2010-10-18 | 2017-03-17 | 대우조선해양 주식회사 | Submerged arc welding machine capable of weld-line trace and image processing method for weld-line trace of submerged arc welding machine |
CN103008881A (en) * | 2012-12-05 | 2013-04-03 | 中国电子科技集团公司第四十五研究所 | Seam tracking method based on template matching |
CN103170778B (en) * | 2013-04-01 | 2015-06-24 | 大连海事大学 | Welded joint automatic tracking system |
-
2017
- 2017-04-24 CN CN201710272076.3A patent/CN107248170B/en not_active Expired - Fee Related
Also Published As
Publication number | Publication date |
---|---|
CN107248170A (en) | 2017-10-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111462135A (en) | Semantic mapping method based on visual S L AM and two-dimensional semantic segmentation | |
CN109903313B (en) | Real-time pose tracking method based on target three-dimensional model | |
CN101488187B (en) | System and method for deformable object recognition | |
CN107993258B (en) | Image registration method and device | |
Wu et al. | VH-HFCN based parking slot and lane markings segmentation on panoramic surround view | |
Zhang et al. | An image stitching algorithm based on histogram matching and SIFT algorithm | |
US6219462B1 (en) | Method and apparatus for performing global image alignment using any local match measure | |
CN107248170B (en) | Weld joint tracking method based on image matching | |
JP2004117078A (en) | Obstacle detection device and method | |
Park et al. | Texture-less object tracking with online training using an RGB-D camera | |
CN111126174A (en) | Visual detection method for robot to grab parts | |
Schmidt et al. | Comparative assessment of point feature detectors in the context of robot navigation | |
CN104715487B (en) | A kind of sub-pixel edge detection method based on Zernike pseudo-matrix | |
US20180174328A1 (en) | Turning radius-based corner detection algorithm | |
CN111915485B (en) | Rapid splicing method and system for feature point sparse workpiece images | |
CN104408696A (en) | Image splicing method aiming at side-scan sonar imaging features | |
CN104200492A (en) | Automatic detecting and tracking method for aerial video target based on trajectory constraint | |
Ansari | A review on SIFT and SURF for underwater image feature detection and matching | |
CN104392428A (en) | Splicing system for side-scan sonar images | |
CN112991159B (en) | Face illumination quality evaluation method, system, server and computer readable medium | |
Amiri et al. | RASIM: a novel rotation and scale invariant matching of local image interest points | |
CN106778831B (en) | Rigid body target on-line feature classification and tracking method based on Gaussian mixture model | |
Guo et al. | An automated registration of RS images based on SURF and piecewise linear transformation | |
Hayashi et al. | Texture Overlay onto Non-rigid Surface using Commodity Depth Camera. | |
Layek et al. | Remote distance measurement from a single image by automatic detection and perspective correction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200626 |
|
CF01 | Termination of patent right due to non-payment of annual fee |