CN112330678B - Product edge defect detection method - Google Patents
Product edge defect detection method Download PDFInfo
- Publication number
- CN112330678B CN112330678B CN202110015305.XA CN202110015305A CN112330678B CN 112330678 B CN112330678 B CN 112330678B CN 202110015305 A CN202110015305 A CN 202110015305A CN 112330678 B CN112330678 B CN 112330678B
- Authority
- CN
- China
- Prior art keywords
- template
- edge
- detected
- edge information
- small
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 78
- 238000001514 detection method Methods 0.000 title claims abstract description 51
- 230000009466 transformation Effects 0.000 claims abstract description 110
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 107
- 238000005070 sampling Methods 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000005457 optimization Methods 0.000 claims abstract description 8
- 230000002159 abnormal effect Effects 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 8
- 238000005520 cutting process Methods 0.000 claims description 7
- 230000011218 segmentation Effects 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000008901 benefit Effects 0.000 abstract description 7
- 230000008859 change Effects 0.000 abstract description 6
- 238000005286 illumination Methods 0.000 abstract description 5
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000005259 measurement Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 6
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 5
- 229910052710 silicon Inorganic materials 0.000 description 5
- 239000010703 silicon Substances 0.000 description 5
- 239000000126 substance Substances 0.000 description 4
- 238000011524 similarity measure Methods 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 208000003464 asthenopia Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G06T3/02—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Abstract
The application discloses a product edge defect detection method. The method comprises the following steps: step 1, inputting an image; step 2, preprocessing an image; step 3, coarse matching of templates; step 4, segmenting the edge information to be detected; step 5, establishing a template data structure; step 6, down-sampling the edge information to be detected; step 7, matching in the template; step 8, calculating affine transformation parameters; step 9, template fine matching; step 10 edge defect detection. The method adopts the edge information of the binary image, is insensitive to illumination change and edge gray scale, and has the advantages of stability and reliability; the template matching parameters are iterated from coarse to fine by utilizing edge information down sampling, so that the calculation speed is high; the precise matching mode based on the RANSAC principle and the iterative optimization of parameters has the advantage of high detection precision; the template matching mode of the sectional matching has the advantage of tensile resistance.
Description
Technical Field
The invention relates to a defect detection method for cover plates and plate products, in particular to a rapid stretch-resistant edge defect detection method.
Background
Various shapes of edges are often included in industrial products, such as circular, rectangular, and a combination of straight and circular arcs, among others. The edge is an important characteristic of industrial products, the quality of the industrial products is directly influenced, and the defects of the edge of the common products are mostly serious defects of the products, such as bud defects, saw teeth, edge breakage and the like, which directly cause the product to be scrapped. Edge defect detection of a product is an important step for ensuring the quality of the product.
The edge defect detection mainly detects the consistency of the edge of the industrial product, namely detects whether the edge of the industrial product has defects such as convex marks, dents and the like. Specifically, for example, edge defect detection is performed on the mobile phone outer screen and the integrated circuit silicon wafer, the mobile phone outer screen and the integrated circuit silicon wafer generally have regular edges, the mobile phone outer screen has a rectangular edge, and the integrated circuit silicon wafer has a combined edge of an arc and a straight line.
If the edge of the mobile phone outer screen has defects, the assembly of the mobile phone outer screen is influenced, and even the mobile phone cannot be normally used; if the edge of the integrated circuit silicon wafer has defects, the manufacturing of the integrated circuit is affected, and even the quality of the chip based on the integrated circuit silicon wafer is reduced. Therefore, edge defect detection of industrial products is an important link in industrial production.
In the prior art, some edge defect detection methods are manual detection. A worker generally inspects an image of an object to be detected in a visual inspection mode, and checks whether a dent or a convex defect exists, however, the detection mode depends on visual inspection of human eyes, the precision is difficult to guarantee, and visual fatigue is easily caused by long-time inspection, so that the detection efficiency is reduced, and even wrong detection and missed detection are caused.
In the prior art, some methods adopt computer vision edge defect detection to detect. The research universality aiming at the edge defect detection algorithm at home and abroad is not strong, one type of algorithm adopts a straight line fitting mode to compare the distance from an edge point to a fitting straight line to detect the edge defect of a straight line region; one type of algorithm compares the change of the front area and the rear area by adopting a switching operation mode to detect the edge defect of a complete area with smooth edge; and the other algorithm adopts a template matching mode to compare the edge defects of the different detection target images to be detected and the template.
However, the above methods are not suitable for defect detection of stretch type edges.
Disclosure of Invention
The invention aims to solve the technical problem that a computer vision detection method in the prior art is not suitable for detecting defects of stretching edges, and provides a product edge defect detection method. The detection method comprises the steps of obtaining affine transformation parameters according to rough matching of minimum circumscribed rectangle features of edge points, calculating the similarity of a template and a to-be-detected object from rough to fine according to a similarity measurement function and by using edge information of different down-sampling scales, updating the affine transformation parameters, updating the template and the to-be-detected pixel-level affine transformation parameters again by using RANSAC and the similarity measurement function, optimizing and updating sub-pixel-level affine transformation parameters corresponding to each segment under target precision by overlapping the affine transformation parameters near the pixel-level affine transformation parameters by using segmented to-be-detected edge information, and realizing edge defect detection of edge stretching products by using a template matching mode of each segment of affine transformation parameters and segmented matching.
In order to solve the technical problem, the invention provides a product edge defect detection method, which comprises the following steps:
step 3, coarse template matching: calculating the coarse matching of the minimum circumscribed rectangle features of the edge points to obtain affine transformation parameters;
step 5, creating a template data structure: creating edge information of the template small graph into a data structure of KD-Tree (k-dimensional Tree);
step 6, edge information to be detected is subjected to down-sampling: acquiring edge information of each section of the small graph to be detected with different down-sampling scales;
and 7, matching in the template: calculating edge information of each section of the small graph to be detected under the template rough matching affine transformation parameters; calculating the similarity between each section of edge information of the small graph to be detected and the edge information of the template small graph and updating corresponding affine transformation parameters;
step 9, template fine matching: calculating edge information of each section of the small graph to be detected under random consistency sampling affine transformation parameters; performing iterative optimization on affine transformation parameters corresponding to edge information of each section of the small graph to be tested under the condition of updating target precision near the pixel level affine transformation parameters;
Preferably, all edges or partial edge regions in the image are selected as template minimaps using one or more tools of rectangle, circle, polygon.
Preferably, the determining the coordinate transformation in the suspected region of the target image further comprises: and mapping the ROI coordinates under the connected coordinate system to the region to be detected.
Preferably, the preprocessing the image further comprises: and carrying out binarization on the template small image and the small image to be detected by adopting fixed threshold binarization or adaptive threshold binarization, wherein the edge information comprises edge point coordinates and a gradient direction.
Preferably, the adjusting the edge information further comprises: screening and sorting contours, eliminating image boundary contour points, adjusting contour sequence to make contours continuous, and determining the corresponding relation between a template and edge information to be detected.
Preferably, the template coarse matching further comprises the following steps:
a. respectively solving the minimum external rectangle of the edge points of the template small graph and the small graph to be detected, and the coordinates of the center point and the deflection angle characteristics of the minimum external rectangle;
b. calculating rough matching affine transformation parameters by using the coordinates of the central point of the minimum circumscribed rectangle and the deflection angle;
c. and updating the edge information of the small graph of the template through affine transformation parameters.
Preferably, the cutting mode for cutting the edge information to be measured is equal division of partial coincidence.
Preferably, the method for making the template thumbnail edge information template in the matching of the templates further comprises: and creating the edge information of the template small graph into a KD-Tree data structure.
Preferably, edge information corresponding to the maximum similarity measurement function is determined by using a random consistency sampling principle, so as to update affine transformation parameters under the current precision.
Preferably, the affine transformation parameters are iteratively optimized and updated near the pixel-precision affine transformation parameters to the sub-pixel-precision affine transformation parameters.
Preferably, the method for detecting the edge defects in the suspected region of the target image comprises the following steps:
a. mapping each section of edge information of the small picture to be detected to a template small picture coordinate system by using affine transformation parameters corresponding to each section of edge information of the small picture to be detected;
b. matching the edge of the small graph to be detected and the edge of the template small graph in a segmentation manner to find abnormal edge points of the small graph to be detected;
c. clustering and growing abnormal points and removing out-of-range abnormal points;
d. completing the defects and calculating the defect characteristics;
e. merging coincident defects and features.
Preferably, the down-sampling mode further comprises: the segment edge information is downsampled to 1/2,1/4 data size.
The beneficial effects of the invention include: the method is based on the edge information of a product image, an edge information template is manufactured by acquiring the edge information of the template small image, the ROI of a target image to be detected is determined by utilizing coordinate transformation to acquire the small image to be detected, the edge information to be detected with different down-sampling scales is manufactured by acquiring the edge information of the small image to be detected, the rapid, stable and real-time online edge defect detection is realized aiming at displacement, rotation, scaling, illumination change, edge stretching and the like of the target image, and the method can be applied to the occasions of performing the edge defect detection in an online or offline state through machine vision.
The rapid stretch-resistant edge defect method based on template matching adopts edge information of a binary image, is insensitive to illumination change and edge gray, and has the advantages of stability and reliability; the template matching parameters are iterated from coarse to fine by utilizing edge information down sampling, so that the calculation speed is high; the precise matching mode which utilizes the RANSAC (random consistent sampling) principle and parameter iterative optimization has the advantage of high detection precision; the template matching mode of the sectional matching has the advantage of tensile resistance.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, it is obvious that the drawings in the following description are only a part of the embodiments or prior art, and other similar or related drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a product defect detection method according to an embodiment of the present invention (top);
FIG. 2 is a flow chart of a product defect detection method according to an embodiment of the present invention (below);
FIG. 3 is a small drawing of a framed template according to an embodiment of the present invention; wherein, a template small image 301 and a target image space 302 are selected;
FIG. 4 is a diagram of mapping ROI coordinates to a region to be inspected in the connected coordinate system according to the embodiment of the present invention; the method comprises the following steps of obtaining a ROI coordinate mapping 401, a small graph to be detected 402, a connected coordinate system XOY 403 and a target image space 404 in the connected coordinate system;
FIG. 5 is a diagram of edge information of a small graph of a template according to an embodiment of the present invention; wherein, the edge point coordinates and gradient direction 501 of the template small graph;
FIG. 6 is a diagram illustrating minimum circumscribed rectangles of edge points of a small graph to be measured according to an embodiment of the present invention; the minimum external rectangle center point coordinate 601 of the edge point of the small graph to be detected and the minimum external rectangle center point deflection angle 602 of the edge point of the small graph to be detected are included.
FIG. 7 is a diagram of partially overlapping and dividing edge information of a small graph to be measured into a plurality of equal parts according to an embodiment of the present invention; the edge information of the small graph to be detected is partially overlapped and divided into equal parts 701;
FIG. 8 is a graph of edge information of different down-sampling scales of the small graph to be measured according to the embodiment of the present invention; the method comprises the following steps that original outline information 801 of a small graph to be tested, downsampling outline information 802 of the small graph 1/2 to be tested, and downsampling outline information 803 of the small graph 1/4 to be tested;
fig. 9 is a flowchart of an affine transformation parameter updating strategy according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to examples. The present invention will be described in further detail below to make the objects, aspects and advantages of the present invention clearer and more clear, but the present invention is not limited to these examples.
The invention provides a rapid stretch-resistant edge defect detection method based on template matching. The method comprises the steps of preparing an edge information template by obtaining edge information of a template small picture based on image edge information, determining a target image region to be detected ROI by coordinate transformation to obtain a small picture to be detected, preparing edge information to be detected with different down-sampling scales by obtaining the edge information of the small picture to be detected, obtaining affine transformation parameters according to the minimum external rectangle feature rough matching of edge points, roughly and precisely calculating the similarity between the template and the region to be detected by the edge information with different down-sampling scales according to a similarity measurement function and updating the affine transformation parameters, updating the template and the pixel level affine transformation parameters to be detected again by using RANSAC (random consistent sampling) and the similarity measurement function, optimizing and updating sub-pixel level affine transformation parameters corresponding to each segment under target precision by using the segmented edge information to be detected to be overlapped near the pixel level affine transformation parameters, and realizing the edge information in the target image region to be detected region by the template matching mode of each segment of affine transformation parameters and segmented matching And detecting defects.
The method can realize rapid, stable and real-time online edge defect detection aiming at displacement, rotation, scaling, illumination change, edge stretching and the like of the target image, and can be applied to the occasions of performing the edge defect detection in an online or offline state through machine vision.
The invention discloses a product edge defect detection method, which comprises the following steps:
step 3, coarse template matching: calculating the coarse matching of the minimum circumscribed rectangle features of the edge points to obtain affine transformation parameters;
step 5, creating a template data structure: creating edge information of the template small graph into a data structure of KD-Tree (k-dimensional Tree);
step 6, edge information to be detected is subjected to down-sampling: acquiring edge information of each section of the small graph to be detected with different down-sampling scales;
and 7, matching in the template: calculating edge information of each section of the small graph to be detected under the template rough matching affine transformation parameters; calculating the similarity between each section of edge information of the small graph to be detected and the edge information of the template small graph and updating corresponding affine transformation parameters;
step 9, template fine matching: calculating edge information of each section of the small graph to be detected under random consistency sampling affine transformation parameters; performing iterative optimization on affine transformation parameters corresponding to edge information of each section of the small graph to be tested under the condition of updating target precision near the pixel level affine transformation parameters;
Preferably, all or part of the edge regions in the image which are obvious are selected as the template thumbnail by using any one of the tools of rectangle, circle, polygon and the like.
Preferably, the coordinate transformation is to map the ROI coordinates under a connected coordinate system to the region to be detected.
Preferably, the preprocessing is to binarize the template small graph and the small graph to be detected by using fixed threshold binarization or adaptive threshold binarization, and the edge information includes edge point coordinates and gradient directions.
Preferably, the adjusting of the edge information includes contour screening and sorting, image boundary contour point elimination, contour sequence adjustment to make the contours continuous, and determination of the corresponding relationship between the template and the edge information to be measured by using an icp (iterative closed point) algorithm.
Preferably, the coarse matching employs the following steps: a. respectively solving the minimum external rectangle of the edge points of the template small graph and the small graph to be detected, and the coordinates of the center point and the deflection angle characteristics of the minimum external rectangle; b. calculating rough matching affine transformation parameters by using the coordinates of the central point of the minimum circumscribed rectangle and the deflection angle; c. and updating the edge information of the small graph of the template through affine transformation parameters.
Preferably, the cutting pattern is a partially overlapping bisection.
Preferably, the template is made by creating edge information of the template thumbnail into a data structure of a KD-Tree (k-dimensional Tree), and the downsampling is performed by downsampling each segment of edge information into 1/2,1/4 data size.
Preferably, the similarity measurement function is the sum of the number of edge points, the distance sum and the sum of gradient direction angles which satisfy a certain condition, and the update affine transformation parameter strategy is that the similarity measurement function under the large-scale down-sampling is larger than the original similarity measurement function, the similarity measurement function under the upper-level down-sampling is calculated until the similarity measurement function under the non-down-sampling is larger than the original similarity measurement function, and the affine transformation parameter is updated.
Preferably, edge information corresponding to the maximum similarity metric function is determined by using a RANSAC (random consistent sampling) principle, so as to update affine transformation parameters at the current precision.
Preferably, the affine transformation parameters are iteratively optimized and updated near the pixel-precision affine transformation parameters to the sub-pixel-precision affine transformation parameters.
Preferably, the method for detecting the edge defects in the suspected region of the target image comprises the following steps: a. mapping each section of edge information of the small picture to be detected to a template small picture coordinate system by using affine transformation parameters corresponding to each section of edge information of the small picture to be detected; b. matching the edge of the small graph to be detected and the edge of the template small graph in a segmentation manner to find abnormal edge points of the small graph to be detected; c. clustering and growing abnormal points and removing out-of-range abnormal points; d. completing the defects and calculating the defect characteristics; e. merging coincident defects and features.
As shown in fig. 1 and fig. 2, which are flow charts (top and bottom) of the product defect detection method of the present embodiment; fig. 1 and fig. 2 are combined together to form a complete flow chart of the product defect detection method. The specific process comprises the following steps:
step 3, coarse template matching: minimum circumscribed rectangle for calculating edge pointsObtaining affine transformation parameters by coarse matching of features
；
Step 5, creating a template data structure: creating the edge information of the template small graph into a data structure of KD-Tree (k-dimensional Tree)
；
Step 6, edge information to be detected is down-sampled: obtaining edge information of each section of small graph to be detected with different down-sampling scales
；
And 7, matching in the template: calculating edge information of each section of small graph to be detected under template rough matching affine transformation parameters
(ii) a According to the similarity measurement function, calculating the similarity between each section of edge information of the small graph to be detected and the edge information of the template small graph from coarse to fine according to the edge information of different down-sampling scales
Updating the corresponding affine transformation parameters;
Step 9, template fine matching: calculating edge information of each section of small graph to be measured under random consistency sampling affine transformation parameters
(ii) a Affine transformation parameters corresponding to edge information of each section of small graph to be tested under the condition of updating target precision through iterative optimization near pixel-level affine transformation parameters
；
The following is a detailed description of the above steps:
step 1: image input
Fig. 3 is a block diagram of a template according to an embodiment of the present invention. Wherein 301 is a small frame template image, and 302 is a target image space. In the embodiment of the invention, the image input is from a linear array camera image sensor in a machine vision system, and an image with a clearer edge and a centered target object is selected as a template image in the input image. Using any geometric tool such as rectangle, circle, polygon, etc. to frame all edges or part of edge area in template image as template small picture under connected coordinate system XOY
Recording geometric tool coordinates (x, y) under a connected body coordinate system XOY, and taking a subsequent input image as a target image
。
Step 2: image pre-processing
In the embodiment of the present invention, the region to be measured corresponding to the template thumbnail may appear at any position of the target image, but the position of the region to be measured under the connected coordinate system is fixed, so as shown in fig. 4, the present invention maps the geometric tool coordinate (x, y) coordinate of the framed template thumbnail to the coordinate (x ', y') under the connected coordinate system XOY in the target image to determine the region to be measured, and obtains the thumbnail to be measured through (x ', y')/the region to be measured
。
In the embodiment of the invention, although the template small image and the small image to be detected are both small images, the data volume is still large, the template matching is mostly useless data, the edge information of the image is the key and important data of the template matching, and the good template matching can be realized. As shown in FIG. 5, the invention adopts fixed threshold binarization or adaptive threshold binarization to obtain the binarized images of the template small graph and the small graph to be measured, and in order to avoid the edge information inconsistency caused by illumination change, edge stretching and the like, the edge information such as the edge point of the template small graph, the edge point coordinate p '(x', y ') and the gradient direction e' (x ', y') is obtained in the binarized image of the template small graph
Acquiring edge information such as edge points, edge point coordinates p (x, y), gradient direction e (x, y) and the like of the small graph to be detected from the binary image of the small graph to be detected
。
The edge information of the template small picture and the small picture to be detected acquired in the embodiment of the invention has useless information such as image boundary points, error information such as false edge points and asymmetric information such as the small picture to be detected and the template small picture wheelThe contour point information does not correspond, etc. The invention further adjusts the acquired edge information, comprising contour screening and sorting, eliminating image boundary contour points, adjusting contour sequence to make the contour continuous, determining the corresponding relation between the to-be-detected and template edge information by utilizing ICP (iterative close Point) algorithm, acquiring the adjusted template small picture edge information
And edge information of the small graph to be tested
。
And step 3: coarse template matching
Although the data volume used for template matching in the embodiment of the invention is small enough relative to the data volume of the image pyramid, the running time of the data volume directly used for template matching is still long. As shown in FIG. 6, the affine transformation parameter of the template rough matching is calculated by obtaining the minimum circumscribed rectangle center point coordinate (x, y) and the deflection angle theta of the edge point of the small graph to be detected, the minimum circumscribed rectangle center point coordinate (x ', y ') and the deflection angle theta ' of the edge point of the template small graph, and the scaling ratio k of the minimum circumscribed rectangle long side L ' of the edge point P ' of the template small graph and the minimum circumscribed rectangle long side L of the edge point P of the small graph to be detected
. The calculation formula of the template rough matching affine transformation parameters is as follows:
wherein the content of the first and second substances,
and 4, step 4: segmenting edge information to be measured
In the embodiment of the invention, certain stretching exists on the edge of the image, the template matching mode of integral matching is adopted for integral matching to be optimal, but the matching of partial positions can be deviatedLeading to false detection of the final edge defect. As shown in FIG. 7, the invention cuts the outline information of the small graph to be measured by adopting a partially overlapped cutting mode
Several equal parts { P1，P2，…，PnAnd calculating affine transformation parameters of all the segments by using a template matching mode of segment matching to avoid segmentation to defects.
And 5: creating template data structures
In the embodiment of the invention, in order to realize the quick search and matching from the edge information of the small graph to be detected to the edge information of the template small graph, the edge information of the template small graph is created into a data structure of KD-Tree (k-dimensional Tree)
。
Step 6: edge information downsampling to be measured
Fig. 8 is a diagram of edge information of different down-sampling scales of a small graph to be measured according to an embodiment of the present invention. The original outline information 801 of the small graph to be tested, the downsampling outline information 802 of the small graph 1/2 to be tested, and the downsampling outline information 803 of the small graph 1/4 to be tested. In the embodiment of the invention, in order to quickly realize the iterative optimization of affine transformation parameters from coarse to fine and reduce the data volume, the edge point P of the small graph to be detected is down-sampled by P1/2And P1/4Data volume
。
And 7: match in template
In the embodiment of the invention, a template matching mode of segmentation matching is adopted to calculate the similarity measurement function S { SN, SD, S theta } of each segment
Updating affine transformation parameters segment by using affine transformation parameter updating strategy
As shown in fig. 9, the specific implementation steps are as follows:
a. calculating template rough matching affine transformation parameter T0Current segment edge information coordinate P of lower to-be-detected small picture0(x, y) and gradient direction e0（x，y）。
b. Searching edge information coordinate P of corresponding template small graph0' (x ', y ') and gradient direction e0’（x’，y’）。
c. Calculating a similarity measure function S for a current segment0{SN0，SD0，Sθ0}：
Wherein, SN0Is a coordinate P0(x, y) to the coordinate P0' (x ', y ') corresponding coordinate points having a distance less than distance0The number of edge points; SD0Is a coordinate P0(x, y) to the coordinate P0' (x ', y ') corresponding coordinate points having a distance less than distance0The sum of the distances of (a); s theta0Is a coordinate P0(x, y) to the coordinate P0' (x ', y ') corresponding coordinate points having a distance less than distance0The sum of the included angles in the gradient directions.
d. Coarse matching of affine transformation parameters [ theta ] in a template0，x0，y0Nearby with a large step size [ Delta theta ]1，Δx1，Δy1Calculating edge point P by nested loop1/4、P1/2And P1/1S similarity metric function of1/4、S1/2And S1/1If { S }1/4、S1/2And S1/1Are all greater than S0Then update S0Recording the corresponding affine transformation parameters [ theta ]0’，x0’，y0' finding the affine transformation parameter theta corresponding to the maximum similarity metric function01，x01，y01Updating template matching affine transformation parameters
：
T1=T01*T0
Wherein the content of the first and second substances,
and 8: RANSAC for calculating affine transformation parameters
In the embodiment of the invention, the RANSAC (random consistent sampling) principle is utilized to solve the problem that the error exists in updating the template matching affine transformation parameters due to the possible existence of edge defects or interference information and the like in template matching
The method comprises the following specific implementation steps:
a. calculating template matching affine transformation parameter T1Current segment edge information coordinate P of lower to-be-detected small picture1(x, y) and gradient direction e1（x，y）。
b. Searching edge information coordinate P of corresponding template small graph1' (x ', y ') and gradient direction e1’（x’，y’）。
c. Calculating a similarity measure function S for a current segment1{SN1，SD1，Sθ1}：
Wherein, SN1Is a coordinate P1(x, y) to the coordinate P1' (x ', y ') corresponding coordinate points having a distance less than distance1The number of edge points; SD1Is a coordinate P1(x, y) to the coordinate P1' (x ', y ') corresponding coordinate points having a distance less than distance1The sum of the distances of (a); s theta1Is a coordinate P1(x, y) to the coordinate P1' (x ', y ') corresponding coordinate points having a distance less than distance1The sum of the included angles in the gradient directions.
d. Randomly selecting the edge information coordinate P of the current segment of the small picture to be measured1Three points in (x, y) { P1，P2， P3And the coordinates P1' (x, y) corresponding point { P }1’，P2’， P3’}。
Wherein, { P1，P2， P3Is from the coordinate P1(x, y) different parts are randomly selected, and the area of a triangle formed by the three points meets a certain condition.
e. And (3) solving a linear equation set by Gaussian elimination to obtain affine transformation parameters:
AT=B*XT
wherein the content of the first and second substances,
obtaining affine transformation parameters
f. Calculating affine transformation parameters T12Next current segment edge point P1/4、P1/2And P1/1Similarity metric function of { S12 }1/4，S121/2，S121/1If { S12 }1/4，S121/2，S121/1Are all greater than S1Then update S1Recording the corresponding affine transformation parameters T12. Until finding out the similarity measurement function meeting the condition or reaching the maximum affine transformation parameter T corresponding to the similarity measurement function when the random consistency sampling times are reached12。
g. Updating affine transformation parameters
：
T2=T12*T1
And step 9: template fine matching
As shown in FIG. 9, the update affine for the embodiment of the present inventionA transformation parameter policy flow chart. In the embodiment of the invention, a template matching mode of subsection fine matching is adopted, the similarity measurement function S { SN, SD and S theta } of each section is calculated, and the affine transformation parameter T is updated section by utilizing an affine transformation parameter updating strategy
The method comprises the following specific implementation steps:
a. calculating random consistency sampling affine transformation parameter T2Current segment edge information coordinate P of lower to-be-detected small picture2(x, y) and gradient direction e2（x，y）。
b. Searching edge information coordinate P of corresponding template small graph2' (x ', y ') and gradient direction e2’（x’，y’）。
c. Calculating a similarity measure function S for a current segment2{SN2，SD2，Sθ2}：
Wherein, SN2Is a coordinate P2(x, y) to the coordinate P2' (x ', y ') corresponding coordinate points having a distance less than distance2The number of edge points; SD2Is a coordinate P2(x, y) to the coordinate P2' (x ', y ') corresponding coordinate points having a distance less than distance2The sum of the distances of (a); s theta2Is a coordinate P2(x, y) to the coordinate P2' (x ', y ') corresponding coordinate points having a distance less than distance2The sum of the included angles in the gradient directions.
d. Sampling affine transformation parameters [ theta ] at random consistency2，x2，y2A small step size [ Delta theta ] is adopted around2，Δx2，Δy2Nested loop computation edge point P21/4、P21/2And P21/1Similarity metric function of { S2 }1/4，S21/2，S21/1If { S2 }1/4，S21/2，S21/1Are all greater than S2Then update S2Recording corresponding affine variationsChange parameter [ theta ]2’，x2’，y2' finding the affine transformation parameter theta corresponding to the maximum similarity metric function23，x23，y23Updating template matching affine transformation parameters:
T3=T23*T2
wherein the content of the first and second substances,
step 10: edge defect detection
In the embodiment of the invention, the edge defect detection is realized by adopting a template matching mode of sectional matching
The method comprises the following specific implementation steps:
a. calculating template fine matching affine transformation parameter T3Current segment edge information coordinate P of lower to-be-detected small picture3（x，y）。
b. Searching edge information coordinate P of corresponding template small graph3’（x’，y’）。
c. Calculating the average distance deviation:
wherein n is a number satisfying
The number of the edge points of (a),
distance, of point p to point p3Is a fixed value.
d. Finding satisfaction conditions
The abnormal edge point of (1), wherein distance4Is a fixed value.
e. Cluster growthSatisfies the conditions
And eliminating the boundary-crossing abnormal edge points.
f. And completing the information of the defect abnormal points, and calculating the defect characteristics such as length, width, area, coordinates and the like of the defect.
g. Merge satisfaction condition
Defect d of1And defect d2And outputting the edge defect detection result.
Although the present invention has been described with reference to a few embodiments, it should be understood that the present invention is not limited to the above embodiments, but rather, the present invention is not limited to the above embodiments, and those skilled in the art can make various changes and modifications without departing from the scope of the invention.
Claims (11)
1. A method for detecting edge defects of a product is characterized by comprising the following steps:
step 1, image input: the method comprises the steps of (1) including a template small picture and a target image;
step 2, image preprocessing: mapping the ROI coordinates under the connected coordinate system to a region to be detected, and determining the region to be detected of a target image to obtain a small image to be detected; preprocessing images of the template small image and the small image to be detected to obtain image edge information; adjusting the edge information to enable the template edge information to correspond to the edge information to be detected;
step 3, coarse template matching: calculating the coarse matching of the minimum circumscribed rectangle features of the edge points to obtain affine transformation parameters;
step 4, segmenting the edge information to be detected: cutting edge information of the small graph to be detected into a plurality of equal parts which are partially overlapped;
step 5, creating a template data structure: creating edge information of the template small graph into a data structure of the KD-Tree;
step 6, edge information to be detected is subjected to down-sampling: acquiring edge information of each section of the small graph to be detected with different down-sampling scales;
and 7, matching in the template: calculating edge information of each section of the small graph to be detected under the template rough matching affine transformation parameters; calculating the similarity between each section of edge information of the small graph to be detected and the edge information of the template small graph and updating corresponding affine transformation parameters;
step 8, affine transformation parameters are calculated: calculating edge information of each section of the small graph to be detected under the matched affine transformation parameters in the template; calculating the similarity between each section of edge information of the small graph to be detected and the edge information of the template small graph under random consistent sampling and updating affine transformation parameters corresponding to each section of edge information of the small graph to be detected;
step 9, template fine matching: calculating edge information of each section of the small graph to be detected under random consistency sampling affine transformation parameters; performing iterative optimization on affine transformation parameters corresponding to edge information of each section of the small graph to be tested under the condition of updating target precision near the pixel level affine transformation parameters;
step 10, edge defect detection: calculating edge information of each section of the small graph to be detected under the condition that the template is matched with the affine transformation parameters accurately; and searching the edge information of the corresponding template small graph to realize the detection of the edge defect in the target image to be detected.
2. The method for detecting edge defects of products according to claim 1, wherein all edges or part of edge regions in the images are selected as template minimaps by using one or more tools of rectangles, circles and polygons.
3. The method for detecting edge defects of products according to claim 1, wherein the step 2 image preprocessing further comprises: and carrying out binarization on the template small image and the small image to be detected by adopting fixed threshold binarization or adaptive threshold binarization, wherein the edge information comprises edge point coordinates and a gradient direction.
4. The method of claim 1, wherein the adjusting the edge information further comprises: screening and sorting contours, eliminating image boundary contour points, adjusting contour sequence to make contours continuous, and determining the corresponding relation between a template and edge information to be detected.
5. The method of claim 1, wherein the rough template matching further comprises:
a. respectively solving the minimum external rectangle of the edge points of the template small graph and the small graph to be detected, and the coordinates of the center point and the deflection angle characteristics of the minimum external rectangle;
b. calculating rough matching affine transformation parameters by using the coordinates of the central point of the minimum circumscribed rectangle and the deflection angle;
c. and updating the edge information of the small graph of the template through affine transformation parameters.
6. The method for detecting the edge defect of the product as claimed in claim 1, wherein the dividing manner for dividing the edge information to be detected is equal division of partial coincidence.
7. The method for detecting edge defects of products according to claim 1, wherein the manner of making the template small-image edge information template in the matching of the templates further comprises: and creating the edge information of the template small graph into a KD-Tree data structure.
8. The product edge defect detection method of claim 1, wherein edge information corresponding to a maximum similarity metric function is determined by using a random consistency sampling principle, so as to update affine transformation parameters at the current precision.
9. The product edge defect detection method of claim 1, wherein affine transformation parameters under sub-pixel precision are updated by iteration affine transformation parameter optimization near pixel precision affine transformation parameters.
10. The method for detecting the edge defect of the product according to claim 1, wherein the step of detecting the edge defect in the target image to be detected comprises the following steps:
a. mapping each section of edge information of the small picture to be detected to a template small picture coordinate system by using affine transformation parameters corresponding to each section of edge information of the small picture to be detected;
b. matching the edge of the small graph to be detected and the edge of the template small graph in a segmentation manner to find abnormal edge points of the small graph to be detected;
c. clustering and growing abnormal points and removing out-of-range abnormal points;
d. completing the defects and calculating the defect characteristics;
e. merging coincident defects and features.
11. The method of claim 1, wherein the down-sampling further comprises: the segment edge information is downsampled to 1/2,1/4 data size.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110015305.XA CN112330678B (en) | 2021-01-07 | 2021-01-07 | Product edge defect detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110015305.XA CN112330678B (en) | 2021-01-07 | 2021-01-07 | Product edge defect detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112330678A CN112330678A (en) | 2021-02-05 |
| CN112330678B true CN112330678B (en) | 2021-04-02 |
Family
ID=74301939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110015305.XA Active CN112330678B (en) | 2021-01-07 | 2021-01-07 | Product edge defect detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112330678B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113063731B (en) * | 2021-03-24 | 2023-01-20 | 上海晨兴希姆通电子科技有限公司 | Detection system and detection method for rotary disc type glass cover plate silk screen printing |
| CN112797900B (en) * | 2021-04-07 | 2021-07-06 | 中科慧远视觉技术(北京)有限公司 | Multi-camera plate size measuring method |
| CN113160161B (en) * | 2021-04-14 | 2023-07-11 | 歌尔股份有限公司 | Method and device for detecting defects at edge of target |
| CN113237888B (en) * | 2021-05-21 | 2022-06-14 | 哈尔滨工业大学 | Large-caliber optical element damage point online and offline dark field image matching method |
| CN113409295B (en) * | 2021-06-30 | 2024-03-29 | 北京兆维电子(集团)有限责任公司 | CELL-based edge defect detection method |
| CN113487561B (en) * | 2021-07-02 | 2023-06-30 | 成都唐源电气股份有限公司 | Pantograph foreign matter detection method and device based on gray gradient abnormal voting |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500448A (en) * | 2013-09-25 | 2014-01-08 | 华南理工大学 | Method for estimating affine transformation model parameters of high density packaging component |
| CN104134220A (en) * | 2014-08-15 | 2014-11-05 | 北京东方泰坦科技股份有限公司 | Low-altitude remote sensing image high-precision matching method with consistent image space |
| CN105701473A (en) * | 2016-01-15 | 2016-06-22 | 深圳市未来媒体技术研究院 | Method for matching palm print image detail point characteristics |
| CN107543828A (en) * | 2017-08-25 | 2018-01-05 | 广东工业大学 | A kind of Surface Flaw Detection method and system |
| CN111292364A (en) * | 2020-01-21 | 2020-06-16 | 天目爱视（北京）科技有限公司 | Method for rapidly matching images in three-dimensional model construction process |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5031641B2 (en) * | 2008-03-31 | 2012-09-19 | 富士通株式会社 | Pattern alignment method, verification method, and verification device |
| CN105930858B (en) * | 2016-04-06 | 2021-03-26 | 吴晓军 | Rapid high-precision geometric template matching method with rotation and scaling functions |
| EP3454128B1 (en) * | 2017-09-12 | 2020-01-29 | IMEC vzw | A method and system for detecting defects of a lithographic pattern |
| CN110567965A (en) * | 2019-07-31 | 2019-12-13 | 华南理工大学 | Smartphone glass cover plate edge visual defect detection method |
| CN111369545B (en) * | 2020-03-10 | 2023-04-25 | 创新奇智(重庆)科技有限公司 | Edge defect detection method, device, model, equipment and readable storage medium |
-
2021
- 2021-01-07 CN CN202110015305.XA patent/CN112330678B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500448A (en) * | 2013-09-25 | 2014-01-08 | 华南理工大学 | Method for estimating affine transformation model parameters of high density packaging component |
| CN104134220A (en) * | 2014-08-15 | 2014-11-05 | 北京东方泰坦科技股份有限公司 | Low-altitude remote sensing image high-precision matching method with consistent image space |
| CN105701473A (en) * | 2016-01-15 | 2016-06-22 | 深圳市未来媒体技术研究院 | Method for matching palm print image detail point characteristics |
| CN107543828A (en) * | 2017-08-25 | 2018-01-05 | 广东工业大学 | A kind of Surface Flaw Detection method and system |
| CN111292364A (en) * | 2020-01-21 | 2020-06-16 | 天目爱视（北京）科技有限公司 | Method for rapidly matching images in three-dimensional model construction process |
Non-Patent Citations (2)
| Title |
|---|
| 机械零件影像边缘检测技术研究及测量系统开发;陈栋梁;《中国优秀硕士学位论文全文数据库 信息科技辑》;20180415(第04期);第I138-2283页 * |
| 融合灰度特征与形状特征的图像配准方法;刘干;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160415(第04期);第I138-1260页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112330678A (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112330678B (en) | Product edge defect detection method | |
| CN110472674B (en) | Template matching algorithm based on edge and gradient characteristics | |
| CN105913415B (en) | A kind of image sub-pixel edge extracting method with extensive adaptability | |
| CN115861291B (en) | Chip circuit board production defect detection method based on machine vision | |
| TWI532116B (en) | Method and system for wafer registration | |
| CN110260818B (en) | Electronic connector robust detection method based on binocular vision | |
| CN113340909B (en) | Glue line defect detection method based on machine vision | |
| CN114862855B (en) | Textile defect detection method and system based on template matching | |
| CN114627080B (en) | Vehicle stamping accessory defect detection method based on computer vision | |
| CN111861979A (en) | Positioning method, positioning equipment and computer readable storage medium | |
| CN114022483B (en) | Injection molding flash area identification method based on edge characteristics | |
| CN114972575A (en) | Linear fitting algorithm based on contour edge | |
| CN115096206A (en) | Part size high-precision measurement method based on machine vision | |
| CN115311279A (en) | Machine vision identification method for warp and weft defects of fabric | |
| Wang et al. | Lane boundary detection based on parabola model | |
| CN114549400A (en) | Image identification method and device | |
| CN107808165B (en) | Infrared image matching method based on SUSAN corner detection | |
| CN109148433B (en) | Method and apparatus for determining dimensions of an integrated circuit device | |
| CN111795970B (en) | Irregular contour defect detection method | |
| JP2008116206A (en) | Apparatus, method, and program for pattern size measurement | |
| CN109934817B (en) | Method for detecting malformation of external contour of fruit body | |
| CN114863492B (en) | Method and device for repairing low-quality fingerprint image | |
| CN114937211B (en) | Remote sensing image change detection method based on invariant object | |
| CN112907510B (en) | Surface defect detection method | |
| CN112508084B (en) | Machine vision detection method based on fuzzy similarity measurement in complex industrial environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |