WO2020133046A1

WO2020133046A1 - Defect detection method and device

Info

Publication number: WO2020133046A1
Application number: PCT/CN2018/124287
Authority: WO
Inventors: 李洪杰
Original assignee: 深圳配天智能技术研究院有限公司
Priority date: 2018-12-27
Filing date: 2018-12-27
Publication date: 2020-07-02
Also published as: CN111758024A; CN111758024B

Abstract

A defect detection method and device, the method comprising: determining an area of interest in an image to be detected (11); dividing the area of interest into multiple caliper areas which are independent from one another and which have the same size (12); performing perpendicular edge detection on each caliper area, a perpendicular edge being a line segment that is in a caliper area and that is perpendicular to a preset edge of the caliper area (13); and acquiring defect information of the area of interest on the basis of the perpendicular edge (14). By using the described method, defect detection may be accurately and effectively implemented, thus improving defect detection efficiency.

Description

Defect detection method and device

【Technical field】

The present application relates to the technical field of image processing, and in particular to a defect detection method and device.

【Background technique】

In actual production applications, it is necessary to rely on machine vision to inspect the surface of the article to confirm whether there are defects, such as: whether there are pits, protrusions and damage on the surface of the workpiece; therefore, defect detection in production applications, especially It has significant use value in production line operation. Defects will appear inconsistent with the surrounding plane in the imaging of the visual system. The visual system uses this uncoordinated location to locate, and then uses relevant algorithms to measure these features to complete the location and measurement of the defect.

The inventor of the present application discovered in the long-term research and development that although there are many edge detection algorithms in the current image processing, such as the Canny algorithm, Sobel algorithm, and Laplacian, these detection algorithms can detect the edges in the image, and the effect is excellent; but these The algorithm can only extract the edge information, but cannot filter and decompose the extracted edges, and it is difficult to accurately and effectively implement defect detection. At the same time, the current defect detection is based on the entire area to be detected, and the calculation efficiency is low.

[Invention content]

The main problem solved by this application is to provide a defect detection method and device, which can accurately and effectively realize defect detection and improve defect detection efficiency.

In order to solve the above technical problems, the technical solution adopted in this application is to provide a defect detection method, which includes: determining the region of interest in the image to be detected; dividing the region of interest into a plurality of independent caliper regions of the same size ; Perform vertical edge detection on each caliper area, and the vertical edge is a line segment in the caliper area perpendicular to the preset edge of the caliper area; based on the vertical edge, obtain defect information of the area of interest.

To solve the above technical problems, another technical solution adopted by the present application is to provide a defect detection device, which includes an input unit, a processor, and an output unit connected in sequence; wherein, the input unit is used to acquire an image to be detected; the processor To determine the region of interest in the image to be detected; divide the region of interest into multiple independent caliper regions of the same size; perform vertical edge detection on each caliper region, the vertical edge is the preset side of the caliper region and the caliper region Vertical line segments; obtaining defect information of the region of interest based on vertical edges; output unit for outputting defect information.

Through the above solution, the beneficial effect of the present application is: by extracting the region of interest in the image to be detected, and then dividing the region of interest into a plurality of caliper regions, the vertical edge detection is performed on the image in each caliper region, thereby obtaining multiple The vertical edge information in each caliper area, and then obtaining the defects of the image to be detected according to the vertical edge information in each caliper area, can accurately and effectively realize the defect detection, and reduce the amount of processing operations and improve the defect detection efficiency.

[Description of the drawings]

In order to more clearly explain the technical solutions in the embodiments of the present invention, the drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For a person of ordinary skill in the art, without paying any creative work, other drawings can be obtained based on these drawings. among them:

1 is a schematic flowchart of an embodiment of a defect detection method provided by this application;

2 is a schematic flowchart of another embodiment of a defect detection method provided by this application;

3 is a schematic diagram of a region of interest having multiple calipers in another embodiment of a defect detection method provided by this application;

4 is a schematic diagram of pixels corresponding to step 24 in another embodiment of the defect detection method provided by this application;

5 is a schematic diagram of a first type of defect feature and a fitted line in another embodiment of the defect detection method provided by this application;

6 is a schematic diagram of a second type of defect characteristics and a fitted line in another embodiment of the defect detection method provided by this application;

7 is a schematic structural diagram of an embodiment of a defect detection device provided by the present application.

【detailed description】

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work fall within the scope of protection of the present application.

Referring to FIG. 1, FIG. 1 is a schematic flowchart of an embodiment of a defect detection method provided by the present application. The method includes:

Step 11: Determine the region of interest in the image to be detected.

The target object to be detected is converted into a digital image by a camera device, that is, an image to be inspected. The image to be inspected is an image with defects. For example, for the flatness inspection of a part surface, the image to be inspected is a projection image of the surface of the part There are defects such as pits or protrusions on the projected image of the part surface; the image to be inspected can be a grayscale image, a color image or a depth image. To facilitate processing, the color image or depth image can be converted into a grayscale image first.

After acquiring the image to be inspected, in order to obtain the defect in the image to be inspected, determine the region of interest (ROI, Region of Interest) in the image to be inspected to reduce the processing time and increase the accuracy; ROI is the area to be processed, which The shape can be rectangular or fan-shaped.

Further, in order to judge whether the setting of the ROI is reasonable, a cross-border inspection of the ROI needs to be performed, that is, whether the ROI exceeds the boundary of the image to be detected, and if the ROI does not exceed the boundary of the image to be detected, step 12 is performed; if the ROI exceeds the image to be detected , The ROI can be redefined until the set ROI does not exceed the boundary.

Step 12: Divide the region of interest into multiple independent caliper regions of the same size.

Divide the area of interest into multiple caliper areas, each with a preset interval or no interval, and the size and shape are equal, the shape of the caliper area is similar to the ROI, for example, if the ROI is rectangular, the caliper area The shape is a high-narrow rectangle; if the ROI is fan-shaped, the caliper area is also high-narrow fan-shaped, and the information such as the position and size of the caliper area is marked.

Step 13: Perform vertical edge detection on each caliper area.

The vertical side is a line segment perpendicular to the preset side of the caliper area in the caliper area.

An edge is a geometric feature of a group of connected pixels. Usually, they are located where the pixel gray transitions between two different areas; the edge detection algorithm can be used to edge detect the image in each caliper area to obtain each caliper area The edge information within the edge; where the edge detection algorithm is realized by obtaining the first derivative, second derivative and gradient of the neighboring pixels. In order to simplify the derivation process, the convolution operation method can be used to perform the convolution operation. The edge The information includes the location of the edge and the pixel value.

Step 14: Obtain the defect information of the region of interest based on the vertical edges.

The obtained multiple edge information is screened to eliminate false defects and obtain true and real defect information.

In order to perform defect detection on the image to be inspected, first extract the area of interest in the image to be inspected, then divide the area of interest into multiple caliper areas, and perform vertical edge detection on the image in each caliper area to obtain multiple caliper areas In the vertical edge of the inside, the defect of the image to be detected is obtained according to the vertical edge in each caliper area, thereby accurately and effectively realizing defect detection, and reducing the amount of processing operations and improving the efficiency of defect detection.

Referring to FIG. 2, FIG. 2 is a schematic flowchart of another embodiment of a defect detection method provided by the present application. The method includes:

Step 21: Determine the region of interest in the image to be detected.

Step 21 is similar to step 11 in the above embodiment, and will not be repeated here.

Step 22: Divide the region of interest into multiple caliper regions that are independent of each other and have the same size.

The width of the caliper area in the direction of the preset side is greater than the height in the direction of the vertical side; the user can set the width, height, initial offset, and repeat offset of the caliper area as needed, or set it as the default value; Move to the distance between the starting position of the first caliper area and the starting position of the area of interest in the direction of the vertical edge. The first caliper area is the caliper area closest to the starting position of the area of interest, and the repeated offset is The distance of the starting position of the adjacent caliper region in the direction of the preset side; the width of the caliper region in the direction of the preset side is set equal to the width of the ROI in the direction of the preset side.

In a specific embodiment, as shown in FIG. 3, the ROI is a rectangular region, the direction of the vertical side is the direction of horizontal to the right, that is, the positive direction of the Y axis of the image, and the direction of the preset side is vertically downward The direction, that is, the positive direction of the X axis of the image, the initial offset is 0, that is, the starting position of the first caliper area coincides with the starting position of the ROI, and the repeated offset can be 5 pixels to achieve discrete sampling; The height of the caliper area and the value of the repeated offset are automatically adjusted according to the parameters given by the user to adaptively adjust the size of the caliper area so that the caliper area can cover the entire area of interest. In other embodiments, for example, when the ROI is a fan-shaped region, the direction of the vertical side may be set as the circumferential direction of the fan-shaped region, and the direction of the preset side may be set as the radial direction of the fan-shaped region.

Step 23: Compress the pixel information in each caliper area in a direction perpendicular to the preset side to obtain a one-dimensional vector corresponding to each caliper area.

In order to obtain the one-dimensional vector corresponding to each caliper area, the gray value of each row of pixels in each caliper area is summed and averaged in the direction perpendicular to the preset side, and the pixel information includes the pixel value in the caliper area; Ground, as shown in FIG. 4, for the caliper area with 4*3 pixels, the pixel values in the caliper area are summed and averaged, that is, the pixel value A=(a1+a2+a3)/3, B=(b1 +b2+b3)/3, C=(c1+c2+c3)/3 and D=(d1+d2+d3)/3, by summing the pixel values along the direction of the preset edge, the The pixel values in the caliper area are compressed into a one-dimensional column vector; in addition, for convenience of processing, the one-dimensional column vector can be transposed, and finally a one-dimensional row vector can be obtained.

Step 24: Perform vertical edge detection on each one-dimensional vector to obtain edge points in each caliper area.

The line segment that passes through the edge point in the caliper area and is perpendicular to the preset edge is the vertical edge; for the compressed one-dimensional row vector, the edge detection algorithm is used for edge detection; specifically, the first-order derivation of each one-dimensional row vector or Perform convolution operation; when using convolution operation to process the one-dimensional row vector, the size of the convolution kernel can be 1*3 or 1*5; perform non-extreme suppression processing on the result after derivation or convolution , Screen out the non-edge points; then according to the contrast at the screened edge points, obtain the contrast at the screened edge points, threshold the obtained contrast, and preserve the edge points whose contrast meets the preset threshold conditions; Specifically, the preset threshold value ranges from 0 to 255, which is used as the basis for edge point screening. This parameter is used to threshold the image after vertical edge detection to filter out edge points with a contrast less than this threshold. In a specific embodiment, the preset threshold is 25.

Step 25: Select the best vertical edge from the vertical edges in the caliper area.

The line segment set in the direction perpendicular to the preset side in each caliper area with an edge point is taken as the vertical edge in the caliper area, for example, as shown in FIG. 4, if the position of the pixel value c2 is an edge point, the vertical The pixel value of the edge is [c1, c2, c3].

After the vertical edge detection in the caliper area, through the process of non-extreme suppression and thresholding, there may be several vertical edges left in the caliper area; how to filter these vertical edges is an important process. Reasonable, it will have a great influence on the fitting of ideal vertical edges and the results of defect detection, so the method of screening is particularly important.

In this embodiment, in order to screen out the real defects of the image to be detected, each vertical edge is scored according to the contrast and position degree of each vertical edge; specifically, the contrast score and the position degree score can be linearly combined to obtain a score value; Then select the vertical edge with the highest score as the best vertical edge.

Among them, the higher the contrast, the higher the score, the position is the order value of the vertical edges found in the preset direction, and the earlier the vertical edge is found, the higher the score.

Step 26: Take the point on the best vertical edge in the caliper area for line fitting to obtain a fitted line.

Take the midpoint on the best vertical edge for line fitting. Specifically, select the midpoint of the best vertical edge as the fitting point, and use a fitting algorithm to perform fitting based on multiple fitting points. In one embodiment, The least square method is used for straight line fitting or curve fitting to obtain a fitting line, which is an ideal defect-free edge. In the fitting process, you can first fit based on all the fitting points on the best vertical edge to get a fitting line, then calculate the shortest distance between each fitting point and the fitting line, and filter out the shortest The best vertical edge corresponding to the fitting point whose distance is greater than the first distance threshold is to filter out pixels farther away from the fitting line, so as not to affect the fitting effect. The positions of these filtered pixels are generally defects. Further, the retained best vertical edge is used to re-fit the line, and the fitted line is replaced by the re-fitted line before re-fitting the line.

Step 27: Mark defects in the caliper area according to the relative positional relationship between the best vertical edge and the fitted line.

In order to detect defects, each caliper area can be traversed to determine whether the best vertical edge exists in each caliper area; if there is no vertical edge in the caliper area, the caliper area is marked as having the first type of defect characteristics; The best vertical edge exists in the caliper area, and the best vertical edge is selected from the vertical edges in the caliper area.

Further, it is determined whether the shortest distance between the fitting point on the best vertical edge and the fitting line is greater than a preset second distance threshold. If the shortest distance between the fitting point on the best vertical edge and the fitting line is greater than the preset second distance threshold, the caliper area is marked as having the second type of defect feature; otherwise, it is considered that this caliper area There are no defects, ie non-defective areas, with true edges.

In a specific embodiment, the first type of defects may be gap features, and the second type of defects may be other defect features that are not gap features. For example, for a standard surface, such as a flat surface and a round surface, due to various reasons, the surface appears convex or missing, these abnormal surface features are collectively referred to as defects; for larger defects, it is called the first A type of defect feature can also be called a gap feature. A gap feature is a special form of defect; for bumps and smaller defects, it is called a second type of defect feature.

In addition, after marking the first type of defect features or the second type of defect features, the defect parameters in the defect area can be counted; specifically, the defect parameters of the first type of defect features and/or the second type of defect features can be calculated , The defect parameters include at least one of the width along the direction of the vertical edge, the height along the direction of the preset edge, and the area; where the width can be an integer multiple of the repeated offset, and the defect height is the furthest from the fitting line at the defect The distance from the fitting point to the fitting line, the gap height is set by the user or is the default value.

For example, as shown in Figure 5, there are three fitting points A, B, and C. Since there are no fitting points in the third caliper area and the fourth caliper area, it is determined that these two areas have the first type Defect characteristics; or, as shown in FIG. 6, after screening, 5 fitting points AE are obtained, and based on these 5 points, the fitting line in the figure is obtained, according to the distance between the fitting point and the fitting line, thus It is determined that the fitting points C and D are defect points, the fitting points A, B and E are true edge points, and the second type of defect feature is an area composed of fitting points C, D and fitting edges.

After acquiring the region of interest in the image to be detected, differentiate the region of interest along the direction perpendicular to the preset side, divide the region of interest into multiple caliper regions, and then pair the region along the direction perpendicular to the preset side The pixels in the caliper area are compressed to obtain an edge row vector, and then the first-order derivation is performed to obtain the edges that meet the requirements in each caliper area. Compared with the prior art, this embodiment can reduce the complexity of the edges. The detection is decomposed into simple one-dimensional line vector edge detection, which reduces the complexity of the algorithm and saves processing time.

In addition, this embodiment uses a non-extreme suppression algorithm and thresholding to perform the first screening to obtain an accurate edge, obtain the contrast at the edge, and obtain the edge pixel value corresponding to the contrast, and then analyze the obtained edge Score for the second screening, get the best vertical edge in the caliper area, get the fitted edge by fitting the points on the best vertical edge, and finally filter based on the distance from the fitted point to the fitted edge Identify the first type of defect characteristics and the second type of defect characteristics; use the "differential" concept to decompose the edge line segment to be detected into multiple small edge line segments in order to screen out the most reasonable small edge line segments for fitting and defect detection. Scoring edges, realizing the secondary screening of edge line segments, accurately and effectively detecting real defects, and reducing the amount of processing operations, improving defect detection efficiency; while ordinary edge detection can only find edge information, it cannot perform edge information Scoring and screening, unable to complete line fitting and defect detection.

Referring to FIG. 7, FIG. 7 is a schematic structural diagram of an embodiment of a defect detection device provided by the present application. The device includes an input unit 71, a processor 72, and an output unit 73 connected in sequence.

Among them, the input unit 71 is used to acquire the image to be detected; the processor 72 is used to determine the region of interest in the image to be detected; the region of interest is divided into a plurality of independent caliper regions of the same size; each caliper region is vertically For edge detection, the vertical edge is a line segment in the caliper area perpendicular to the preset edge of the caliper area; the defect information of the region of interest is acquired based on the vertical edge; and the output unit 73 is used to output defect information.

In the several embodiments provided in this application, it should be understood that the disclosed method and device may be implemented in other ways. For example, the device implementations described above are only schematic. For example, the division of modules or units is only a division of logical functions. In actual implementation, there may be other divisions, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software function unit.

The above are only examples of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by the description and drawings of this application, or directly or indirectly used in other related technical fields, The same reason is included in the patent protection scope of this application.

Claims

A defect detection method, which includes:

Determine the region of interest in the image to be detected;

Dividing the region of interest into a plurality of independent caliper regions of the same size;

Performing vertical edge detection on each of the caliper areas, the vertical edge being a line segment in the caliper area that is perpendicular to the preset edge of the caliper area;

Obtain defect information of the region of interest based on the vertical edge.
The defect detection method according to claim 1, wherein after the region of interest in the image to be detected is determined, before the region of interest is divided into a plurality of independent caliper regions of the same size, include:

Checking whether the region of interest exceeds the boundary of the image to be detected;

If the region of interest does not exceed the boundary of the image to be detected, perform the step of dividing the region of interest into a plurality of independent caliper regions of the same size;

If the region of interest exceeds the boundary of the image to be detected, the region of interest is determined again.
The defect detection method according to claim 1, wherein

The width of the caliper area in the direction of the preset side is greater than the height in the direction of the vertical side.
The defect detection method according to claim 1, wherein the vertical edge detection of each caliper area includes:

Compressing the pixel information in each caliper area in a direction perpendicular to the preset side to obtain a one-dimensional vector corresponding to each caliper area;

Vertical edge detection is performed on each of the one-dimensional vectors to obtain edge points in each caliper area, and a line segment passing through the edge point and perpendicular to the preset edge in the caliper area is the vertical edge.
The defect detection method according to claim 4, wherein the pixel information in each caliper area is compressed in a direction perpendicular to the preset side to obtain a one-dimensional vector corresponding to each caliper area, include:

The gray value of each row of pixels in the direction perpendicular to the preset side in each caliper area is summed and averaged.
The defect detection method according to claim 4, wherein the performing edge detection on each of the one-dimensional vectors to obtain edge points in each caliper area includes:

Perform a first-order derivation of each of the one-dimensional vectors;

Carry out non-extreme value suppression on the derivative results and filter out non-edge points;

Obtain the contrast at the filtered edge points, and retain the edge points whose contrast meets the preset threshold condition.
The defect detection method according to claim 1, wherein the acquiring defect information of the region of interest based on the vertical edge includes:

Selecting the best vertical edge from the vertical edges in the caliper area;

Take a point on the best vertical edge in the caliper area for line fitting to obtain a fitted line;

Mark the defect area of the caliper area according to the relative positional relationship between the best vertical edge and the fitting line.
The defect detection method according to claim 7, wherein the selecting the best vertical edge from the vertical edges in the caliper area includes:

Scoring each vertical edge according to the contrast and position degree of each vertical edge, where the position degree is an order value of finding the vertical edges in a preset direction;

The highest vertical edge is selected as the best vertical edge.
The defect detection method according to claim 7, wherein the midpoint on the best vertical edge in the caliper area is taken for line fitting.
The defect detection method according to claim 7, wherein after performing line fitting on the point on the best vertical edge in the caliper area to obtain a fitted line, the method further comprises:

Calculating the shortest distance between each fitting point and the fitting line;

Filtering out the best vertical edge corresponding to the fitting point where the shortest distance is greater than the first distance threshold;

The line fitting is performed again using the retained best vertical edge, and the fitting line obtained before replacing the line fitting is replaced with the obtained fitting line.
The defect detection method according to claim 7, wherein before selecting the best vertical edge from the vertical edges in the caliper area, the method further comprises:

Determine whether the best vertical edge exists in each caliper area, and if the best vertical edge does not exist, mark the caliper area as having the first type of defect feature;

If there is the best vertical edge, the best vertical edge is selected from the vertical edges in the caliper area.
The defect detection method according to claim 11, wherein the step of defect marking the caliper area according to the relative positional relationship between the best vertical edge and the fitting line further comprises:

Determine whether the shortest distance between the fitting point on the best vertical edge and the fitting line is greater than a preset second distance threshold;

If it is greater than the preset second distance threshold, the caliper area is marked as having a second type of defect feature.
The defect detection method according to claim 12, further comprising:

Calculate the defect parameters of the first-type defect feature and/or the second-type defect feature.
The defect detection method according to claim 13, wherein the defect parameter includes at least one of a defect width in the direction of the vertical side, a defect height in the direction of the preset side, and a defect area.
A defect detection device, characterized in that it comprises an input unit, a processor and an output unit connected in sequence;

Wherein, the input unit is used to acquire the image to be detected; the processor is used to determine the region of interest in the image to be detected; the region of interest is divided into a plurality of independent caliper regions of the same size; Each caliper area is subjected to vertical edge detection, and the vertical edge is a line segment in the caliper area perpendicular to the preset edge of the caliper area; based on the vertical edge, the defect information of the region of interest is obtained; the output unit is used to Output the defect information.