CN113688807B

CN113688807B - Self-adaptive defect detection method, device, recognition system and storage medium

Info

Publication number: CN113688807B
Application number: CN202111243916.6A
Authority: CN
Inventors: 张武杰
Original assignee: Casi Vision Technology Luoyang Co Ltd; Casi Vision Technology Beijing Co Ltd
Current assignee: Casi Vision Technology Luoyang Co Ltd; Casi Vision Technology Beijing Co Ltd
Priority date: 2021-10-26
Filing date: 2021-10-26
Publication date: 2022-02-08
Anticipated expiration: 2041-10-26
Also published as: CN114581742A; CN114140679B; CN113688807A; CN114140679A; CN114581742B

Abstract

The application provides a self-adaptive defect detection method, a self-adaptive defect detection device, a self-adaptive defect identification system and a storage medium, wherein the method comprises the following steps: the method comprises the steps of extracting a target detection area from a reference image of a detected target based on an interested area, cutting the reference image from a preset distance outside the target detection area to completely obtain the target detection area, filling a non-target detection area, calculating a mean image and a variance image of the filled image to be detected, calculating a high threshold value image and a low threshold value image through a high threshold value scale coefficient, a low threshold value scale coefficient, a minimum reference deviation and a maximum gray variance, then respectively making difference images with the filled image to be detected through the high threshold value image and the low threshold value image to obtain a high threshold value connected domain and a low threshold value connected domain, and finally screening a part of the low threshold value connected domain, which has intersection with the high threshold value connected domain, to obtain a defect connected domain set, so that abnormal defect image segmentation of the illumination unevenness image is realized.

Description

Self-adaptive defect detection method, device, recognition system and storage medium

Technical Field

The application relates to the field of machine vision, in particular to a self-adaptive defect detection method, a self-adaptive defect detection device, a self-adaptive defect identification system and a storage medium.

Background

The AOI (Automated Optical Inspection) device has a wide application range in industrial scenes, and especially plays an important role in detecting appearance defects of industrial products. AOI is an important tool for liberating a large number of quality inspection workers, reducing staff, improving efficiency, improving the productivity of enterprises, improving the yield of products and improving the product competitiveness of companies. In different industrial scenes, the AOI device usually has different product forms, and the detection principle may be to highlight the defects of the product by means of a reasonable optical design scheme, acquire the product image to a computer by means of an image acquisition device, calculate the positions and sizes of the defects by means of a reasonable image processing algorithm, further acquire texture features of the defects, and classify and grade the defects by combining the sizes and the texture features of the defects. In the technical field of display panels, various defects such as scratches, concave-convex points, edge breakage and the like on the panel are generally required to be detected, and image segmentation is a common method for positioning the defects.

In the prior art, common image segmentation methods include a fixed threshold method, an adaptive threshold method and the like, and under the condition that a texture background is complex, deep learning methods such as a convolutional neural network and the like are also utilized. However, due to the influence of the film coating process on the panel and the transmission stability, the image of the product shows larger uneven brightness, the defects of scratches and the like on the surface of the product show the characteristics of discontinuity, poor contrast consistency and the like due to grade difference, and the image is difficult to be normalized by the image preprocessing method, so that the conventional fixed threshold method and the adaptive threshold method cannot well realize the segmentation of the defects in such a scene, and further cannot realize the accurate detection of the defects. Although deep learning methods such as convolutional neural networks can solve the above problems to some extent, they usually do not give priority to the methods because they require many training samples and more computation resources and computation time during computation.

Disclosure of Invention

In view of this, an object of the present application is to provide a self-adaptive defect detecting method, a device, an identification system, and a storage medium, which can adopt a self-adaptive threshold segmentation method in a defect detecting process in an uneven-illumination industrial scene, so as to solve a problem of how to perform image segmentation on abnormal defects of an uneven-brightness image in the prior art. In addition, the method and the device further fuse the segmented results, fuse a plurality of discontinuous segmentation results into the same segmentation result, can effectively overcome the problem of image segmentation difficulty caused by uneven brightness, and improve the accuracy of abnormal defect detection based on image segmentation.

In order to solve the foregoing technical problem, in a first aspect, an embodiment of the present application provides a method for adaptive defect detection, where the method includes:

acquiring a reference image of a measured target, and establishing a reference coordinate system by taking the measured target in the reference image as an object;

setting an interested region for a detected target in the reference image, and extracting a target detection region according to the interested region; the region of interest is associated with the reference coordinate system;

acquiring a positive external rectangle of the target detection area, and cutting an image based on the external expansion preset distance of the positive external rectangle to obtain an image to be detected;

filling a non-target detection area in the image to be detected to obtain a filled image to be detected;

calculating a mean image and a variance image of the filled image to be detected;

calculating a high threshold value image and a low threshold value image according to the mean value image and the variance image and a preset high threshold value scale coefficient, a preset low threshold value scale coefficient, a preset minimum standard deviation and a preset maximum gray variance;

calculating a high threshold difference image and a low threshold difference image according to the filled image to be detected, the high threshold image and the low threshold image;

extracting a high-threshold set of connected components from the high-threshold difference image, extracting a low-threshold set of connected components from the low-threshold difference image, and

aiming at the low-threshold connected domain set, screening out a low-threshold connected domain which has intersection with a high-threshold connected domain in the high-threshold connected domain set to obtain a defect connected domain set;

the high threshold difference image is obtained by making a difference between the filled image to be detected and the high threshold image, and the low threshold difference image is obtained by making a difference between the filled image to be detected and the low threshold image.

In some embodiments, the acquiring a reference image of a target to be measured and creating a reference coordinate system with the target to be measured in the reference image as an object includes:

acquiring an image of a detected target as a reference image;

establishing a reference coordinate system by a preset coordinate system establishing method by taking a measured target in the reference image as an object; the preset coordinate system creating method comprises straight line fitting or template matching.

In some embodiments, the filling the non-target detection area in the image to be detected to obtain a filled image to be detected includes:

determining a filling mode of a non-target detection area in the image to be detected according to the gray level characteristics of the defects in the image to be detected; the filling mode comprises gray level mirror image filling and/or fixed gray level filling;

and filling the non-target detection area in the image to be detected according to the filling mode to obtain the filled image to be detected.

In some embodiments, the calculating the mean image and the variance image of the filled image to be measured includes:

setting the size of a neighborhood window according to the size of the target defect;

and calculating the mean image and the variance image of the filled image to be detected according to the size of the neighborhood window.

In some embodiments, the extracting the high-threshold connected component set from the high-threshold difference image and the extracting the low-threshold connected component set from the low-threshold difference image comprises:

extracting connected domains with the gray values smaller than 0 in the high-threshold difference image to obtain a high-threshold connected domain set;

and extracting the connected domain with the gray value smaller than 0 in the low threshold difference image to obtain a low threshold connected domain set.

In some embodiments, the screening out, for the low-threshold connected domain set, a low-threshold connected domain that intersects with a high-threshold connected domain in the high-threshold connected domain set to obtain a defect connected domain set includes:

searching an intersection region of the high threshold connected domain set and the low threshold connected domain set;

and taking all low-threshold connected domains containing the intersection region as a defect connected domain set.

sequentially traversing each low-threshold connected domain in the low-threshold connected domain set, and judging whether an intersection exists between the low-threshold connected domain and a high-threshold connected domain in the high-threshold connected domain set;

if the intersection exists between the low threshold connected domain and the high threshold connected domain in the high threshold connected domain set, the low threshold connected domain is determined as a defect connected domain;

if the low threshold connected domain and the high threshold connected domain in the high threshold connected domain set do not have an intersection, the low threshold connected domain is removed from the low threshold connected domain set;

and when the judgment and analysis of all the low threshold connected domains are finished, taking the low threshold connected domain set which finishes the rejecting operation as a defect connected domain set.

In some embodiments, before the calculating the mean image and the variance image of the filled image to be measured, the method further includes:

determining the defect type of the defect in the target detection area;

and if the defect type of the defect in the target detection area is a dark defect, continuing to perform detection calculation.

In some embodiments, further comprising:

and if the defect type of the defect in the target detection area is a bright defect, performing gray inversion on the filled image to be detected.

In a second aspect, an embodiment of the present application provides a defect fusion method, where the method includes:

extracting a defect connected domain set based on a target image to be detected;

extracting linear connected domains from the defect connected domain set, and marking the remaining defect connected domains as point connected domains;

randomly selecting two linear connected domains from all linear connected domains as linear connected domain pairs, and calculating a first angle and a first distance between central axes of the two linear connected domains in each linear connected domain pair; the first distance includes at least one type of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance;

clustering each linear connected domain pair according to a first angle and a first distance corresponding to each linear connected domain pair to obtain a linear connected domain set corresponding to the linear connected domain pair;

aiming at each linear connected domain set, calculating a minimum external rectangle corresponding to the linear connected domain set, and merging the dot-shaped connected domains with the distance from the minimum external rectangle to the minimum external rectangle being less than a preset threshold value into the linear connected domain set to obtain linear defects and residual dot-shaped connected domains;

and clustering the residual punctiform connected domains to obtain dense point defects and isolated point defects.

In some embodiments, extracting linear connected domains from the set of defect connected domains and marking the remaining defect connected domains as dotted connected domains comprises:

calculating the second moment of each defect connected domain according to the area and the central coordinate of each defect connected domain in the defect connected domain set;

calculating the linearity of the defect connected domain according to the second moment of the defect connected domain;

judging whether the linearity of the defect connected domain reaches a preset linearity or not;

if the linearity of the defect connected domain reaches the preset linearity, extracting the defect connected domain from the defect connected domain set and marking the defect connected domain as a linear connected domain;

and after all the defect connected domains are calculated, marking the remaining defect connected domains in the defect connected domain set as point connected domains.

In some embodiments, the formula of the second moment of the defect connected domain is as follows:

；

wherein, F is the area of the defect connected domain, (x, y) is the coordinate of any pixel point in the defect connected domain, (x)₀，y₀) Is the center coordinate of the defect connected domain.

In some embodiments, the formula for the linearity of the defect connected domain is as follows:

l=[(m₂₀-m₀₂)²+4*m₁₁*m₁₁]/(m₂₀+m₀₂)² ；

wherein l represents the linearity of the defect connected domain, m₂₀Is the central second moment in the vertical direction, m₀₂Is the central second step distance, m, of the horizontal direction₁₁Is the comprehensive central second moment of the horizontal direction and the vertical direction.

In some embodiments, clustering each pair of linear connected domains according to a first angle and a first distance corresponding to each pair of linear connected domains to obtain a set of linear connected domains corresponding to the pair of linear connected domains, includes:

calculating a second angle and a second distance between the central axis of any other linear connected domain and the central axis of the linear connected domain associated with all the linear connected domain pairs aiming at each linear connected domain pair, and judging whether at least one group of the second angle and the second distance are respectively smaller than or equal to a first angle and a first distance corresponding to the linear connected domain pair; the second distance comprises a distance type which is the same as a distance type comprised by the first distance;

if at least one group of the second angle and the second distance is smaller than or equal to the first angle and the first distance corresponding to the pair of linear connected domains, merging the other linear connected domains into the set of linear connected domains corresponding to the pair of linear connected domains, and associating the other linear connected domains with the pair of linear connected domains.

In some embodiments, the clustering the remaining dotted connected domains to obtain dense point defects and isolated point defects includes:

generating a rectangular area with a preset size by taking each residual dotted connected domain as a center, and determining that the rectangular area containing the residual dotted connected domains exceeding a preset number is a dense rectangular area;

and merging the dense rectangular areas containing the same residual dot-shaped connected domains, clustering all the residual dot-shaped connected domains in each merged dense rectangular area into a dense point defect, and determining the residual dot-shaped connected domains not contained in the dense rectangular areas as isolated point defects.

In a third aspect, an embodiment of the present application provides a defect fusion method, where the method includes:

screening the defect connected domains in the defect connected domain set according to the linearity to obtain a linear connected domain and a dotted connected domain;

selecting a plurality of groups of linear connected domains from the linear connected domains, and clustering according to the angle and the distance between the central axes of the linear connected domains in each group of linear connected domains to obtain a linear connected domain set; each group of linear connected domains comprises two linear connected domains; the distance comprises at least one of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance;

aiming at each linear connected domain set, calculating a first minimum circumscribed rectangle of the linear connected domain set, and merging the dot-shaped connected domains with the distance between the dot-shaped connected domains and the edge of the first minimum circumscribed rectangle being smaller than a preset threshold value into the linear connected domain set to obtain linear defects and residual dot-shaped connected domains;

according to the coordinates of the central point of each remaining point-like connected domain, creating a second minimum circumscribed rectangle containing the central points of all the remaining point-like connected domains, and generating a dense point search region according to the length of the second minimum circumscribed rectangle and the width of a search window with a preset size;

and scanning the second minimum circumscribed rectangle by using a dense point search area containing a search window to obtain dense point defects and isolated point defects.

In some embodiments, the scanning the second minimum bounding rectangle with the dense point search region including the search window to obtain the dense point defect and the isolated point defect includes:

the dense point searching area is arranged at the upper end inside the second minimum circumscribed rectangle, and the dense point searching area is scanned from left to right through the searching window to obtain a dense point area at the current position of the dense point searching area;

determining the moving distance of the dense point searching area and moving according to the quantity of the dense point areas contained in the current position of the dense point searching area;

and when the scanning of the second minimum circumscribed rectangle is finished, determining the residual dot-shaped connected domain in each scanned dense dot region as a dense dot defect, and determining the residual dot-shaped connected domain which is not in the dense dot region as an isolated dot defect.

In some embodiments, the determining and moving the moving distance of the dense point search area according to the number of dense point areas included in the current position of the dense point search area includes:

if the current position of the dense point search area comprises at least one dense point area, moving the dense point search area downwards to a first target position, and scanning the dense point search area from left to right by using the search window; the first target position is the position from the upper edge of the dense point searching area to the center point of the nearest remaining point-shaped connected domain below the current position of the dense point searching area.

if the current position of the dense point searching area does not contain the dense point area, moving the dense point searching area downwards to a second target position, and scanning the dense point searching area from left to right by using the searching window; and the second target position is the position of moving the upper edge of the dense point searching area to the central point of a residual dotted connected domain closest to the lower part of the current position of the upper edge.

In some embodiments, the setting the dense point search area at the upper end inside the second minimum circumscribed rectangle, and scanning the dense point search area from left to right with the search window to obtain the dense point area at the current position of the dense point search area includes:

moving the search window to the position where the left edge of the search window is located at the center point of the first remaining dotted connected domain from left to right in the dense point search area, and judging whether the number of the center points of the remaining dotted connected domains contained in the search window exceeds a preset number;

determining whether the current position of the search window is treated as a dense point region based on the determination result;

and when the scanning of the current position of the dense point searching area is finished, taking all the scanned dense point areas as the dense point areas of the current position of the dense point searching area.

In some embodiments, the step of determining whether the current position of the search window is treated as a dense point region based on the determination result includes:

if the number of the central points of the remaining point-like connected domains contained in the search window exceeds a preset number, determining the current position of the search window as a dense point region;

moving the search window to a third target position rightwards, and judging whether the number of the central points of the remaining point-like connected domains contained in the search window of the current position exceeds a preset number; and the third target position is the position from the left edge of the search window to the center point of the nearest residual dotted connected domain on the right side of the current position of the search window.

if the number of the center points of the remaining point-like connected domains contained in the search window does not exceed the preset number, moving the search window to a fourth target position to judge whether the number of the center points of the remaining point-like connected domains contained in the search window at the current position exceeds the preset number; the fourth target position is the center point of a residual dot-shaped connected domain which is closest to the right of the current position of the left edge after the left edge of the search window is moved. In a fourth aspect, an embodiment of the present application provides an adaptive defect detection apparatus, including:

the acquisition module is used for acquiring a reference image of a measured target and establishing a reference coordinate system by taking the measured target in the reference image as an object;

the extraction module is used for setting an interested region for a detected target in the reference image and extracting a target detection region according to the interested region; the region of interest is associated with the reference coordinate system;

the cutting module is used for acquiring a positive external rectangle of the target detection area and cutting an image based on the external expansion preset distance of the positive external rectangle to obtain an image to be detected;

the filling module is used for filling a non-target detection area in the image to be detected to obtain a filled image to be detected;

the first calculation module is used for calculating a mean image and a variance image of the filled image to be detected;

the second calculation module is used for calculating to obtain a high threshold value image and a low threshold value image according to the mean value image and the variance image and a preset high threshold value scale coefficient, a preset low threshold value scale coefficient, a preset minimum standard deviation and a preset maximum gray variance;

the third calculation module is used for calculating a high threshold difference image and a low threshold difference image according to the filled image to be detected, the filled high threshold image and the filled low threshold image, extracting a high threshold connected domain set from the high threshold difference image and extracting a low threshold connected domain set from the low threshold difference image; the high threshold difference image is obtained by making a difference between the filled image to be detected and the high threshold image, and the low threshold difference image is obtained by making a difference between the filled image to be detected and the low threshold image;

and the screening module is used for screening out the low-threshold connected domain which has intersection with the high-threshold connected domain in the high-threshold connected domain set aiming at the low-threshold connected domain set to obtain a defect connected domain set.

In a fifth aspect, an embodiment of the present application provides a defect fusion apparatus, including:

the marking module is used for extracting a linear connected domain from a defect connected domain set and marking the remaining defect connected domains as point connected domains, wherein the defect connected domain set is extracted based on a target image to be detected;

the calculation module is used for randomly selecting two linear connected domains from all linear connected domains as linear connected domain pairs and calculating a first angle and a first distance between central axes of the two linear connected domains in each linear connected domain pair; the first distance comprises at least one of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance;

the clustering module is used for clustering each linear connected domain pair according to the first angle and the first distance corresponding to the linear connected domain pair to obtain a linear connected domain set corresponding to the linear connected domain pair;

the linear defect module is used for calculating a minimum external rectangle corresponding to each linear connected domain set, and merging the dot-shaped connected domains with the distance from the minimum external rectangle to the minimum external rectangle being less than a preset threshold value into the linear connected domain set to obtain a linear defect and residual dot-shaped connected domains;

and the point defect module is used for clustering the residual point connected domains to obtain dense point defects and isolated point defects.

In a sixth aspect, an embodiment of the present application provides a defect fusion apparatus, including:

the classification module is used for screening defect connected domains in a defect connected domain set according to linearity to obtain a linear connected domain and a dotted connected domain, wherein the defect connected domain set is extracted based on a target image to be detected;

the linear clustering module is used for selecting a plurality of groups of linear connected domains from the linear connected domains, and clustering according to the angle and the distance between the central axes of the linear connected domains in each group of linear connected domains to obtain a linear connected domain set; each group of linear connected domains comprises two linear connected domains; the distance comprises at least one of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance;

the merging module is used for calculating a first minimum external rectangle of each linear connected domain set, merging the dot-shaped connected domains with the distance between the dot-shaped connected domains and the edge of the first minimum external rectangle being smaller than a preset threshold value into the linear connected domain set, and obtaining linear defects and residual dot-shaped connected domains;

the creating module is used for creating a second minimum circumscribed rectangle containing all the remaining point-like connected domain center points according to the coordinates of each remaining point-like connected domain center point, and generating a dense point searching region according to the length of the second minimum circumscribed rectangle and the width of a searching window with a preset size;

and the scanning module is used for scanning the second minimum circumscribed rectangle by using a dense point searching area containing a searching window to obtain dense point defects and isolated point defects.

In a seventh aspect, an embodiment of the present application provides a recognition system, including a camera, an image processing device, and a detection recognition device, where the detection recognition device is configured to implement the steps of the method in any one of the first aspect, the second aspect, or the third aspect.

In an eighth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps of the method of any one of the first aspect, the second aspect, or the third aspect.

The self-adaptive defect detection method provided by the embodiment of the application extracts a target detection area from a reference image of a detected target based on an interested area, cuts the reference image from a preset distance outside the target detection area to completely obtain the target detection area, fills a non-target detection area, calculates a mean image and a variance image of the filled image to be detected, calculates a high threshold value image and a low threshold value image through a high threshold value scale coefficient, a low threshold value scale coefficient, a minimum reference deviation and a maximum gray variance to obtain a high threshold value image and a low threshold value image, respectively makes a difference image with the filled image to be detected through the high threshold value image and the low threshold value image to obtain a high threshold value connected area and a low threshold value connected area, and finally screens a part of the low threshold value connected area, which has intersection with the high threshold value connected area, to obtain a defect connected area set. According to the self-adaptive defect detection method provided by the embodiment of the application, the abnormal defect image segmentation of the image with uneven illumination is realized by combining the high and low sensitivity scale coefficients and the forms of local mean values and variances, the segmentation strategy can also realize accurate detection on the abnormal defect with local contrast only having a plurality of gray differences, the problem of image segmentation difficulty caused by uneven brightness is effectively solved, and the accuracy of the abnormal defect detection based on image segmentation is improved. In addition, in order to perform more high-precision screening and grade identification on the defects, the defects are subjected to fusion clustering, and are fused into linear defects, dense point defects and isolated point defects, so that the method has great value for subsequent product quality judgment.

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a schematic flowchart of a method for adaptive defect detection according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram of a target detection area provided in an embodiment of the present application;

fig. 3 is a schematic flowchart of a defect fusion method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an axis in a linear communication domain provided by an embodiment of the present application;

FIG. 5 is a schematic view of the distance between the central axes according to the embodiment of the present application

Fig. 6 is a schematic flowchart of another defect fusion method provided in the embodiment of the present application;

fig. 7 is a schematic diagram of a dense point search area in cluster scanning provided in the embodiment of the present application;

FIG. 8 is a schematic diagram of an adaptive defect detection apparatus according to an embodiment of the present disclosure;

FIG. 9 is a schematic view of a defect fusion apparatus according to an embodiment of the present disclosure;

fig. 10 is a schematic view of another defect fusion apparatus provided in the embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a schematic flowchart of a method for adaptive defect detection according to an embodiment of the present disclosure. The embodiment of the application provides a self-adaptive defect detection method, as shown in fig. 1, comprising the following steps:

step S101, acquiring a reference image of a measured target, and creating a reference coordinate system by taking the measured target in the reference image as an object;

step S102, setting an interested area for a detected target in the reference image, and extracting a target detection area according to the interested area; the region of interest is associated with the reference coordinate system;

s103, acquiring a positive external rectangle of the target detection area, and cutting an image based on the external expansion preset distance of the positive external rectangle to obtain an image to be detected;

step S104, filling a non-target detection area in the image to be detected to obtain a filled image to be detected;

step S105, calculating a mean image and a variance image of the filled image to be detected;

step S106, calculating to obtain a high threshold value image and a low threshold value image according to the mean value image and the variance image and a preset high threshold value scale coefficient, a preset low threshold value scale coefficient, a preset minimum standard deviation and a preset maximum gray variance;

step S107, calculating a high threshold difference image and a low threshold difference image according to the filled image to be detected, the high threshold image and the low threshold image, extracting a high threshold connected domain set from the high threshold difference image, and extracting a low threshold connected domain set from the low threshold difference image; the high threshold difference image is obtained by making a difference between the filled image to be measured and the high threshold image, and the low threshold difference image is obtained by making a difference between the filled image to be measured and the low threshold image;

and S108, screening out a low-threshold connected domain which has intersection with a high-threshold connected domain in the high-threshold connected domain set aiming at the low-threshold connected domain set to obtain a defect connected domain set. Specifically, a reference image of a target to be measured is acquired, and then a reference coordinate system describing the position and orientation of the target object in the reference image is created with the target object as an object for positioning and calculation in subsequent image processing.

For a target detection area of a detected target, an area of interest needs to be set first, the area of interest contains all the target detection area of the detected target, and then the target detection area is extracted by a binarization method. The interested area is associated with a reference coordinate system, and when the pose of the reference coordinate system changes, the interested area changes synchronously.

Fig. 2 is a schematic diagram of a target detection area provided in an embodiment of the present application. In this embodiment, it is intended to detect defects in a window area of a glass cover plate of a mobile phone, and in order to obtain the area, we first generate a quadrangle outside the glass cover plate of the mobile phone in fig. 2 as an ROI, and simultaneously extract the window area by using a binarization method, such as an area surrounded by cross lines on the edge of the glass cover plate of the mobile phone in fig. 2.

In order to reduce the amount of calculation for defect detection, images unrelated to the target detection area should be minimized, but the integrity of the target detection area should be ensured. Therefore, in the embodiment of the application, the right external rectangle of the target detection area is calculated first, then the preset distance is expanded outwards to obtain a new rectangle, and the new rectangle is used for cutting the reference image to obtain the image to be detected. And removing interference generated by uneven brightness in the image in a filling mode for a non-target detection area in the image to be detected to obtain the filled image to be detected.

The method for extracting the abnormal defects comprises the steps of reducing the gray value of a mean image of a filled image to be detected based on a variance image, a high threshold scale coefficient, a low threshold scale coefficient, a minimum standard deviation and a maximum gray variance to obtain a high threshold image and a low threshold image, calculating difference images of the high threshold image, the low threshold image and the filled image to be detected respectively, judging that the abnormal defects possibly exist if a connected domain with the gray value smaller than 0 appears in the difference images, namely, areas smaller than the mean value after the gray value reduction in the filled image to be detected, and screening the areas to determine the abnormal defects. The specific method comprises the following steps:

i) and calculating the mean image and the variance image of the filled image to be detected according to the size of the neighborhood window. The size of the neighborhood window can be determined according to the size of the target defect, generally, the larger the size of the defect is, the larger the size of the neighborhood window is to be set, but the size of the neighborhood window is not sensitive to the size of the defect, but the operation efficiency is influenced, and the larger the window is, the lower the operation efficiency is. The size of the neighborhood window conforming to the measured target can be set according to needs, for example, the size of the neighborhood window can be set to be 11 for a mobile phone cover plate. When the mean image and the variance image of the filled image to be detected are calculated, the integral image can be reasonably used for calculation in order to improve the calculation efficiency;

ii) setting a high threshold scale factor k^highLow threshold scale factor k^lowMinimum reference deviation t_offsetAnd the maximum gray variance sigma of the image to be measured_maxPerforming the high threshold image M by the following formula_t ^highAnd a low threshold image M_t ^lowThe calculation of (2):

where row is the pixel row coordinate, col is the pixel column coordinate,

for the gray threshold where the high threshold image is located at the pixel location (row, col),

for gray threshold values, μ, of low threshold image at pixel position (row, col)_{（row，col）}Is the gray value, σ, of the mean image at the pixel position (row, col)_{（row，col）}Is the gray value of the variance image at the pixel location (row, col). High threshold scale factor k^highAnd a low threshold coefficient k^lowThe value range of (1) is (0).

The setting of the high threshold scale coefficient and the low threshold scale coefficient affects the detection precision of the abnormal defect, and the smaller the two scale coefficients are, the higher the detection precision is. For example, the high threshold scaling factor may be set to 0.1 and the low threshold scaling factor may be set to 0.05. The minimum reference deviation is set to avoid instability due to the gray variance approaching 0 in the case where the average gray of the image is very small, and optionally, the minimum reference deviation is set to-2. Maximum gray variance σ_maxThe influence on the detection effect is small, the maximum can be set to 128, and the default is usually setIs set to 128;

iii) passing the filled image under test and the high threshold image M_t ^highPerforming pixel-by-pixel difference to obtain a high threshold difference image; through the filled image to be measured and the low threshold value image M_t ^lowAnd carrying out difference pixel by pixel to obtain a low threshold difference image. Then extracting connected domains with the gray values smaller than 0 from the high-threshold difference image to obtain a high-threshold connected domain set, and extracting connected domains with the gray values smaller than 0 from the low-threshold image to obtain a low-threshold connected domain set;

iv) searching the intersection of the high-threshold connected domain set and the low-threshold connected domain set, and taking the low-threshold connected domain containing the intersection of the high-threshold connected domain set and the low-threshold connected domain set as a defect connected domain set. Illustratively, the final eligible connected domain is extracted from the set of low threshold connected domains. The calculation method comprises the steps of sequentially traversing each connected domain in the low-threshold connected domain set, judging whether the connected domain has intersection with the high-threshold connected domain, if not, removing the connected domain from the low-threshold connected domain set, and recording the remaining low-threshold connected domains as a defect connected domain set after the removal is finished

。

The method is used for detecting darker defects, and if brighter defects are detected, the filled image to be detected is subjected to gray inversion and then subsequent detection.

In some embodiments, the step S101 of acquiring a reference image of a target to be measured, and creating a reference coordinate system with the target to be measured in the reference image as an object includes:

step a1, acquiring an image of a detected target as a reference image;

step a2, using the measured target in the reference image as the object, and creating a reference coordinate system by a preset coordinate system creation method; the preset coordinate system creation method comprises straight line fitting or template matching.

Specifically, the method for creating the reference coordinate system may be line fitting, template matching, or other methods, and may be selected according to adaptability to the target to be measured. Taking a measured object as a mobile phone cover plate as an example, a reference coordinate system can be created by adopting a straight line fitting method, fitting a straight line by utilizing the vertical edge of the cover plate of the mobile phone cover plate image, fitting a straight line by utilizing the upper edge of the cover plate of the mobile phone cover plate image, establishing the reference coordinate system by utilizing the intersection point of the two straight lines as a coordinate origin and taking any one of the two straight lines as a horizontal coordinate axis or a vertical coordinate axis.

In some embodiments, the step S104 of filling the non-target detection area in the image to be detected to obtain a filled image to be detected includes:

b1, determining a filling mode of a non-target detection area in the image to be detected according to the gray level characteristics of the defects in the image to be detected; the filling mode comprises gray level mirror image filling and fixed gray level filling;

and b2, filling the non-target detection area in the image to be detected according to the filling mode to obtain the filled image to be detected.

Specifically, there are three main cases of defect gray scale characteristics: the neighborhood is always dark, the neighborhood is always bright, and the neighborhood grayscale is uncertain.

When the defect gray scale is characterized in that the neighborhood is always darker, a method of fixing gray scale values can be adopted to fill the non-target detection area with the fixed gray scale value (such as 0) lower than the defect gray scale; when the defect gray scale is characterized in that the neighborhood is always brighter, a method of fixing gray scale values can be adopted to fill the non-target detection area with the fixed gray scale value (such as 255) higher than the defect gray scale value; when the defect gray scale is characterized in that the neighborhood gray scale is uncertain, a gray scale mirror image method can be adopted to mirror-copy the gray scale image of the adjacent part of the non-target detection area.

Fig. 3 is a schematic flowchart of a defect fusion method according to an embodiment of the present application. The embodiment of the application also provides a defect fusion method, which is used for further defect classification and aggregation of the defect connected domain set obtained by defect detection. The defect connected domain set is extracted based on the target image to be detected, and the method for extracting the defect connected domain set based on the target image to be detected includes, but is not limited to, the above adaptive defect detection method. As shown in fig. 3, the method includes:

step S201, extracting linear connected domains from the defect connected domain set, and marking the remaining defect connected domains as point connected domains;

s202, two linear connected domains are randomly selected from all linear connected domains to serve as linear connected domain pairs, and a first angle and a first distance between central axes of the two linear connected domains in each linear connected domain pair are calculated; the first distance includes at least one of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance;

step S203, clustering is carried out on each linear connected domain pair according to a first angle and a first distance corresponding to the linear connected domain pair to obtain a linear connected domain set corresponding to the linear connected domain pair;

step S204, aiming at each linear connected domain set, calculating a minimum external rectangle corresponding to the linear connected domain set, and merging the dot-shaped connected domains with the distance from the minimum external rectangle to the minimum external rectangle being less than a preset threshold value into the linear connected domain set to obtain linear defects and residual dot-shaped connected domains;

and S205, clustering the residual punctate connected domains to obtain dense point defects and isolated point defects.

Specifically, in order to perform more accurate screening and grade determination on the defects so as to facilitate quality analysis of the detected target after defect detection, after the defect connected domain set of the detected target is obtained, fusion clustering can be performed on the defect connected domains in the defect connected domain set.

Firstly, calculating the linearity of each connected domain in the defect connected domain set, marking the connected domain with the linearity larger than a linearity threshold value as a linear connected domain, and marking the remaining connected domains with the linearity smaller than the linearity threshold value as point connected domains. The linearity is calculated by using the second-order center distance of all pixel positions in the defect connected domain, the value range is (0, 1), the larger the value is, the better the linearity is, and the linearity calculation formula is as follows:

l=[(m₂₀-m₀₂)²+4*m₁₁*m₁₁]/(m₂₀+m₀₂)²；

wherein l represents the linearity of the defect connected domain, m₂₀Is the central second moment in the vertical direction, m₀₂Is the central second step distance, m, of the horizontal direction₁₁The second order moment is the comprehensive central second order moment of the horizontal direction and the vertical direction, and the calculation formula of the second order moment is as follows:

；

Then, performing fusion clustering of the linear connected domain, specifically comprising the following steps:

i) two linear connected domains are arbitrarily selected as a pair of linear connected domains, central axes are respectively calculated for the two linear connected domains in each group of linear connected domain pairs, fig. 4 is a schematic diagram of the central axes of the linear connected domains provided by the application, as shown in fig. 4, two curves represent the two linear connected domains, and two straight dotted lines are the central axes of the two linear connected domains respectively. Next, a first angle and a first distance between the central axes of the two linearly connected domains are calculated. The first angle theta is the angle between the extensions of the two central axes. Illustratively, the first distance includes a vertical distance, a translation distance, a relative distance, an offset distance and a straight-line distance, and fig. 5 is a schematic diagram of the distance between the central axes provided by the embodiment of the present application, as shown in fig. 5, wherein the vertical distance d₁Is the central axis l₁Near the central axis l₂End point of (d) towards the central axis l₂(or the central axis l₂Extension of) the length of the perpendicular drawn; distance d of translation₂Is the central axis l₁Near the central axis l₂End point of (d) towards the central axis l₂(or the central axis l₂Extension line of) from the foot to the central axis l by making a perpendicular line₂Near the central axis l₁The distance between the end points of (a); the relative distance is the central axis l₁Length and translation distance d of₂The ratio of (A) to (B); offset distance d₃Is the central axis l₁Away from the central axis l₂End point of (d) towards the central axis l₂(or the central axis l₂Extension of) the length of the perpendicular drawn; linear distance d₄Is the central axis l₁And the neutral axis l₂The distance between the end points close to each other. The above-mentioned central axis l₁And l₂Only in order to distinguish the two central axes, when the first distance is actually calculated, the two central axes need to be mutually subjected to the first distance, and a group of distance data with relatively small numerical values is selected as the first distance.

ii) clustering each group of linear connected domain pairs, and merging the linear connected domains meeting the first angle and the first distance together to obtain a linear connected domain set. If an intersection exists between the two linear connected domain sets after combination, combining the two linear connected domain sets;

iii) calculating a minimum circumscribed rectangle for each linear connected domain set, sequentially calculating the distance from each point-like connected domain to each minimum circumscribed rectangle, and if the minimum circumscribed rectangle exists, merging the current point-like connected domain into the linear connected domain set corresponding to the minimum circumscribed rectangle with the closest distance, wherein the distance between the current point-like connected domain and the minimum circumscribed rectangle is smaller than a preset threshold value; and if the minimum external rectangle with the distance between the current point-like connected domain and the current point-like connected domain smaller than the preset threshold value does not exist, determining the current point-like connected domain as the residual point-like connected domain.

After the steps are completed, a plurality of linear defects and residual point-like connected domains are obtained, then the residual point-like connected domains are clustered, the plurality of residual point-like connected domains which are distributed densely are combined into a dense point defect, and other residual point-like connected domains which are distributed dispersedly are determined as isolated point defects.

In some embodiments, the step S203, for each pair of linear connected components, clustering according to the first angle and the first distance corresponding to the pair of linear connected components to obtain a set of linear connected components corresponding to the pair of linear connected components, includes:

step c1, calculating a second angle and a second distance between the central axis of any other linear connected domain and the central axis of the linear connected domain associated with all the linear connected domain pairs, and judging whether at least one group of the second angle and the second distance are respectively smaller than or equal to the first angle and the first distance corresponding to the linear connected domain pair; the second distance includes the same distance type as the first distance;

and c2, if at least one group of the second angle and the second distance is smaller than or equal to the first angle and the first distance corresponding to the pair of linear connected domains, merging the other linear connected domains into the set of linear connected domains corresponding to the pair of linear connected domains, and associating the other linear connected domains with the pair of linear connected domains.

Specifically, when the corresponding first angle and first distance are clustered according to the linear connected domain, the central axis l of any one of the other linear connected domains is selected first₃Then respectively calculating the central axis l₃Axis l in two linear connected domains aligned with linear connected domain₁And l₂The second angle and the first angle are the same angle, and the distance type included in the second distance is the same as the distance type included in the first distance. If the central axis l₃And the central axis l₁And l₂The second angle of any one of the first and second angles is less than or equal to the first angle, and the second distance is less than or equal to the first distance, then the central axis l is connected to₃And adding the corresponding linear connected domain into the linear connected domain set corresponding to the linear connected domain pair. And the central axis l₃Also participate in subsequent clustering, i.e. the central axis of the next selected other linear connected domain is calculated except for the central axis l₁And l₂In addition to the second angle and the second distance, also calculate the angle l with the central axis₃And (4) the second angle and the second distance, and the steps are circulated until the clustering judgment of all other linear connected domains is completed.

In some embodiments, the step S205 of clustering the remaining dotted connected domains to obtain dense point defects and isolated point defects includes:

d1, generating a rectangular area with a preset size by taking each residual dotted connected domain as a center, and determining that the rectangular area containing the residual dotted connected domains in excess of a preset number is a dense rectangular area;

and d2, merging the dense rectangular areas containing the same residual dot-shaped connected domains, clustering all the residual dot-shaped connected domains in each merged dense rectangular area into a dense point defect, and determining the residual dot-shaped connected domains not contained in the dense rectangular area as isolated point defects.

Specifically, when performing clustering on the remaining dot-like connected domains, a dense dot rule is defined, that is, if the number of remaining dot-like connected domains included in a rectangular region with a preset size created with one remaining dot-like connected domain as a center exceeds a preset number, the rectangular region is determined to be a dense rectangular region. The preset size and the preset number need to be set according to the defect classification criteria of the product, for example, the preset size of the mobile phone cover plate may be set to 100 × 100, and the preset number is 5.

Then, a rectangular area with a preset size is created by taking each remaining point-like connected domain as a center, the number of the remaining point-like connected domains in each rectangular area is counted, and the rectangular areas with the number exceeding the preset number are marked as dense rectangular areas.

And then, checking whether the same residual dot-shaped connected domains exist between the dense rectangular regions, namely whether an intersection exists, if so, combining the two dense rectangular regions into one dense rectangular region until all the dense rectangular regions are independent, and clustering the residual dot-shaped connected domains in each independent dense rectangular region to serve as a dense dot defect. And the remaining point-like connected domains which are not subjected to dense point clustering are isolated point defects.

Fig. 6 is a schematic flowchart of another defect fusion method provided in the embodiment of the present application. The embodiment of the application also provides a defect fusion method, which is used for further defect classification and aggregation of the defect connected domain set obtained by defect detection. The defect connected domain set is extracted based on the target image to be detected, and the method for extracting the defect connected domain set based on the target image to be detected includes, but is not limited to, the above adaptive defect detection method. As shown in fig. 6, the method includes:

s301, screening defect connected domains in the defect connected domain set according to linearity to obtain linear connected domains and dotted connected domains;

s302, selecting a plurality of groups of linear connected domains from the linear connected domains, and clustering according to the angle and the distance between the central axes of the linear connected domains in each group of linear connected domains to obtain a linear connected domain set; each group of linear connected domains comprises two linear connected domains; the distance includes at least one of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance; preferably, the distance includes a vertical distance, a translational distance, a relative distance, an offset distance, and a straight distance.

Step S303, calculating a first minimum circumscribed rectangle of each linear connected domain set, and merging the dot-shaped connected domains with the distance between the dot-shaped connected domains and the edge of the first minimum circumscribed rectangle being smaller than a preset threshold value into the linear connected domain set to obtain linear defects and residual dot-shaped connected domains;

step S304, according to the coordinates of the central point of each remaining point-like connected domain, creating a second minimum circumscribed rectangle containing the central points of all the remaining point-like connected domains, and according to the length of the second minimum circumscribed rectangle and the width of a search window with a preset size, generating a dense point search region;

step S305, scanning the second minimum circumscribed rectangle with a dense point search area including a search window to obtain a dense point defect and an isolated point defect.

Specifically, defect connected domains in the defect connected domain set are screened according to the linearity, the defect connected domains with the linearity reaching a linearity threshold value are determined as linear connected domains, and the defect connected domains with the linearity not reaching the linearity threshold value are determined as point connected domains.

And obtaining a plurality of groups of linear connected domains by randomly selecting two linear connected domains for many times, calculating the angle and the distance of the central axis of the two linear connected domains in each group of linear connected domains, and clustering each group of linear connected domains according to the angle and the distance to obtain a linear connected domain set. Drawing a minimum circumscribed rectangle, namely a first minimum circumscribed rectangle, for each linear connected domain set, merging the dot-shaped connected domains which are less than a preset threshold value from the edge of the first minimum circumscribed rectangle into the linear connected domain set, and determining the linear connected domain set after the merging of the dot-shaped connected domains as a linear defect.

And then carrying out dense aggregation on the remaining point-like connected domains, wherein the dense aggregation of the remaining point-like connected domains firstly needs to determine a minimum circumscribed rectangle containing all the center points of the remaining point-like connected domains, namely a second minimum circumscribed rectangle, according to the coordinates of the center point of each remaining point-like connected domain. Then, a dense point search area with the length of the second minimum circumscribed rectangle and the width of a search window with a preset size is created in the second minimum circumscribed rectangle, and a search window with a preset size sliding left and right exists in the dense point search area. And performing aggregation scanning by moving the dense point search area up and down in the second minimum circumscribed rectangle and moving the search window left and right in the dense point search area to find out the dense point defects, wherein the residual dot-like connected domains which are not aggregated in the dense point defects are used as isolated point defects.

In some embodiments, the step S305 of scanning the second minimum bounding rectangle with the dense point search area including the search window to obtain the dense point defect and the isolated point defect includes:

e1, arranging the dense point searching area at the upper end of the inside of the second minimum circumscribed rectangle, and scanning the dense point searching area from left to right by the searching window to obtain a dense point area at the current position of the dense point searching area;

step e2, if the current position of the dense point search area includes at least one dense point area, moving the dense point search area downwards to a first target position, and scanning the dense point search area from left to right with the search window; the first target position is the central point of a residual dot-shaped connected domain which is closest to the lower part of the current position of the dense point searching region after the upper edge of the dense point searching region moves to;

step e3, if the current position of the dense point searching area does not include a dense point area, moving the dense point searching area downwards to a second target position, and scanning the dense point searching area from left to right by the searching window; the second target position is the central point of a residual dot-shaped connected domain which is closest to the current position of the upper edge after the upper edge of the dense point searching region moves to the current position of the upper edge;

step e4, when the scanning of the second minimum circumscribed rectangle is completed, determining the remaining dot-like connected domains in each scanned dense dot region as a dense dot defect, and determining the remaining dot-like connected domains not in the dense dot region as isolated dot defects.

Fig. 7 is a schematic diagram of a dense point search area in cluster scanning provided in an embodiment of the present application. Specifically, in the second minimum bounding rectangle, the dense point search area slides from top to bottom for cluster scanning, and the sliding rule of the dense point search area is as follows:

i) the initial position of the dense point searching area is that the upper edge of the dense point searching area is superposed with the upper edge of the second minimum circumscribed rectangle;

ii) after each movement of the dense point search area, scanning the internal search window once from left to right;

iii) at the current position of the dense point search area, after the search window finishes scanning from left to right, according to whether the dense point area is scanned in the whole process, as shown in the left side of fig. 7, if at least one dense point area is scanned, moving down the dense point search area (left one in fig. 7) to enable the upper edge of the moved dense point search area to be located at the position of the central point of the nearest remaining point-like communication domain below the dense point search area before moving (left two in fig. 7); as shown in fig. 7 right, if the dense point area is not scanned, the dense point search area (right (r)) of fig. 7) is moved down, and the upper edge of the dense point search area is moved down to the position of the center point of the remaining point-like connected domain nearest below the current position of the upper edge (right ((r)) of fig. 7.

And moving the dense point search area according to the sliding rule until the scanning of the whole second minimum circumscribed rectangle is finished, if a plurality of dense point areas are found, extracting the residual dot-shaped connected domains in the dense point areas, clustering the residual dot-shaped connected domains in each dense point area together to determine the dense dot-shaped connected domain, and scanning again until no new dense point area exists after the complete second minimum circumscribed rectangle is scanned. And after the scanning is stopped, determining the residual dot-shaped connected domains still in the second minimum circumscribed rectangle as isolated dot-shaped connected domains.

In some embodiments, in the step e1, the step of disposing the dense point search area at the upper end of the inside of the second minimum circumscribed rectangle, and scanning the dense point search area from left to right with the search window to obtain the dense point area at the current position of the dense point search area includes:

step f1, moving the search window to the position where the left edge of the search window is located at the center point of the first remaining point-like connected domain from left to right in the dense point search area, and judging whether the number of the center points of the remaining point-like connected domains contained in the search window exceeds a preset number;

step f2, if the number of the center points of the remaining point-like connected domain contained in the search window exceeds the preset number, determining the current position of the search window as a dense point region, moving the search window to the third target position rightwards, and judging whether the number of the center points of the remaining point-like connected domain contained in the search window at the current position exceeds the preset number; the third position is that the left edge of the search window moves to the center point of a nearest residual point-like connected domain on the right side of the current position of the search window;

step f3, if the number of the center points of the remaining point-like connected domain contained in the search window does not exceed the preset number, moving the search window to the fourth target position to judge whether the number of the center points of the remaining point-like connected domain contained in the search window at the current position exceeds the preset number; the fourth target position is a center point of a residual dotted connected domain which is closest to the right of the current position of the left edge after the left edge of the search window moves;

step f4, when the scanning of the current position of the dense point searching area is completed, taking all the scanned dense point areas as the dense point areas of the current position of the dense point searching area.

Specifically, after the dense point search area is set or moved, the dense point area of the dense point search area needs to be scanned by sliding the search window from left to right in the dense point search area, and if the determination condition of the dense point area is that the number of the center points of the remaining punctate communication domains in the search window exceeds a preset number, it is determined that the area included in the current search window is a dense point area. The sliding rule of the search window in the dense point search area is as follows:

i) after the dense point search area is set or moved, the initial scanning position of the search window is that the left edge of the search window is superposed with the central point of the residual point-shaped communication area at the leftmost position in the current position of the dense point search area;

ii) the search window slides from left to right, and the calculation of the sliding step length is influenced by whether the current position is judged to be a dense point region: if the current position of the search window is determined as a dense point area, the sliding step length of the search window is the distance from the center point of the remaining point-like communication domain with the nearest distance to the right side of the right edge of the current position of the search window to the left edge of the search window; if the current position of the search window is not determined as the dense point area, the sliding step length of the search window is the distance from the center point of the residual point-shaped connected domain with the nearest distance to the left edge of the search window on the right side of the left edge of the current position of the search window to the left edge of the search window.

And sliding the search window according to the sliding rule until the search window slides to the rightmost end in the dense point search area, finishing scanning of the current position of the dense point search area, moving the dense point search area, and resetting the position of the search window to the leftmost end of the dense point search area to perform scanning according to the sliding rule.

Fig. 8 is a schematic diagram of an adaptive defect detection apparatus according to an embodiment of the present application. An embodiment of the present application provides a self-adaptive defect detection apparatus, as shown in fig. 8, including:

an obtaining module 40, configured to obtain a reference image of a target to be measured, and create a reference coordinate system with the target to be measured in the reference image as an object;

an extraction module 41, configured to set an area of interest for a target to be detected in the reference image, and extract a target detection area according to the area of interest; the region of interest is associated with the reference coordinate system;

the cutting module 42 is configured to obtain a positive circumscribed rectangle of the target detection area, and perform image cutting based on a preset distance of outward expansion of the positive circumscribed rectangle to obtain an image to be detected;

a filling module 43, configured to fill a non-target detection area in the to-be-detected image to obtain a filled to-be-detected image;

a first calculating module 44, configured to calculate a mean image and a variance image of the filled image to be measured;

a second calculating module 45, configured to calculate to obtain a high threshold image and a low threshold image according to the mean image and the variance image, and a preset high threshold scale coefficient, a preset low threshold scale coefficient, a preset minimum standard deviation, and a preset maximum gray variance;

a third calculating module 46, configured to calculate a high-threshold difference image and a low-threshold difference image according to the filled image to be measured, the high-threshold image, and the low-threshold image, extract a high-threshold connected domain set from the high-threshold difference image, and extract a low-threshold connected domain set from the low-threshold difference image; the high threshold difference image is obtained by making a difference between the filled image to be measured and the high threshold image, and the low threshold difference image is obtained by making a difference between the filled image to be measured and the low threshold image;

and the screening module 47 is configured to screen out, for the low-threshold connected domain set, a low-threshold connected domain having an intersection with a high-threshold connected domain in the high-threshold connected domain set, so as to obtain a defect connected domain set.

Fig. 9 is a schematic view of a defect fusion apparatus according to an embodiment of the present application. An embodiment of the present application provides a defect fusion apparatus, which can be applied to the above-mentioned adaptive defect detection apparatus to further classify and aggregate defects of a defect connected domain set obtained by defect detection, as shown in fig. 9, including:

a marking module 50, configured to extract a linear connected domain from the defect connected domain set, and mark the remaining defect connected domains as point connected domains;

the calculation module 51 is configured to arbitrarily select two linear connected domains from all the linear connected domains as a linear connected domain pair, and calculate a first angle and a first distance between central axes of the two linear connected domains in each linear connected domain pair; the first distance includes at least one of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance; preferably, the first distance includes a vertical distance, a translational distance, a relative distance, an offset distance, and a straight distance;

a clustering module 52, configured to cluster, for each pair of linear connected domains, a first angle and a first distance corresponding to the pair of linear connected domains to obtain a set of linear connected domains corresponding to the pair of linear connected domains;

a linear defect module 53, configured to calculate, for each linear connected domain set, a minimum circumscribed rectangle corresponding to the linear connected domain set, and merge a dotted connected domain whose distance from the minimum circumscribed rectangle is smaller than a preset threshold into the linear connected domain set, so as to obtain a linear defect and remaining dotted connected domains;

and a dotted defect module 54, configured to cluster the remaining dotted connected domains to obtain dense point defects and isolated point defects.

Fig. 10 is a schematic view of another defect fusion apparatus provided in the embodiment of the present application. The embodiment of the present application provides a defect fusion apparatus, which can be applied to the above-mentioned adaptive defect detection apparatus to further classify and aggregate defects of a defect connected domain set obtained by defect detection, as shown in fig. 10, including:

the classification module 60 is configured to screen the defect connected domains in the defect connected domain set according to linearity to obtain a linear connected domain and a dotted connected domain;

the line clustering module 61 is used for selecting a plurality of groups of linear connected domains from the linear connected domains, and clustering according to the angle and the distance between the central axes of the linear connected domains in each group of linear connected domains to obtain a linear connected domain set; each group of linear connected domains comprises two linear connected domains; the distance includes at least one of a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance; preferably, the distance includes a vertical distance, a translational distance, a relative distance, an offset distance, and a linear distance;

a merging module 62, configured to calculate, for each linear connected domain set, a first minimum circumscribed rectangle of the linear connected domain set, and merge a dotted connected domain whose distance from an edge of the first minimum circumscribed rectangle is smaller than a preset threshold into the linear connected domain set, so as to obtain a linear defect and remaining dotted connected domains;

a creating module 63, configured to create a second minimum circumscribed rectangle including all the remaining dotted-connected domain center points according to the coordinates of each remaining dotted-connected domain center point, and generate a dense point search region according to the length of the second minimum circumscribed rectangle and the width of a search window of a preset size;

the scanning module 64 is configured to scan the second minimum circumscribed rectangle by using the dense point search region including the search window to obtain a dense point defect and an isolated point defect.

Corresponding to the adaptive defect detection method in fig. 1, the defect fusion method in fig. 3, and the defect fusion method in fig. 6, an embodiment of the present application further provides an identification system, which includes a camera, an image processing device, and a detection and identification device, where the detection and identification device is used to implement the adaptive defect detection method and the two defect fusion methods, so as to solve the problem of how to perform image segmentation on abnormal defects of an image with uneven brightness in the prior art.

Corresponding to an adaptive defect detection method in fig. 1, an adaptive defect fusion method in fig. 3, and a defect fusion method in fig. 6, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, where the computer program is executed by a processor to perform the steps of the adaptive defect detection method and the two defect fusion methods.

Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc., when a computer program on the storage medium is executed, the above-mentioned adaptive defect detection method and two defect fusion methods can be executed, so as to solve the problem of image segmentation on abnormal defects of uneven-brightness images in the prior art, the defect detection method proposed in the embodiment of the present application realizes the abnormal defect image segmentation on uneven-illumination images by combining high-low sensitivity scale coefficients and the forms of local mean and variance, the segmentation strategy can also realize accurate detection on abnormal defects with local contrast having only a few gray differences, effectively overcomes the problem of image segmentation difficulty caused by uneven brightness, improves the accuracy of abnormal defect detection based on image segmentation, and performs cluster fusion on defects, to facilitate higher accuracy screening and grade determination.

In the embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus once an item is defined in one figure, it need not be further defined and explained in subsequent figures, and moreover, the terms "first", "second", "third", etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.

Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the present disclosure, which should be construed in light of the above teachings. Are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An adaptive defect detection method, comprising:

2. The method of claim 1, wherein the obtaining a reference image of the target object and creating a reference coordinate system with the target object in the reference image as an object comprises:

acquiring an image of a detected target as a reference image;

3. The method of claim 1, wherein the filling the non-target detection area in the image to be detected to obtain the filled image to be detected comprises:

4. The method of claim 1, wherein the computing the mean image and the variance image of the filled image under test comprises:

5. The method of claim 1, wherein extracting a high-threshold set of connected components from the high-threshold difference image and extracting a low-threshold set of connected components from the low-threshold difference image comprises:

6. The method of claim 1, wherein the screening out, for the set of low-threshold connected domains, low-threshold connected domains that intersect with a high-threshold connected domain in the set of high-threshold connected domains to obtain a set of defect connected domains comprises:

7. The method of claim 1, wherein the screening out, for the set of low-threshold connected domains, low-threshold connected domains that intersect with a high-threshold connected domain in the set of high-threshold connected domains to obtain a set of defect connected domains comprises:

8. The method of claim 1, further comprising, prior to said computing the mean image and the variance image of the filled image under test:

determining the defect type of the defect in the target detection area;

9. The method of claim 8, further comprising:

10. An adaptive defect detection apparatus, comprising:

11. A recognition system comprising a camera, image processing means and detection recognition means, characterized in that said detection recognition means are adapted to implement the steps of the method according to any of the preceding claims 1-9.

12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of the preceding claims 1-9.