CN115393290A

CN115393290A - Edge defect detection method, device and equipment

Info

Publication number: CN115393290A
Application number: CN202210946591.6A
Authority: CN
Inventors: 徐卫东
Original assignee: Shenzhen Xinshizhi Technology Co ltd
Current assignee: Shenzhen Xinshizhi Technology Co ltd
Priority date: 2022-08-09
Filing date: 2022-08-09
Publication date: 2022-11-25

Abstract

The embodiment of the invention discloses an edge defect detection method, an edge defect detection device and computer equipment, wherein the method comprises the following steps: acquiring a target image of a cover plate to be detected, wherein the target image is an image of the cover plate to be detected; based on a first defect detection method, performing defect detection on the target image to obtain a first edge defect in the target image; classifying the defects of the first edge defects in the target image based on a second defect detection method, and determining second edge defects in the target image; the second defect detection method is a defect detection method based on deep learning; and screening the first edge defect and the second edge defect, and determining a defect detection result of the cover plate to be detected. By adopting the method and the device, the accuracy of detecting the edge defects of the mobile phone cover plate can be improved.

Description

Edge defect detection method, device and equipment

Technical Field

The invention relates to the technical field of industrial automation machine vision detection, in particular to an edge defect detection method, an edge defect detection device, computer equipment and a computer readable storage medium.

Background

With the rapid development of manufacturing industry, the requirements of people on the quality of products are increasingly improved, and the surface quality of the products has more and more important influence on the commercial value of the products. The surface defect detection becomes an important link in the product quality control process, and the product quality and the production efficiency can be effectively improved. Surface defect detection is widely used in the fields of cloth flaw detection, workpiece surface quality detection, aerospace and the like. The traditional algorithm can work well for occasions with rule defects and simpler scenes, but is not applicable to occasions with unobvious characteristics, various shapes and more disordered scenes. In recent years, machine learning based recognition algorithms have become more sophisticated and many companies have begun to try to apply deep learning algorithms to industrial applications.

The machine vision surface defect detection technology based on optical image sensing gradually replaces a manual visual detection method, becomes an important means for surface defect detection, and has the advantages of automation, non-contact, high speed, high precision, good stability and the like. The appearance of the technology greatly improves the efficiency of production operation, avoids influencing the accuracy of detection results due to operation conditions, subjective judgment and the like, realizes better and more accurate surface defect detection and more rapid identification of the surface defect of a product. The detection of the surface defect of the product belongs to one of machine vision technologies, namely, the function of simulating human vision by using computer vision is utilized to acquire, process and calculate images from specific real objects and finally carry out actual detection, control and application. The surface defect detection of the product is an important part of machine vision detection, and the accuracy of the detection directly influences the final quality of the product.

In the field of industrial inspection, edge detection is an important part of inspecting product defects. However, due to the edge diversity of the product, the 2.5D arc surface or the 3D arc surface, or the conditions of the R angle, the U-shaped groove, the S-shaped angle and the like after the process grinding after the laser cutting, the difficulty of edge detection is increased. In addition, most of the edge detection is performed in a backlight field, and a conventional digital image processing technology is generally applied. However, only products which are easy to image optically, such as edge defects, can be processed in a backlight field, and the detection of more process morphological defects cannot be dealt with. In the related technical solution of edge detection, due to the background interference or the lack of algorithm precision, the accuracy of edge detection is not high, for example, there is a significant deficiency in the accuracy of edge defect detection for a mobile phone cover plate.

Disclosure of Invention

In view of the foregoing, it is necessary to provide an edge defect detection method, apparatus, computer device and computer readable storage medium.

In a first aspect of the invention, there is provided a method of edge defect detection, the method comprising:

acquiring a target image of a cover plate to be detected, wherein the target image is an image of the cover plate to be detected;

based on a first defect detection method, performing defect detection on the target image to obtain a first edge defect in the target image;

classifying the defects of the first edge defects in the target image based on a second defect detection method, and determining second edge defects in the target image; the second defect detection method is a defect detection method based on deep learning;

and screening the first edge defect and the second edge defect, and determining a defect detection result of the cover plate to be detected.

Optionally, the method further includes:

preprocessing the target image based on a preset image filtering algorithm; the preset image filtering algorithm comprises one or more combinations of mean filtering, median filtering, markov random field filtering, simulated annealing filtering, dynamic threshold segmentation, linear filtering, mathematical morphology filtering, gradient processing, self-adaptive filtering and Hilbert filtering.

Optionally, the target image is a grayscale image; after the step of obtaining the target image of the cover plate to be detected, the method further comprises the following steps: and obtaining an ROI (region of interest) in the target image, wherein the ROI comprises ROI image areas corresponding to the vertical edge, the horizontal edge and the R-angle edge of the cover plate to be detected in the target image.

Optionally, the step of performing defect detection on the target image based on the first defect detection method to obtain a first edge defect in the target image further includes: performing first defect detection on ROI image areas of vertical edges and horizontal edges of the cover plate to be detected to obtain first edge defects in the target image, wherein the first edge defects comprise edge defects in the vertical direction and/or the horizontal direction; and performing first defect missing on the edge of the R corner of the cover plate to be detected to obtain a first edge defect in the target image, wherein the first edge defect comprises an edge defect on the R corner.

Optionally, the step of performing first defect detection on the ROI image region of the vertical edge and the horizontal edge of the cover plate to be detected to obtain a first edge defect in the target image further includes: according to a preset first gray threshold, carrying out binarization processing on the target image to obtain a binarization image corresponding to the target image; performing expansion processing on the binary image corresponding to the target image to obtain a corresponding first mask image; performing edge detection on the target image to obtain a first defect image in the target image; carrying out 8-neighborhood contour tracking processing on the target image, determining at least one inflection point, and carrying out fitting operation on the at least one inflection point; performing defect detection based on the fitting result to obtain a second defect image; and determining a first defect feature in the vertical direction and/or the horizontal direction based on the first defect image, the second defect image and the first mask image.

Optionally, the step of performing a first defect missing on the R-corner edge of the cover plate to be detected to obtain a first edge defect in the target image further includes: performing first nonlinear gradient processing on the target image to obtain a first nonlinear gradient map; wherein the first non-linear gradient process is a kernel 15 x 15 non-linear gradient process; performing binarization processing on the first nonlinear gradient map according to a preset first gradient threshold value to obtain a binarization image corresponding to the first nonlinear gradient map; performing defect merging processing according to a preset first pixel radius on the basis of a binary image corresponding to the nonlinear gradient map, and determining a second mask image according to an image after defect merging; performing second nonlinear gradient processing on the target image to obtain a second nonlinear gradient map; wherein the second non-linear gradient treatment is a kernel 3 x 3 non-linear gradient treatment; performing binarization processing on the second nonlinear gradient map according to a preset second gradient threshold value to obtain a binarization image corresponding to the second nonlinear gradient map; performing contour tracking processing and cavity detection processing on the binary image corresponding to the second nonlinear gradient map, and performing defect merging processing on the basis of a preset second pixel radius to obtain a third defect image; determining a first edge defect at the R corner based on a third defect image and the second mask image.

Optionally, the deep learning model is obtained by training a clip thumbnail containing a defect of a training sample; the deep learning model is a Squeezenet model, the Squeezenet model comprises a Squeeze layer and an expanded layer, the Squeeze layer comprises a convolution layer of 1 × 1 convolution kernel, and the expanded layer comprises a convolution layer of 1 × 1 convolution kernel and a convolution layer of 3 × 3 convolution kernel; the step of classifying the first edge defect in the target image based on the second defect detection method, and determining the second edge defect in the target image, further includes: inputting the first edge defect into the Squeeze layer, then inputting the output result of the Squeeze layer into the Expand layer, and taking the output result of the Expand layer as the second edge defect.

Optionally, the step of screening the first edge defect and the second edge defect to determine the defect detection result of the cover plate to be detected further includes: and screening the first edge defect and the second edge defect based on preset feature screening expressions corresponding to different classification defects, and determining a defect detection result under each classification defect.

In a second aspect of the present invention, there is provided an edge defect detecting apparatus, the apparatus comprising:

the device comprises an image acquisition module, a detection module and a processing module, wherein the image acquisition module is used for acquiring a target image of a cover plate to be detected, and the target image is an image of the cover plate to be detected;

the first defect detection module is used for carrying out defect detection on the target image based on a first defect detection method to obtain a first edge defect in the target image;

the second defect detection module is used for classifying the defects of the first edge defects in the target image based on a second defect detection method and determining second edge defects in the target image; the second defect detection method is a defect detection method based on deep learning;

and the defect fusion screening module is used for screening the first edge defect and the second edge defect and determining the defect detection result of the cover plate to be detected.

In a third aspect of the invention, there is provided a computer apparatus comprising a processor and a memory for storing a computer program; the processor is configured to perform the steps of the edge defect detection method according to the first aspect as described above according to the computer program.

In a fourth aspect of the invention, a computer-readable storage medium is provided, for storing a computer program for performing the steps of the edge defect detection method according to the first aspect.

By adopting the embodiment of the invention, the following beneficial effects are achieved:

after the edge defect detection method, the edge defect detection device, the computer equipment and the computer readable storage medium are adopted, under the condition that the edge characteristics of the glass cover plate of the mobile phone need to be detected, a corresponding target image is obtained firstly, then the defect detection is carried out on the target image based on a first defect detection method, and a first edge defect in the target image is obtained; classifying the defects of the first edge defect in the target image based on a second defect detection method, and determining a second edge defect in the target image; the second defect detection method is a defect detection method based on deep learning; and screening the first edge defect and the second edge defect, and determining a defect detection result of the cover plate to be detected. That is, through the edge defect detection method, the edge defect detection device, the computer equipment and the computer readable storage medium, the initial defect classification is carried out based on the traditional first defect detection method, and the defects are further classified based on the second defect detection method of the deep learning, so that the advantages of the traditional algorithm and the deep learning algorithm are combined, the edge detection accuracy is improved, and the yield of the mobile phone glass cover plate is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Wherein:

FIG. 1 is a schematic flow chart diagram of a method for edge defect detection according to an embodiment;

FIGS. 2 (A) - (F) are schematic diagrams illustrating an embodiment of performing a first defect detection on a target image to obtain an image of a first edge defect in a vertical/horizontal direction;

FIGS. 3 (A) - (F) are schematic diagrams illustrating second defect detection on a target image to obtain an image of a first edge defect at an R corner in one embodiment;

FIG. 4 is a flowchart illustrating an embodiment of obtaining a first edge feature in a vertical/horizontal direction based on a first defect detection method;

FIG. 5 is a flowchart illustrating an embodiment of obtaining a second edge feature at an R-corner based on a second defect detection method;

FIG. 6 is a diagram illustrating the structure of the Squeezenet model in one embodiment;

FIG. 7 is a diagram illustrating the calculation process of the Squeezenet model in one embodiment;

FIG. 8 is a diagram illustrating a process for defect screening based on feature screening expressions in one embodiment;

FIG. 9 is a schematic diagram of an edge defect detecting apparatus according to an embodiment;

FIG. 10 is a schematic structural diagram of a computer device for executing the edge defect detection method in one embodiment.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.

In the embodiment of the invention, in order to detect the edge defect of the mobile phone cover plate, an edge defect detection method is provided by acquiring the image of the cover plate to be detected and then detecting the edge defect based on the image, so as to detect whether the edge of the mobile phone cover plate has the defect.

Specifically, referring to fig. 1, a schematic flow chart of the edge defect detection method provided in the embodiment of the present invention is shown, where the edge defect detection method includes steps S101 to S104 shown in fig. 1:

step S101: acquiring a target image of a cover plate to be detected, wherein the target image is an image of the cover plate to be detected.

In this embodiment, the cover plate to be detected is a glass cover plate of a mobile phone, which is a cover plate in the production of the glass cover plate of the mobile phone on a production line. Here, image acquisition needs to be performed on the cover plate to be detected to perform defect detection. Wherein, an image acquisition device (such as a CCD device) can be arranged on the production line to acquire the image of the cover plate to be detected so as to detect the defects. The collected day is an image collected by a cover plate to be detected in a dark field lighting mode, and the target image is a gray image.

Further, in this embodiment, for the cover plate to be detected, edge defect detection needs to be performed on the edge in the vertical direction, the edge in the horizontal direction, and the edge at the R corner (rounded corner), respectively. For the defect detection by partition, it is necessary to extract ROI regions in the vertical direction, horizontal direction, and R-angle, respectively, and then perform defect detection based on the ROI regions, respectively. That is to say, for the acquired target image, the ROI areas in the target image are acquired from the directions of the vertical edge, the horizontal edge and the R-corner edge, respectively, and the ROI areas include ROI image areas in the target image corresponding to the vertical edge, the horizontal edge and the R-corner edge of the cover plate to be detected.

Further, in order to improve the accuracy of subsequent image processing, in this step, the target image needs to be preprocessed based on a preset image filtering algorithm to remove useless information in the picture, and meanwhile, the picture does not lose useful detail information. The preset image filtering algorithm includes one or more combinations of mean filtering, median filtering, markov random field filtering, simulated annealing filtering, dynamic threshold segmentation, linear filtering, mathematical morphology filtering, gradient processing, adaptive filtering, and Hilbert filtering, which is not limited in this embodiment.

Step S102: and based on a first defect detection method, performing defect detection on the target image to acquire a first edge defect in the target image.

It should be noted that in this embodiment, two defect detection methods need to be adopted for detecting the edge defect of the cover plate to be detected, where the two defect detection methods include a first defect detection method and a second defect detection method, the first defect detection method is a conventional defect detection method, and the second defect detection method is a defect detection method based on deep learning. That is, in the present embodiment, the defect detection is performed by the conventional defect detection method and the defect detection method based on deep learning, respectively, so as to improve the accuracy of the edge defect detection.

Specifically, in this step, the first defect detection method is a conventional edge defect detection method, and completes detection of edge defects in each ROI region in the target image to obtain the first edge defect in the target image.

Edge information is a very representative image feature. The purpose of edge detection is to find a set formed by pixels with severe brightness change in an image, and the image is often a contour. If the edges in the image can be accurately measured and located, it means that the actual object can be located and measured, including the area of the object, the diameter of the object, the shape of the object, etc. can be measured. In image acquisition, there are the following 4 cases that form an edge when represented in an image. 1) A discontinuity in depth; 2) Discontinuities in surface orientation; 3) Different object materials (which results in different reflection coefficients of light); 4) The illumination is different in the scene.

In a specific embodiment, the first defect detection is performed on the target image, and the obtained defect features in the target image include, but are not limited to, one or more of position information, defect area, perimeter, length and width, roundness, ellipse flatness, aspect ratio, duty ratio, and other edge defect features such as contour features and gray scale features.

The above-described plurality of edge defect features will be described below.

Position information: representing coordinate (x, y) distribution information of the defects on the glass image, and being beneficial to distinguishing whether the defects are in the glass surface or at the glass edge;

defect area: an effective pixel area representing a defect (s = Pi R or s = minimum circumscribed rectangle w h);

length and width: represents the length and width of the minimum circumscribed rectangle;

roundness: let p be the center point (mass point) of the region, p _ i be all pixels on the outline, and F be the area of the outline (the number of pixels in the outline here, not the area of the region surrounded by the outline);

oval flatness: the ratio of the major axis to the minor axis of the circumscribed ellipse of the defect;

aspect ratio: representing the ratio of the length to the width of the minimum bounding rectangle of the defect;

duty ratio: the ratio of the effective pixel area representing the defect to the minimum circumscribed rectangle area.

In the process of detecting the first edge defect, the defect detection needs to be performed from the vertical direction, the horizontal direction and the R angle respectively, because the defect characteristics of the R angle in the vertical direction and the horizontal direction are different, different defect detection methods need to be adopted to improve the accuracy of the edge defect detection.

Specifically, the step of performing defect detection on the target image based on the first defect detection method to obtain a first edge defect in the target image further includes: performing first defect detection on ROI image areas of vertical edges and horizontal edges of the cover plate to be detected to obtain first edge defects in the target image, wherein the first edge defects comprise edge defects in the vertical direction and/or the horizontal direction; and performing first defect missing on the edge of the R corner of the cover plate to be detected to obtain a first edge defect in the target image, wherein the first edge defect comprises an edge defect on the R corner.

How the detection is performed from edge defect detection in the vertical and horizontal directions, and edge defect detection at the R-angle, respectively, is explained below.

Specifically, in the vertical and/or horizontal direction, in the process of detecting a first edge defect, binarization processing is performed on the target image according to a preset first gray threshold value to obtain a binarization image corresponding to the target image; performing expansion processing on the binary image corresponding to the target image to obtain a corresponding first mask image; performing edge detection on the target image to obtain a first defect image in the target image; carrying out 8-neighborhood contour tracking processing on the target image, determining at least one inflection point, and carrying out fitting operation on the at least one inflection point; detecting the defects based on the fitting result to obtain a second defect image; and determining a first defect characteristic in the vertical direction and/or the horizontal direction based on the first defect image, the second defect image and the first mask image.

Assuming that the width of the target image is W and the height is H, the target image is a set of pixel points I (I, j), which is shown in fig. 2 (a).

And carrying out binarization on the target image by using a preset first gray threshold value T1 to obtain a binarized image corresponding to the target image.

And performing expansion processing on the binary image corresponding to the target image. The binarized image corresponding to the target image is regarded as a defect Map, then merging processing is performed on the defects in the binarized image with a preset expansion radius (for example, 10 pixels), and then a corresponding area Mask (first Mask image) is determined for an area corresponding to a defect with a maximum outline selected from the defects after the merging processing, as shown in fig. 2 (B).

Performing edge detection on the target image to obtain a first defect image corresponding to the target image, where a Prewitt edge detection operator may be used to process the target image, and perform horizontal filtering processing and then vertical filtering processing to generate a first defect image Prewitt _ Map, as shown in fig. 2 (C). In the Prewitt edge detection operator, the central gradient is estimated by using the gray values of 8 pixel points in the surrounding neighborhood, and the corresponding gradient calculation formula is as follows:

the convolution kernel of the Prewitt edge detection operator is as follows:

performing 8-neighborhood contour tracking processing on a target image, searching for an Open _ Line graph, specifically referring to fig. 2 (D), performing inflection point detection (Ramer-Douglas-Peucker algorithm, also called a larmer-Douglas-pocker algorithm or an iterative adaptive point algorithm) on disconnected Open Line lines, recursively obtaining at least one inflection point, sequencing contour lines corresponding to the inflection point, and calculating a reference Line _0 by means of piecewise fitting; comparing the points on the contour line corresponding to the at least one inflection point with the reference, and performing defect preselection based on a preset threshold T2 to generate a second defect image Fit _ Map, as shown in fig. 2 (E). The Prewitt _ Map and the Fit _ Map are ored, and then the first mask image is anded, and the small defect outliers are filtered to obtain a defect Result _ Map (first defect feature), as shown in fig. 2 (F).

Further, referring to fig. 4, a schematic flow chart of the above-mentioned first defect detection method for obtaining the first edge feature in the vertical/horizontal direction is shown.

Specifically, on defect edge detection at an R-angle, in a first edge defect detection process, a first nonlinear gradient processing is performed on the target image to obtain a first nonlinear gradient map; wherein the first non-linear gradient process is a kernel 15 x 15 non-linear gradient process; performing binarization processing on the first nonlinear gradient map according to a preset first gradient threshold value to obtain a binarization image corresponding to the first nonlinear gradient map; performing defect merging processing according to a preset first pixel radius on the basis of a binary image corresponding to the nonlinear gradient map, and determining a second mask image according to an image after defect merging; performing second nonlinear gradient processing on the target image to obtain a second nonlinear gradient map; wherein the second non-linear gradient treatment is a kernel 3 x 3 non-linear gradient treatment; performing binarization processing on the second nonlinear gradient map according to a preset second gradient threshold value to obtain a binarization image corresponding to the second nonlinear gradient map; performing contour tracking processing and cavity detection processing on the binary image corresponding to the second nonlinear gradient map, and performing defect merging processing on the basis of a preset second pixel radius to obtain a third defect image; determining a first edge defect at the R corner based on a third defect image and the second mask image.

Assuming that the width of the target image is W and the height is H, the target image is a set of pixel points I (I, j), which is shown in fig. 3 (a).

The first nonlinear gradient processing of 15 × 15 kernels is performed on the target image to obtain a first nonlinear gradient Map _ Grad _0, which is specifically shown in fig. 3 (B). Wherein, by way of

The gradient value at each coordinate point is calculated to determine a corresponding gradient Map, i.e. a first non-linear gradient Map _ Grad _0, where gu and gv represent the gradient values of the coordinate point in the x and y directions, respectively.

And binarizing the first nonlinear gradient Map _ Grad _0 by using a preset first gradient threshold value T1 to obtain a binarized image corresponding to the first nonlinear gradient Map.

The binarized image corresponding to the first nonlinear gradient Map is regarded as a defect Map1, a defect merging processing operation with a preset first pixel radius (for example, 10 pixels) is performed, and then a defect with a maximum contour is selected based on the defect after the merging processing to determine a second Mask image Mask, which is specifically shown in fig. 3 (C).

Performing 3 × 3 kernel second nonlinear gradient processing on the target image to obtain a second nonlinear gradient Map _ Grad _1, which is specifically shown in fig. 3 (D).

And carrying out binarization processing on the second nonlinear gradient Map _ Grad _1 by using a preset second gradient threshold value T2 to obtain a binarization image corresponding to the second nonlinear gradient Map.

Carrying out Contour tracing processing on the binary image corresponding to the second nonlinear gradient Map to obtain a Contour Map Contour _ Map; specifically, refer to fig. 3 (E). And then, performing hole detection processing, and performing run-length code defect merging based on a preset second pixel radius (for example, the radius is 1) to obtain a third defect image Map2. The Map2 and the second Mask image Mask are used to perform an and operation, and then large defects are filtered, that is, a first edge defect Result _ Map is generated, which is the first edge defect on the R corner, as shown in fig. 3 (F).

Further, referring to fig. 5, a schematic flow chart of obtaining the second edge feature on the R corner based on the second defect detection method is shown.

Step S103: classifying the defects of the first edge defects in the target image based on a second defect detection method, and determining second edge defects in the target image;

the second defect detection method is a defect detection method based on deep learning.

For some defects under the condition of complex background interference, the effect detected by using the traditional method is not good, so that after the image is obtained, a deep learning model is further required to be further utilized to train some defects independently aiming at some specific defects, and therefore some specific defects can be detected and classified. In a specific embodiment, the deep learning networks commonly used are ThunderNet, peloenet, yolo series, etc. This embodiment is not limited.

Specifically, after the first edge defect is calculated, it is necessary to further classify the defect feature corresponding to the first edge defect. The defect feature classification is selected here, because the dimension of the original image data is too large, the image with the pixel size of the clip thumbnail (256 × 256 or 64 × 64) for extracting the defect is selected to perform model training, and the calculated amount of the model training is reduced by using a lightweight network, wherein the common networks include Yolo-tiny, mobileNet, squeezeNet, shuffleNet and the like.

Specifically, unlike the conventional convolution method, the core of the Squeezenet is a Fire module (Fire module), which is composed of two layers, namely, a Squeeze layer and an expanded layer, as shown in fig. 6, the Squeeze layer is a convolution layer with 1 × 1 convolution kernel, and the expanded layer is a convolution layer with 1 × 1 convolution kernel and 3 × 3 convolution kernel.

And in the process of further classifying the first edge defect through the Squeezenet model to obtain a corresponding second edge defect, inputting the first edge defect into the Squeeze layer, then inputting the output result of the Squeeze layer into the expanded layer, and taking the output result of the expanded layer as the second edge defect.

Specifically, as shown in fig. 7, in the Squeeze layer, the first edge defect of W × H × M is input to the convolution layer of 1 × 1 convolution kernel in the Squeeze layer to obtain the feature map of W × H × s1, then the feature map of W × H × s1 is input to the convolution layers of 1 × 1 and 3 convolution kernels in the Expand layer to obtain the feature maps corresponding to H × M × e1 and H × M × e3, respectively, and the contact operation is performed to obtain the feature map of H × M (e 1+ e 3).

In the training log of the Squeezenet model, the accuracy rate quickly reaches 1, and the loss value is very small and close to 0.

That is, in this step, after the edge defect is preliminarily classified by using the conventional defect detection method in step S102, deep learning classification needs to be performed according to the defect clip minimap, and the deep learning classification is further applied to the squeezet model to classify the edge defect, so as to improve the accuracy of classification detection of the edge defect.

Step S104: and screening the first edge defect and the second edge defect, and determining a defect detection result of the cover plate to be detected.

In this embodiment, it is necessary to perform comprehensive defect screening by integrally combining the results of the first defect detection method and the second defect detection method to obtain a final defect detection result. Specifically, for a plurality of preset classification defects, the first edge defect and the second edge defect need to be screened based on the corresponding preset feature screening expressions, and the defect detection result under each classification defect is determined. The classification defects of each type can be classified more carefully and accurately, and the classification defects are screened through the expression, so that the speed is very high, and the rapid demand of actual production can be completely met. In a specific embodiment, please refer to fig. 8, which shows a schematic diagram of the defect screening process based on the feature screening expression.

Further, referring to fig. 9, a schematic structural diagram of an edge defect detecting apparatus is shown, where as shown in fig. 9, the edge defect detecting apparatus includes:

the image acquisition module 101 is configured to acquire a target image of a cover plate to be detected, where the target image is an image of the cover plate to be detected;

a first defect detection module 102, configured to perform defect detection on the target image based on a first defect detection method, to obtain a first edge defect in the target image;

the second defect detection module 103 is configured to classify a first edge defect in the target image based on a second defect detection method, and determine a second edge defect in the target image; the second defect detection method is a defect detection method based on deep learning;

and the defect fusion screening module 104 is configured to screen the first edge defect and the second edge defect, and determine a defect detection result of the cover plate to be detected.

In an optional embodiment, the image obtaining module 101 is further configured to pre-process the target image based on a preset image filtering algorithm; the preset image filtering algorithm comprises one or more combinations of mean filtering, median filtering, markov random field filtering, simulated annealing filtering, dynamic threshold segmentation, linear filtering, mathematical morphology filtering, gradient processing, self-adaptive filtering and Hilbert filtering.

In an alternative embodiment, the target image is a grayscale image; the image obtaining module 101 is further configured to obtain an ROI in the target image, where the ROI includes ROI image regions corresponding to a vertical edge, a horizontal edge, and an R-angle edge of the cover plate to be detected in the target image.

In an optional embodiment, the first defect detection module 102 is further configured to perform first defect detection on an ROI image region of a vertical edge and a horizontal edge of the cover plate to be detected, so as to obtain a first edge defect in the target image, where the first edge defect includes an edge defect in a vertical direction and/or a horizontal direction; and performing first defect missing on the edge of the R corner of the cover plate to be detected to obtain a first edge defect in the target image, wherein the first edge defect comprises an edge defect on the R corner.

In an optional embodiment, the first defect detection module 102 is further configured to perform binarization processing on the target image according to a preset first gray threshold, so as to obtain a binarized image corresponding to the target image; performing expansion processing on the binary image corresponding to the target image to obtain a corresponding first mask image; performing edge detection on the target image to obtain a first defect image in the target image; carrying out 8-neighborhood contour tracking processing on the target image, determining at least one inflection point, and carrying out fitting operation on the at least one inflection point; performing defect detection based on the fitting result to obtain a second defect image; and determining a first defect characteristic in the vertical direction and/or the horizontal direction based on the first defect image, the second defect image and the first mask image.

In an optional embodiment, the first defect detecting module 102 is further configured to perform a first nonlinear gradient process on the target image to obtain a first nonlinear gradient map; wherein the first non-linear gradient process is a kernel 15 x 15 non-linear gradient process; performing binarization processing on the first nonlinear gradient map according to a preset first gradient threshold value to obtain a binarization image corresponding to the first nonlinear gradient map; performing defect merging processing according to a preset first pixel radius on the basis of a binary image corresponding to the nonlinear gradient map, and determining a second mask image according to an image after defect merging; performing second nonlinear gradient processing on the target image to obtain a second nonlinear gradient map; wherein the second non-linear gradient treatment is a kernel 3 x 3 non-linear gradient treatment; performing binarization processing on the second nonlinear gradient map according to a preset second gradient threshold value to obtain a binarization image corresponding to the second nonlinear gradient map; performing contour tracking processing and cavity detection processing on the binary image corresponding to the second nonlinear gradient map, and performing defect merging processing based on a preset second pixel radius to obtain a third defect image; determining a first edge defect at the R corner based on a third defect image and the second mask image.

In an optional embodiment, the deep learning model is obtained by training a clip thumbnail containing defects of a training sample; the deep learning model is an Squeezenet model, the Squeezenet model comprises a Squeeze layer and an expanded layer, the Squeeze layer comprises a convolution layer of 1 × 1 convolution kernels, and the expanded layer comprises a convolution layer of 1 × 1 convolution kernels and a convolution layer of 3 × 3 convolution kernels; the second defect detection module 103 is further configured to input the first edge defect into the Squeeze layer, and then input an output result of the Squeeze layer into an extended layer, where the output result of the extended layer is used as the second edge defect.

In an optional embodiment, the defect fusion filtering module 104 is further configured to filter the first edge defect and the second edge defect based on preset feature filtering expressions corresponding to different classification defects, and determine a defect detection result under each classification defect.

FIG. 10 is a diagram illustrating an internal structure of a computer device for implementing the edge defect detection method in one embodiment. The computer device may specifically be a terminal, and may also be a server. As shown in fig. 10, the computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program which, when executed by the processor, causes the processor to carry out the above method. The internal memory may also have stored therein a computer program which, when executed by the processor, causes the processor to perform the method described above. It will be appreciated by those skilled in the art that the configuration shown in fig. 10 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

After the edge defect detection method, the edge defect detection device, the computer equipment and the computer-readable storage medium are adopted, under the condition that the edge characteristics of the glass cover plate of the mobile phone need to be detected, a corresponding target image is obtained firstly, then the defect detection is carried out on the target image based on a first defect detection method, and a first edge defect in the target image is obtained; classifying the defects of the first edge defects in the target image based on a second defect detection method, and determining second edge defects in the target image; the second defect detection method is a defect detection method based on deep learning; and screening the first edge defect and the second edge defect, and determining the defect detection result of the cover plate to be detected. That is, through the edge defect detection method, the edge defect detection device, the computer equipment and the computer readable storage medium, the initial defect classification is carried out based on the traditional first defect detection method, and the defects are further classified based on the second defect detection method of the deep learning, so that the advantages of the traditional algorithm and the deep learning algorithm are combined, the edge detection accuracy is improved, and the yield of the mobile phone glass cover plate is improved.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.

Claims

1. An edge defect detection method, the method comprising:

classifying the defects of the first edge defect in the target image based on a second defect detection method, and determining a second edge defect in the target image; the second defect detection method is a defect detection method based on deep learning;

2. The edge defect detection method of claim 1, further comprising:

3. The edge defect detection method of claim 1, wherein the target image is a grayscale image;

after the step of obtaining the target image of the cover plate to be detected, the method further comprises the following steps:

and obtaining an ROI (region of interest) in the target image, wherein the ROI comprises ROI image areas corresponding to the vertical edge, the horizontal edge and the R-angle edge of the cover plate to be detected in the target image.

4. The edge defect detection method of claim 3, wherein the step of performing defect detection on the target image based on the first defect detection method to obtain the first edge defect in the target image further comprises:

performing first defect detection on ROI (region of interest) image areas of the vertical edge and the horizontal edge of the cover plate to be detected to obtain first edge defects in the target image, wherein the first edge defects comprise edge defects in the vertical direction and/or the horizontal direction;

and performing first defect missing on the edge of the R corner of the cover plate to be detected to obtain a first edge defect in the target image, wherein the first edge defect comprises an edge defect on the R corner.

5. The edge defect detection method according to claim 4, wherein the step of performing the first defect detection on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be detected to obtain the first edge defect in the target image further comprises:

according to a preset first gray threshold, carrying out binarization processing on the target image to obtain a binarization image corresponding to the target image;

performing expansion processing on the binary image corresponding to the target image to obtain a corresponding first mask image;

performing edge detection on the target image to obtain a first defect image in the target image;

carrying out 8-neighborhood contour tracking processing on the target image, determining at least one inflection point, and carrying out fitting operation on the at least one inflection point; detecting the defects based on the fitting result to obtain a second defect image;

and determining a first defect feature in the vertical direction and/or the horizontal direction based on the first defect image, the second defect image and the first mask image.

6. The edge defect detecting method according to claim 4, wherein the step of performing a first defect missing on the R-corner edge of the cover plate to be detected to obtain a first edge defect in the target image further comprises:

performing first nonlinear gradient processing on the target image to obtain a first nonlinear gradient map; wherein the first non-linear gradient process is a kernel 15 x 15 non-linear gradient process; performing binarization processing on the first nonlinear gradient map according to a preset first gradient threshold value to obtain a binarization image corresponding to the first nonlinear gradient map; performing defect merging processing according to a preset first pixel radius on the basis of a binary image corresponding to the nonlinear gradient map, and determining a second mask image according to an image after defect merging;

performing second nonlinear gradient processing on the target image to obtain a second nonlinear gradient map; wherein the second non-linear gradient treatment is a kernel 3 x 3 non-linear gradient treatment; performing binarization processing on the second nonlinear gradient map according to a preset second gradient threshold value to obtain a binarization image corresponding to the second nonlinear gradient map; performing contour tracking processing and cavity detection processing on the binary image corresponding to the second nonlinear gradient map, and performing defect merging processing based on a preset second pixel radius to obtain a third defect image; determining a first edge defect at the R corner based on a third defect image and the second mask image.

7. The edge defect detection method according to claim 1, wherein the deep learning model is obtained by training clip minimaps containing defects of training samples;

the deep learning model is an Squeezenet model, the Squeezenet model comprises a Squeeze layer and an expanded layer, the Squeeze layer comprises a convolution layer of 1 × 1 convolution kernels, and the expanded layer comprises a convolution layer of 1 × 1 convolution kernels and a convolution layer of 3 × 3 convolution kernels;

the step of classifying the first edge defect in the target image based on the second defect detection method, and determining the second edge defect in the target image, further includes:

inputting the first edge defect into the Squeeze layer, then inputting the output result of the Squeeze layer into the expanded layer, and taking the output result of the expanded layer as the second edge defect.

8. The edge defect detecting method of claim 1, wherein the step of screening the first edge defect and the second edge defect to determine the defect detection result of the cover plate to be detected further comprises:

and screening the first edge defect and the second edge defect based on preset feature screening expressions corresponding to different classification defects, and determining a defect detection result under each classification defect.

9. An edge defect detection apparatus, comprising:

the device comprises an image acquisition module, a detection module and a display module, wherein the image acquisition module is used for acquiring a target image of a cover plate to be detected, and the target image is an image of the cover plate to be detected;

10. A computer device comprising a memory and a processor, the memory having executable code that when run on the processor implements the edge defect detection method of any of claims 1 to 8.