CN114693685A - Unsupervised defect detection model training method and defect detection method - Google Patents
Unsupervised defect detection model training method and defect detection method Download PDFInfo
- Publication number
- CN114693685A CN114693685A CN202210617738.7A CN202210617738A CN114693685A CN 114693685 A CN114693685 A CN 114693685A CN 202210617738 A CN202210617738 A CN 202210617738A CN 114693685 A CN114693685 A CN 114693685A
- Authority
- CN
- China
- Prior art keywords
- image
- defect
- sample
- segmentation
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
A method for training unsupervised defect detection model includes using sample to train defect detection model, forming partial sample pair by normal sample image and artificial abnormal sample image applying artificial noise to normal sample image, forming partial sample pair by two identical normal sample images without need of real abnormal sample image and corresponding label. During training, the artificial abnormal sample image or the normal sample image is input into the model and is reconstructed through the reconstruction module, a reconstruction loss function is constructed according to the reconstructed image and the normal sample image in the corresponding sample pair for training, the reconstructed image and the input artificial abnormal sample image or the normal sample image are spliced and then subjected to defect segmentation, the refined reconstruction effect of the normal image is improved, meanwhile, the difference between the abnormal image and the reconstructed image is increased, the false detection of the normal image is reduced, and the detection performance is improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a training method and a defect detection method of an unsupervised defect detection model.
Background
The defect detection of industrial images usually depends on manual classification of a large number of images, selects abnormal images which do not meet requirements, and labels corresponding defect information to train a model, so that the purpose of defect detection is achieved. With the continuous improvement of the consumption demand on the quality requirement of industrial products, industrial image data has three characteristics: rareness, imbalance, and diversity.
(1) The rare property: the number of the abnormal images is reflected, in the actual industrial production, along with the technical progress of the production process, most of the produced products belong to perfect standard products, only a small part of the products are defective products, the number of the products is rare, the abnormal images shot by the defective products are rare, the abnormal images need to be manually selected by people familiar with the products, if the defects are detected by adopting a supervision technology, special technical personnel are required to mark all the abnormal images with accurate pixel level marks, the cost is high, and the abnormal images are difficult to obtain;
(2) unbalance: the method mainly has two aspects, namely on the image level, the rarity causes the number of abnormal images to be less than that of normal images; secondly, on the pixel level, the proportion of the defect area in one image to the whole image is small; the two layers of the image level and the pixel level act together to cause the proportion of normal data and defective data in industrial image data to be seriously unbalanced;
(3) diversity: in industrial production, the defect types are numerous, the differences of the sizes, the shapes, the positions, the color textures and the like are large, and the distribution of defect data is inconsistent statistically; in a continuous production process, new types of defects may appear with unpredictability, and therefore, the types of defects cannot be completely summarized in a statistical manner, presenting diversity.
The model is trained in a supervision mode to detect defects, sufficient samples are needed, type balance among the samples and comprehensive coverage of the types are guaranteed, and the technical requirements of supervision algorithms cannot be met due to the three characteristics of industrial image data.
Disclosure of Invention
The invention provides an unsupervised defect detection model training method and a defect detection method, which aim to overcome the limitation of a supervised algorithm in defect detection.
According to a first aspect, an embodiment provides a training method of an unsupervised defect detection model, the defect detection model includes a feature extractor, a feature compression module, an image reconstruction module, and a defect segmentation module, and the training method of the defect detection model includes:
acquiring a normal sample image set, wherein the normal sample image is an image of a defect-free standard substance;
extracting a predetermined ratio from the normal sample image setpOf the normal sample imagexFor the extracted normal sample imagexApplying artificial noise to obtain artificial abnormal sample image
And forming sample pairs with the original image
For the non-extracted normal sample imagexCopying to obtain a copied image, and forming a sample pair with the original image
Forming a training sample set by all sample pairs;
obtaining sample pairs from the training sample set, when obtaining the sample pairs
Then the artificial abnormal sample image is taken
Inputting the initial features into the feature extractor to obtain initial featuresαWhen obtaining the sample pair
Then any one of the normal sample images is takenxInputting the initial features into the feature extractor to obtain initial featuresα；
Characterizing the initial featureαInput into the feature compression module to obtain compressed featuresβ；
Characterizing said compressionβInput into the image reconstruction module to obtain a reconstructed imagey；
If the obtained sample is a pair
Then an image will be reconstructedyWith artificial abnormal sample images
Splicing on the channels to obtain a merged image, if the obtained merged image is a sample pair
Then an image will be reconstructedyAnd the normal sample imagexSplicing the channels to obtain a merged image;
inputting the merged image into the defect segmentation module to obtain a defect prediction imagez；
From the reconstructed imageyAnd the normal sample image in the corresponding sample pairxConstructing a reconstruction loss functionL r Predicting the image based on the defectzAnd corresponding defective label imagesmConstructing a segmentation loss functionL s From the reconstruction loss functionL r And a segmentation loss functionL s Constructing a total loss functionL total ；
According to the total loss functionL total And training the defect detection model to obtain corresponding model parameters.
In one embodiment, the feature extractor is a pre-trained neural network, the initial featuresαA feature map comprising one or more network layer outputs of the pre-trained neural network.
In one embodiment, in the training process, the parameters of the feature extractor are kept unchanged, and after the parameters of the other parts except the feature extractor in the defect detection model are updated for a preset number of times, the feature extractor starts to update the parameters.
In one embodiment, the feature compression module is to compress the initial features byαCompressing to obtain compression characteristicsβ：
Presetting a compression sizes；
The initial characteristic is measuredαIs resized to the compressed sizes；
Splicing the feature graphs after the size adjustment on the channel to form a feature group;
dividing each feature map in the feature group into a plurality of non-overlapping small blocks, and taking the pixel average value of each small block to form a new feature map;
channel compression of feature sets to obtain compressed featuresβ。
In one embodiment, the image reconstruction module includes one or more deconvolution layers, a batch normalization layer, and a ReLU activation function.
In one embodiment, the reconstruction loss functionL r The method specifically comprises the following steps:
whereinλ 1Andλ 2in order to be a preset balance coefficient,L MSE represents the root mean square error, and
whereiny i Representing a reconstructed imageyTo (1) aiThe value of each of the pixels is calculated,x i representing corresponding normal sample imagesxTo (1) aiThe value of each of the pixels is calculated,nrepresenting the number of pixels;
L SSIM represents a loss of structural similarity, an
Wherein
Whereinμ y Andμ x respectively reconstructed imagesyAnd corresponding normal sample imagesxThe average value of the pixels of (a),
and
respectively as reconstructed imagesyAnd corresponding normal sample imagexThe variance of the pixels of (a) is,σ yx for reconstructing imagesyAnd corresponding normal sample imagesxCovariance of c1And c2Is a preset constant.
In one embodiment, the segmentation loss functionL s The method specifically comprises the following steps:
whereinδ 1Andδ 2are respectively a preset weight coefficient,L cross represents a cross-entropy loss, and
whereinz i Predictive picture representing defectszTo (1) aiThe predicted value of each pixel is calculated,m i representing corresponding defect label imagesmTo (1) aiThe label value of each pixel is set to,nrepresenting the number of pixels;
L dice represents the loss of the cross-over ratio, and
whereinz d Predictive picture representing defectszIn the area of the image that is predicted to be defective,m d representing defective label imagesmIs marked as a defective area.
In one embodiment, the total loss functionL total The method specifically comprises the following steps:
according to a second aspect, an embodiment provides an unsupervised defect detection method, comprising:
acquiring an image to be detected of a detected object;
inputting the image to be detected into a defect detection model trained by the training method according to the first aspect to obtain a defect prediction image;
and performing threshold segmentation on the defect prediction image by using a set segmentation threshold to obtain a defect segmentation image.
In one embodiment, the segmentation threshold is set by:
obtaining a verification set, wherein the verification set is composed of normal sample images;
inputting each normal sample image in the verification set into the defect detection model respectively to obtain a defect prediction image of each normal sample image;
obtaining the maximum predicted value of all pixels of all defect predicted imagesS maxAnd minimum predicted valueS min；
From maximum predicted valueS maxStarting and gradually reducing the step length delta, performing threshold segmentation on each defect prediction image by taking the current numerical value as a segmentation threshold, then calculating the false detection rate until the false detection rate is greater than or equal to a preset false detection rate threshold, and taking the segmentation threshold at the moment as a final segmentation threshold, wherein when the threshold segmentation is performed, pixel points with the predicted values greater than the segmentation threshold are determined as defect points, and the calculation formula of the step length delta is as follows:
whereinS step Is a preset number of steps.
According to a third aspect, an embodiment provides a computer-readable storage medium having a program stored thereon, the program being executable by a processor to implement the training method according to the first aspect described above, and/or the defect detection method according to the second aspect described above.
According to the unsupervised defect detection model training method and the defect detection method, only the normal sample image is used for training, the normal sample image with the preset proportion is extracted, the artificial abnormal sample image is obtained by applying artificial noise, the defect detection model is trained by forming the sample pair with the original image, the real abnormal sample image and the corresponding label are not needed, a user can train the model by only providing the normal sample image, the defect detection task is completed, the limitation of a supervision algorithm is overcome, the manpower is saved, the production efficiency of a manufacturer is improved, and the cost reduction and the efficiency improvement are realized. In the training process, the artificial abnormal sample image or the normal sample image is input into the model to be reconstructed through the reconstruction module, a reconstruction loss function is constructed according to the reconstructed image and the normal sample image in the corresponding sample pair for training, the reconstructed image is spliced with the input artificial abnormal sample image or the normal sample image, and then the reconstructed image is input into the defect segmentation module to be subjected to defect segmentation, so that the refined reconstruction effect of the normal image is improved, the difference between the abnormal image and the reconstructed image is increased, the false detection of the normal image is reduced, the precision of the defect segmentation is improved, and the detection performance is improved.
Drawings
FIG. 1 is a diagram illustrating a defect detection model according to an embodiment;
FIG. 2 is a schematic diagram illustrating a training process of a defect detection model according to an embodiment;
FIG. 3 is a flow diagram of a method for training an unsupervised defect detection model according to an embodiment;
FIG. 4 is a flow diagram of an unsupervised defect detection method of an embodiment;
FIG. 5 is a schematic illustration of a defect segmentation map of an embodiment;
FIG. 6 is a flow diagram of a method of setting a segmentation threshold according to one embodiment.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. Wherein like elements in different embodiments are numbered with like associated elements. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, those skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in detail in order to avoid obscuring the core of the present application from excessive description, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning. The term "connected" and "coupled" when used in this application, unless otherwise indicated, includes both direct and indirect connections (couplings).
The invention uses the unsupervised technology to detect the defects of the industrial products, and under the condition of lacking the abnormal images or difficult acquisition of the abnormal images, the unsupervised technology used by the invention can well meet the data requirements of users, does not need real abnormal images and corresponding labels, and can train the model only by using normal sample images to complete the defect detection task.
At present, most of methods for defect detection by adopting an unsupervised technology adopt an image reconstruction method, and a general processing mode is to use a normal sample image as a training sample, construct a network model to extract characteristics of the normal sample image, encode the characteristics, obtain a hidden layer representation, and decode the hidden layer representation to complete image reconstruction. During reasoning, the defects are positioned according to the pixel-by-pixel difference between the reconstructed image and the input image by utilizing the principle that the network model cannot be trained by using the abnormal image and cannot well recover the defect region in the image. However, the hidden layer representation of the network model usually only accepts output features from the directly upper layer, the generalization and representation capability of the features is limited, the reconstruction result of a normal image is sometimes fuzzy and the quality is not fine enough, and sometimes a defect region can be better reconstructed due to the inherent generalization of the network model, the difference between an input image and a reconstructed image is not large, the reconstruction error is small, and the defect detection is not facilitated. The reconstruction effect on the normal image is poor, and the recovery capability of the defect area is equivalent, so that the detection performance is reduced.
The invention provides a novel unsupervised defect detection method based on an image reconstruction technology, which can increase the difference between an abnormal image and a reconstructed image thereof while improving the refined reconstruction effect of a normal image, and combine the reconstructed image and an input image for defect segmentation, thereby being beneficial to realizing accurate detection of surface defects of industrial products and improving the detection performance. The defect detection method can directly output the defect segmentation result in an end-to-end mode, meanwhile, the training of the model can be completed by forming the normal sample images into the training samples in a certain mode without any real abnormal samples during training, and technicians do not need to carry out pixel-level marking on the sample images, so that the data burden of product manufacturers is reduced, the labor is saved, the production efficiency of the manufacturers is improved, and the cost reduction and the efficiency improvement are realized.
The structure of the defect detection model of the present invention will be explained. Referring to fig. 1, a defect detection model in an embodiment of the inventionFIncluding a feature extractorfFeature compression modulegImage reconstruction moduleRAnd a defect segmentation moduleS。
Feature extractorfExtracting features of an input image to obtain initial featuresαFor processing of subsequent modules. In one embodiment, the feature extractorfA pre-trained neural network can be used, which is trained through a common image data set, such as ImageNet, and has good feature extraction capability. Can be derived from pre-trained nervesObtaining feature maps of outputs in one or more network layers of a network to compose initial featuresαWhen a plurality of feature maps are employed, multi-scale features of the input image can be obtained.
Feature compression modulegFor compressing and integrating the extracted image features, more effective feature representation is extracted to obtain compressed featuresβModule for image reconstructionRAnd (6) carrying out reconstruction. In one embodiment, the feature extractorfInitial characteristics of the outputαWhen the feature graph is a multi-layer feature graph, the feature compression modulegThe initial characteristics can be matched in the following mannerαCompressing to obtain a compression characteristicβ: first, a compression size is set in advancesIn a specific embodiment, the size may be compressedsSet to one quarter of the input image resolution; then for the initial characteristicsαIs adjusted, the target size of the adjustment is the compressed sizesResizing each feature map to a compressed sizesEssentially, the method comprises the steps of performing feature alignment operation to obtain a multi-layer feature map with uniform resolution; splicing the feature graphs after the size adjustment on the channels to form a feature group; for each feature map in the feature group, performing a compression operation spatially, specifically: partitioning the feature map into non-overlapping blocks, in a specific embodiment, the number of the blocks is 1024, and the size of the blocks is 4 × 4, and then performing an averaging operation on all pixels in each block to obtain a compressed feature map, that is, taking the pixel average value of each block to form a new feature map, so that the size of the feature map is compressed; after all feature graphs in the feature group are subjected to space compression operation, channel compression is carried out on the feature group, and the number of channels of the feature group is reduced to obtain compressed featuresβChannel compression may be achieved by, for example, inputting the feature set into a 1 × 1 convolutional layer. Wherein averaging is performed on pixels within the patch, the feature variations on the feature map can be smoothed, making the generated features more robust to noise input, and secondly reducing the amount of computation.
Image reconstruction moduleRMay be a neural network modelIncluding one or more deconvolution layers, Batch Normalization (BN), and ReLU activation functions. Image reconstruction moduleRFor using compression characteristicsβAnd performing image reconstruction, wherein the reconstruction aims to map the input image onto the normal image.
Input defect segmentation module after splicing reconstructed image and input image on channelSPerforming defect segmentation, and defect segmentation moduleSAnd outputting a defect prediction image, wherein each pixel of the defect prediction image has a prediction value for representing the probability that the pixel is predicted to belong to a defect point, the prediction value can be a numerical value between 0 and 1, and the defect prediction image is subjected to threshold segmentation to obtain a defect area. Defect segmentation moduleSIt may be a neural network model, for example, a classical UNet network structure may be used.
During the training process, the module is reconstructed for the imageRAnd a defect segmentation moduleSThe two parts are respectively designed with corresponding loss functions for training, and the overall training process can refer to fig. 2. Integral structure and characteristic compression module of model in the inventiongAnd an image reconstruction moduleRAnd a defect segmentation moduleSThe design of the loss function of (2) is critical. Referring to fig. 3, a training method of the defect detection model of the present invention is described below, and in one embodiment, the method includes steps 110 to 190, which are described in detail below.
Step 110: and acquiring a normal sample image set, wherein the normal sample image set consists of normal sample images, and the normal sample images are images of defect-free standard products.
Step 120: and constructing sample pairs to form a training sample set.
Extracting a predetermined ratio from a set of normal sample imagespOf the normal sample imagexFor the extracted normal sample imagexApplying artificial noise to obtain artificial abnormal sample image
And forming sample pairs with the original image
For the normal sample image not extractedxCopying to obtain a copy image, and forming a sample pair with the original image
A training sample set is composed of all sample pairs.
Step 130: obtaining initial featuresα。
Sample pairs are obtained from a set of training samples. The sample pair obtained may be
Or
When obtaining the sample pair
Then the artificial abnormal sample image is taken
Input feature extractorfTo obtain initial characteristicsαWhen obtaining the sample pair
Then any one of the normal sample images is takenxInput feature extractorfTo obtain initial characteristicsα。
Step 140: initial characterizationαInput feature compression modulegTo obtain compression characteristicsβ。
Step 150: feature of compressionβInput image reconstruction moduleRTo obtain a reconstructed imagey。
Step 160: will reconstruct the imageyAnd splicing with the input image of the model to obtain a combined image.
If the sample pairs obtained in step 130 are
Then an image will be reconstructedyWith artificial abnormal sample images
Splicing on the channel to obtain a merged image
If the obtained sample is a pair
Then an image will be reconstructedyAnd the normal sample imagexSplicing on the channel to obtain a merged image
。
Step 170: inputting merged image into defect segmentation moduleSTo obtain a defect prediction imagez. Defect predictive imagezThe defect segmentation map may be obtained by performing threshold segmentation through a preset segmentation threshold, and the region predicted as a defect may be obtained from the defect segmentation map.
Step 180: from the reconstructed imageyAnd the normal sample image in the corresponding sample pairxConstructing a reconstruction loss functionL r Predicting the image based on the defectzAnd corresponding defective label imagesmConstructing a segmentation loss functionL s From the reconstruction loss functionL r And a segmentation loss functionL s Constructing a total loss functionL total 。
Reconstructing an imageyAnd the normal sample image in the corresponding sample pairxIn reconstructing the loss functionL r Under the guidance of (2) an image reconstruction moduleRAnd (5) training. In the technical scheme of the invention, the image reconstruction moduleRIt should implement the defect detection model regardless of the inputFWhether the image is a normal image or an abnormal image passes through an image reconstruction moduleRCan be mapped onto the normal image after the reconstruction operation. In the training process, if the input is manual exceptionConstant sample image
Then image reconstruction moduleRIs to reconstruct the imageyNormal sample image mapping to corresponding sample pairxEssentially, the denoising capability of a noise sample is learned, so that the input abnormal image can be restored to a normal image; if the input is a normal sample imagexThen image reconstruction moduleRIs to reconstruct the imageyMapping to Normal sample imagexItself (also corresponding to the normal sample image in the corresponding sample pairx) It is essential to learn the fine reconstruction capability for normal samples.
Defect prediction imagezAnd corresponding defective label imagesmIn dividing the loss functionL s Guided defect segmentation moduleSAnd (5) training. In the case of normal sample imagexApplying artificial noise to obtain artificial abnormal sample image
While the defect label image can be generated at the same time, the defect label image is easy to understandm. Partitioning modules for defectsSIt should be able to accurately identify defects, and eventually to obtain an accurate defect segmentation map when the input includes an abnormal image, and eventually to obtain a segmentation result without defects at the pixel level when the input includes only a normal image.
Based on the above-mentioned objectives, the applicant has designed a corresponding reconstruction loss functionL r And a segmentation loss functionL s . In one embodiment, the loss function is reconstructedL r The method is characterized by comprising two parts, wherein the first part adopts root mean square error, and the expression is as follows:
whereiny i Representing reconstructed imagesyTo (1) aiThe value of each of the pixels is calculated,x i representing corresponding normal sample imagesxTo (1) aiThe value of each of the pixels is calculated,nrepresenting the number of pixels.
The second part is structural similarity loss, and the expression is as follows:
wherein
Whereinμ y Andμ x respectively reconstructed imagesyAnd corresponding normal sample imagesxThe average value of the pixels of (a),
and
respectively reconstructed imagesyAnd corresponding normal sample imagesxThe variance of the pixels of (a) is,σ yx for reconstructing imagesyAnd corresponding normal sample imagexCovariance of c1And c2Is a preset constant, c1、c2Two smaller constants may be used to avoid the denominator zero-division operation. Function(s)hReconstructed image jointly measured from three aspects of brightness, contrast and structureyAnd corresponding normal sample imagesxThe image reconstruction quality can be effectively improved due to the similarity of the image reconstruction method.
Reconstruction loss functionL r Is a weighted sum of two parts:
whereinλ 1Andλ 2for a predetermined balancing factor, an example isλ 1=0.3，λ 2=0.7。
In one embodiment, the loss function is partitionedL s By cross entropy lossL cross Loss of cross-over ratioL dice Two parts are formed. Cross entropy lossL cross The expression of (a) is:
whereinz i Predictive picture representing defectszTo (1) aiThe predicted value of each pixel is calculated,m i representing corresponding defect label imagesmTo (1)iThe label value of each pixel is set to,nrepresenting the number of pixels. Since it is determined whether the pixel point is a defect point and belongs to the second category, the label value can be set to 0 or 1, the positive category (i.e., the defect point) is 1, and the negative category is 0.
Loss of cross-over ratioL dice The expression of (a) is:
whereinz d Predictive picture representing defectszIn the area of the image that is predicted to be defective,m d representing defective label imagesmIs marked as a defective area.
Segmentation loss functionL s Is a weighted sum of two parts:
whereinδ 1Andδ 2respectively, are preset weight coefficients, one example beingδ 1=0.5，δ 2=0.5。
For total loss functionL total In one embodiment, the reconstruction may be a loss functionL r And a segmentation loss functionL s Added as a function of total lossL total Namely:
step 190: according to the total loss functionL total For defect detection modelFTraining to obtain corresponding model parameters to complete the defect detection modelFAnd (4) training.
In the early stage of integral model training, the feature extractorfThe parameters of the model can be kept in a frozen state, do not participate in the training of the model and do not update the parameters, when the parameters of other parts in the model are updated for preset times, the training state tends to be stable, and the feature extractorfThen the deblocking is started to update the parameters to improve the feature extractorfFeature extraction capability on images.
The defect detection model can be used for defect detection after training of the defect detection model is completed, the invention provides an unsupervised defect detection method, defects of a product are detected by using the defect detection model trained by the training method of any embodiment of the invention, please refer to fig. 4, and the method in one embodiment comprises steps 210-230, which are specifically described below.
Step 210: and acquiring an image to be detected of the detected object.
The detected object can be a product in an industrial production process, such as a mechanical part, an electronic component and the like. Any image pickup device such as a camera can be used for shooting the detected object to obtain the image to be detected.
Step 220: the image to be detected is input into a defect detection model trained by the training method of any embodiment of the invention to obtain a defect prediction image.
Step 230: and performing threshold segmentation on the defect prediction image by using the set segmentation threshold to obtain a defect segmentation image. The final result is shown in fig. 5, and the positions of the defects can be known from the defect segmentation map.
The invention adopts peopleThe normal sample image is subjected to data expansion for the manufactured abnormity, and the training of the defect detection model is completed according to the data expansion, but the artificial abnormity is not some or several real defects, but is sample data deviating from the normality of the normal sample image, so that the segmentation threshold value directly set according to the sample image is inaccurate, and the detection performance of the defect detection model is reduced. Therefore, in an embodiment of the present invention, a method for setting a segmentation threshold is provided, after training of a defect detection model is completed, by controlling a false detection rate of a normal sample image (step (c)) (False Positive Rate, FPR) To obtain an accurate segmentation threshold. Referring to fig. 6, an embodiment of the method for setting the segmentation threshold includes steps 231 to 237, which are described in detail below.
Step 231: and acquiring a verification set, wherein the verification set consists of normal sample images.
Step 232: and respectively inputting the normal sample images in the verification set into the trained defect detection model to obtain a defect prediction image of each normal sample image.
Step 233: obtaining maximum predicted value of pixel levelS maxAnd minimum predicted valueS minI.e. the maximum prediction value of all pixels of all defect prediction imagesS maxAnd minimum predicted valueS min。
Step 234: performing threshold segmentation on each defect prediction image by using the current value as a segmentation threshold, and then calculating the false detection rateFPR. Wherein the initial value of the current value is the maximum predicted valueS max. When threshold segmentation is performed, pixel points whose predicted values are greater than the segmentation threshold can be determined as defective points.
Step 235: and judging whether the false detection rate is greater than or equal to a preset false detection rate threshold, if so, executing step 236, otherwise, executing step 237. In a specific embodiment, the false detection rate threshold may be set to 0.01.
Step 236: the division threshold at this time is set as the final division threshold.
Step 237: the current value is subtracted by the preset step Δ and the process returns to step 234. The calculation formula of the step length delta is as follows:
whereinS step Is a preset number of steps.
Due to technical limitations, in order to ensure the detection rate of abnormal images, the similar methods commonly used in the industry generally adopt a mode of reducing a segmentation threshold, so that the false detection rate of normal images is too high, and in practical application, secondary detection is often needed. The method for setting the segmentation threshold according to the false detection rate of the normal sample image can improve the detection performance of the defect while keeping a low false detection rate on the normal image.
According to the unsupervised defect detection model training method and the unsupervised defect detection model defect detection method, only the normal sample image is used for training, the artificial abnormal sample image is obtained by applying artificial noise to the normal sample image, the real abnormal sample image and the corresponding label are not needed, a user can train the model by only providing the normal sample image, the defect detection task is completed, the limitation of a supervision algorithm is overcome, the manpower is saved, the production efficiency of a manufacturer is improved, and the cost reduction and efficiency improvement are realized. In the training process, the artificial abnormal sample image or the normal sample image is input into the model and is reconstructed through the reconstruction module, and the normal sample image is used as a learning target for image reconstruction, so that the normal image or the abnormal image input into the defect detection model can be mapped onto the normal image after image reconstruction operation, the refined reconstruction quality of the normal image is improved, and the difference between the abnormal image and the reconstructed image is increased; the reconstructed image is spliced with the input image, and the input defect segmentation module performs defect segmentation, so that the position information of the defect is reserved, the false detection of a normal image is reduced, the accurate positioning of the position of the defect is realized, the precision of the defect segmentation is improved, and the detection performance is improved.
The method for setting the segmentation threshold value can reduce the false detection rate, and can acquire the segmentation threshold value by controlling the false detection rate of the normal sample image, so that a user can set the false detection rate differently according to the requirements of the user, and the defects of different products can be detected more conveniently.
Those skilled in the art will appreciate that all or part of the functions of the methods in the above embodiments may be implemented by hardware, or may be implemented by a computer program. When all or part of the functions of the above embodiments are implemented by a computer program, the program may be stored in a computer-readable storage medium, and the storage medium may include: a read only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to realize the above functions. For example, the program may be stored in a memory of the device, and when the program in the memory is executed by the processor, all or part of the functions described above may be implemented. In addition, when all or part of the functions in the above embodiments are implemented by a computer program, the program may be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a removable hard disk, and may be downloaded or copied to a memory of a local device, or may be version-updated in a system of the local device, and when the program in the memory is executed by a processor, all or part of the functions in the above embodiments may be implemented.
The present invention has been described in terms of specific examples, which are provided to aid understanding of the invention and are not intended to be limiting. For a person skilled in the art to which the invention pertains, several simple deductions, modifications or substitutions may be made according to the idea of the invention.
Claims (11)
1. A method for training an unsupervised defect detection model is characterized in that the defect detection model comprises a feature extractor, a feature compression module, an image reconstruction module and a defect segmentation module, and the method for training the defect detection model comprises the following steps:
acquiring a normal sample image set, wherein the normal sample image is an image of a defect-free standard substance;
extracting a predetermined ratio from the normal sample image setpOf the normal sample imagexFor the extracted normal sample imagexApplying artificial noise to obtain artificial abnormal sample image
And forming sample pairs with the original image
For the normal sample image not extractedxCopying to obtain a copied image, and forming a sample pair with the original image
Forming a training sample set by all sample pairs;
obtaining sample pairs from the training sample set, when obtaining the sample pairs
Then the artificial abnormal sample image is taken
Inputting the initial features into the feature extractor to obtain initial featuresαWhen obtaining the sample pair
Then any one of the normal sample images is takenxInputting the initial features into the feature extractor to obtain initial featuresα；
Characterizing the initial featureαInput into the feature compression module to obtain compressed featuresβ；
Characterizing said compressionβInput into the image reconstruction module to obtain a reconstructed imagey；
If the obtained sample is a pair
Then an image will be reconstructedyWith artificial abnormal sample images
Splicing on the channels to obtain a merged image, if the obtained merged image is a sample pair
Then an image will be reconstructedyAnd the normal sample imagexSplicing the channels to obtain a merged image;
inputting the merged image into the defect segmentation module to obtain a defect prediction imagez；
From the reconstructed imageyAnd the normal sample image in the corresponding sample pairxConstructing a reconstruction loss functionL r Predicting the image based on the defectzAnd corresponding defective label imagesmConstructing a segmentation loss functionL s From the reconstruction loss functionL r And a segmentation loss functionL s Constructing a total loss functionL total ；
According to the total loss functionL total And training the defect detection model to obtain corresponding model parameters.
2. The training method of claim 1, wherein the feature extractor is a pre-trained neural network, the initial features beingαA feature map comprising one or more network layer outputs of the pre-trained neural network.
3. The training method according to claim 2, wherein the parameters of the feature extractor are kept unchanged during the training process, and when the parameters of the other parts of the defect detection model except the feature extractor are updated for a preset number of times, the feature extractor starts to update the parameters.
4. The training method of claim 2, wherein the feature compression module applies to the initial features byαPerforming compression to obtain compression characteristicsβ：
Presetting a compression sizes；
Characterizing the initial featureαIs resized to the compressed sizes；
Splicing the feature graphs after the size adjustment on the channel to form a feature group;
dividing each feature map in the feature group into a plurality of non-overlapping small blocks, and taking the pixel average value of each small block to form a new feature map;
channel compression of feature sets to obtain compressed featuresβ。
5. The training method of claim 1, wherein the image reconstruction module comprises one or more of a deconvolution layer, a batch normalization layer, and a ReLU activation function.
6. The training method of claim 1, wherein the reconstruction loss functionL r The method specifically comprises the following steps:
whereinλ 1Andλ 2in order to be a preset balance coefficient,L MSE represents the root mean square error, and
whereiny i Representing reconstructed imagesyTo (1) aiThe value of each of the pixels is calculated,x i graph representing corresponding normal samplesImage (A)xTo (1) aiThe value of each of the pixels is calculated,nrepresenting the number of pixels;
L SSIM represents a loss of structural similarity, an
Wherein
Whereinμ y Andμ x respectively reconstructed imagesyAnd corresponding normal sample imagesxThe average value of the pixels of (a),
and
respectively reconstructed imagesyAnd corresponding normal sample imagesxThe variance of the pixels of (a) is,σ yx for reconstructing imagesyAnd corresponding normal sample imagesxCovariance of c1And c2Is a preset constant.
7. The training method of claim 1, wherein the segmentation loss functionL s The method specifically comprises the following steps:
whereinδ 1Andδ 2are respectively a preset weight coefficient,L cross represents a cross-entropy loss, and
whereinz i Predictive picture representing defectszTo (1) aiThe predicted value of each pixel is calculated,m i representing corresponding defect label imagesmTo (1) aiThe label value of each pixel is set to,nrepresenting the number of pixels;
L dice represents the loss of the cross-over ratio, and
whereinz d Predictive picture representing defectszIn the area of the image that is predicted to be defective,m d representing defective label imagesmIs marked as a defective area.
8. The training method of claim 1, wherein the total loss functionL total The method specifically comprises the following steps:
9. an unsupervised defect detection method, comprising:
acquiring an image to be detected of a detected object;
inputting the image to be detected into a defect detection model trained by the training method according to any one of claims 1 to 8 to obtain a defect prediction image;
and performing threshold segmentation on the defect prediction image by using a set segmentation threshold to obtain a defect segmentation image.
10. The defect detection method of claim 9, wherein the segmentation threshold is set by:
obtaining a verification set, wherein the verification set is composed of normal sample images;
inputting each normal sample image in the verification set into the defect detection model respectively to obtain a defect prediction image of each normal sample image;
obtaining the maximum predicted value of all pixels of all defect predicted imagesS maxAnd minimum predicted valueS min；
From maximum predicted valueS maxStarting and gradually reducing the step length delta, performing threshold segmentation on each defect prediction image by taking the current numerical value as a segmentation threshold, then calculating the false detection rate until the false detection rate is greater than or equal to a preset false detection rate threshold, and taking the segmentation threshold at the moment as a final segmentation threshold, wherein when the threshold segmentation is performed, pixel points with the predicted values greater than the segmentation threshold are determined as defect points, and the calculation formula of the step length delta is as follows:
whereinS step Is a preset number of steps.
11. A computer-readable storage medium, characterized in that the medium has stored thereon a program which is executable by a processor to implement the training method of any one of claims 1-8 and/or the defect detection method of any one of claims 9-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210617738.7A CN114693685B (en) | 2022-06-02 | 2022-06-02 | Unsupervised defect detection model training method and defect detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210617738.7A CN114693685B (en) | 2022-06-02 | 2022-06-02 | Unsupervised defect detection model training method and defect detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114693685A true CN114693685A (en) | 2022-07-01 |
| CN114693685B CN114693685B (en) | 2022-08-09 |
Family
ID=82130901
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210617738.7A Active CN114693685B (en) | 2022-06-02 | 2022-06-02 | Unsupervised defect detection model training method and defect detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114693685B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114841294A (en) * | 2022-07-04 | 2022-08-02 | 杭州德适生物科技有限公司 | Classifier model training method and device for detecting chromosome structure abnormality |
| CN115239719A (en) * | 2022-09-22 | 2022-10-25 | 南昌昂坤半导体设备有限公司 | Defect detection method, system, electronic device and storage medium |
| CN115331077A (en) * | 2022-08-22 | 2022-11-11 | 北京百度网讯科技有限公司 | Training method of feature extraction model, target classification method, device and equipment |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103440654A (en) * | 2013-08-27 | 2013-12-11 | 南京大学 | LCD foreign body defect detection method |
| CN106803244A (en) * | 2016-11-24 | 2017-06-06 | 深圳市华汉伟业科技有限公司 | Defect identification method and system |
| CN110796637A (en) * | 2019-09-29 | 2020-02-14 | 郑州金惠计算机系统工程有限公司 | Training and testing method and device of image defect detection model and storage medium |
| CN112446869A (en) * | 2020-11-27 | 2021-03-05 | 鹏城实验室 | Unsupervised industrial product defect detection method and device based on deep learning |
| CN112837295A (en) * | 2021-02-01 | 2021-05-25 | 北京工业大学 | Rubber glove defect detection method based on generation of countermeasure network |
| CN112884721A (en) * | 2021-02-01 | 2021-06-01 | 成都市览图科技有限公司 | Anomaly detection method and system and computer readable storage medium |
| CN113674263A (en) * | 2021-08-27 | 2021-11-19 | 浙江捷瑞电力科技有限公司 | Small sample defect detection method based on generation type countermeasure network |
| CN114092386A (en) * | 2020-05-26 | 2022-02-25 | 富士通株式会社 | Defect detection method and apparatus |
| CN114299034A (en) * | 2021-12-30 | 2022-04-08 | 杭州海康威视数字技术股份有限公司 | Defect detection model training method, defect detection method and device |
-
2022
- 2022-06-02 CN CN202210617738.7A patent/CN114693685B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103440654A (en) * | 2013-08-27 | 2013-12-11 | 南京大学 | LCD foreign body defect detection method |
| CN106803244A (en) * | 2016-11-24 | 2017-06-06 | 深圳市华汉伟业科技有限公司 | Defect identification method and system |
| CN110796637A (en) * | 2019-09-29 | 2020-02-14 | 郑州金惠计算机系统工程有限公司 | Training and testing method and device of image defect detection model and storage medium |
| CN114092386A (en) * | 2020-05-26 | 2022-02-25 | 富士通株式会社 | Defect detection method and apparatus |
| CN112446869A (en) * | 2020-11-27 | 2021-03-05 | 鹏城实验室 | Unsupervised industrial product defect detection method and device based on deep learning |
| CN112837295A (en) * | 2021-02-01 | 2021-05-25 | 北京工业大学 | Rubber glove defect detection method based on generation of countermeasure network |
| CN112884721A (en) * | 2021-02-01 | 2021-06-01 | 成都市览图科技有限公司 | Anomaly detection method and system and computer readable storage medium |
| CN113674263A (en) * | 2021-08-27 | 2021-11-19 | 浙江捷瑞电力科技有限公司 | Small sample defect detection method based on generation type countermeasure network |
| CN114299034A (en) * | 2021-12-30 | 2022-04-08 | 杭州海康威视数字技术股份有限公司 | Defect detection model training method, defect detection method and device |
Non-Patent Citations (1)
| Title |
|---|
| JUHUA LIU ET AL: "Multistage GAN for fabric defect detection", 《IEEE TRANSACTION ON IMAGE PROCESS》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114841294A (en) * | 2022-07-04 | 2022-08-02 | 杭州德适生物科技有限公司 | Classifier model training method and device for detecting chromosome structure abnormality |
| CN115331077A (en) * | 2022-08-22 | 2022-11-11 | 北京百度网讯科技有限公司 | Training method of feature extraction model, target classification method, device and equipment |
| CN115331077B (en) * | 2022-08-22 | 2024-04-26 | 北京百度网讯科技有限公司 | Training method of feature extraction model, target classification method, device and equipment |
| CN115239719A (en) * | 2022-09-22 | 2022-10-25 | 南昌昂坤半导体设备有限公司 | Defect detection method, system, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114693685B (en) | 2022-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114693685B (en) | Unsupervised defect detection model training method and defect detection method | |
| CN111383209B (en) | Unsupervised flaw detection method based on full convolution self-encoder network | |
| CN106846316B (en) | Automatic identification method for typical defect images in GIS | |
| CN108230278B (en) | Image raindrop removing method based on generation countermeasure network | |
| CN112818827B (en) | Method for judging stage temperature control point in tobacco leaf baking process based on image recognition | |
| CN106096655B (en) | A kind of remote sensing image airplane detection method based on convolutional neural networks | |
| CN114663346A (en) | Strip steel surface defect detection method based on improved YOLOv5 network | |
| CN111047543A (en) | Image enhancement method, device and storage medium | |
| WO2024021461A1 (en) | Defect detection method and apparatus, device, and storage medium | |
| CN107644203B (en) | Feature point detection method for shape adaptive classification | |
| CN115994900A (en) | Unsupervised defect detection method and system based on transfer learning and storage medium | |
| CN115731166A (en) | High-voltage cable connector polishing defect detection method based on deep learning | |
| CN115880170A (en) | Single-image rain removing method and system based on image prior and gated attention learning | |
| CN111428730B (en) | Weak supervision fine-grained object classification method | |
| CN111462002A (en) | Underwater image enhancement and restoration method based on convolutional neural network | |
| CN110660048A (en) | Leather surface defect detection algorithm based on shape characteristics | |
| CN114202491A (en) | Method and system for enhancing optical image | |
| CN112258592A (en) | Method and related device for generating human face visible light pattern | |
| CN117237279A (en) | Blind quality evaluation method and system for non-uniform distortion panoramic image | |
| CN110766708A (en) | Image comparison method based on contour similarity | |
| CN115619796A (en) | Method and device for obtaining photovoltaic module template and nonvolatile storage medium | |
| CN109741313A (en) | The non-reference picture quality appraisement method of independent component analysis and convolutional neural networks | |
| CN115760695A (en) | Image anomaly identification method based on depth vision model | |
| CN115272203A (en) | No-reference image quality evaluation method based on deep learning | |
| CN113744199A (en) | Image damage detection method, electronic device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |