CN115409822A - Industrial part surface anomaly detection method based on self-supervision defect detection algorithm - Google Patents

Abstract

The invention discloses an industrial part surface anomaly detection method based on an automatic supervision defect detection algorithm, which comprises the following steps of: acquiring normal sample image data of an industrial part; carrying out data enhancement processing on the normal sample image data; constructing an abnormal defect detection model; inputting the image of the industrial part to be detected into an abnormal defect detection model, judging whether the image of the industrial part to be detected is an abnormal image, and then obtaining a defect activation diagram of the abnormal image in the last group of feature diagrams of the model through Grad-CAM to realize defect positioning. The defect data can be prevented from being collected by enhancing the normal sample image data to generate the defect data; the ability of positioning the defects can be improved by using the formed CAM-loss as a loss function; by introducing an ECA-net attention mechanism into a backbone network, the receptive field can be increased, and the algorithm classification capability and defect data discrimination capability are effectively improved; the invention only needs normal image data and does not need marking, thereby saving labor cost and showing better detection capability.

Description

Industrial part surface anomaly detection method based on self-supervision defect detection algorithm

Technical Field

The invention belongs to the field of image detection, and particularly relates to an industrial part surface anomaly detection method based on an automatic supervision defect detection algorithm.

Background

The image anomaly detection means that only normal pictures are used for identifying anomalies in the training stage. In a real scene, abnormal samples are very rare and various, and it is not easy to collect a large number of possible samples. Therefore, typical surveillance methods cannot be applied directly to these scenarios. In contrast, it is relatively much easier to collect normal training samples. In recent years, due to the potential application of anomaly detection in the detection of manufacturing defects in the industrial field, researchers have attracted extensive attention.

At present, a monitoring mode is mostly used in a mainstream industrial defect detection method, a large amount of defect data needs to be collected in the mode, but products are influenced by the market in actual production, an enterprise is difficult to ensure that all kinds of products are produced in the same time period, the number of the defect products is small, and the types of the defects are large. When various defects are collected comprehensively and the number of the defects meets the requirements of a deep learning algorithm, the time span is long, the labeled data is adjusted greatly after the defect data are collected, and a large amount of data can be labeled by a lot of manpower.

Disclosure of Invention

In view of the problems in the prior art, the present invention aims to provide a method for detecting surface anomalies of industrial parts based on a self-supervision defect detection algorithm. The detection method only needs normal image data and does not need labeling, so that the labor cost is saved, and the detection capability is better.

In order to achieve the above purpose, the invention adopts the following technical scheme:

the industrial part surface anomaly detection method based on the self-supervision defect detection algorithm comprises the following steps:

s1, acquiring normal sample image data of an industrial part;

s2, performing data enhancement processing on the acquired normal sample image data of the industrial part to acquire abnormal data, and forming an automatic supervision training set by the acquired abnormal data and the normal sample image data;

s3, building a convolutional neural network model introducing an ECA-net attention mechanism, wherein ResNet is used as a backbone network in the convolutional neural network model;

s4, inputting the self-supervision training set into the constructed convolutional neural network model for training, and taking the trained model as an abnormal defect detection model;

and S5, inputting the image of the industrial part to be detected into an abnormal defect detection model, judging whether the image of the industrial part to be detected is an abnormal image or not through the model, and then obtaining a defect activation diagram of the abnormal image in the last group of feature diagrams of the model through Grad-CAM (Grad-CAM) to realize the positioning of the surface defects of the industrial part.

Preferably, in step S2, the data enhancement processing adopts a cutpast and cutpast-scar data enhancement method, and after data enhancement, two abnormal data are generated for each normal sample image data, wherein the abnormal data obtained by the cutpast data enhancement method are used for simulating a large defect; abnormal data obtained by adopting a Cutpase-scar data enhancement method is used for simulating tiny defects; and then marking the obtained abnormal data and the normal sample image data, wherein the normal sample image data is marked as 0, the abnormal data is marked as 1, and an automatic supervision training set is formed.

Preferably, an improved cross entropy function is adopted as a loss function in the process of training the convolutional neural network model, and the improved cross entropy function is defined as CAM-loss which is calculated by adopting the following formula (1):

CAM-loss＝10 ^-5 ×L _cam+ L _ce (1)

wherein L is _ce Representing the cross entropy loss function, L _cam As a regularization term, a distance constraint function between the CAAM and the CAM is expressed, and is calculated by the following formula (2):

wherein H and W represent the height and width of the characteristic diagram, respectively, and l ₂ Representing Euclidean distance, CAM ₁ Representing an abnormal class activation graph, and CAAM representing the sum of the last group of feature graphs. The CAM-loss can drive the convolutional neural network to express the characteristics of the target class and inhibit the characteristics of the non-target class or the background, so that the characteristic capability with higher discriminative power is obtained.

Preferably, the defect activation map of the abnormal image obtained by the Grad-CAM in the last group of feature maps of the model is used for realizing the positioning of the surface defects of the industrial part, and the method specifically comprises the following steps:

s51, calculating the weights of the defects at different positions of the last group of characteristic diagrams by adopting the gradient of network back propagation;

s52, weighting and summing the last group of feature maps of the network by using the obtained weights to obtain an abnormal activation map, and positioning the surface defects of the industrial parts.

Compared with the prior art, the invention has the following beneficial effects:

(1) According to the method, the defect data is generated by performing data enhancement on the normal sample image data of the industrial part, so that the collection of the defect data is avoided; by adding an ECA-net attention mechanism in a backbone network, the receptive field can be increased, and the algorithm classification capability and defect data discrimination capability are effectively improved; the ability to locate defects can be improved by using the constructed CAM-loss as a loss function.

(2) The detection method only needs normal image data and does not need marking, thereby saving labor cost and showing better detection capability of the surface defects of the industrial parts.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flow chart of the method for detecting surface abnormality of an industrial part based on an automatic supervision defect detection algorithm;

FIG. 2 is a schematic diagram of a process for generating abnormal data from normal image data;

FIG. 3 is a schematic diagram of a process of inputting an auto-supervised training set into a constructed convolutional neural network model for training;

FIG. 4 is a schematic diagram of a process for generating an abnormal activation map;

FIG. 5 is a view showing the positioning effect obtained in example 2;

FIG. 6 is a view showing the positioning effect obtained in example 2;

FIG. 7 is a view showing the positioning effect obtained in example 2;

fig. 8 is a view showing the positioning effect obtained in example 2.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details.

Example 1

Referring to fig. 1, the embodiment provides an industrial part surface anomaly detection method based on an auto-supervision defect detection algorithm, which includes the following steps:

s1, obtaining a normal sample image;

s2, performing data enhancement processing on the acquired normal sample image data to acquire abnormal data, and then forming a training set by the acquired abnormal data and the normal sample image data;

specifically, a Cutpase and a Cutpase-scar data enhancement method are adopted to carry out data enhancement processing on the acquired normal sample image data, and after the data enhancement, two abnormal data are generated on each normal sample image data, wherein the abnormal data acquired by the Cutpase data enhancement method are used for simulating large defects; abnormal data obtained by adopting a Cutpase-scar data enhancement method is used for simulating tiny defects; and then marking the obtained abnormal data and the normal sample image data, wherein the normal sample image data is marked as 0, the abnormal data is marked as 1, and an automatic supervision training set is formed.

Referring to fig. 2, the above-mentioned cutcast is a data enhancement method that can create abnormal data, and is used for simulating a large defect. Compared with CutOut and Randomerasing, the method for enhancing CutPaste data is simpler; there are two differences compared to CutMix: on one hand, cutMix directly pastes image blocks to a normal image by using the existing image label and MixUp in a target; on the other hand, cutMix extracts rectangular tiles from an image and pastes to random locations in another image. Cutcast-scar is a data enhancement method derived from cutcast, which is used to simulate small defects,

s3, building a convolutional neural network model introducing an ECA-net attention mechanism, wherein ResNet is used as a backbone network in the convolutional neural network model;

because the convolution operation processes the local neighborhood in space or time, in the repeated application of the operation, the remote dependency relationship can be captured only by gradually propagating signals through data, and the problems of low calculation efficiency and difficult optimization of a neural network exist in the repeated local convolution operation. In view of this, the present invention introduces an ECA-net attention mechanism, which can help the neural network to have a larger receptive field, so that the neural network can learn to use global information to selectively emphasize informative features and suppress less useful features, thereby improving the ability of the convolutional neural network to capture anomalies. To obtain global information, the ECA-net attention module compresses global spatial information into channels and generates a one-dimensional weight matrix by using global average pooling, where the global average pooling process of channels is denoted by G (X), which is calculated by the following equation:

where X represents the normal profile matrix, X ∈ R (W × H × C), where H, W and C in turn represent the height, width, and channel of the element profile.

To exploit the information aggregated in the compression operation described above, a second convolution operation is then performed, which is intended to selectively emphasize useful features and suppress less useful features using global spatial information. Therefore, the channel attention weight matrix ω storing the different channel weight values of the image can be obtained by the following formula:

ω＝σ(C1D _k (G(X)))

in the formula, C1D represents a 1D convolution, and σ represents a linear unit after correction.

The ECA-net attention mechanism can realize information interaction between channels through one-dimensional convolution with a convolution kernel of k, and finally realizes the recalibration of the network to the characteristics. Specifically, the attention direction of the network can be adjusted by multiplying the obtained channel attention weight matrix ω by the normal feature map matrix X, and the method adopts

Specifically, the formula is shown as follows:

in the formula (I), the compound is shown in the specification,

representing the channel multiplication between feature maps ω and X, in particular, the ECA-net attention module with k =3 achieves similar results with SE-net, while having lower model complexity, where efficiency and effectiveness are guaranteed by properly capturing local cross-channel interactions.

The invention also introduces CAM-loss as a loss function of a convolutional neural network model process built by training, the CAM-loss is different from other loss functions introducing additional constraints on the basis of cross entropy, a CAM-loss constraint CAAM is closer to the CAM of the target class, the characteristics of the target class are well expressed, and the characteristics of the non-target class are inhibited. CAM-loss is applicable to any CNN architecture,with negligible additional parameters and computational power. CAM represents a spatial difference region for identifying a specific class, CAAM represents a final set of feature map sums, and a distance constraint between CAAM and CAM is defined as L _cam ，L _cam Represented by the following formula:

wherein H and W represent the height and width of the characteristic diagram, respectively, and l ₂ Representing the euclidean distance. CAM ₁ An exception class activation map is represented.

CAM-loss is the cross entropy loss function L _ce The calculation formula of (2) is as follows:

CAM-loss＝βL _cam+ L _ce

in the formula, L _ce Representing a cross entropy loss function; beta means that L is added from the beta-th training _cam ，L _cam It is considered necessary to add the CAM loss function at an appropriate time during training, since β requires a lot of experimentation and is less effective in an unsupervised network. In view of this, the present invention provides L _cam Added as a regularization term to the cross-entropy loss function L _ce And then the CAM-loss is calculated by the following formula:

CAM-loss＝10 ^-5 ×L _cam +L _ce

the last linear layer is added on the basis of the average pool layer, and the backbone network is not directly classified, so that the operation can better improve the classification of the network. The CAM-loss constructed by the method can be used as a loss function to drive the convolutional neural network to express the characteristics of the target category and inhibit the characteristics of the non-target category or the background, so that the characteristic capability with higher discriminative power is obtained.

S4, inputting the self-supervision training set into the constructed convolutional neural network model for training, and taking the trained model as an abnormal defect detection model;

in particular, referring to FIG. 3, the convolutional neural network incorporating the ECA-net attention mechanism is named ECA-ResNet and the self-supervised training is pooled and exportedInputting into ECA-ResNet, outputting the final group of feature graph sets, and sequentially passing through GAP (global average pooling) and full Connected Layer (parameter W) _i ¹ (i belongs to (0,n))), and a detection head (MLP _ head) of the multilayer perceptron, and outputs a judgment result P of the picture to be detected ₁ And P ₂ And a genuine label (1,0); then, P ₁ 、P ₂ Constructing a Cross Entropy loss function (Cross Encopy) together with a real label (1,0), and simultaneously performing Euclidean distance operation on CAAM and CAM to obtain L _cam Finally, cross Entropy loss functions (Cross Encopy) and 10 are used ^-5 ×L _cam The sum of which constitutes the CAM-loss as a loss function. The CAM-loss can drive the constructed convolutional neural network to express the characteristics of the target category and inhibit the characteristics of the non-target category or the background, so that the characteristic capability with more discriminative power is obtained, and an algorithm operation process is completed. And then updating parameters of each network layer through back propagation, and obtaining a Model capable of classifying normal and abnormal data after 256 times of training, namely obtaining an abnormal defect detection Model.

Wherein the target of the self-supervised training (i.e. the training-derived model) is defined as L _cp ，L _cp Is represented by the following formula:

L _cp ＝E _x∈X [CL(g(x)，0)+CL(g(CP(x))，1)+CL(g(CPS(x))，1)]

in the formula, X represents normal data, CP (. Quadrature.) represents CutPase, CPS (. Quadrature.) represents CutPase-scar, g represents convolutional neural network, and CL (. Quadrature.) represents CAM-loss.

And S5, inputting the image of the industrial part to be detected into an abnormal defect detection model, judging whether the image of the industrial part to be detected is an abnormal image or not through the model, and then obtaining a defect activation diagram of the abnormal image in the last group of feature diagrams of the model through Grad-CAM to realize the positioning of the surface defects of the industrial part.

In order to better understand and intuitively explain CNN and make better decision on a model, the invention adopts Grad-weighted Class Activation Mapping (Grad-CAM) to locate abnormal images.

Specifically, referring to fig. 4, inputting an image to be detected into an abnormal defect detection model, judging whether the image to be detected is an abnormal image or not through the model, outputting a label 1 for abnormal data and a label 0 for a normal picture, and then positioning the image with the label 1 through the Grad-CAM, namely, the Grad-CAM calculates the weight of the defect at different positions of the last group of feature maps by adopting a gradient of network back propagation; and performing weighted summation on the last group of feature maps of the network by using the obtained weights to obtain an abnormal activation map, so as to realize defect positioning, and in order to obtain a category 1 positioning map (such as CutPaste, which uses 1 to represent a defect type), compared with the existing CAM, the CNN of any structure can be visualized by using Grad-CAM under the condition of not modifying the network structure or retraining.

In the invention, the data enhancement by adopting the CutPaste is not perfect simulation of real defects, but finds irregularity in the learning process and well summarizes invisible abnormity; and the ECA-net is introduced into the backbone network in the process of building the neural network model, so that the learning process can be simplified, and the representation capability of the network is obviously enhanced. In addition, grad-CAM is used for realizing image anomaly positioning; for better positioning, L _cam As a regular term, a new loss function CAM-loss is constructed, and the CAM-loss can drive a backbone network to express the characteristics of the target category and inhibit the characteristics of the non-target category or the background, so that more discriminative characteristic representation is obtained; and the constructed CAM-loss can be applied to any CNN architecture, thereby effectively enhancing the compactness in the class and the separability between the classes.

Example 2

(1) Acquiring normal industrial part image data, performing data increasing processing, and then constructing a self-supervision training set;

(2) Network training: the batch size is set to 32, the input image size is 256 × 256, and the optimizers for both the generator and discriminator are SGD. In order to avoid increasing the amount of calculation, the parameter k of the ECA network is set to 3, the learning rate is set to 0.03, and models of 256 generations are trained, and the detection method of the invention is performed on the NVIDIA Quadro P4000 GPU by using pytorech.

(3) Abnormality detection: the inventive detection method is applied on a MVTec AD dataset, which is a comprehensive dataset for anomaly detection in industrial detection images, containing images of five texture classes (i.e. carpet, grid, leather, tile and wood) and images of ten object classes (i.e. bottles, cables, capsules, hazelnuts, metal nuts, pills, screws, toothbrushes, transistors and zippers).

(4) And (3) quantitative performance evaluation: the detection method of the invention is compared with the RIAD and the MemSTC Net, and the AUC for evaluating the image-level abnormality detection performance is reported, wherein a Gaussian density estimator (GDE is used for normal/abnormal classification, the Area (AUC) under the receiving operation characteristic is calculated by gradually changing the threshold value of A, the higher the AUC is, the better the performance of the method is, and the quantitative performance evaluation result is shown in the table 1.

TABLE 1

As can be seen from the results in Table 1, the detection method of the present invention achieves the best performance in terms of average AUC, which verifies the effectiveness of the network proposed by the present invention on industrial detection images. The detection method realizes the highest ROC-AUC average value which is 2 percent higher than that of the prior most advanced method. RIAD and MemSTC Net perform better on objects like screws, toothbrushes, transistors etc. However, the detection method of the present invention is superior to other methods in 11 classes and has very good performance.

Ablation experiment: an ablation study was performed on the MVTecAD dataset with ResNet plus MLP projection head above the mean cell layer, and compared to the detection method of the present invention, with the results shown in table 2.

TABLE 2

As can be seen from the results in Table 2, the detection method of the invention realizes the highest average value of ROC-AUC, which is 5 percent higher than that of the ResNet method. The ResNet method has better performance on object classes (such as screws and toothbrushes), and the detection method has better performance on other classes. Furthermore, a comparison of the two methods in terms of position is required.

Referring to FIGS. 5-8, which are graphs of the localization effects of the above two methods, it can be seen from the results of FIGS. 5-8 that the detection method of the present invention exhibits better locations in most categories, including the category with lower AUC.

In conclusion, the invention constructs an automatic supervision network for image anomaly detection, namely an anomaly defect detection model, and evaluates the anomaly defect detection model on a real MVTec anomaly detection data set, and the evaluation result shows that the detection method realizes an advanced image anomaly detection result which is 1 percentage point better than that of the existing method and is better than that of the existing method in most categories. Compared with ResNet, the detection method of the invention better realizes the abnormal positioning task.

The present invention is not limited to the above-described embodiments, and those skilled in the art will be able to make various modifications without creative efforts from the above-described conception, and fall within the scope of the present invention.

Claims (4)

1. The industrial part surface anomaly detection method based on the self-supervision defect detection algorithm is characterized by comprising the following steps of:

s1, acquiring normal sample image data of an industrial part;

s2, performing data enhancement processing on the acquired normal sample image data of the industrial part to acquire abnormal data, and forming an automatic supervision training set by the acquired abnormal data and the normal sample image data;

s3, building a convolutional neural network model introducing an ECA-net attention mechanism, wherein ResNet is used as a backbone network in the convolutional neural network model;

s4, inputting the self-supervision training set into the constructed convolutional neural network model for training, and taking the trained model as an abnormal defect detection model;

and S5, inputting the image of the industrial part to be detected into an abnormal defect detection model, judging whether the image of the industrial part to be detected is an abnormal image or not through the model, and then obtaining a defect activation diagram of the abnormal image in the last group of feature diagrams of the model through Grad-CAM to realize the positioning of the surface defects of the industrial part.

2. The method for detecting the surface abnormality of the industrial part based on the self-supervision defect detection algorithm according to the claim 1, characterized in that in the step S2, the data enhancement processing adopts a Cutpase and Cutpase-scar data enhancement method, and after data enhancement, each normal sample image data generates two abnormal data, wherein the abnormal data obtained by the Cutpase data enhancement method is used for simulating a large defect; abnormal data obtained by adopting a Cutpase-scar data enhancement method is used for simulating tiny defects; and then marking the obtained abnormal data and the normal sample image data, wherein the normal sample image data is marked as 0, the abnormal data is marked as 1, and an automatic supervision training set is formed.

3. The method for detecting surface anomalies of industrial parts based on an unsupervised defect detection algorithm according to claim 1, characterized by using an improved cross-entropy function as a loss function in the process of training the convolutional neural network model, and defining the improved cross-entropy function as a CAM-loss, wherein the CAM-loss is calculated by the following formula (1):

CAM-loss＝10 ^-5 ×L _cam+ L _ce (1)

wherein L is _ce Representing the cross entropy loss function, L _cam As a regularization term, a distance constraint function between the CAAM and the CAM is expressed, and is calculated by the following formula (2):

wherein H and W represent the height and width of the characteristic diagram, respectively, and l ₂ Representing Euclidean distance, CAM ₁ Representing an abnormal class activation graph, and CAAM representing the sum of the last group of feature graphs.

4. The method for detecting the surface abnormality of the industrial part based on the self-supervision defect detection algorithm according to claim 1, wherein in the step S5, the defect activation map of the last group of feature maps of the abnormal image obtained by the Grad-CAM model is used for realizing the positioning of the surface defect of the industrial part, and the method comprises the following specific steps:

s51, calculating the weights of the defects at different positions of the last group of characteristic diagrams by adopting the gradient of network back propagation;

s52, weighting and summing the last group of feature maps of the network by using the obtained weights to obtain an abnormal activation map, and positioning the surface defects of the industrial parts.

Images