WO2021253632A1

WO2021253632A1 - Cloth defect detection method based on adversarial neural network, and terminal and storage medium

Info

Publication number: WO2021253632A1
Application number: PCT/CN2020/111475
Authority: WO
Inventors: 周凯; 吴小飞; 庞凤江; 武艳萍; 彭其栋; 张帆
Original assignee: 深圳新视智科技术有限公司
Priority date: 2020-06-19
Filing date: 2020-08-26
Publication date: 2021-12-23
Also published as: CN111724372A

Abstract

Disclosed are a cloth defect detection method based on an adversarial neural network, and a terminal and a storage medium. The detection method comprises the following steps: step 1: training a GAN model according to a positive sample of cloth, and outputting the GAN model after the training is completed; step 2: sending a detection image sample of the cloth to the GAN model, and the GAN model reconstructing the detection image sample, so as to generate a good reconstructed image sample; and step 3: performing structural similarity comparison between the detection image sample and the reconstructed image sample, and determining, according to a difference degree obtained by means of comparison, whether the detection image sample is defective.

Description

Fabric defect detection method, terminal and storage medium based on confrontation neural network

References to related applications

This disclosure requires all rights and interests of the invention patent application filed with the State Intellectual Property Office of the People's Republic of China on June 19, 2020, with the application number 202010566989.8 and the invention title "Fabric defect detection method, terminal and storage medium based on anti-neural network" , And incorporate its entire content into this article by reference.

field

The present disclosure generally relates to the technical field of cloth defect detection, and more specifically to a cloth defect detection method, terminal, and storage medium based on an adversarial neural network.

background

With the rapid economic development, my country's manufacturing industry is also developing rapidly, and products can be manufactured on a large scale every day for market demand. The increase in the number and variety of products has led to increasing requirements for product quality, and products with beautiful appearances are more likely to be favored by consumers. The surface quality of the product will affect the commercial value of the product, and the flaws in the appearance will directly cause the depreciation of the commercial value of the product. The surface quality of the product has an important influence on the direct use or deep processing of the product. In the production process of the product, due to the influence of equipment and technology, different kinds of defects often appear on the surface of the product. For example, defects such as holes, stains, lack of warp, and weft breaks in the fabric, the surface quality not only affects the appearance of the product itself, but also may affect the functional characteristics of the product itself, causing significant losses to the enterprise. Therefore, it is very necessary to detect the surface defects of the product, and it is particularly important to design a surface defect detection method with real-time and effective surface defects.

The traditional inspection method of cloth inspection is a manual inspection method, that is, under certain light source conditions, workers directly observe the cloth to be tested, and judge whether the cloth is defective or not based on intuitive experience. This method is very easy to be affected by human subjective factors and cause false detection or missed detection, and the detection efficiency is very low, and it is not suitable for the needs of modern large-scale production. In addition, the test results and data of manual testing cannot be automatically saved, which is inconvenient for subsequent data query, and it is also difficult to guide future cloth production.

The core of the cloth defect detection system lies in the defect detection algorithm. The current common defect detection algorithms can be roughly divided into the following categories: statistics, spectra, models, learning and structure. Statistical algorithm is to realize defect detection by expressing the spatial distribution of image gray value, mainly including co-occurrence matrix, mathematical morphology, etc. For the fabric defect detection of mathematical morphology analysis, the pre-trained Gabor wavelet network is used to extract important features, and the defect detection rate is higher than 90%. Spectral algorithms use different ways to express the characteristics of images to complete the detection of defects in the image, including Fourier transform, wavelet transform, and Gabor transform algorithms. There is also the idea of Fourier transform for defect detection. First, the histogram equalization is used to enhance the contrast of the fabric image, and then the central frequency spectrum is calculated by the two-point FFT, and a better defect detection result is obtained. The structural algorithm is based on the texture structure of the detected object, and the corresponding algorithm is designed according to different texture structures. In recent years, saliency analysis algorithms are also commonly used in cloth detection. A context-based local texture saliency analysis algorithm uses local binary to extract texture features and applies them to texture image detection. The better effect proves the algorithm’s effectiveness. Effectiveness.

Learning methods are mainly Neural Network (Neural Network) and deep learning, BP artificial neural network and defect detection method based on convolutional neural network. This method can realize defect detection and location on images with texture structure and special structure, and the performance is far Better than traditional detection methods. The above detection methods have achieved good results in specific applications, but still have some shortcomings. For example, although the detection method based on Gabor transform can provide a joint spatial representation capability, the calculation of the filter template is complicated and the amount of calculation is amazing. The detection method based on mathematical morphology can detect the size and shape of the defect very well, and show the defect very well, but it does not have enough visual effect support. The detection process of the above methods is roughly image preprocessing, target extraction, feature selection and pattern classification. Each link will affect the recognition rate of the model, especially when the amount of data is large and complex, the difficulty of feature selection will also be Increase, to a certain extent, will restrict the generalization ability of the algorithm. In addition, corresponding detection algorithms need to be designed according to different detection requirements, which not only requires a large amount of prior knowledge, but also considers the efficiency and effectiveness of the detection method, which has great limitations to a certain extent. Deep learning can directly learn two-dimensional images, reducing image preprocessing, without manually extracting features, and can automatically learn more appropriate features layer by layer, greatly reducing the impact of human factors.

The existing deep learning detection algorithms mainly include: Faster Rcnn, SSD, Yolov3, etc. These algorithms are good for general scenes, but they are not effective in the cloth industry. The main reason is that there are too many types of cloth and different defect shapes. And for the complex and ever-changing background of plaid patterns and printing, there is no good solution so far. Deep learning has requirements for data samples. The collection of hundreds of fabric defects is time-consuming and laborious. In the final fabric manufacturing process, processes such as shaping, mercerizing, and liquid ammonia are very fast, and deep learning has high computational complexity. Very challenging.

Overview

The present disclosure provides a cloth defect detection method, a terminal and a storage medium based on an adversarial neural network.

On the one hand, the present disclosure provides a cloth defect detection method based on an adversarial neural network, which includes: Step 1: Train a GAN model based on the positive samples of the cloth, and output the GAN model after the training is completed; Step 2: Take the detected image samples of the cloth Send to the GAN model, the GAN model reconstructs the detected image sample to generate a good reconstructed image sample; and Step 3: Compare the detected image sample with the reconstructed image sample for structural similarity, according to the difference in comparison Degree to determine whether the detected image sample is defective.

In some embodiments, offline training is used when training the GAN model.

In some embodiments, the GAN model is a DCGAN model.

In some embodiments, both the generator and discriminator in the DCGAN model use batch normalization; stride convolution is used in the discriminator to replace spatial pooling, and the deconvolution layer in the generator uses microstepping Amplitude convolution; remove the fully connected layer in the deeper architecture of the DCGAN model; use the Tanh activation function in the output layer of the generator, use the ReLU activation function in the other layers of the generator, and use the leaky ReLU activation function in the discriminator.

In some embodiments, a hidden variable input is added to the generator input in the DCGAN model. The discriminator not only needs to participate in the true and false judgment, but also needs to seek mutual information with the hidden variable.

In some embodiments, the DCGAN model updates the generator and discriminator based on mutual information.

In some embodiments, the characteristics of structural similarity contrast include brightness difference, contrast difference, and structural difference.

In some embodiments, the brightness difference is estimated as:

The contrast difference is estimated as:

The structural difference is estimated as:

In some real-time schemes, u _x and u _y represent the mean value of image block x and image block y, _{respectively, and σ x} and σ _y represent the standard deviation of image block x and image block y, respectively.

and

Respectively represent the variance of the image block x and the image block y, σ _xy represents the covariance, C ₁ =(K ₁ ×L) ² , C ₂ =(K ₂ ×L) ² , C ₃ =0.5C ₂ , take K ₁ =0.01, K ₂ =0.03, L is the dynamic range of the pixel value, L=255, the structural similarity comparison formula is:

M(x,y)=L(x,y)×C(x,y)×S(x,y).

On the other hand, the present disclosure also provides a terminal. The terminal includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor. The processor executes the computer program when the computer program is executed. The above detection method.

In another aspect, the present disclosure also provides a storage medium, the storage medium including computer instructions, when the computer instructions run on a computer, the computer executes the above detection method.

Brief description of the drawings

Fig. 1 is a flowchart of a detection method in an embodiment of the present disclosure.

Fig. 2 is a flowchart of the training phase of the detection method in an embodiment of the present disclosure.

FIG. 3 is a schematic diagram of the operation of generating a confrontation network in an embodiment of the present disclosure.

Fig. 4 is a structure diagram of a DCGAN model in an embodiment of the present disclosure.

Fig. 5 is a structural diagram of a model with hidden variables added in an embodiment of the present disclosure.

Figure 6 is a flow chart of the testing phase of the detection method in an embodiment of the present disclosure.

Fig. 7 is a picture of an inspection image sample of cloth in an embodiment of the present disclosure.

FIG. 8 is a picture of a good reconstructed image sample after reconstruction of a GAN model in an embodiment of the present disclosure.

Detail

The present disclosure will be described in detail below with reference to the drawings and specific embodiments.

Please refer to FIG. 1. On the one hand, the present disclosure provides a cloth defect detection method based on an adversarial neural network.

In some embodiments, the testing method includes training and testing phases, which specifically include the following steps:

Step 1: Train the GAN model according to the positive samples of the cloth, and output the GAN model after the training is completed. Please refer to Figure 2. Figure 2 is a flow chart of the training phase. The GAN model is trained after collecting positive samples of cloth and preprocessing the positive samples. After the training is completed, the GAN model is output. The training of the GAN model adopts offline training. The detection method adopts the collection of positive samples to train the model, which can avoid the problems of a wide variety of cloth defects and difficult collection, and is more in line with the actual production situation.

Please refer to Figure 3. The GAN (Generative Adversarial Networks) model is inspired by the two-player zero-sum game in game theory. The two game parties in GAN are respectively composed of generators (G) and The discriminator (discriminative model, D) acts as. The generator generates a pseudo sample through the input non-standard image (noise data), the pseudo sample generated by the generator and the real sample of the collected cloth are sent to the discriminator, and the discriminator needs to distinguish between the fake sample and the real sample , The GAN model updates the model according to the judgment result. Therefore, the generator needs to output as close to the real sample as possible to fool the discriminator, and the discriminator needs to distinguish as far as possible whether the input sample is a real sample or a fake image generated by the generator. The final solution of the game is the Nash equilibrium. The ultimate goal is to hope that the discriminator’s discriminative ability is strong enough, and when the pseudo samples of the generator are input into the discriminator, it is difficult for the discriminator to judge whether it is true or false, that is Maximizing the discriminating ability of the discriminator, minimizing the probability of judging the output image of the generator as a false image. Therefore, the discriminator must maximize the following loss function, divide the real samples into positive examples as much as possible, and generate negative examples as possible:

Where p _r represents the true sample distribution, and p _g represents the sample distribution generated by the generator. For the generator, there are two loss functions, namely:

Combining formula 1 and formula 2, the entire training process can be expressed by the following formula:

In some embodiments, the GAN model adopts the DCGAN (Deep Convolution Generative Adversarial Networks) model, which can improve the effect through the powerful feature extraction capabilities of the convolutional layer. Please refer to Figure 4, which is a diagram of the DCGAN model structure. Compared with the GAN model, the model structure of DCGAN has the following characteristics:

1. Batch normalization is used on both the generator and discriminator to solve the problem of poor initialization, while ensuring that the gradient is propagated to each layer, and it can also prevent the generator from taking all The samples all converge to the same point;

2. Use strided convolutions in the discriminator model to replace spatial pooling, while the deconvolution layer in the generator model uses fractional strided convolutions; and

3. For deeper architectures, removing the fully connected layer increases the stability of the model. The Tanh activation function is used in the output layer of the generator, the Re LU activation function is used in other layers, and the leaky ReLU activation function is used in the discriminator.

Please refer to FIG. 5, the present disclosure adds a latent variable C_vector input on the DCGAN model in order to better reconstruct the data. The hidden variables and noise data are sent to the generator, which generates pseudo samples, and sends the pseudo samples and real samples to the generator. The generator not only participates in the true and false judgment, but also needs to seek mutual information with the hidden variables, And update the generator and the discriminator according to the mutual information, so that more hidden variable information is retained in the generated image of the generator. In this solution, by collecting positive samples and training the network according to the above model, a trained GAN model can be obtained. The testing phase shown in Figure 6 can be started, and the testing phase includes step 2 and step 3.

Step 2: Send the test image sample of the cloth to the GAN model, and the GAN model reconstructs the test image sample to generate a good reconstructed image sample. Please refer to Figures 7 and 8, the pictures of the detected image samples of the cloth in Figure 7. The cloth in Figure 7 is defaced, and Figure 7 is sent to the GAN model, which reconstructs Figure 7 , Generate a reconstructed sample picture (Figure 8), the cloth in Figure 8 does not have the stains in the original Figure 7.

Step 3: Perform a structural similarity comparison between the detected image sample and the reconstructed image sample, and determine whether the detected image sample is defective according to the degree of difference of the contrast. Structural similarity (structural similarity index, SSIM) is an index that measures the similarity of two images. It measures the similarity of images from three different perspectives: brightness, contrast, and structure. Among them, the brightness difference is estimated as:

The contrast difference is estimated as:

The structural difference is estimated as:

In Formula 5 to Formula 7, u _x and u _y represent the mean value of image block x and image block y, _{respectively, and σ x} and σ _y represent the standard deviation of image block x and image block y, respectively.

and

Respectively represent the variance of image block x and image block y, σ _xy represents covariance, in order to avoid the case where the denominator is 0, a total of 3 constants _{C 1} , C ₂ , C _{3 are set} _{, where C 1} =(K ₁ ×L) ² , C ₂ =(K ₂ ×L) ² , C ₃ =0.5C ₂ , generally take K ₁ =0.01, K ₂ =0.03, L is the dynamic range of the pixel value, L=255.

The structural similarity comparison combines the brightness difference, contrast difference and structure difference, and the calculation formula is as follows:

M(x,y)=L(x,y)×C(x,y)×S(x,y) (Formula 8)

Figure 7 and Figure 8 are selected for comparison. Since Figure 8 is a good image generated by the GAN model, if the difference between Figure 7 and Figure 8 reaches a predetermined value, it can be judged that there is a defect in Figure 7, if it does not reach, then you can The sample in Figure 7 is judged to be a good product.

In some embodiments, a positive sample of cloth is used to train the GAN model, and during testing, a sample of the detected image of the cloth (for example, Figure 7) is sent to the GAN model for data reconstruction, and GAN is used where there are defects. Network generation capability, generated reconstructed image samples (such as Figure 8), and then through SSIM similarity feature extraction, calculate the degree of difference between the test image sample and the reconstructed image sample, and judge the output test image sample according to the degree of difference In order to achieve the purpose of detecting defects, it can detect defects such as color difference, holes, flying flowers, wrong warp, wrong weft, crumpled and stains on the cloth. It is suitable for cotton, linen, non-woven, printed cloth, Various types of fabrics such as denim fabrics and woven fabrics.

It can be understood that the GAN model and the method for determining structural similarity described in the embodiments of the present disclosure are only used to illustrate the embodiment of the detection method, and it is not the only implementation mode. The GAN model can also be related to various GANs. The improvement of the model is substituted, and the method of judging the structural similarity can be replaced by the improved method related to the structural similarity, all of which are within the protection scope of this scheme.

On the other hand, the present disclosure provides a terminal. The terminal includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor. The processor implements the foregoing when the computer program is executed. Detection method. In some embodiments, the terminal may include hardware such as a cloth transport mechanism, an industrial camera, a screen, and a printing device. In some embodiments, when performing cloth inspection, the cloth is installed and placed on the cloth conveying mechanism, and the cloth conveying mechanism moves and stretches the cloth. The industrial camera collects images of the cloth, and the collected image passes through the above The detection method is used for detection. The detection results and data can be recorded and sent to the cloud for storage, and can also be displayed on the screen or printed out by a printing device. This facilitates data statistics and collation, and helps to improve the process and improve the fabric based on the subsequent data. Rate.

In another aspect, the present disclosure provides a storage medium. The storage medium includes computer instructions. When the computer instructions run on a computer, the computer executes the above detection method.

In some embodiments, the present disclosure trains and generates a GAN model through a positive sample of cloth, and then uses the GAN model to reconstruct the test image sample of the cloth to generate a reconstructed image sample. The samples are compared by structural similarity to judge whether there are defects. It can detect defects such as color difference, holes, flying, wrong warp, wrong weft, wrinkle, and stains. It is suitable for cotton, linen, non-woven fabric, printed fabric, denim, etc. Various types of fabrics, such as fabrics and woven fabrics, can well solve the problems of difficulty in collecting defective samples and intricate fabric backgrounds in the actual production process. The detection speed and detection accuracy can meet the requirements of actual production testing.

The above are only the preferred embodiments of the present disclosure and are not used to limit the present disclosure. Any modification, equivalent replacement and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure. Inside.

Claims

The cloth defect detection method based on the confrontation neural network includes the following steps:

Step 1: Train the GAN model according to the positive samples of the cloth, and output the GAN model after the training is completed;

Step 2: Send the test image sample of the cloth to the GAN model, and the GAN model reconstructs the test image sample to generate a good reconstructed image sample; and

Step 3: Perform a structural similarity comparison between the detected image sample and the reconstructed image sample, and determine whether the detected image sample is defective according to the degree of difference of the contrast.
The method for detecting cloth defects based on a confrontational neural network according to claim 1, wherein the training of the GAN model adopts offline training.
The method for detecting cloth defects based on an adversarial neural network according to claim 1 or 2, wherein the GAN model is a DCGAN model.
The cloth defect detection method based on the adversarial neural network according to claim 3, wherein in the DCGAN model, both the generator and the discriminator use batch normalization; and the stride convolution is used in the discriminator instead Spatial pooling, the deconvolution layer in the generator uses micro-stride convolution; the fully connected layer is removed in the deeper architecture of the DCGAN model; the Tanh activation function is used in the output layer of the generator, The other layers of the generator use the ReLU activation function, and the leaky ReLU activation function is used in the discriminator.
The method for detecting cloth defects based on the adversarial neural network according to claim 3 or 4, wherein in the DCGAN model, a hidden variable input is added to the generator input, and the discriminator not only needs to participate in the true and false judgment, but also needs to cooperate with the hidden variable input. Variables seek mutual information.
The method for detecting cloth defects based on an adversarial neural network according to any one of claims 3 to 5, wherein the DCGAN model updates the generator and the discriminator based on the mutual information.
The method for detecting cloth defects based on an anti-neural network according to any one of claims 1 to 6, wherein the features of the structural similarity comparison include brightness differences, contrast differences, and structural differences.
8. The method for detecting cloth defects based on an adversarial neural network according to claim 7, wherein the brightness difference is estimated as:

The contrast difference is estimated as:

The structural difference is estimated as:

Among them, u x and u y represent the mean values of image block x and image block y, respectively, and σ x and σ y represent the standard deviation of image block x and image block y, respectively.
and
Respectively represent the variance of the image block x and the image block y, σ xy represents the covariance, C 1 =(K 1 ×L) 2 , C 2 =(K 2 ×L) 2 , C 3 =0.5C 2 , take K 1 =0.01, K 2 =0.03, L is the dynamic range of the pixel value, L=255, the structural similarity comparison formula is:

M(x,y)=L(x,y)×C(x,y)×S(x,y).
A terminal, which includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and when the processor executes the computer program, it implements what is described in any one of claims 1-8 The method described.
A storage medium, which includes computer instructions, which when run on a computer, cause the computer to execute the method described in any one of claims 1-8.