CN112164033A - Abnormal feature editing-based method for detecting surface defects of counternetwork texture - Google Patents

Abnormal feature editing-based method for detecting surface defects of counternetwork texture Download PDF

Info

Publication number
CN112164033A
CN112164033A CN202010962259.XA CN202010962259A CN112164033A CN 112164033 A CN112164033 A CN 112164033A CN 202010962259 A CN202010962259 A CN 202010962259A CN 112164033 A CN112164033 A CN 112164033A
Authority
CN
China
Prior art keywords
image
texture
abnormal
feature
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010962259.XA
Other languages
Chinese (zh)
Inventor
杨华
尹周平
周勤远
宋开友
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN202010962259.XA priority Critical patent/CN112164033A/en
Publication of CN112164033A publication Critical patent/CN112164033A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G06T7/41—Analysis of texture based on statistical description of texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G06T7/41—Analysis of texture based on statistical description of texture
    • G06T7/44—Analysis of texture based on statistical description of texture using image operators, e.g. filters, edge density metrics or local histograms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G06T2207/20032—Median filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Abstract

The invention belongs to the technical field of image processing, and particularly discloses a method for detecting surface defects of an anti-network texture based on abnormal feature editing, which comprises the following steps: acquiring a non-defective good product image and a corresponding defect image to jointly form an image data set; constructing an antagonistic network, wherein the antagonistic network comprises a generator and a discriminator, the generator is used for extracting image characteristics, detecting abnormal characteristics, and editing the abnormal characteristics by adopting normal characteristics to obtain a reconstructed image; the discriminator is used for discriminating the good product image and the reconstructed image; training the countermeasure network through an image data set according to a pre-constructed optimization target to obtain a reconstructed image generation model; and inputting the image to be detected into the reconstructed image generation model to obtain a corresponding reconstructed image, and further obtaining the texture surface defects according to the image to be detected and the corresponding reconstructed image. The invention has higher detection precision for defects with different shapes, sizes and contrasts on different texture surfaces.

Description

Abnormal feature editing-based method for detecting surface defects of counternetwork texture
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a network texture resisting surface defect detection method based on abnormal feature editing.
Background
In the industrial manufacturing field, due to factors such as the quality of raw materials and complicated manufacturing processes, texture defects may be generated on the surface of products, such as mobile phone screens, wood, textiles, and tiles. Texture defects refer to local areas of irregular brightness variation or texture disruption. These texture defects directly affect the user experience, and in order to control the product quality, all types of texture defects should be effectively controlled in the manufacturing process, so defect detection is the basis and key for promoting the whole industrial manufacturing industry.
The image recognition technology only needs to acquire texture images through a CCD camera, then positions the texture images to the positions needing to be interested by utilizing an image processing algorithm, and detects defects accurately. Therefore, the defect detection method based on the image technology can meet the requirement of industrial automation.
At present, the detection of the texture surface defects has the following three difficulties: the texture surface defect detection is still a very challenging problem because the texture types are various, that is, various regular and irregular textures exist, the defects are various, that is, various defects may appear on the same texture surface, and the defect samples in the industry are few and difficult to collect and label. Some existing research results can only partially meet the requirements of precision and robustness of texture surface defect detection. Therefore, it is necessary to provide an unsupervised texture defect detection algorithm, which is suitable for detecting multiple defects on multiple texture surfaces.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention provides an anti-network texture surface defect detection method based on abnormal feature editing, and aims to adopt a strategy of paired input, add abnormality in a training stage, and detect and edit the utilized abnormality, so that the reconstruction of the defects is inhibited, the detection rate of the defects is improved, and meanwhile, the precision of texture detail reconstruction is further improved by adopting a pixel-level anti-learning mode.
In order to achieve the above object, the present invention provides a method for detecting surface defects of an anti-network texture based on abnormal feature editing, which comprises the following steps:
s1, acquiring a defect-free good product image and a defect image corresponding to the good product image, wherein the defect image and the good product image jointly form an image data set;
s2, constructing a countermeasure network edited based on abnormal features, wherein the countermeasure network comprises a generator and a discriminator, the generator is used for extracting image features of a good product image and a defect image, detecting the abnormal features in the good product image and the defect image, editing the abnormal features by adopting normal features, and reconstructing the abnormal features according to the processed features to generate a reconstructed image; the discriminator is used for discriminating the good product image and the reconstructed image on each pixel;
s3, training the confrontation network edited based on abnormal features through an image data set according to a pre-constructed optimization target, and performing confrontation learning by a generator and a discriminator during training so as to obtain a reconstructed image generation model;
s4, inputting the image to be detected into the reconstructed image generation model to obtain a corresponding reconstructed image, and further obtaining texture surface defects according to the image to be detected and the corresponding reconstructed image to complete the texture surface defect detection of the image to be detected.
More preferably, in S2, the image features are extracted by 5 convolutional layers, where the step size of the first convolutional layer is 1, the number of channels is 16, the step sizes of the following three convolutional layers are 2, the number of channels is increased by a multiple of 2, and the step size of the fifth convolutional layer is 1.
Preferably, in S2, the method for detecting abnormal features by using a clustering method based on center constraint specifically includes the following steps:
(1) with C ═ C1,c2,…,cKDenotes a cluster center, calculates a cluster center ckAnd image feature fiResidual error R betweenikFurther obtaining the distance of each image feature to the clustering center and passing through the minimum distance diAnd obtaining a cluster to which the image features belong, wherein the specific calculation formula is as follows:
Rik＝fi-ck
Figure BDA0002680965400000031
wherein i belongs to 1,2, …, N; k is 1,2, …, K; n represents the number of image features, and K represents the number of clustering centers;
(2) calculating the center boundary T of each clustercAnd will be spaced from the cluster center by more than TcAs an abnormal feature, the central boundary TcIs calculated as follows:
Figure BDA0002680965400000032
wherein σdIs the standard deviation of the distance of the image features from the cluster center.
Preferably, in S2, when editing the abnormal features, the abnormal features are convolved with the normal background features to obtain matching scores, the matching scores are normalized into a score map through softmax operation, corresponding normal features are obtained by combining the score map and the background features, and finally the abnormal features are replaced with the normal features to complete the editing of the abnormal features.
As a further preferred, the calculation formula of the optimization objective, i.e. the trained joint loss L, is as follows:
L＝λ1Lrec+λ2Lclu+λ3Ladv
wherein L isrecFor texture reconstruction loss, LcluTo cluster the losses, LadvDetermining a loss for the pixel; lambda [ alpha ]1、λ2And λ3The weights are texture reconstruction loss, clustering loss, and pixel discrimination loss, respectively.
As a further preferred, the clustering penalty Lclu＝Lc+LklWherein:
Lcfor the central constraint, the calculation formula is as follows:
Figure BDA0002680965400000033
Lklto aid in scoring, specifically based on the residual R between the cluster center and the image featureikCalculating a distance score SikFurther, a target score S 'is obtained'ikThen using KL divergence to obtain an auxiliary score LklThe specific calculation formula is as follows:
Figure BDA0002680965400000034
Figure BDA0002680965400000041
Figure BDA0002680965400000042
wherein α ═ { α ═ α1,α2,…,αKThe "learning smoothing factor" is the learning smoothing factor,
Figure BDA0002680965400000043
further preferably, the texture reconstruction lossLose Lrec＝Lrec_o+Lrec_d+Lrec_mThe specific calculation formula is as follows:
Figure BDA0002680965400000047
Figure BDA0002680965400000048
Figure BDA0002680965400000044
wherein: i isoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbFor reconstructed images corresponding to defect maps, ImIs a mask image; beta is the amplification factor, and beta is the amplification factor,
Figure BDA0002680965400000045
w and H are the width and height of the image respectively.
More preferably, the pixel discrimination loss L isadvIs calculated as follows:
Figure BDA0002680965400000046
wherein, IoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbA reconstructed image corresponding to the defect map; w and H are the width and height of the image respectively.
Preferably, in S4, the method for obtaining the texture surface defect according to the image to be detected and the reconstructed image includes the following steps:
(1) the image to be detected and the corresponding reconstructed image are differenced to obtain a residual error image Ires：
Ires＝|Id-Idb|
Wherein, IdFor the image to be detected, IdbFor reconstitutionAn image;
(2) performing median filtering on the residual image, and then performing double-threshold processing and morphological closing operation on the residual image to obtain a binary image I containing texture surface defectsbin(i,j):
Figure BDA0002680965400000051
Wherein i belongs to (1, …, W), j belongs to (1, …, H), W and H are the width and height of the image respectively; t is1、T2Are two thresholds from the residual map.
Further preferably, the threshold value T is set to be lower than the threshold value T1、T2Is calculated as follows:
Figure BDA0002680965400000052
where μ and σ are the mean value and standard deviation of the residual map, respectively, and are the division sensitivity control coefficients.
Generally, compared with the prior art, the above technical solution conceived by the present invention mainly has the following technical advantages:
1. the invention provides an unsupervised defect detection method, which provides abnormity for model training by adopting artificially manufactured defect graphs and corresponding good product graphs as paired input in an off-line training stage, and is beneficial to the model to process abnormal characteristics in a testing stage; during online detection, only the images are input into a trained network model, corresponding texture background images are reconstructed, and the two images are subjected to subtraction and post-processing, so that defects can be detected; the method solves the problem of low detection precision caused by different types, large-scale change, low contrast, irregular brightness change, variable shape, lack of samples and the like in the texture defect detection algorithm.
2. The method for detecting the defects of the network texture surface based on abnormal characteristic editing is not limited by texture types and defect types, has higher detection precision on the defects with different sizes and different contrasts on different texture surfaces, and is suitable for detecting various defects on various texture surfaces.
3. The method adopts a clustering method based on central constraint to detect the abnormal features of the hidden space, enables the extracted features to be more discriminative while detecting the abnormal features, edits the detected abnormal features to inhibit the reconstruction of the defects and improve the detection rate of the defects.
4. The method utilizes a plurality of convolutional layers to extract the characteristics of the image, and extracts the characteristics of deeper images by increasing the number of channels of the convolutional layers and setting the step length of convolution operation; in addition, by adopting pixel level discrimination, the reconstruction precision of texture background details can be further improved, and the over-detection rate is reduced while the defect detection rate is improved.
5. In order to accurately reconstruct the texture background image, the texture reconstruction loss, the clustering loss and the pixel discrimination loss are jointly adopted, and the whole model is optimized through a multi-task loss function.
Drawings
FIG. 1 is a diagram of an anomaly feature editing-based countermeasure network model according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an abnormal feature detection module according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an abnormal feature editing module according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a pixel-level decision module according to an embodiment of the invention;
FIG. 5 is a schematic diagram of a defect detection stage according to an embodiment of the present invention;
fig. 6 is a schematic diagram illustrating the effect of detecting surface defects of different textures in an embodiment of the present invention, where (a), (c), (e), (g), (i), and (k) are images to be detected, and (b), (d), (f), (h), (j), and (l) are corresponding detection results.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment of the invention provides a method for detecting the surface defects of counternetwork textures based on abnormal feature editing, which comprises an off-line training stage and an on-line detection stage:
the offline training phase comprises:
s1, artificially generating defects:
on a defect-free good product chart IoGenerating a corresponding artificial defect map IdWherein, in the step (A),
Figure BDA0002680965400000071
W,H,C0respectively representing the width, height and channel number of an input image; the defect map and the good map together form an image data set;
and S2, constructing the countermeasure network based on abnormal feature editing, wherein the countermeasure network comprises a generator and a discriminator as shown in figure 1. The generator comprises a feature extraction module FEM, an abnormal feature detection module AFDM, an abnormal feature editing module GCFEM and a feature decoding module FDM, wherein the feature extraction module is used for extracting the hidden spatial features of the input texture image; the abnormal feature detection module detects abnormal features by adopting a clustering method based on central constraint; the abnormal characteristic editing module uses the cosine similarity of the abnormal characteristic and the normal characteristic to carry out weighting construction to construct a normal characteristic to replace the abnormal characteristic, thereby inhibiting the reconstruction of the defect; the feature decoding module is used for reconstructing the texture background and reconstructing the hidden space features into the texture picture by using a plurality of convolution layers symmetrical to the feature extraction module. The discriminator comprises a pixel-level discrimination module PDM for discriminating the good images and the reconstructed images on each pixel, and the texture background reconstruction precision is further improved. Specifically, the method comprises the following steps:
s21, a feature extraction module:
obtaining the input hidden space characteristics by using 5 convolution layers, wherein the convolution kernels of all the layers except the last layer have the sizeAre all 3 × 3; the step size and the number of channels of the first convolutional layer are 1 and C, respectively1(in the present embodiment, C116), the step size of the following three convolutional layers is all 2, the number of channels is incremented by a multiple of 2, and finally, the convolutional kernel size of the fifth layer is 1 × 1, the step size is 1. Therefore, the feature extraction module will IoAnd IdAs input and to propose a hidden spatial feature F, wherein
Figure BDA0002680965400000072
C is set to 10.
S22, an abnormal feature detection module:
the hidden spatial feature F extracted by the feature extraction module can be regarded as a set of local features, that is, F ═ F1,f2,…,fNAnd is and
Figure BDA0002680965400000073
the purpose of the abnormal feature editing module is to use K centers and an adaptive threshold value TcTo learn the distribution of the hidden spatial features, then those distances exceed TcIs considered to be a characteristic of the anomaly. Notably, K centers and an adaptive threshold TcAre learnable parameters whose values are affected only by non-defective good maps and are updated as clustering is lost. To achieve this goal, different types of local features in the texture image should be separated and clustered into K clusters. However, the original implicit spatial distribution of the texture image is cluttered, which is very difficult for clustering.
The invention adopts a clustering method based on center constraint to detect abnormal features, and specifically comprises the following steps as shown in figure 2: with C ═ C1,c2,…,cKDenotes the cluster center, where c1∈RC×1. Assuming that all normal hidden spatial features are distributed around the corresponding center, the abnormal features mapped from the defect are far from the center, cluster center ckAnd image feature fiResidual error R betweenikThe calculation is as follows:
Rik＝fi-ck
wherein i belongs to 1,2, …, N; k is 1,2, …, K; n represents the number of image features. Then using the distance score SikTo measure each implicit spatial feature fiWhether the degree of belonging to the cluster center is normal or not, that is, the closer the normal feature is to the corresponding center, the farther the abnormal feature is from the center, and the calculation formula is as follows:
Figure BDA0002680965400000081
wherein α ═ { α ═ α1,α2,…,αKIs a learned smoothing factor, it is noted that the smoothing factor controls the distance, updating the parameters due to subsequent clustering losses. Each normal feature fiA distance d to C, and a minimum distance diIt is indicated to which cluster the feature belongs:
Figure BDA0002680965400000082
center boundary T for each clustercCalculated by the following equation:
Figure BDA0002680965400000083
wherein σdIs the standard deviation of the distances of the image features to the cluster centers, so those distances exceed TcIs considered to be an anomalous feature.
S23, an abnormal characteristic editing module:
for detected abnormal features, how to edit them to suppress reconstruction of defects is crucial for texture surface defect detection. The anomaly feature editing module replaces the feature that has been replaced with the ordered global semantic information from the normal background feature. As shown in fig. 3, the normal background features are sorted according to their similarity to the abnormal features to be edited, and then combined into a new feature to replace the abnormal features. Each detected abnormal feature can be regarded as a convolution kernel of 1 multiplied by 1, a matching score is obtained by convolution of the abnormal feature and a normal background feature, then the matching score is normalized into a score map by softmax operation, then the corresponding normal feature is obtained by combining the score map and the background feature, and finally the obtained normal feature is used for replacing the corresponding abnormal feature so as to achieve the purpose of editing the abnormal feature.
S24, a feature decoding module:
the feature decoding module reconstructs the hidden spatial features F into texture pictures by using a plurality of convolution layers which are symmetrical to the feature extraction module. Similarly, the convolution kernels of all layers are 3 × 3, except for the first layer, which has a convolution kernel size of 1 × 1. Similar to the U-net structure, the feature decoding module directly reconstructs the texture background of the features of each layer in the coding module through jumping connection.
As shown in FIG. 1, the model transforms a texture image IoOr IdAs input, further output the corresponding texture background image IobOr IdbWherein
Figure BDA0002680965400000091
The FEM extracts the input convolution features as texture expressions, and then the FDM reconstructs these texture expressions out of the texture background, which can be expressed as follows:
Figure BDA0002680965400000092
Figure BDA0002680965400000093
wherein the content of the first and second substances,
Figure BDA0002680965400000094
representing a function incorporating a feature extraction module, a feature decoding module, an abnormal feature detection module and a feature editing module, thetaTRepresenting the parameters of these modules.
S25, a pixel level judging module:
as shown in fig. 4, it is composed of 4 convolutional layers and 4 transposed convolutional layers, the convolution kernel of each convolutional layer is 3 × 3, the step size is 2, the number of filter kernels of the first layer is 32, the following convolution kernels sequentially increase by a multiple of 2, the convolution kernel of each transposed convolutional layer is 3 × 3, the step size is 2, the number of filter kernels of the first layer is 256, the following convolution kernels sequentially decrease by a multiple of 2, and the number of filter kernels of the last layer is 1.
When judging, the real good product is mapped IoReconstructed image Idb、IobThese three types of samples are used as input to discriminate between true and false at each pixel of the image, where IoIs a true sample, IdbAnd IobIs a false sample whose output is a discriminant score that predicts the likelihood that each pixel in the image is from the original image distribution, thereby achieving pixel-level discrimination.
And S3, training the confrontation network edited based on the abnormal features through the image data set according to the pre-constructed optimization target, and performing confrontation learning by the generator and the discriminator during training to obtain a reconstructed image generation model.
Specifically, for accurately reconstructing the texture background map, the calculation formula of the optimization target, i.e. the trained joint loss L, is as follows:
L＝λ1Lrec+λ2Lclu+λ3Ladv
wherein L isrecFor texture reconstruction loss, LcluTo cluster the losses, LadvDetermining a loss for the pixel; lambda [ alpha ]1、λ2And λ3Three lost weights, respectively; more specifically:
(1) clustering loss Lclu＝Lc+Lkl：
In the clustering process of step S22, the mainstream depth clustering method neglects to reduce the intra-class gap and increase the inter-class gap, which results in a less than ideal clustering effect. To solve this problem and accurately detect the abnormalityThe constant characteristic makes the original distribution in FIG. 2 approach the target distribution, and proposes an auxiliary score LklAnd a central constraint Lc。
The purpose of this auxiliary score is to make the distance d more compact, thereby increasing the discriminability of the hidden spatial features, so by calculating SikAnd normalized on each cluster center to get a target score S'ik：
Figure BDA0002680965400000101
Wherein the content of the first and second substances,
Figure BDA0002680965400000102
the Kullback-Leibler (KL) divergence is then used to minimize the distance score and the auxiliary target score:
Figure BDA0002680965400000111
to further reduce intra-class similarity and increase inter-class similarity, the following central constraints are proposed:
Figure BDA0002680965400000112
central constraint LcThe hidden space features extracted by the feature extraction module can be more representative by enabling different centers to be more dispersed and similar features to be more compact, and meanwhile, the detection precision of the abnormal features is also improved.
(2) Texture reconstruction loss Lrec＝Lrec_o+Lrec_d+Lrec_m：
For texture reconstruction, there are three penalties: l isrec_o、Lrec_dAnd Lrec_mThe purpose of these penalty functions is to minimize the gap between the input image and the reconstructed texture background image by mean square error:
Figure BDA0002680965400000113
Figure BDA0002680965400000114
Figure BDA0002680965400000115
wherein: II-2Is represented by L2A norm; i ismFor the mask image, β is the magnification factor,
Figure BDA0002680965400000116
sum (-) is the summation operation.
(3) The pixel discrimination loss LadvIn the determination process performed in step S24, the calculation formula is as follows:
Figure BDA0002680965400000117
the online detection phase comprises:
s4, as shown in FIG. 5, the image I to be detecteddInputting the reconstructed image into a reconstructed image generation model, extracting hidden space features, detecting abnormal features, editing abnormal features, and decoding the edited features to obtain a corresponding reconstructed image IdbAnd further obtaining the texture surface defects according to the image to be detected and the corresponding reconstructed image.
Specifically, firstly, the image I to be detected is detecteddWith corresponding reconstructed image IdbMaking difference to obtain residual error image Ires：
Ires＝|Id-Idb|
Then, carrying out median filtering operation on the residual image to remove noise therein, and then carrying out double-threshold processing and morphology closing operation on the residual image to obtain a binary image I of a final detection resultbin：
Figure BDA0002680965400000121
Wherein i ∈ (1, …, W), j ∈ (1, …, H), and T ∈1And T2Is from IresCan be regarded as a boundary of the residual map, the defect is outside this boundary. Assuming that the residuals follow a gaussian distribution, the threshold is calculated as follows:
Figure BDA0002680965400000122
where μ and σ are the mean value and standard deviation from the residual map, respectively, and are the division sensitivity control coefficients. The image to be detected and the detected binary image are shown in fig. 6.
According to the method for detecting the surface defects of the counternetwork texture based on the abnormal feature editing, which is provided by the invention, the strategy of paired input is adopted, the abnormality is added in the training stage, and the abnormality is processed by using the abnormal feature detection module and the abnormal feature editing module, so that the reconstruction of the defects is inhibited, and the detection rate of the defects is improved; the pixel-level confrontation learning mode is carried out by adopting the pixel-level discrimination module, so that the texture background reconstruction precision can be further improved, and the overdetection rate can be reduced while the detection rate of texture defects is improved.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1. A method for detecting the surface defects of counternetwork textures based on abnormal feature editing is characterized by comprising the following steps:
s1, acquiring a defect-free good product image and a defect image corresponding to the good product image, wherein the defect image and the good product image jointly form an image data set;
s2, constructing a countermeasure network edited based on abnormal features, wherein the countermeasure network comprises a generator and a discriminator, the generator is used for extracting image features of a good product image and a defect image, detecting the abnormal features in the good product image and the defect image, editing the abnormal features by adopting normal features, and reconstructing the abnormal features according to the processed features to generate a reconstructed image; the discriminator is used for discriminating the good product image and the reconstructed image on each pixel;
s3, training the confrontation network edited based on abnormal features through an image data set according to a pre-constructed optimization target, and performing confrontation learning by a generator and a discriminator during training so as to obtain a reconstructed image generation model;
s4, inputting the image to be detected into the reconstructed image generation model to obtain a corresponding reconstructed image, and further obtaining texture surface defects according to the image to be detected and the corresponding reconstructed image to complete the texture surface defect detection of the image to be detected.
2. The method for detecting surface defects of anti-net texture based on abnormal feature editing as claimed in claim 1, wherein in S2, the image features are extracted by 5 convolutional layers, wherein the step size of the first convolutional layer is 1, the number of channels is 16, the step size of the following three convolutional layers is 2, the number of channels is increased by a multiple of 2, and the step size of the fifth convolutional layer is 1.
3. The method for detecting surface defects of antagonistic network textures based on abnormal feature editing as claimed in claim 1, wherein in S2, a clustering method based on central constraint is adopted for abnormal feature detection, specifically comprising the following steps:
(1) with C ═ C1,c2,…,cKDenotes a cluster center, calculates a cluster center ckAnd image feature fiResidual error R betweenikFurther obtaining the distance of each image feature to the clustering center and passing through the minimum distance diAnd obtaining a cluster to which the image features belong, wherein the specific calculation formula is as follows:
Rik＝fi-ck
Figure FDA0002680965390000021
wherein i belongs to 1,2, …, N; k is 1,2, …, K; n represents the number of image features, and K represents the number of clustering centers;
(2) calculating the center boundary T of each clustercAnd will be spaced from the cluster center by more than TcAs an abnormal feature, the central boundary TcIs calculated as follows:
Figure FDA0002680965390000022
wherein σdIs the standard deviation of the distance of the image features from the cluster center.
4. The method for detecting surface defects of antagonistic network textures based on abnormal feature editing as claimed in claim 1, wherein in S2, when abnormal feature editing is performed, the abnormal feature is convolved with the normal background feature to obtain a matching score, the matching score is normalized into a score map by softmax operation, a corresponding normal feature is obtained by combining the score map and the background feature, and finally the abnormal feature is replaced by the normal feature to complete the editing of the abnormal feature.
5. The method for detecting the surface defects of the counternetwork textures based on the abnormal feature editing as claimed in claim 3, wherein the optimization objective, namely the calculation formula of the trained joint loss L, is as follows:
L＝λ1Lrec+λ2Lclu+λ3Ladv
wherein L isrecFor texture reconstruction loss, LcluTo cluster the losses, LadvDetermining a loss for the pixel; lambda [ alpha ]1、λ2And λ3The weights are texture reconstruction loss, clustering loss, and pixel discrimination loss, respectively.
6. The method of claim 5, wherein the cluster loss L is a loss of similarity between the net texture surface defect and the cluster loss Lclu＝Lc+LklWherein:
Lcfor the central constraint, the calculation formula is as follows:
Figure FDA0002680965390000023
Lklto aid in scoring, specifically based on the residual R between the cluster center and the image featureikCalculating a distance score SikFurther, a target score S 'is obtained'ikThen using KL divergence to obtain an auxiliary score LklThe specific calculation formula is as follows:
Figure FDA0002680965390000031
Figure FDA0002680965390000032
Figure FDA0002680965390000033
wherein α ═ { α ═ α1,α2,…,αKThe "learning smoothing factor" is the learning smoothing factor,
Figure FDA0002680965390000034
7. the method of claim 5, wherein the anomaly-based editing for detecting the surface defects of the network texture is performed by a computerThe texture reconstruction loss Lrec＝Lrec_o+Lrec_d+Lrec_mThe specific calculation formula is as follows:
Figure FDA0002680965390000035
Figure FDA0002680965390000036
Figure FDA0002680965390000037
wherein: i isoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbFor reconstructed images corresponding to defect maps, ImIs a mask image; beta is the amplification factor, and beta is the amplification factor,
Figure FDA0002680965390000038
w and H are the width and height of the image respectively.
8. The method of claim 5, wherein the pixel discrimination loss L is a penalty L for detecting the surface defects of the network texture based on abnormal feature editingadvIs calculated as follows:
Figure FDA0002680965390000039
Figure FDA0002680965390000041
wherein, IoIs a good product picture IobIs a reconstructed image corresponding to the good image IdbA reconstructed image corresponding to the defect map; w and H are the width and height of the image respectively.
9. The method for detecting surface defects of counternetwork textures based on abnormal feature editing as claimed in claim 1, wherein in S4, texture surface defects are obtained according to the image to be detected and the reconstructed image, and the method specifically comprises the following steps:
(1) the image to be detected and the corresponding reconstructed image are differenced to obtain a residual error image Ires：
Ires＝|Id-Idb|
Wherein, IdFor the image to be detected, IdbTo reconstruct an image;
(2) performing median filtering on the residual image, and then performing double-threshold processing and morphological closing operation on the residual image to obtain a binary image I containing texture surface defectsbin(i,j):
Figure FDA0002680965390000042
Wherein i belongs to (1, …, W), j belongs to (1, …, H), W and H are the width and height of the image respectively; t is1、T2Are two thresholds from the residual map.
10. The method of claim 9, wherein the threshold T is set as a threshold T for detecting surface defects of the texture of the resist web based on abnormal feature editing1、T2Is calculated as follows:
Figure FDA0002680965390000043
where μ and σ are the mean value and standard deviation of the residual map, respectively, and are the division sensitivity control coefficients.
CN202010962259.XA 2020-09-14 2020-09-14 Abnormal feature editing-based method for detecting surface defects of counternetwork texture Pending CN112164033A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010962259.XA CN112164033A (en) 2020-09-14 2020-09-14 Abnormal feature editing-based method for detecting surface defects of counternetwork texture

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010962259.XA CN112164033A (en) 2020-09-14 2020-09-14 Abnormal feature editing-based method for detecting surface defects of counternetwork texture

Publications (1)

Publication Number Publication Date
CN112164033A true CN112164033A (en) 2021-01-01

Family

ID=73857448

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010962259.XA Pending CN112164033A (en) 2020-09-14 2020-09-14 Abnormal feature editing-based method for detecting surface defects of counternetwork texture

Country Status (1)

Country Link
CN (1) CN112164033A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112700432A (en) * 2021-01-12 2021-04-23 华中科技大学 Texture surface defect detection method and system based on abnormal synthesis and decomposition
CN113516650A (en) * 2021-07-30 2021-10-19 深圳康微视觉技术有限公司 Circuit board hole plugging defect detection method and device based on deep learning
CN113838040A (en) * 2021-09-29 2021-12-24 西安工程大学 Detection method for defect area of color texture fabric
CN113947567A (en) * 2021-09-23 2022-01-18 西安电子科技大学广州研究院 Defect detection method based on multi-task learning
CN114757900A (en) * 2022-03-31 2022-07-15 启东新朋莱纺织科技有限公司 Artificial intelligence-based textile defect type identification method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110969606A (en) * 2019-11-29 2020-04-07 华中科技大学 Texture surface defect detection method and system
CN110992354A (en) * 2019-12-13 2020-04-10 华中科技大学 Abnormal region detection method for countering self-encoder based on introduction of automatic memory mechanism

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110969606A (en) * 2019-11-29 2020-04-07 华中科技大学 Texture surface defect detection method and system
CN110992354A (en) * 2019-12-13 2020-04-10 华中科技大学 Abnormal region detection method for countering self-encoder based on introduction of automatic memory mechanism

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
H. YANG等: ""An Anomaly Feature-Editing-Based Adversarial Network for Texture Defect Visual Inspection"", 《IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112700432A (en) * 2021-01-12 2021-04-23 华中科技大学 Texture surface defect detection method and system based on abnormal synthesis and decomposition
CN112700432B (en) * 2021-01-12 2022-07-05 华中科技大学 Texture surface defect detection method and system based on abnormal synthesis and decomposition
CN113516650A (en) * 2021-07-30 2021-10-19 深圳康微视觉技术有限公司 Circuit board hole plugging defect detection method and device based on deep learning
CN113516650B (en) * 2021-07-30 2023-08-25 深圳康微视觉技术有限公司 Circuit board hole plugging defect detection method and device based on deep learning
CN113947567A (en) * 2021-09-23 2022-01-18 西安电子科技大学广州研究院 Defect detection method based on multi-task learning
CN113947567B (en) * 2021-09-23 2024-03-22 西安电子科技大学广州研究院 Defect detection method based on multitask learning
CN113838040A (en) * 2021-09-29 2021-12-24 西安工程大学 Detection method for defect area of color texture fabric
CN114757900A (en) * 2022-03-31 2022-07-15 启东新朋莱纺织科技有限公司 Artificial intelligence-based textile defect type identification method

Similar Documents

Publication Publication Date Title
CN112164033A (en) Abnormal feature editing-based method for detecting surface defects of counternetwork texture
Gao et al. A generative adversarial network based deep learning method for low-quality defect image reconstruction and recognition
CN108398268B (en) Bearing performance degradation evaluation method
CN104992223B (en) Intensive Population size estimation method based on deep learning
CN105224872B (en) A kind of user's anomaly detection method based on neural network clustering
CN110009013A (en) Encoder training and characterization information extracting method and device
Markou et al. A neural network-based novelty detector for image sequence analysis
CN103366180A (en) Cell image segmentation method based on automatic feature learning
CN109389171B (en) Medical image classification method based on multi-granularity convolution noise reduction automatic encoder technology
CN111079539B (en) Video abnormal behavior detection method based on abnormal tracking
CN110287770B (en) Water individual target matching identification method based on convolutional neural network
CN109815967A (en) CNN ship seakeeping system and method based on Fusion Features
CN110473231A (en) A kind of method for tracking target of the twin full convolutional network with anticipation formula study more new strategy
CN111145145B (en) Image surface defect detection method based on MobileNet
CN108550163A (en) Moving target detecting method in a kind of complex background scene
CN108171710A (en) A kind of image segmentation processing method
CN107609588A (en) A kind of disturbances in patients with Parkinson disease UPDRS score Forecasting Methodologies based on voice signal
CN110969606A (en) Texture surface defect detection method and system
CN114972312A (en) Improved insulator defect detection method based on YOLOv4-Tiny
CN114998602A (en) Domain adaptive learning method and system based on low confidence sample contrast loss
CN113361346B (en) Scale parameter self-adaptive face recognition method for replacing adjustment parameters
CN114580934A (en) Early warning method for food detection data risk based on unsupervised anomaly detection
CN114049305A (en) Distribution line pin defect detection method based on improved ALI and fast-RCNN
CN105160336B (en) Face identification method based on Sigmoid functions
CN116823983A (en) One-to-many style handwriting picture generation method based on style collection mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210101

RJ01 Rejection of invention patent application after publication