CN112700432B - Texture surface defect detection method and system based on abnormal synthesis and decomposition - Google Patents

Texture surface defect detection method and system based on abnormal synthesis and decomposition Download PDF

Info

Publication number
CN112700432B
CN112700432B CN202110033380.9A CN202110033380A CN112700432B CN 112700432 B CN112700432 B CN 112700432B CN 202110033380 A CN202110033380 A CN 202110033380A CN 112700432 B CN112700432 B CN 112700432B
Authority
CN
China
Prior art keywords
abnormal
defect
image
texture
decomposition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110033380.9A
Other languages
Chinese (zh)
Other versions
CN112700432A (en
Inventor
杨华
宋开友
尹周平
侯岳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN202110033380.9A priority Critical patent/CN112700432B/en
Publication of CN112700432A publication Critical patent/CN112700432A/en
Application granted granted Critical
Publication of CN112700432B publication Critical patent/CN112700432B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Abstract

The invention discloses a texture surface defect detection method and system based on abnormal synthesis and decomposition, and belongs to the field of image processing. The invention constructs a defect generation network guided by segmentation, can generate a large number of defect samples similar to real defects by using a small number of real defect training samples, and simultaneously provides an abnormal synthesis method based on Gaussian sampling, which can randomly synthesize abnormal negative samples by using defect-free positive samples, thereby solving the problem of small number of defect samples in industry and further improving the defect detection precision; according to the method, the abnormal negative sample is decomposed into the texture background image and the abnormal mask image by adopting the abnormal decomposition network, so that the defect can be effectively inhibited from being reconstructed into the texture background, the texture background reconstruction precision is improved, the defect area can be accurately segmented, the residual image and the abnormal segmented image are fused, the defect detection rate is improved, and the defect overdetection rate is reduced.

Description

Texture surface defect detection method and system based on abnormal synthesis and decomposition
Technical Field
The invention belongs to the field of image processing, and particularly relates to a texture surface defect detection method and system based on abnormal synthesis and decomposition.
Background
In the field of industrial manufacturing, various industrial products have different raw material quality types and complex production and manufacturing process. It is difficult to avoid various surface defects on the surface of products such as textiles, novel display devices, ceramics, steel, and the like. Surface defects refer to local areas that differ from the surrounding texture and pattern, or local areas with irregular brightness variations. Because the surfaces of various products often present different texture characteristics, the texture surface defects can directly reduce the product quality and influence the user experience. To improve production quality, all types of surface defects should be tightly controlled during the manufacturing process. The texture surface defect detection technology based on machine vision is the most important technology for detecting the texture surface defects, and intelligent manufacturing is realized by utilizing an artificial intelligence technology. Therefore, texture surface defect detection based on artificial intelligence is the basis and key of the whole manufacturing industry.
In the manufacturing process of various industrial products, the texture surface defects have the following characteristics: different types, large-scale change, low contrast, irregular brightness change and variable shapes, and meanwhile, the number of defective products relative to good products is extremely small, so that the number of available defect samples is small, and great difficulty is brought to visual detection. Therefore, texture surface defect detection algorithms remain a challenging task in industrial product quality control.
At present, a large number of algorithms are proposed to solve the problem of detecting the texture surface defects. The existing method can only be suitable for one or more types of textures (such as only a display device can be detected, and the method cannot be suitable for the surface of wood), can only detect fixed type texture defects (such as only high contrast defects can be detected, and low contrast defects cannot be detected), and is difficult to solve all conditions. Therefore, a more robust and more adaptive deep learning texture surface defect detection algorithm needs to be provided, which is suitable for various textures and has better robustness for different types of texture surface defects.
Disclosure of Invention
In view of the above defects or improvement needs in the prior art, the present invention provides a texture surface defect detection method and system based on abnormal synthesis and decomposition, which aims to improve the accuracy of surface defect detection.
In order to achieve the above object, the present invention provides a texture surface defect detection method based on abnormal synthesis and decomposition, comprising:
s1, constructing a defect generation network for segmentation guidance; said defectThe generation network comprises a generator and a discriminator of the segmentation guide; the generator is used for synthesizing the first type negative sample image IcnConversion into a first type abnormal negative sample image I more similar to a real defect imagen(ii) a The first type of synthetic negative sample image IcnFrom defect-free textured background IpAnd a defect image IdDefective label image I oflBy regional superposition synthesis; the partition-guided discriminator is used for respectively discriminating abnormal negative sample images InWith the real defect image IdCarrying out defect segmentation;
s2, aiming at defect segmentation, training a defect generation network by using a counterstudy method;
s3, converting the second type of synthetic negative sample image I 'by using a trained generator'cnObtaining a second abnormal negative sample image I'n(ii) a The second type of synthetic negative sample image is composed of a defect-free texture background IpWith an anomalous mask image I generated by random samplingaBy regional superposition synthesis;
s4, constructing an abnormal decomposition network and using a defect-free texture background IpAnd a second type abnormal negative sample image I'nAs a training set, training an abnormal decomposition network; the abnormal decomposition network is used for decomposing the defect image into a texture background image and an abnormal image;
and S5, inputting the texture surface image to be detected into the trained abnormal decomposition network to obtain a corresponding texture background image and an abnormal image, and fusing a residual image between the defect image and the texture background image with the abnormal image to obtain a defect detection result and obtain a defect detection result.
Further, a first type of synthetic negative sample image IcnFrom defect-free textured background IpAnd defective label image IlPerforming regional superposition synthesis by using the following expression;
Icn(x,y)＝(1-Il(x,y))·Ip(x,y)+λ·Il(x,y)
where λ represents a random variable, x 1., W, y 1., H, W, H represent the image width and height, respectively.
Further, the defect generation network is trained using the following loss function:
Figure BDA0002893176620000031
Figure BDA0002893176620000032
wherein L isdis(θsd) Representing the loss function of the training arbiter, Lgen(θg) A loss function representing a training generator;
Figure BDA0002893176620000033
respectively representing expectation, matrix dot product, L1、L2Norm, λa、λbRepresenting the weight corresponding to each loss.
Further, the random sampling comprises Gaussian sampling, student t distribution sampling and uniform sampling.
Further, the training process of the abnormal decomposition network comprises the following steps:
defect-free textured background I using an encoderpCoded as corresponding implicit spatial features Zp(ii) a Using a feature embedding encoder to embed a spatial feature ZpEncoding into corresponding texture features
Figure BDA0002893176620000034
Decoding texture features with a decoder
Figure BDA0002893176620000035
Obtaining a defect-free textured background IpIs reconstructed image of
Figure BDA0002893176620000036
Utilizing an encoder to obtain a second type abnormal negative sample image I'nCoded as corresponding implicit spatial features ZnAnd abnormal feature Za(ii) a Using feature embedding encoders to embed hidden spatial featuresSign ZnEncoding into corresponding texture features
Figure BDA0002893176620000037
Make it
Figure BDA0002893176620000038
And
Figure BDA0002893176620000039
sharing a feature distribution space; decoding texture features with a decoder
Figure BDA00028931766200000310
Correspondingly obtaining a second abnormal negative sample image I'nReconstructed textured background image
Figure BDA00028931766200000311
Decoding of exception features Z with exception decoderaObtaining an abnormal foreground image
Figure BDA00028931766200000312
Respectively processing the second type abnormal negative sample image I 'by a discriminator of segmentation guidance'nWith reconstructed textured background image
Figure BDA00028931766200000313
Carrying out defect segmentation;
an encoder, a feature embedding encoder, a decoder and a discriminator are trained using a counterlearning method.
Further, the anomaly decomposition network is trained using the following loss function:
Figure BDA0002893176620000041
Lrrepresenting a reconstruction loss for training an encoder, a feature embedding encoder and a decoder; l iszRepresenting implicit spatial feature constraint loss; for training the encoder and the feature embedding encoder; l isaIndicating anomalous decodingA penalty for training the encoder and the anomaly decoder; l isadRepresents the discrimination loss of the discriminator, LagArbiter generated losses for training an encoder, a feature-embedded encoder and a decoder, lambda1,λ2,λ3,λ4The weights of the four losses are respectively.
In general, the above technical solutions contemplated by the present invention can achieve the following advantageous effects compared to the prior art.
(1) The invention constructs a defect generation network guided by segmentation, can generate a large number of defect samples similar to real defects by using a small number of real defect training samples, and can randomly synthesize abnormal negative samples by using defect-free positive samples by the abnormal synthesis method based on Gaussian sampling, thereby solving the problem of small quantity of defect samples in industry and further improving the defect detection precision.
(2) The existing defect detection method comprises an unsupervised learning method and a supervised learning method, wherein the unsupervised learning method only utilizes non-defective good images for training and utilizes background reconstruction to detect defects, but because defects are not used in the training process, the defect feature expression cannot be learned, the method can only be suitable for one or more types of textures (such as only a display device can be detected and the surface of wood cannot be suitable), only can detect fixed types of texture defects (such as only high-contrast defects can be detected and low-contrast defects cannot be detected), and all conditions are difficult to solve; the supervised learning method needs a large amount of labeled defect samples for training, and the industrial field is difficult to collect a large amount of defect samples, so the method cannot be applied to the industrial field; according to the method, the abnormal negative sample is decomposed into the texture background image and the abnormal mask image by adopting the abnormal decomposition network, so that the defect can be effectively inhibited from being reconstructed into the texture background, the texture background reconstruction precision is improved, the defect area can be accurately segmented, the residual image and the abnormal segmented image are fused, the defect detection rate is improved, and the defect overdetection rate is reduced.
Drawings
FIG. 1 is a schematic flow chart of a texture surface defect detection method based on abnormal composition and decomposition according to the present invention;
FIG. 2 is a schematic diagram of a feature embedding encoder implemented in accordance with the present invention;
FIG. 3 is a comparison of defect detection effects.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Abnormal negative sample image InCan be decomposed into defect-free texture background IpAnd abnormal image IaThe assumptions of (a) are as follows:
In(x,y)＝Ib(x,y)·Ip(x,y)+Ia(x,y)·If(x,y) (0-1)
wherein, In,Ip,Ib,Ia,If∈RW×H×1W and H respectively indicate an image width and a height, and in this study, 256 pixels are used, x is 1. I isaRepresenting an abnormal mask image, IfRepresenting the corresponding abnormal region image, IbRepresenting a textured background mask image, i.e. Ib(x,y)＝1-Ia(x,y)。
Based on the above assumptions, as shown in fig. 1, the present invention proposes a texture surface defect detection method based on abnormal synthesis and decomposition, which simulates the abnormal synthesis in the decomposition process based on the abnormal synthesis assumption in formula (0-1). Specifically, the method comprises a training process and a testing process, wherein the training process comprises three steps: the method comprises the steps of dividing guided defect generation network, abnormal synthesis based on Gaussian sampling and abnormal decomposition network. The defect generation network comprises a generator and a discriminator, and the generator can utilize a defect-free texture background I through a counterstudy processpGenerating an anomalous negative sample image In(ii) a The abnormal synthesis step based on Gaussian sampling is realized by random Gaussian sampling and generationGenerating a plurality of abnormal negative sample images I'nThe abnormal decomposition network converts the abnormal negative sample image I'nDecomposition into textured background images
Figure BDA0002893176620000061
And abnormal foreground image
Figure BDA0002893176620000062
Realizing an abnormal decomposition process; the abnormal decomposition network after training is used in the test process to obtain the texture defect image IdDecomposition into textured background images
Figure BDA0002893176620000063
And abnormal segmentation of images
Figure BDA0002893176620000064
And by fusing texture defect images IdWith the texture background image
Figure BDA0002893176620000065
And detecting the texture surface defects by the residual image and the abnormal segmentation image. The proposed algorithm is explained in detail below.
(1) Split-guided defect generation network
The segmentation-guided defect generation network is used to generate a large number of abnormal negative examples similar to real defects.
Firstly, a defect-free texture background IpAnd a defect image IdDefective label image I oflSynthesizing a synthesized negative sample image I by regional superpositioncn：
Icn(x,y)＝(1-Il(x,y))·Ip(x,y)+λ·Il(x,y) (0-2)
Wherein IlCorresponds to I in the formula (0-1) with a random variable λ (0 < λ < 1)aAnd If。
Then, the generator of the segmentation-guided defect generation network will synthesize a negative sample image IcnConversion into an anomalous negative sample image InTo make it match with the real defect mapLike more similarly:
In＝fg(Icn；θg) (0-3)
wherein In∈RW×H×1，fg(. o) and θgRepresenting the function and the parameters of the generator, respectively. Abnormal negative sample image InShould be in IlThe corresponding masked areas generate defects close to real defects and keep other texture areas unchanged. Thus, the segmentation-guided counterlearning approach presented herein provides different optimization objectives for mask regions versus texture regions. Abnormal negative sample image InWith real defect image IdThe discriminator division defects that are simultaneously input to the division guide:
Figure BDA0002893176620000066
wherein Ins,Ids∈RW×H×1Representing the output pixel-by-pixel prediction result image, fsd(. o) and θsdRespectively representing functions and parameters of the discriminators of the segmentation guide. Purpose of the generator is IlThe corresponding mask region generates a defect close to a real defect, and the discriminator for segmentation guidance discriminates the defect as the real defect.
The pixel-by-pixel cross-entropy loss function is used to perform a segmentation-guided countermeasure learning process:
Figure BDA0002893176620000071
Figure BDA0002893176620000072
wherein L isdis(θsd) Representing the loss function of the training arbiter, Lgen(θg) A loss function representing a training generator;
Figure BDA0002893176620000073
respectively representing expectation, matrix dot product, L1、L2Norm, λa、λbRepresenting the weight corresponding to each loss. To further ensure the abnormal negative sample image I generatednTexture region and defect-free positive sample image IpConsistently, the reconstruction loss is also used as an auxiliary loss training generator in equations (0-6).
(2) Anomaly synthesis based on Gaussian sampling
As shown in fig. 1, the abnormal synthesis method based on gaussian sampling may randomly synthesize an abnormal negative sample using a defect-free positive sample, and the defect-free positive sample is fused with an abnormal mask image of the random gaussian sampling to obtain a synthesized abnormal negative sample; and then, converting the synthesized abnormal negative sample into an abnormal negative sample which is closer to the real defect sample by using a generator in the trained segmentation-guided defect generation network.
The first step of the anomaly synthesis method based on gaussian sampling is gaussian sampling. In this step, an abnormal mask image IaGenerated by random Gaussian sampling and matched with a defect-free texture background IpFusing and synthesizing the image Icn. Based on the assumption that each defect region obeys Gaussian distribution, two-dimensional Gaussian function is used for randomly sampling an abnormal weight map
Figure BDA0002893176620000074
Figure BDA0002893176620000075
Wherein the content of the first and second substances,
Figure BDA0002893176620000076
further:
Figure BDA0002893176620000077
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA00028931766200000811
is a parameter of each two-dimensional gaussian function, i 1a，NaIndicating the number of defective areas. In this study, the random sampling range of each parameter is: a is more than or equal to 0.1i≤2，
Figure BDA0002893176620000082
-90°≤θi≤90°，1≤NaLess than or equal to 3. By contrast ratio A to defectiCenter position of defect
Figure BDA0002893176620000083
Shape of defect
Figure BDA0002893176620000084
Defect angle thetaiThe random sampling of (2) can be carried out on a large number of defect samples.
By intercepting
Figure BDA0002893176620000085
The ith anomaly mask can be obtained:
Figure BDA0002893176620000086
where max (-) denotes maximum operation, TaThe threshold value is set to 0.7, and in practical application, the threshold value is set according to the requirement of defect contrast. Then, the ith abnormal region image
Figure BDA0002893176620000087
Can be obtained by fusion
Figure BDA0002893176620000088
And defect-free positive sample IpObtaining:
Figure BDA0002893176620000089
where α ═ {1, -1}, is used to obtain bright defects or dark defects. The anomaly image and the anomaly mask image can be constructed by fusing all sampled regions:
Figure BDA00028931766200000810
finally, a synthetic abnormal image I can be obtainedcn. Thus, based on Gaussian sampling, by fusing defect-free positive sample images IpWith randomly sampled abnormal mask image Ia。
In the semi-supervised mode of the method, the anomaly synthesis method based on Gaussian sampling further performs anomaly transformation. A generator for generating a trained segmentation-guided defect generation network, synthesizing the abnormal image I by the formula (0-3)cnTransition to an anomalous negative sample image InSo that the image is closer to the real defect and a large amount of abnormal negative sample images can be synthesized. Gaussian sampling is easier to operate, and in practical application, other random sampling methods can be used for generating abnormal negative samples.
(3) Exception decomposition network
As shown in fig. 1, the abnormal decomposition network is composed of five modules: the device comprises an encoder, a characteristic embedding encoder, a decoder, an abnormal decoder and a partition guiding discriminator, wherein an abnormal negative sample is decomposed into a texture background image and an abnormal mask image, and the texture background accurate reconstruction and the defect area accurate partition can be realized through a partition counterstudy mechanism, so that the defect detection precision is greatly improved.
Firstly, the defect-free texture background I is embedded into the encoder and the decoder through the encoder and the featurespIs used to learn the texture feature representation of the implicit spatial nature. By means of an encoder, IpIs coded as a hidden spatial feature Zp：
Zp＝fe(Ip；θe) (0-12)
Wherein the content of the first and second substances,
Figure BDA0002893176620000091
Wl,Hl,Clrespectively representing hidden spacesWidth, height and number of vias of the inter-feature, fe(. cndot.) with θeRepresenting the functions and parameters of the encoder, respectively. The proposed feature embedding encoder will be ZpEncoding into texture features
Figure BDA0002893176620000092
Figure BDA0002893176620000093
Wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0002893176620000094
ft(. o) and θtRepresenting the function and parameter of the feature embedding encoder, respectively. The texture features are then decoded by a decoder
Figure BDA0002893176620000095
A defect-free textured background I can be obtainedpIs reconstructed image of
Figure BDA0002893176620000096
Figure BDA0002893176620000097
Wherein the content of the first and second substances,
Figure BDA0002893176620000098
fd(. o) and θdRepresenting the functions and parameters of the decoder, respectively. The texture feature distribution can be learned through a reconstruction process formed by an encoder, a feature embedding encoder and a decoder:
Figure BDA0002893176620000099
at the same time, with a defect-free textured background IpCorresponding paired abnormal negative sample InEncoding into implicit spatial features Z by an encodernAnd abnormal feature Za：
Zn,Za＝fe(In；θe) (0-16)
Wherein, the first and the second end of the pipe are connected with each other,
Figure BDA00028931766200000910
Znis further input into a feature embedding encoder to be encoded into texture features
Figure BDA00028931766200000911
Figure BDA00028931766200000912
Wherein the content of the first and second substances,
Figure BDA00028931766200000913
using central consistency constraints, enforcing
Figure BDA00028931766200000914
And
Figure BDA00028931766200000915
share a feature distribution space:
Figure BDA0002893176620000101
wherein Z belongs to
Figure BDA0002893176620000102
And
Figure BDA0002893176620000103
union of (1), NbRepresenting the batch size during random gradient descent training. Finally, the abnormal decoder and the decoder are respectively paired
Figure BDA0002893176620000104
And ZaDecoding, abnormal negative samplesInI.e. decomposed into a textured background image
Figure BDA0002893176620000105
And abnormal foreground image
Figure BDA0002893176620000106
Figure BDA0002893176620000107
Wherein the content of the first and second substances,
Figure BDA0002893176620000108
fad(. cndot.) with θadRespectively representing the function and parameters of the anomalous decoder.
Finally, an anomalous negative example InWith reconstructed textured background image
Figure BDA0002893176620000109
The discriminator division defect input to the division guide:
Figure BDA00028931766200001010
wherein the content of the first and second substances,
Figure BDA00028931766200001011
is the output pixel-by-pixel prediction result. The pixel-by-pixel cross-entropy loss function is used to perform a segmentation-guided countermeasure learning process:
Figure BDA00028931766200001012
for decoding out abnormal foreground image
Figure BDA00028931766200001013
The pixel-by-pixel cross-entropy penalty is used for the segmentation penalty:
Figure BDA00028931766200001014
by combining equations (0-15) (0-18) (0-21) (0-22), the anomaly decomposition network can be trained through a multitask antagonistic learning process:
Figure BDA00028931766200001015
wherein λ is1,λ2,λ3,λ4Four lost weights respectively.
The purpose of the feature embedding encoder is to make the texture features of the defect-free positive sample image and the defect negative sample image share a common feature space, thereby forcing the model to efficiently separate the texture background image from the defect negative sample image.
The structure of the feature-embedded encoder is shown in fig. 2. Hidden spatial feature ZpAnd ZnDimension C that can be considered as an input imagelLocal feature set Z ofp＝{f1,...,fNWhere N ═ Wl×HlThe total number of local features. D ═ D1,...,dKDenotes a set of learnable word vectors of number K (K16 in this study). s ═ s1,...,sKAnd σ ═ σ }1,...,σKThe learnable smoothing factor and the diagonal covariance between the word vectors, respectively. The residual vector between the local features and the word vector can be calculated as: r isik＝fi-dkWherein, i is 1, 1., N, K is 1, 1., K. The weight assigned to each local feature to each word vector may be calculated as:
Figure BDA0002893176620000111
the first and second order residual coding vectors for each word vector may be computed as:
Figure BDA0002893176620000112
wherein the content of the first and second substances,
Figure BDA0002893176620000113
and
Figure BDA0002893176620000114
first and second order coded vectors, respectively.
Figure BDA0002893176620000115
And
Figure BDA0002893176620000116
quilt L2The code vectors concatenated into each word vector after normalization:
Figure BDA0002893176620000117
concatenating the code vectors of all word vectors, i.e.
Figure BDA0002893176620000118
A residual coding vector can be obtained, which represents the unordered texture feature expression. Finally, e is input to a fully-connected layer to reduce its dimension to ClAnd the obtained result vectors e' and Z are usedpMultiplying by channels to obtain final texture feature expression:
Figure BDA0002893176620000119
(4) actual defect detection process
After the model optimization is completed, the abnormal decomposition network can be used for detecting texture defects. As shown in FIG. 1, in the defect detection stage, a texture defect image I is first obtaineddInputting the abnormal decomposition network to obtain a texture background image
Figure BDA0002893176620000121
And abnormal segmentation of images
Figure BDA0002893176620000122
Then, the residual image and the abnormal segmentation image are fused to detect the texture surface defects:
Figure BDA0002893176620000123
wherein 0 < lambdafIs less than 1. The effect of defect detection is shown in fig. 3. The first column is the original image, the second column is the true value image, the third column is the detection result of the existing unsupervised defect detection method AE-SSIM which shows the best performance, and the fourth column is the semi-supervised mode detection result image.
The texture surface defect detection algorithm based on the abnormal synthesis and decomposition network provided by the invention utilizes the provided segmentation-guided defect generation network and a small amount of real defect training samples to generate a large amount of defect samples similar to real defects; the abnormal synthesis method based on Gaussian sampling can randomly synthesize abnormal negative samples by using defect-free positive samples; and finally, decomposing the abnormal negative sample into a texture background image and an abnormal mask image by the proposed abnormal decomposition network. When the detection is carried out, only an abnormal decomposition network is needed, and the defect image is decomposed into a texture background image and an abnormal image; and fusing a residual image between the defect image and the texture background image with the abnormal image, so that the defect can be accurately detected.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (7)

1. A texture surface defect detection method based on abnormal synthesis and decomposition is characterized by comprising the following steps:
s1, constructing a defect generation network for segmentation guidance; the defect generationThe network comprises a generator and a discriminator of the segmentation guide; the generator is used for synthesizing the first type negative sample image IcnConversion into a first type abnormal negative sample image I more similar to a real defect imagen(ii) a The first type of synthetic negative sample image IcnFrom defect-free textured background IpAnd a defect image IdDefective label image I oflBy regional superposition synthesis; the partition-guided discriminator is used for respectively discriminating abnormal negative sample images InWith the real defect image IdCarrying out defect segmentation;
s2, aiming at defect segmentation, training a defect generation network by using a counterstudy method;
s3, converting the second type of synthetic negative sample image I 'by using a trained generator'cnObtaining a second abnormal negative sample image I'n(ii) a The second type of synthetic negative sample image is composed of a defect-free texture background IpAbnormal mask image I generated by random samplingaBy regional superposition synthesis;
s4, constructing an abnormal decomposition network and using a defect-free texture background IpAnd a second-type abnormal negative sample image I'nTraining an abnormal decomposition network as a training set; the abnormal decomposition network is used for decomposing the defect image into a texture background image and an abnormal image;
and S5, inputting the texture surface image to be detected into the trained abnormal decomposition network to obtain a corresponding texture background image and an abnormal image, and fusing a residual image between the defect image and the texture background image with the abnormal image to obtain a defect detection result and obtain a defect detection result.
2. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in claim 1, wherein the first type synthesized negative sample image IcnFrom defect-free textured background IpAnd defective label image IlPerforming regional superposition synthesis by using the following expression;
Icn(x,y)＝(1-Il(x,y))·Ip(x,y)+λ·Il(x,y)
where λ represents a random variable, x 1., W, y 1., H, W, H represent the image width and height, respectively.
3. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in claim 1 or 2, characterized in that the defect generation network is trained by using the following loss function:
Figure FDA0002893176610000021
Figure FDA0002893176610000022
wherein L isdis(θsd) Representing the loss function of the training arbiter, Lgen(θg) A loss function representing a training generator;
Figure FDA0002893176610000023
*,||·||1,||·||2respectively representing expectation, matrix dot product, L1、L2Norm, λa、λbRepresenting the weight corresponding to each loss.
4. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in any one of claims 1 to 3, wherein the random sampling comprises Gaussian sampling, student's t-distribution sampling, and uniform sampling.
5. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in any one of claims 1-4, wherein the abnormal decomposition network training process is as follows:
defect-free textured background I using an encoderpCoded as corresponding implicit spatial features Zp(ii) a Using a feature embedding encoder to embed a spatial feature ZpEncoding into corresponding texture features
Figure FDA0002893176610000024
Decoding texture features with a decoder
Figure FDA0002893176610000025
Obtaining a defect-free textured background IpIs reconstructed image of
Figure FDA0002893176610000026
Utilizing an encoder to convert the second type abnormal negative sample image I'nCoded as corresponding implicit spatial features ZnAnd abnormal feature Za(ii) a Using a feature embedding encoder to embed a spatial feature ZnEncoding into corresponding texture features
Figure FDA0002893176610000027
Make it
Figure FDA0002893176610000028
And
Figure FDA0002893176610000029
sharing a feature distribution space; decoding texture features with a decoder
Figure FDA00028931766100000210
Correspondingly obtaining a second abnormal negative sample image I'nReconstructed textured background image
Figure FDA00028931766100000211
Decoding of exception features Z with exception decoderaObtaining an abnormal foreground image
Figure FDA00028931766100000212
Respectively processing the second type abnormal negative sample image I 'by a discriminator of segmentation guidance'nWith reconstructed textured background image
Figure FDA00028931766100000213
Carrying out defect segmentation;
an encoder, a feature embedding encoder, a decoder and a discriminator are trained using a counterlearning method.
6. The texture surface defect detection method based on abnormal synthesis and decomposition as claimed in claim 5, characterized in that the abnormal decomposition network is trained by using the following loss function:
Figure FDA0002893176610000031
Lrrepresenting reconstruction losses for training the encoder, the feature embedding encoder and the decoder; l iszRepresenting implicit spatial feature constraint loss; for training the encoder and the feature embedding encoder; l isaRepresenting an anomalous decoding loss for training the encoder and the anomalous decoder; l is a radical of an alcoholadRepresents the discrimination loss of the discriminator, LagArbiter generated losses for training an encoder, a feature-embedded encoder and a decoder, lambda1,λ2,λ3,λ4The weights of the four losses are respectively.
7. A texture surface defect detection system based on abnormal synthesis and decomposition, comprising: a computer-readable storage medium and a processor;
the computer-readable storage medium is used for storing executable instructions;
the processor is used for reading executable instructions stored in the computer-readable storage medium and executing the texture surface defect detection method based on abnormal synthesis and decomposition of any one of claims 1 to 6.
CN202110033380.9A 2021-01-12 2021-01-12 Texture surface defect detection method and system based on abnormal synthesis and decomposition Active CN112700432B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110033380.9A CN112700432B (en) 2021-01-12 2021-01-12 Texture surface defect detection method and system based on abnormal synthesis and decomposition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110033380.9A CN112700432B (en) 2021-01-12 2021-01-12 Texture surface defect detection method and system based on abnormal synthesis and decomposition

Publications (2)

Publication Number Publication Date
CN112700432A CN112700432A (en) 2021-04-23
CN112700432B true CN112700432B (en) 2022-07-05

Family

ID=75513933

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110033380.9A Active CN112700432B (en) 2021-01-12 2021-01-12 Texture surface defect detection method and system based on abnormal synthesis and decomposition

Country Status (1)

Country Link
CN (1) CN112700432B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113658167B (en) * 2021-08-24 2024-03-26 凌云光技术股份有限公司 Training image generation method and device containing defects
CN113947567B (en) * 2021-09-23 2024-03-22 西安电子科技大学广州研究院 Defect detection method based on multitask learning
CN115409694B (en) * 2022-11-03 2023-01-13 季华实验室 Semantic-guided defect image generation method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5737072A (en) * 1991-08-22 1998-04-07 Kla Instruments Corporation Automated photomask inspection apparatus and method
CN106023158A (en) * 2016-05-10 2016-10-12 浙江科技学院 SD-OCT-image-based nacre layer defect identification method for fresh water non-nucleated pearl
CN107563355A (en) * 2017-09-28 2018-01-09 哈尔滨工程大学 Hyperspectral abnormity detection method based on generation confrontation network
CN110969606A (en) * 2019-11-29 2020-04-07 华中科技大学 Texture surface defect detection method and system
CN112164033A (en) * 2020-09-14 2021-01-01 华中科技大学 Abnormal feature editing-based method for detecting surface defects of counternetwork texture

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7274995B2 (en) * 2003-11-19 2007-09-25 Honeywell International Inc. Apparatus and method for identifying possible defect indicators for a valve

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5737072A (en) * 1991-08-22 1998-04-07 Kla Instruments Corporation Automated photomask inspection apparatus and method
CN106023158A (en) * 2016-05-10 2016-10-12 浙江科技学院 SD-OCT-image-based nacre layer defect identification method for fresh water non-nucleated pearl
CN107563355A (en) * 2017-09-28 2018-01-09 哈尔滨工程大学 Hyperspectral abnormity detection method based on generation confrontation network
CN110969606A (en) * 2019-11-29 2020-04-07 华中科技大学 Texture surface defect detection method and system
CN112164033A (en) * 2020-09-14 2021-01-01 华中科技大学 Abnormal feature editing-based method for detecting surface defects of counternetwork texture

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
An Anomaly Feature-Editing-Based Adversarial Network for Texture Defect Visual Inspection;Hua Yang et al.;《IEEE Transactions on Industrial Informatics》;20200811;第2220 - 2230页 *

Also Published As

Publication number Publication date
CN112700432A (en) 2021-04-23

Similar Documents

Publication Publication Date Title
CN112700432B (en) Texture surface defect detection method and system based on abnormal synthesis and decomposition
Ulyanov et al. It takes (only) two: Adversarial generator-encoder networks
CN110570396B (en) Industrial product defect detection method based on deep learning
CN108648197B (en) Target candidate region extraction method based on image background mask
CN107967695B (en) A kind of moving target detecting method based on depth light stream and morphological method
CN108108751A (en) A kind of scene recognition method based on convolution multiple features and depth random forest
CN109886970A (en) The detection dividing method and computer storage medium of target object in terahertz image
CN110969606B (en) Texture surface defect detection method and system
CN112070727B (en) Metal surface defect detection method based on machine learning
CN113160123B (en) Leather defect detection method, system and device based on YOLOv5
CN114677346B (en) Method for detecting end-to-end semi-supervised image surface defects based on memory information
CN110930378B (en) Emphysema image processing method and system based on low data demand
CN114463759A (en) Lightweight character detection method and device based on anchor-frame-free algorithm
CN114565594A (en) Image anomaly detection method based on soft mask contrast loss
CN115131313A (en) Hyperspectral image change detection method and device based on Transformer
CN117409192B (en) Data enhancement-based infrared small target detection method and device
CN116883933A (en) Security inspection contraband detection method based on multi-scale attention and data enhancement
CN117011274A (en) Automatic glass bottle detection system and method thereof
CN116452408A (en) Transparent liquid sensing method based on style migration
CN115546171A (en) Shadow detection method and device based on attention shadow boundary and feature correction
Saito et al. Image processing method for automatic measurement of number of DNA breaks
CN114758123A (en) Remote sensing image target sample enhancement method
CN114666571A (en) Video sensitive content detection method and system
CN111429436A (en) Intrinsic image analysis method based on multi-scale attention and label loss
CN115035315B (en) Ceramic tile color difference grading detection method and system based on attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant