CN116580267A - Defect sample generation method and device, electronic equipment and storage medium - Google Patents

Defect sample generation method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116580267A
CN116580267A CN202310621403.7A CN202310621403A CN116580267A CN 116580267 A CN116580267 A CN 116580267A CN 202310621403 A CN202310621403 A CN 202310621403A CN 116580267 A CN116580267 A CN 116580267A
Authority
CN
China
Prior art keywords
defect
sample
time step
noise
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310621403.7A
Other languages
Chinese (zh)
Inventor
黄开竹
杨超龙
杨关禹
赵伟光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Duke Kunshan University
Original Assignee
Duke Kunshan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Duke Kunshan University filed Critical Duke Kunshan University
Priority to CN202310621403.7A priority Critical patent/CN116580267A/en
Publication of CN116580267A publication Critical patent/CN116580267A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Testing Or Measuring Of Semiconductors Or The Like (AREA)

Abstract

The application discloses a method and a device for generating a defect sample, electronic equipment and a storage medium; the method comprises the following steps: randomly sampling standard Gaussian noise, taking the noise distribution obtained by sampling as the noise characteristic of the current time step, inputting the noise characteristic and a semantic tag corresponding to the target defect into a defect generation network, and predicting the diffusion noise distribution of the last time step; randomly sampling based on the disturbance noise distribution to obtain the diffusion noise of the last time step; according to the diffusion noise and the noise characteristics of the current time step, calculating the noise characteristics of the last time step; repeating the above operation until the noise characteristic of the 0 th time step is calculated; generating a local defect image corresponding to the target defect based on the noise characteristic of the 0 th time step; inputting the image into a defect fusion network to generate a defect sample corresponding to the target defect. The embodiment of the application can generate the high-fidelity local defect and naturally blend the local defect into a normal sample.

Description

Defect sample generation method and device, electronic equipment and storage medium
Technical Field
The embodiment of the application relates to the technical field of artificial intelligence, in particular to a method and a device for generating a defect sample, electronic equipment and a storage medium.
Background
In recent years, defect detection has been of great importance in many fields. In the manufacturing industry, defect detection can help enterprises to find defects on a production line in time, and defective products are prevented from flowing into the market. The method can ensure the product quality, improve the user satisfaction and reduce the cost and loss of enterprises. Or in the medical field, the defect detection can help doctors to find focus and abnormality of diseases in time, thereby improving diagnosis accuracy and treatment effect. For example, in the field of medical imaging, defect detection may help doctors find lesions such as tumors, vascular lesions, etc., thereby aiding diagnosis and treatment.
Currently, the detection of defects by deep learning technology is the mainstream technology. However, the supervised approach based on deep learning requires a large amount of labeling data to train the model, but in practice, labeling data tends to be scarce, especially in the case of defect data. First, since true defect samples tend to be more difficult to obtain than normal samples, the number of samples for defect detection tends to be limited, which can limit the performance of the defect detection system. Through the defect sample generation technology, a large number of defect samples can be synthesized, so that the defect detection system can learn the characteristics and modes of defects more comprehensively, and the detection performance is improved.
Currently, there are some fault sample generation methods based on unsupervised learning, for example, a method based on generating a countermeasure network (GAN), a variational self-encoder (VAE), and the like. These methods may increase the number of defect samples by randomly generating data, but due to uncertainty in data generation, the generated defect samples may not be accurate and reliable enough and there is some defect bias. For this reason, researchers have proposed a Diffusion Model (Diffusion Model) to alleviate the problems of the generative Model described above. The diffusion model is a probability-based generation model for generating samples with highly realistic properties. The basic idea of the diffusion model is to consider the sample as an initial state and gradually diffuse the sample from a random state to a real sample space through a series of random, reverse diffusion processes. Compared with the traditional generation model, the diffusion model has the following advantages: the diffusion operation is reversible, so that the generation process of the sample can be precisely controlled, and a high-quality sample can be generated; the random walk in the diffusion process enables the model to have certain adaptability to different data.
However, the difficulty of directly generating the defect sample based on the generation model is high, mainly the occupied area of the defect part is small, the model is difficult to concentrate on the generation of the defect, and the network may pay more attention to the generation of the sample background. Thus, it is difficult to directly generate highly realistic defect samples from the above-described individual generation models.
Disclosure of Invention
The application provides a method, a device, an electronic device and a storage medium for generating a defect sample, which can generate a high-fidelity local defect and naturally blend the high-fidelity local defect into a normal sample.
In a first aspect, an embodiment of the present application provides a method for generating a defect sample, where the method includes:
randomly sampling standard Gaussian noise, and taking the noise distribution obtained by random sampling as the noise characteristic f of the current time step T And characterizing the noise characteristic f of the current time step T Inputting semantic tags y corresponding to pre-acquired target defects into a pre-trained defect generation network CUNet, and generating through the pre-trained defectsThe network CUNet predicts the distribution epsilon of the diffusion noise of the time step which is the last time step of the current time step θ (f T ,T,y)；
Distribution epsilon of disturbance noise based on the last time step θ (f T T, y) randomly sampling to obtain the diffusion noise of the last time step; and based on the diffuse noise of the last time step and the noise characteristic f of the current time step T Calculating the noise characteristic f of the last time step T-1 The method comprises the steps of carrying out a first treatment on the surface of the Repeating the above operation with the last time step as the current time step until the noise characteristic f of the 0 th time step is calculated 0 ；
Noise characteristics f based on time step 0 0 Generating a local defect image corresponding to the target defect;
inputting the local defect image into a pre-trained defect fusion network, and generating a defect sample corresponding to the target defect through the defect fusion network.
In a second aspect, an embodiment of the present application further provides a device for generating a defect sample, where the device includes: the system comprises a prediction module, a calculation module, a generation module and a fusion module; wherein,,
the prediction module is used for randomly sampling standard Gaussian noise, and taking the distribution of noise obtained by random sampling as the noise characteristic f of the current time step T And characterizing the noise characteristic f of the current time step T Inputting semantic tags y corresponding to pre-acquired target defects into a pre-trained defect generation network CUNet, and predicting the distribution epsilon of diffusion noise of the last time step of the current time step through the pre-trained defect generation network CUNet θ (f T ,T,f)；
The calculation module is used for calculating the distribution epsilon of disturbance noise based on the last time step θ (f T T, y) randomly sampling to obtain the diffusion noise of the last time step; and based on the diffuse noise of the last time step and the noise characteristic f of the current time step T Calculating the noise characteristic f of the last time step T-1 The method comprises the steps of carrying out a first treatment on the surface of the Repeating the above operation with the last time step as the current time step until the noise characteristic f of the 0 th time step is calculated 0 ；
The generation module is used for generating the noise characteristic f based on the 0 th time step 0 Generating a local defect image corresponding to the target defect;
the fusion module is used for inputting the local defect image into a pre-trained defect fusion network, and generating a defect sample corresponding to the target defect through the defect fusion network.
In a third aspect, an embodiment of the present application provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method for generating a defect sample according to any embodiment of the present application.
In a fourth aspect, an embodiment of the present application provides a storage medium having stored thereon a computer program, which when executed by a processor, implements the method for generating a defect sample according to any embodiment of the present application.
The embodiment of the application provides a method, a device, electronic equipment and a storage medium for generating a defect sample, which are characterized in that firstly, standard Gaussian noise is randomly sampled, the noise distribution obtained by random sampling is used as the noise characteristic of a current time step, the noise characteristic of the current time step and a semantic tag corresponding to a pre-acquired target defect are input into a pre-trained defect generation network, and the diffusion noise distribution of the last time step of the current time step is predicted through the pre-trained defect generation network; then randomly sampling based on the distribution of disturbance noise of the previous time step to obtain the diffusion noise of the previous time step; according to the diffusion noise of the last time step and the noise characteristic of the current time step, calculating the noise characteristic of the last time step of the current time step; taking the last time step as the current time step, and repeatedly executing the operation until the noise characteristic of the 0 th time step is calculated; generating a local defect image corresponding to the target defect based on the noise characteristic of the 0 th time step; and finally, inputting the local defect image into a pre-trained defect fusion network, and generating a defect sample corresponding to the target defect through the defect fusion network. That is, in the present application, the method for generating a defect sample is based on multiple stages, and in the first stage, a local defect image is generated through a defect generation network; in the second stage, highly realistic defect samples are generated by the defect fusion network. And the application gradually diffuses the sample from a random state to a real sample space through a reverse diffusion process. Compared with the traditional generation model, the application has the following advantages: the diffusion operation is reversible, and the generation process of the sample can be precisely controlled, so that a high-quality sample is generated. In the prior art, because the defect part occupies a small sample area, the model is difficult to concentrate on the generation of defects, so the difficulty of directly generating the defect sample based on the generation model is high, and the network may pay more attention to the generation of the sample background. Therefore, compared with the prior art, the method, the device, the electronic equipment and the storage medium for generating the defect sample can generate the high-fidelity local defect and naturally blend the high-fidelity local defect into the normal sample; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Drawings
FIG. 1 is a first flow chart of a method for generating a defect sample according to an embodiment of the present application;
FIG. 2 is a first flow chart of a training method of a defect generation network according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a training method of a defect generation network according to an example of the present application;
FIG. 4 is a second flow chart of a training method of the defect generation network according to the embodiment of the present application;
FIG. 5 is a first flow chart of a training method of a defect fusion network according to an embodiment of the present application;
FIG. 6 is a second flow chart of a training method of a defect fusion network according to an embodiment of the present application;
fig. 7 is a schematic diagram of a training method of a defect fusion network according to an example of the present application;
FIG. 8 is a schematic structural diagram of a device for generating a defect sample according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present application are shown in the drawings.
Example 1
Fig. 1 is a schematic flow chart of a first procedure of a method for generating a defect sample according to an embodiment of the present application, where the method may be performed by a device for generating a defect sample or an electronic device, and the device or the electronic device may be implemented by software and/or hardware, and the device or the electronic device may be integrated into any intelligent device having a network communication function. As shown in fig. 1, the method for generating the defect sample may include the steps of:
s101, randomly sampling standard Gaussian noise, taking the noise distribution obtained by random sampling as the noise characteristic of the current time step, inputting the noise characteristic of the current time step and a semantic tag corresponding to a pre-acquired target defect into a pre-trained defect generation network, and predicting the diffusion noise distribution of the last time step of the current time step through the pre-trained defect generation network.
In this step, the electronic device may randomly sample the standard gaussian noise, and take the noise distribution obtained by random sampling as the noise characteristic f of the current time step T And the noise characteristic f of the current time step T Semantic tags corresponding to pre-acquired target defects y is input into a pre-trained defect generation network CUNet, and the distribution epsilon of the diffusion noise of the time step which is the last time step of the current time step is predicted through the pre-trained defect generation network CUNet θ (f T T, y). Specifically, the electronic device may randomly sample the standard gaussian noise, and record the distribution of the noise obtained by random sampling as the noise characteristic of the T-th time step as f T The method comprises the steps of carrying out a first treatment on the surface of the Then f is carried out T Inputting semantic labels y corresponding to target defects into a pre-trained defect generation network CUNet, and predicting the distribution epsilon of diffusion noise of the T-1 time step through the CUNet θ (f T ,T,y)。
S102, randomly sampling the distribution of disturbance noise based on the previous time step to obtain the diffusion noise of the previous time step; according to the diffusion noise of the last time step and the noise characteristic of the current time step, calculating the noise characteristic of the last time step; and taking the last time step as the current time step, and repeatedly executing the operation until the noise characteristic of the 0 th time step is calculated.
In this step, the electronic device can be based on the distribution ε of the disturbance noise of the previous time step θ (f T T, y) randomly sampling to obtain diffusion noise of the last time step; and based on the diffuse noise of the last time step and the noise characteristics f of the current time step T Calculating the noise characteristic f of the last time step T-1 The method comprises the steps of carrying out a first treatment on the surface of the The previous time step is taken as the current time step, and the operation is repeatedly executed until the noise characteristic f of the 0 th time step is calculated 0 . Specifically, the noise characteristic of the T-1 time step is:wherein,,gaussian parameter beta t Linearly increasing in the range of (0.0001,0.002) in the diffuse t-step indicates that the amount of noise per step increases linearly while the gaussian parameter a t ＝1-β t ；/>
S103, generating a local defect image corresponding to the target defect based on the noise characteristic of the 0 th time step.
In this step, the electronic device may be based on the noise characteristic f of the 0 th time step 0 And generating a local defect image corresponding to the target defect. Specifically, the noise characteristic f of the 0 th time step is obtained by the above cyclic recursion 0 Will f 0 The partial defect image is inputted to a decoder D of a pre-trained hidden layer diffusion model, and the generated partial defect image is expressed as 512×512×3 channels.
S104, inputting the local defect image into a pre-trained defect fusion network, and generating a defect sample corresponding to the target defect through the defect fusion network.
In this step, the electronic device may input the local defect image into a pre-trained defect fusion network, and generate a defect sample corresponding to the target defect through the defect fusion network. The imperfect defect sample is input into a pre-trained defect fusion network, and a highly real defect sample can be generated.
According to the defect sample generation method provided by the embodiment of the application, firstly, random sampling is carried out on standard Gaussian noise, the noise distribution obtained by random sampling is used as the noise characteristic of the current time step, the noise characteristic of the current time step and the semantic label corresponding to the pre-acquired target defect are input into a pre-trained defect generation network, and the diffusion noise distribution of the last time step of the current time step is predicted through the pre-trained defect generation network; then randomly sampling based on the distribution of disturbance noise of the previous time step to obtain the diffusion noise of the previous time step; according to the diffusion noise of the last time step and the noise characteristic of the current time step, calculating the noise characteristic of the last time step of the current time step; taking the last time step as the current time step, and repeatedly executing the operation until the noise characteristic of the 0 th time step is calculated; generating a local defect image corresponding to the target defect based on the noise characteristic of the 0 th time step; and finally, inputting the local defect image into a pre-trained defect fusion network, and generating a defect sample corresponding to the target defect through the defect fusion network. That is, in the present application, the method for generating a defect sample is based on multiple stages, and in the first stage, a local defect image is generated through a defect generation network; in the second stage, highly realistic defect samples are generated by the defect fusion network. And the application gradually diffuses the sample from a random state to a real sample space through a reverse diffusion process. Compared with the traditional generation model, the application has the following advantages: the diffusion operation is reversible, and the generation process of the sample can be precisely controlled, so that a high-quality sample is generated. In the prior art, because the defect part occupies a small sample area, the model is difficult to concentrate on the generation of defects, so the difficulty of directly generating the defect sample based on the generation model is high, and the network may pay more attention to the generation of the sample background. Therefore, compared with the prior art, the method for generating the defect sample can generate the high-fidelity local defect and naturally blend the high-fidelity local defect into the normal sample; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example two
Fig. 2 is a first flow chart of a training method of a defect generation network according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments. As shown in fig. 2, the training method of the defect generation network may include the steps of:
s201, if the defect generation network does not meet a preset first convergence condition, a preset number of defect samples are extracted from a preset local defect set to serve as training samples of a current batch.
In this step, if the defect generating network does not meet the preset first convergence condition, the electronic device may perform the method of generating the local defect set I defect A predetermined number of defective samples are extracted as a current batch of training samples. In particular, the electronic device may be inPre-built local defect set I defect Random batch sampling of 16 training samplesI.e. batch size=16, subscript 0 indicates an initial state. To increase the robustness of the model, each sample in each batch is randomly sampled by the step t of the diffusion (b) E form ({ 1,2, …, T }); wherein T is a predefined maximum time step 200; the step size of diffusion for each sample is not fixed.
S202, randomly selecting a step length from a pre-constructed diffusion step length set as a corresponding step length of each training sample in the training samples of the current batch.
And S203, training the defect generation network by using the training samples of the current batch and the step length corresponding to each training sample, and repeatedly executing the operations until the defect generation network meets the first convergence condition.
In this step, the electronic device may train the defect generating network by using the training samples of the current batch and the step sizes corresponding to the training samples, and repeatedly execute the above operations until the defect generating network meets the first convergence condition. The defect generation network in the embodiments of the present application may be a pre-trained hidden layer diffusion model. Fig. 3 is a schematic diagram of a training method of the defect generation network according to an example of the present application. As shown in fig. 3, the training method of the defect generation network may include the steps of:
step one: defining a maximum time step t=200 of diffusion, gaussian parameter β t Linearly increasing in the range of (0.0001,0.002) in the diffuse t-step indicates that the amount of noise per step increases linearly while the gaussian parameter a t ＝1-β t ；
Step two: from a pre-built local defect set I defect Randomly sampling 16 training samplesI.e. batch size=16, subscript 0 indicates an initial state. To increase the robustness of the model, each sample in each batch is randomly sampled by the step t of the diffusion (b) E form ({ 1,2, …, T }); wherein T is a predefined maximum time step 200; the step size of diffusion for each sample is not fixed.
Step three: inputting the training samples of the current batch into an encoder E of a pre-training hidden layer expansion model, and obtaining hidden layer characteristics corresponding to each training sample through the encoder EExpressed as the number of wide and high channels 64 x 3. For standard Gaussian noise ε (b) N (0, 1) is randomly sampled, and the number of the wide-high channels is 64 multiplied by 3. Then the noise characteristic of step t is: />
Step four: will f t (b) And corresponding semantic tags y (b) Inputting the noise distribution into the CUNet, and predicting the noise distribution of the t step through the CUNet network as follows:where θ is a parameter of the CUNet network. The loss function employed by the CUNet network is: />
Step five: and continuously iterating the operations from the second step to the fourth step, optimizing the CUNet network based on the minimized loss function L in each iteration process, and updating the parameter theta of the CUNet network by using an Adam optimization method in each iteration process until the model converges.
According to the training method of the defect generation network, if the defect generation network does not meet the preset first convergence condition, a preset number of defect samples are extracted from a preset local defect set to serve as training samples of a current batch; then randomly selecting a step length from a pre-constructed diffusion step length set as a corresponding step length of each training sample in the training samples of the current batch; and training the defect generation network by using the training samples of the current batch and the step length corresponding to each training sample, and repeatedly executing the operation until the defect generation network meets the first convergence condition. That is, the application can collect a certain number of defect samples in batches in the local defect set constructed in advance, and train the defect generation network by using each batch of training samples and the step length corresponding to each training sample. Compared with the traditional generation model, the application has the following advantages: the diffusion operation is reversible, and the generation process of the sample can be precisely controlled, so that a high-quality sample is generated. In the prior art, because the defect part occupies a small sample area, the model is difficult to concentrate on the generation of defects, so the difficulty of directly generating the defect sample based on the generation model is high, and the network may pay more attention to the generation of the sample background. Therefore, compared with the prior art, the method for generating the defect sample can generate the high-fidelity local defect and naturally blend the high-fidelity local defect into the normal sample; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example III
Fig. 4 is a second flow chart of a training method of the defect generating network according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments. As shown in fig. 4, the training method of the defect generation network may include the steps of:
s401, marking each defect sample in the pre-constructed real defect sample set, and shearing the real defects in the defect sample set to form an original local defect set.
In this step, the electronic device can perform a pre-built operation on the actual defect sample setMarking each defective sample in the list, and marking the defective sample set +.>To form the original local defect set under real defect shearing>In particular, the electronic device can +.>Marking each defect sample, cutting off the real defects to form original local defect set +.>The cut defect image is expressed as 512×512×3 in the number of wide and high channels.
S402, taking each defect sample in the original local defect set as a first defect sample, taking one defect sample randomly selected in the original defect sample set as a second defect sample corresponding to each first defect sample, and mixing each first defect sample with the corresponding second defect sample to obtain each new virtual sample and a label corresponding to each new virtual sample.
In this step, the electronic device may collect the original local defect setMixing with a randomly selected other sample, the two samples sampled can be expressed separately as: defect a and defect B, respectively, are noted: x is x A And x B The labels being y respectively A And y B . According to the Mixup technique, the new virtual samples and labels are respectively: x is x AB And y AB The method comprises the steps of carrying out a first treatment on the surface of the Wherein x is AB ＝λx A +(1-λ)x B ；y AB ＝λx a +(1-λ)y B The method comprises the steps of carrying out a first treatment on the surface of the Wherein lambda is a compliance BetaA distributed random number. X is x AB 、y AB And the original local defect set->Together as inputs to a pre-trained hidden diffusion model, denoted as I defect . The application provides a method for treating original local defect set +.>And generating a plurality of defects and spliced samples of the defects by adopting a data brightening method based on Mixup so as to improve the diversity of matching of the samples in the data set and semantic annotations, thereby increasing the cognitive information provided by the data for different semantic annotations in the training process.
S403, constructing a local defect set based on each new virtual sample, the label corresponding to each new virtual sample and the original local defect set.
S404, if the defect generation network does not meet a preset first convergence condition, a preset number of defect samples are extracted from the preset local defect set to serve as training samples of the current batch.
S405, randomly selecting a step length from a pre-constructed diffusion step length set as a corresponding step length of each training sample in the training samples of the current batch.
S406, training the defect generation network by using the training samples of the current batch and the step sizes corresponding to the training samples, and repeatedly executing the operations until the defect generation network meets the first convergence condition.
According to the training method of the defect generation network, if the defect generation network does not meet the preset first convergence condition, a preset number of defect samples are extracted from a preset local defect set to serve as training samples of a current batch; then randomly selecting a step length from a pre-constructed diffusion step length set as a corresponding step length of each training sample in the training samples of the current batch; and training the defect generation network by using the training samples of the current batch and the step length corresponding to each training sample, and repeatedly executing the operation until the defect generation network meets the first convergence condition. That is, the embodiment of the application can collect a certain number of defect samples in batches in the local defect set constructed in advance, and train the defect generation network by using each batch of training samples and the step length corresponding to each training sample. Compared with the traditional generation model, the embodiment of the application has the following advantages: the diffusion operation is reversible, and the generation process of the sample can be precisely controlled, so that a high-quality sample is generated. In the prior art, because the defect part occupies a small sample area, the model is difficult to concentrate on the generation of defects, so the difficulty of directly generating the defect sample based on the generation model is high, and the network may pay more attention to the generation of the sample background. Therefore, compared with the prior art, the method for generating the defect sample can generate the high-fidelity local defect and naturally blend the high-fidelity local defect into the normal sample; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example IV
Fig. 5 is a first flow chart of a training method of a defect fusion network according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments. As shown in fig. 5, the training method of the defect fusion network may include the following steps:
s501, if the defect fusion network does not meet a preset second convergence condition, adding the local defect image into a pre-constructed normal sample to obtain an imperfect defect sample set.
S502, randomly selecting an imperfect defect sample from an imperfect defect sample set to serve as a current training sample.
And S503, training the defect fusion network by using the current training sample, and repeatedly executing the operation until the defect fusion network meets a second convergence condition.
In this step, when the electronic device trains the defect fusion network, if the arbiter does not meet the preset requirementIf the third convergence condition is set, the electronic device may input the current training sample into the defect fusion network, and output the generated defect sample x through the defect fusion network fake The method comprises the steps of carrying out a first treatment on the surface of the Then using the generated defect sample x fake Training the discriminator until the discriminator meets a third convergence condition; based on the generated defect sample x fake And training the defect fusion network by the trained discriminators.
Further, the electronic device is using the generated defect sample x fake When training the discriminator, the generated defect sample x is firstly generated fake Is input into a discriminator, through which a loss function L based on a predetermined is determined adv Calculating a generated defect sample x fake Is of (1)Then according to the sample disturbance->In the generated defect sample x fake Determining the most realistic disturbance countermeasure sample x in the neighborhood of (2) adv-p The method comprises the steps of carrying out a first treatment on the surface of the Then in the real defect sample set +.>Randomly sampling to obtain a true defect sample x real The method comprises the steps of carrying out a first treatment on the surface of the And then the true defect sample x real Countering sample x with the truest disturbance adv-p Inputting the binary predicted value into a discriminator, and outputting the binary predicted value through the discriminator; adjusting parameters in the discriminator according to the binary predicted value; wherein, a true defect sample x real The target of the predicted value of (2) is 1; the truest disturbance counter sample x adv-p Is targeted at 0.
Further, the electronic device is based on the generated defect sample x fake When the trained discriminators train the defect fusion network, the generated defect sample x can be firstly generated dake Input into a trained arbiter, through which the training arbiter is based on pre-prediction A predetermined loss function L adv Calculating a generated defect sample x fake Is of (1)Then according to the sample disturbance->In the generated defect sample x fake Determining the least realistic disturbance countermeasure sample x in the neighborhood of (2) adv-F The method comprises the steps of carrying out a first treatment on the surface of the The least realistic disturbance is then opposed to sample x adv-F Inputting the binary predicted value into a discriminator, and outputting the binary predicted value through the discriminator; adjusting parameters in the defect fusion network according to the binary predicted value; wherein the binary predictor is targeted to 1.
According to the training method of the defect fusion network, if the defect fusion network does not meet the preset second convergence condition, a local defect image is added into a pre-constructed normal sample, and an imperfect defect sample set is obtained; then randomly selecting an imperfect defect sample from the imperfect defect sample set as a current training sample; and training the defect fusion network by using the current training sample, and repeatedly executing the operation until the defect fusion network meets a second convergence condition. That is, the embodiment of the application can obtain an imperfect defect sample based on the local defect image and the normal sample, and then train the defect fusion network by using the imperfect defect sample. Compared with the prior art, the method for generating the defect sample can generate the high-fidelity local defect and naturally blend the high-fidelity local defect into the normal sample; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example five
Fig. 6 is a second flow chart of a training method of a defect fusion network according to an embodiment of the present application. Further optimization and expansion based on the above technical solution can be combined with the above various alternative embodiments. As shown in fig. 6, the training method of the defect fusion network may include the following steps:
s601, if the defect generation network does not meet a preset second convergence condition, adding the local defect image into a pre-constructed normal sample to obtain an imperfect defect sample set.
S602, randomly selecting an imperfect defect sample from an imperfect defect sample set as a current training sample.
And S603, if the discriminator does not meet the preset third convergence condition, inputting the current training sample into a defect fusion network, and outputting the generated defect sample through the defect fusion network.
S604, training the discriminator by using the generated defect sample until the discriminator meets a third convergence condition.
And S605, training the defect fusion network based on the generated defect sample and the trained discriminator, and repeating the operation until the defect fusion network meets a second convergence condition.
Fig. 7 is a schematic diagram of a training method of a defect fusion network according to an example of the present application. As shown in fig. 7, the training method of the defect fusion network may include the following steps:
step one: adding the local defect image into a pre-constructed normal sample to obtain an imperfect defect sample setSpecifically, the electronic device may randomly downsample the defects generated by the defect generation network into an image with a width-height channel lower than 512×512, and paste the image onto any position in the normal sample to obtain an imperfect defect sample set, which is recorded as->Expressed as a number of wide and high channels of 512 x 3.
Step two: from the slaveMiddle random sampling training sample->Inputting the defect sample into a defect fusion network UNetResNet34, denoted as F, and obtaining a generated defect sample +.>
Step three: taking FGSM as an example, a defect sample x is generated according to a pair of discriminators P fake Is a loss function L of (2) adv ＝logP(x fake ) Calculating a generated defect sample x fake Is of (1)
Step four: finding disturbance countermeasure sample x that is considered to be the most realistic in the neighborhood of the generated defect sample adv-p ；Wherein, E is a disturbance factor, and the value can be 0.1; sign (·) represents a signed function.
Step five: from a true defect sample setRandomly sampling to obtain a true defect sample x real Will x adv-p And x real Input to the discriminator FCN, denoted as P, output binary predicted values, whose corresponding labels are 0 and 1, respectively. Wherein, the loss function of the discriminator is as follows: />
Step six: a fixed arbiter that recalculates the sample perturbation and finds the perturbation in the neighborhood of the generated defective sample that is considered the least realistic against sample x adv-F ；Optimizing a defect fusion network, wherein the loss function of the defect fusion network is as follows: />
Step seven: and (3) continuously iterating the steps two to six, training the defect fusion network and the discriminant, and updating network parameters by using an Adam optimization method in each iteration process until the model converges.
According to the training method of the defect fusion network, if the defect fusion network does not meet the preset second convergence condition, a local defect image is added into a pre-constructed normal sample, and an imperfect defect sample set is obtained; then randomly selecting an imperfect defect sample from the imperfect defect sample set as a current training sample; and training the defect fusion network by using the current training sample, and repeatedly executing the operation until the defect fusion network meets a second convergence condition. That is, the embodiment of the application can obtain an imperfect defect sample based on the local defect image and the normal sample, and then train the defect fusion network by using the imperfect defect sample. Compared with the prior art, the method for generating the defect sample can generate the high-fidelity local defect and naturally blend the high-fidelity local defect into the normal sample; in addition, the technical scheme of the embodiment of the application is simple and convenient to realize, convenient to popularize and wider in application range.
Example six
Fig. 8 is a schematic structural diagram of a device for generating a defect sample according to an embodiment of the present application. As shown in fig. 8, the defect sample generating device includes: a prediction module 801, a calculation module 802, a generation module 803 and a fusion module 804; wherein,,
the prediction module 801 is configured to randomly sample standard gaussian noise, and take a noise distribution obtained by random sampling as a noise characteristic f of a current time step T And characterizing the noise characteristic f of the current time step T Inputting semantic tags y corresponding to pre-acquired target defects into a pre-trained defect generation network CUNet, and predicting the distribution epsilon of diffusion noise of the last time step of the current time step through the pre-trained defect generation network CUNet θ (f T ,T,y)；
The calculation module 802 is configured to calculate a distribution epsilon of disturbance noise based on the previous time step θ (f T T, y) randomly sampling to obtain the diffusion noise of the last time step; and based on the diffuse noise of the last time step and the noise characteristic f of the current time step T Calculating the noise characteristic f of the last time step T-1 The method comprises the steps of carrying out a first treatment on the surface of the Repeating the above operation with the last time step as the current time step until the noise characteristic f of the 0 th time step is calculated 0 ；
The generating module 803 is configured to generate a noise characteristic f based on the 0 th time step 0 Generating a local defect image corresponding to the target defect;
the fusion module 804 is configured to input the local defect image into a pre-trained defect fusion network, and generate a defect sample corresponding to the target defect through the defect fusion network.
The generating device of the defect sample can execute the method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing the method. Technical details not described in detail in this embodiment may refer to the method for generating a defect sample provided in any embodiment of the present application.
Example seven
Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Fig. 9 shows a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the application. The electronic device 12 shown in fig. 9 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 9, the electronic device 12 is in the form of a general purpose computing device. Components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, a bus 18 that connects the various system components, including the system memory 28 and the processing units 16.
Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Electronic device 12 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by electronic device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from or write to non-removable, nonvolatile magnetic media (not shown in FIG. 9, commonly referred to as a "hard disk drive"). Although not shown in fig. 9, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 18 through one or more data medium interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the application.
A program/utility 40 having a set (at least one) of program modules 42 may be stored in, for example, memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 42 generally perform the functions and/or methods of the embodiments described herein.
The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and/or any devices (e.g., network card, modem, etc.) that enable the electronic device 12 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 22. Also, the electronic device 12 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 over the bus 18. It should be appreciated that although not shown in fig. 9, other hardware and/or software modules may be used in connection with electronic device 12, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing the defect sample generation method provided by the embodiment of the present application.
Example eight
The embodiment of the application provides a computer storage medium.
The computer-readable storage media of embodiments of the present application may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
Note that the above is only a preferred embodiment of the present application and the technical principle applied. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, while the application has been described in connection with the above embodiments, the application is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the application, which is set forth in the following claims.

Claims (10)

1. A method for generating a defect sample, the method comprising:
randomly sampling standard Gaussian noise, and taking the noise distribution obtained by random sampling as the noise characteristic f of the current time step T And characterizing the noise characteristic f of the current time step T Inputting semantic tags y corresponding to pre-acquired target defects into a pre-trained defect generation network CUNet, and predicting the distribution epsilon of diffusion noise of the last time step of the current time step through the pre-trained defect generation network CUNet θ (f T ,T,y)；
Distribution epsilon of disturbance noise based on the last time step θ (f T T, y) randomly sampling to obtain the diffusion noise of the last time step; and based on the diffuse noise of the last time step and the noise characteristic f of the current time step T Calculating the last time stepNoise characteristic f of (2) T-1 The method comprises the steps of carrying out a first treatment on the surface of the Repeating the above operation with the last time step as the current time step until the noise characteristic f of the 0 th time step is calculated 0 ；
Noise characteristics f based on time step 0 0 Generating a local defect image corresponding to the target defect;
inputting the local defect image into a pre-trained defect fusion network, and generating a defect sample corresponding to the target defect through the defect fusion network.
2. The method of claim 1, wherein prior to randomly sampling the standard gaussian noise, the method further comprises:
if the defect generation network CUNet does not meet the preset first convergence condition, the defect generation network CUNet is in the preset local defect set I defect Extracting a preset number of defect samples as training samples of the current batch;
for each training sample in the current batch of training samples, randomly selecting a step length from a pre-constructed diffusion step length set as a corresponding step length of the training sample;
Training the defect generation network CUNet by using the training samples of the current batch and the step length corresponding to each training sample, and repeatedly executing the operations until the defect generation network meets the first convergence condition.
3. The method according to claim 2, characterized in that in the pre-constructed local defect set I defect Before extracting the predetermined number of defect samples as the training samples of the current batch, the method further comprises:
for a pre-constructed real defect sample setMarking each defective sample in the list, and marking the defective sample set +.>To form the original local defect set under real defect shearing>
Collecting said original local defect setRespectively as a first defect sample in said original defect sample set +.>Taking one defect sample randomly selected as a second defect sample corresponding to each first defect sample, and mixing each first defect sample with the corresponding second defect sample to obtain each new virtual sample and a label corresponding to each new virtual sample;
based on each new virtual sample, the label corresponding to each new virtual sample, and the original local defect set Constructing the local defect set I defect 。
4. The method of claim 1, wherein prior to entering the local defect image into a pre-trained defect fusion network, the method further comprises:
if the defect fusion network does not meet the preset second convergence condition, adding the local defect image into a pre-constructed normal sample to obtain an imperfect defect sample setIn the imperfect defect sample set +.>Is selected randomly for an imperfect defect sample +.>As a current training sample;
and training the defect fusion network by using the current training sample, and repeatedly executing the operation until the defect fusion network meets the second convergence condition.
5. The method of claim 4, wherein training the defect fusion network using the current training samples comprises:
if the discriminator does not meet the preset third convergence condition, inputting the current training sample into the defect fusion network, and outputting the generated defect sample x through the defect fusion network fake ；
Using the generated defect sample x fake Training a discriminator until the discriminator meets the third convergence condition;
Based on the generated defect sample x fake And training the defect fusion network by the trained discriminators.
6. The method according to claim 5, wherein the generated defect samples x are used fake Training the arbiter, comprising:
sample the generated defect sample x fake Is input into a discriminator, by means of which it is based on a predetermined loss function L adv Calculating the generated defect sample x fake Is of (1)
Based on the sample disturbanceAt the generated defect sample x fake Determining the most realistic disturbance countermeasure sample x in the neighborhood of (2) adv-p ；
In a real defect sample setRandomly sampling to obtain a true defect sample x real ；
Sample the real defect x real Countering sample x with the truest disturbance adv-p Inputting the binary predicted value into the discriminator, and outputting the binary predicted value through the discriminator; adjusting parameters in the discriminator according to the binary predicted value; wherein the true defect sample x real The target of the predicted value of (2) is 1; the truest disturbance counter-measures the sample x adv-p Is targeted at 0.
7. The method according to claim 6, wherein the defect sample x generated is based on fake And training the defect fusion network by the trained discriminators, including:
sample the generated defect sample x fake Input into the trained discriminators, through which the loss function L is based on a predetermined adv Calculating the generated defect sample x fake Is of (1)
Based on the sample disturbanceAt the generated defect sample x fake Determining the least realistic disturbance countermeasure sample x in the neighborhood of (2) adv-F ；
The least realistic disturbance is opposed to sample x adv-F Is input into the discriminator through theThe discriminator outputs a binary predicted value; adjusting parameters in the defect fusion network according to the binary predicted value; wherein the binary predictor is targeted at 1.
8. A device for generating a defect sample, the device comprising: the system comprises a prediction module, a calculation module, a generation module and a fusion module; wherein,,
the prediction module is used for randomly sampling standard Gaussian noise, and taking the distribution of noise obtained by random sampling as the noise characteristic f of the current time step T And characterizing the noise characteristic f of the current time step T Inputting semantic tags y corresponding to pre-acquired target defects into a pre-trained defect generation network CUNet, and predicting the distribution epsilon of diffusion noise of the last time step of the current time step through the pre-trained defect generation network CUNet θ (f T ,T,y)；
The calculation module is used for calculating the distribution epsilon of disturbance noise based on the last time step θ (f T T, y) randomly sampling to obtain the diffusion noise of the last time step; and based on the diffuse noise of the last time step and the noise characteristic f of the current time step T Calculating the noise characteristic f of the last time step T-1 The method comprises the steps of carrying out a first treatment on the surface of the Repeating the above operation with the last time step as the current time step until the noise characteristic f of the 0 th time step is calculated 0 ；
The generation module is used for generating the noise characteristic f based on the 0 th time step 0 Generating a local defect image corresponding to the target defect;
the fusion module is used for inputting the local defect image into a pre-trained defect fusion network, and generating a defect sample corresponding to the target defect through the defect fusion network.
9. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of generating defect samples of any of claims 1-7.
10. A storage medium having stored thereon a computer program, which when executed by a processor implements the method of generating a defect sample according to any of claims 1 to 7.
CN202310621403.7A 2023-05-30 2023-05-30 Defect sample generation method and device, electronic equipment and storage medium Pending CN116580267A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310621403.7A CN116580267A (en) 2023-05-30 2023-05-30 Defect sample generation method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310621403.7A CN116580267A (en) 2023-05-30 2023-05-30 Defect sample generation method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116580267A true CN116580267A (en) 2023-08-11

Family

ID=87533969

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310621403.7A Pending CN116580267A (en) 2023-05-30 2023-05-30 Defect sample generation method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116580267A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117333740A (en) * 2023-12-01 2024-01-02 东声(苏州)智能科技有限公司 Defect image sample generation method and device based on stable diffusion model
CN117649351A (en) * 2024-01-30 2024-03-05 武汉大学 Diffusion model-based industrial defect image simulation method and device

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117333740A (en) * 2023-12-01 2024-01-02 东声(苏州)智能科技有限公司 Defect image sample generation method and device based on stable diffusion model
CN117333740B (en) * 2023-12-01 2024-04-05 东声(苏州)智能科技有限公司 Defect image sample generation method and device based on stable diffusion model
CN117649351A (en) * 2024-01-30 2024-03-05 武汉大学 Diffusion model-based industrial defect image simulation method and device
CN117649351B (en) * 2024-01-30 2024-04-19 武汉大学 Diffusion model-based industrial defect image simulation method and device

Similar Documents

Publication Publication Date Title
CN111090987B (en) Method and apparatus for outputting information
US10592779B2 (en) Generative adversarial network medical image generation for training of a classifier
US10937540B2 (en) Medical image classification based on a generative adversarial network trained discriminator
US10540578B2 (en) Adapting a generative adversarial network to new data sources for image classification
WO2021103761A1 (en) Compound property analysis method and apparatus, compound property analysis model training method, and storage medium
WO2020087974A1 (en) Model generation method and device
CN116580267A (en) Defect sample generation method and device, electronic equipment and storage medium
CN108416310B (en) Method and apparatus for generating information
WO2022188584A1 (en) Similar sentence generation method and apparatus based on pre-trained language model
Azad et al. Foundational models in medical imaging: A comprehensive survey and future vision
US20200074267A1 (en) Data prediction
CN112308077A (en) Sample data acquisition method, image segmentation method, device, equipment and medium
US20220188575A1 (en) Crowdsourcing to filter out unpopular potential candidate answers
Norris Machine Learning with the Raspberry Pi
Hong et al. Utilizing bots for sustainable news business: Understanding users’ perspectives of news bots in the age of social media
Yu et al. Diffusion-based data augmentation for nuclei image segmentation
US20210150270A1 (en) Mathematical function defined natural language annotation
Cahan et al. Multimodal fusion models for pulmonary embolism mortality prediction
Biró et al. Visual Object Detection with DETR to Support Video-Diagnosis Using Conference Tools
CN113962737A (en) Target recognition model training method and device, and target recognition method and device
US11822896B2 (en) Contextual diagram-text alignment through machine learning
Zhu et al. Medical named entity recognition of Chinese electronic medical records based on stacked Bidirectional Long Short-Term Memory
CN114241411B (en) Counting model processing method and device based on target detection and computer equipment
CN115410642A (en) Biological relation network information modeling method and system
CN111046085B (en) Data tracing processing method and device, medium and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination