CN113610787A

CN113610787A - Training method and device of image defect detection model and computer equipment

Info

Publication number: CN113610787A
Application number: CN202110852005.7A
Authority: CN
Inventors: 林珠; 赵晓萌; 方少亮; 周俊杰; 陈树敏; 杜宝林
Original assignee: Guangdong Science & Technology Infrastructure Center
Current assignee: Guangdong Science & Technology Infrastructure Center
Priority date: 2021-07-27
Filing date: 2021-07-27
Publication date: 2021-11-05

Abstract

The application relates to a training method and device of an image defect detection model, computer equipment and a storage medium. The method comprises the following steps: a plurality of unlabeled generated images belonging to the target domain may be generated by a first generator in the trained feature enhancement model. Thus, the generated images with greatly enhanced characteristics can be obtained, and then the generated images are clustered to obtain the categories corresponding to the generated images respectively; constructing various class confrontation models, and obtaining the trained various class confrontation models based on the generated images of the same class; random variable data and the noise data are acquired and input to the second generators in the various category countermeasure models in a superposition mode, category characteristic images of various categories are generated greatly, the condition that the sample size is insufficient is relieved greatly, the over-fitting condition can be avoided, the trained defect detection models with improved generalization capability are obtained based on the category characteristic images, and the accuracy and the recall rate of micron-scale defect detection are improved greatly.

Description

Training method and device of image defect detection model and computer equipment

Technical Field

The present application relates to the field of image detection technologies, and in particular, to a training method and apparatus for an image defect detection model, a computer device, and a storage medium.

Background

With the development of image detection technology, when defect detection is performed on the surfaces of products in the fields of new energy, semiconductors, panel display, circuit boards and the like, the features of the images of the products can be extracted through an image processing algorithm, and image defects can be identified according to feature information.

However, when the micron-level image detection is performed on the image, the number of samples with micron-level defects is small and unbalanced, so that the defect detection model formed by training is over-fitted, and the accuracy of the micron-level defect detection is greatly reduced.

Disclosure of Invention

In view of the above, it is necessary to provide a training method, an apparatus, a computer device and a storage medium for an image defect detection model.

A method of training an image defect detection model, the method comprising:

acquiring a first generator in the trained feature enhancement model; the feature enhancement model is obtained by pre-training based on source domain sample images and noise data and then re-training based on target domain sample images and noise data; inputting the noise data to the first generator to obtain a plurality of label-free generated images; clustering the generated images to obtain the categories corresponding to the generated images respectively; respectively constructing class confrontation models respectively corresponding to each class, and training the class confrontation models corresponding to the corresponding classes based on the generated image of the same class to obtain the trained class confrontation models respectively corresponding to the classes; acquiring random variable data, and superposing and inputting the noise data and the random variable data to a second generator in each trained class countermeasure model to obtain class characteristic images respectively corresponding to the classes; and constructing a defect detection model, and training the defect detection model based on the plurality of class characteristic images to obtain a trained defect detection model, wherein the trained defect detection model is used for detecting the defects of the to-be-detected image of the target domain.

In one embodiment, the feature enhancement model includes a first generator and a first discriminator, and the pre-training step of the feature enhancement model includes:

acquiring source domain sample images and noise data; performing feature extraction on the current source domain sample image to obtain a current source domain feature map; generating a current source domain training generation image through the first generator based on the noise data and the hidden features of the source domain samples obtained in the previous training iteration; inputting the current source domain feature map and the current source domain training generated image into a first discriminator to obtain a first discrimination result and a current source domain sample hidden feature; and adjusting the model parameters of the feature enhancement model based on the first judgment result, entering next cycle iteration, taking the hidden features of the current source domain sample as the hidden features of the source domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the current source domain sample image for feature extraction, and continuing to execute the step of obtaining the current source domain feature map until a first training stop condition is reached, so as to obtain the pre-trained feature enhancement model.

In one embodiment, the retraining step of the feature-enhanced model includes:

acquiring a target domain sample image and noise data; performing feature extraction on the current target domain sample image to obtain a current target domain feature map; generating a current target domain training generation image through a first generator obtained by pre-training based on the noise data and the target domain sample hidden features obtained by the previous training iteration; inputting the current target domain feature map and the current target domain training generated image into a first discriminator obtained by pre-training to obtain a second discrimination result and a current target domain sample hidden feature; and adjusting model parameters of the feature enhancement model obtained by pre-training based on the second judgment result, entering next cycle iteration, taking the hidden features of the current target domain sample as the hidden features of the target domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the current target domain sample image for feature extraction, and continuing to execute the step of obtaining the current target domain feature map until a second training stop condition is reached, so as to obtain the trained feature enhancement model.

In one embodiment, the performing feature extraction on the current target domain sample image to obtain a current target domain feature map includes:

extracting a target area in the current target area sample image to obtain a target area image; and performing feature extraction on the target area image to obtain a current target area feature map.

In one embodiment, the class confrontation model includes a second generator and a second discriminator, the training of the class confrontation model with the corresponding class is performed on the generated image based on the same class, and the class confrontation model corresponding to each class after training is obtained, including:

acquiring generated images of the same category, and performing feature extraction on the current generated image to obtain a current feature extraction image; scene sample noise data and random variable data are obtained, and a training class characteristic image of the current time is generated through the second generator on the basis of the scene noise data, the random variable data and a semi-supervised sample obtained in the previous training iteration; inputting the current feature extraction image and the current training class feature image into a second discriminator to obtain a third discrimination result and a current semi-supervised sample; and adjusting model parameters of the class countermeasure model based on the third judgment result, entering next cycle iteration, taking the current semi-supervised sample as the semi-supervised sample obtained by the previous training iteration corresponding to the next training iteration, returning the current generated image to perform feature extraction, continuously executing the step of obtaining the current feature extraction image until a third training stop condition is reached, and obtaining the trained class countermeasure model corresponding to the corresponding class.

In one embodiment, the obtaining scene sample noise data comprises:

performing scene prejudgment on the characteristic generating images, and determining a target scene corresponding to each characteristic generating image; scene sample noise data corresponding to the target scene is acquired.

In one embodiment, the inputting the current feature extraction graph and the current training class feature image into the second decision device to obtain a third decision result and a current semi-supervised sample includes:

inputting the current feature extraction image and the current training class feature image into a second discriminator to obtain a third discrimination result and a middle semi-supervised sample; and when the intermediate semi-supervised sample meets the preset threshold range, taking the intermediate semi-supervised sample as the current semi-supervised sample.

An apparatus for training an image defect detection model, the apparatus comprising:

the acquisition module is used for acquiring a first generator in the trained feature enhancement model; the feature enhancement model is obtained by pre-training based on source domain sample images and noise data and then re-training based on target domain sample images and noise data;

a first generating module for inputting the noise data to the first generator to obtain a plurality of label-free generated images;

the clustering module is used for clustering the generated images to obtain the categories corresponding to the generated images respectively;

the first training module is used for respectively constructing class confrontation models respectively corresponding to each class, and training the class confrontation models corresponding to the corresponding classes based on the generated image of the same class to obtain trained class confrontation models respectively corresponding to the classes;

the second generation module is used for acquiring random variable data, and superposing and inputting the noise data and the random variable data to a second generator in each trained class countermeasure model to obtain class characteristic images respectively corresponding to the classes;

and the second training module is used for constructing a defect detection model, training the defect detection model based on the plurality of class characteristic images to obtain a trained defect detection model, and carrying out defect detection on the to-be-detected image of the target domain by using the trained defect detection model.

A computer device comprising a memory storing a computer program and a processor implementing the method of training an image defect detection model according to any one of the above when the computer program is executed.

A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the method for training an image defect detection model according to any one of the above.

According to the training method, the training device, the computer equipment and the storage medium of the image defect detection model, a plurality of label-free generated images belonging to the target domain can be generated through a first generator in a trained feature enhancement model, wherein the feature enhancement model is obtained by pre-training based on the source domain sample image and the noise data and then retraining based on the target domain sample image and the noise data. Thus, the generated images with greatly enhanced characteristics can be obtained, and then the generated images are clustered to obtain the categories corresponding to the generated images respectively; respectively constructing class confrontation models respectively corresponding to each class, and training the class confrontation models corresponding to the corresponding classes based on the generated image of the same class to obtain the trained class confrontation models respectively corresponding to the classes; the method comprises the steps of obtaining random variable data, inputting the noise data and the random variable data into a second generator of each trained class countermeasure model in a superposition mode, obtaining class characteristic images corresponding to all classes respectively, further increasing different classes of images used for defect detection, greatly relieving the condition of insufficient sample amount, avoiding the over-fitting condition, building a defect detection model, training the defect detection model based on a plurality of class characteristic images, obtaining the trained defect detection model, and improving the generalization capability of the defect detection model.

Drawings

FIG. 1 is a diagram illustrating an exemplary embodiment of a training method for an image defect inspection model;

FIG. 2 is a schematic flowchart illustrating a method for training an image defect detection model according to an embodiment;

FIG. 3 is a schematic flow chart diagram illustrating the pre-training step of the feature enhancement model in one embodiment;

FIG. 4 is a schematic flow chart diagram illustrating the retraining step of the feature enhancement model in one embodiment;

FIG. 5 is a schematic flow chart diagram illustrating the steps of generating a feature enhancement model in one embodiment;

FIG. 6 is a flowchart illustrating the steps of obtaining trained confrontation models in one embodiment;

FIG. 7 is a flowchart illustrating the process steps of generating the countermeasure model for each category corresponding to the category in one embodiment;

FIG. 8 is a flowchart illustrating a method for training an image defect detection model according to another embodiment;

FIG. 9 is a block diagram showing an exemplary embodiment of an apparatus for training an image defect detection model;

FIG. 10 is a diagram showing an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The training method of the image defect detection model provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 and the server 104 may be used alone or in combination to perform the training method of the image defect detection model. Taking the training method in which the server executes the image defect detection model alone as an example, the server 104 obtains the first generator in the trained feature enhancement model; after the feature enhancement model is pre-trained based on the source domain sample image and the noise data, the server 104 is retrained based on the target domain sample image and the noise data to obtain the feature enhancement model; the server 104 inputs the noise data to the first generator, resulting in a plurality of unlabeled generated images; the server 104 performs clustering processing on the generated images to obtain categories corresponding to the generated images respectively; the server 104 respectively constructs a class countermeasure model corresponding to each class, and trains the class countermeasure models corresponding to the corresponding classes based on the generated image of the same class to obtain trained class countermeasure models corresponding to the classes; the server 104 acquires random variable data, and superimposes and inputs the noise data and the random variable data to a second generator in each trained class countermeasure model to obtain class characteristic images respectively corresponding to each class; the server 104 constructs a defect detection model, and trains the defect detection model based on the plurality of class feature images to obtain a trained defect detection model, wherein the trained defect detection model is used for detecting defects of the to-be-detected image of the target domain. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.

In an embodiment, as shown in fig. 2, a training method of an image defect detection model is provided, which is described by taking the method as an example applied to a computer device, which may be specifically the terminal or the server in fig. 1, where the training of the image defect detection model includes the following steps:

step S202, a first generator in the trained feature enhancement model is obtained; the feature enhancement model is obtained by pre-training based on source domain sample images and noise data and then re-training based on target domain sample images and noise data.

Wherein the source domain is the knowledge that has been learned, and the target domain is the new knowledge to be learned. The noise data is noise data carrying a plurality of task labels, for example, the noise data carries data of task labels such as incomplete etching, residual copper, needle prick, small burr on the line, small thread, small notch on the line, and the like.

Specifically, the computer device builds a feature enhancement model based on the generated countermeasure network, pre-trains and retrains the feature enhancement model to obtain a trained feature enhancement model, and obtains a first generator in the trained feature enhancement model based on the trained feature enhancement model. The feature enhancement model comprises a first generator and a first discriminator, wherein the first generator is obtained by pre-training based on source domain sample images and noise data and then retraining based on target domain sample images and noise images, the first generator can be a deep convolutional network, and details of a high-resolution picture generated by a countermeasure network can be improved.

For example, the computer device constructs a first generator and a first discriminator based on a confrontation training model, constructs a feature enhancement model based on the first generator and the first discriminator, trains the feature enhancement model based on source domain data of a display panel and target domain data of a circuit board to obtain the trained feature enhancement model, and obtains a first generator in the trained feature enhancement model based on the trained feature enhancement model, wherein the feature enhancement model comprises the first generator and the first discriminator, and the first generator is obtained by pre-training based on source domain sample images and noise data and then retraining based on target domain sample images and noise images.

Step S204 is to input the noise data to the first generator to obtain a plurality of label-free generated images.

Specifically, the computer device acquires a first generator in the trained feature enhancement model, inputs the noise data into the first generator, and performs feature generation to obtain a plurality of label-free generated images. For example, the computer device obtains a first generator in the trained feature enhancement model, where the first generator may be formed based on a deep convolutional network, and noise data with task labels such as pinprick, small burr on line, line young, and the like is subjected to feature generation through the deep convolutional network to obtain a plurality of label-free generated images.

Step S206, clustering processing is carried out on the generated images to obtain the categories corresponding to the generated images respectively.

The clustering process is a process of learning characteristic representation of data and clustering the data based on the characteristic representation, namely, the data are divided into a plurality of classes according to the similarity degree of each object in the data according to a certain evaluation criterion, the objects belonging to the same cluster have high relevance and are similar to each other, and the objects in different clusters have low relevance and are different from each other. Clustering may be achieved using SOM (Self Organizing mapping Maps) algorithm, K-means clustering algorithm, ROCK algorithm, and the like.

Specifically, the computer device acquires a plurality of label-free generated images, performs clustering processing on the plurality of label-free generated images, and obtains a category corresponding to each generated image by using the images with the same characteristics as one category. For example, the computer device obtains a plurality of label-free generated images, performs clustering processing by using a clustering algorithm of an SOM (Self Organizing mapping neural network), inputs the plurality of label-free generated images to an input layer of the SOM, and performs clustering by finding a node which is most matched with the generated images in a hidden layer of the SOM for each generated image, thereby obtaining a category in each input generated image.

And S208, respectively constructing class confrontation models respectively corresponding to each class, and training the class confrontation models corresponding to the corresponding classes based on the generated image of the same class to obtain the trained class confrontation models respectively corresponding to the classes.

The type countermeasure model is a deep learning model for generating countermeasure Networks (GAN), and is used for generating images.

Specifically, the computer device determines the number of categories based on the categories corresponding to the generated images, respectively constructs category confrontation models corresponding to each category based on the number of the categories, and trains the category confrontation models of the corresponding categories based on the generated images of the same category to obtain the trained category confrontation models corresponding to the categories. The trained class countermeasure model can generate an image with high reality based on input noise data.

For example, the computer device constructs a residual copper class countermeasure model corresponding to the residual copper class, a burr class countermeasure model corresponding to the burr class, and a notch class countermeasure model corresponding to the notch class based on the classes, respectively, based on the classes, and inputs the generated image corresponding to the residual copper for training, to obtain a trained residual copper class countermeasure model corresponding to the residual copper class, which can generate an image with residual copper based on the input noise.

Step S210, obtaining random variable data, and inputting the noise data and the random variable data to the second generator in each trained class countermeasure model in a superimposed manner to obtain class feature images corresponding to each class.

Wherein, the random variable data is data related to noise, and the random variable data can be random any noise; the noise data is noise data carrying a plurality of task labels, for example, the noise data carries data of task labels such as incomplete etching, residual copper, needle prick, small burr on the line, small thread, small notch on the line, and the like. The second generator in each class countermeasure model employs a multi-layer convolutional network.

Specifically, the computer equipment acquires random variable data, superposes the noise data and the random variable data and inputs the superposed data and the superposed data to a second generator in each trained class countermeasure model, and the second generator performs convolution operation on images by using a multi-scale convolution kernel to generate class characteristic images respectively corresponding to all classes.

For example, the computer device obtains random variable data, obtains a trained residual copper class countermeasure model, a trained burr class countermeasure model, and a trained notch class countermeasure model, and for each class countermeasure model, superimposes and inputs the random variable data and noise data carrying task labels of pinprick, small burr on line, small line, and the like to a second generator in the trained class countermeasure model corresponding to the class, for example, superimposes and inputs the noise data carrying task labels of pinprick, small burr on line, small line, and the like to the second generator in the trained residual copper class countermeasure model, and obtains a residual copper feature image.

Step S212, a defect detection model is constructed, the defect detection model is trained based on the plurality of class characteristic images, a trained defect detection model is obtained, and the trained defect detection model is used for detecting the defects of the to-be-detected images of the target domain.

Specifically, the computer device builds a defect detection model based on the convolutional neural network, acquires a plurality of category feature images corresponding to each category, trains the defect detection model based on the plurality of category feature images to obtain a trained defect detection model, and the trained defect detection model is used for performing defect detection on an image to be detected in a target domain.

According to the training method of the image defect detection model, a first generator in a trained feature enhancement model can generate a plurality of label-free generated images belonging to a target domain, wherein the feature enhancement model is obtained by pre-training based on source domain sample images and noise data and then retraining based on the target domain sample images and the noise data. Thus, the generated images with greatly enhanced characteristics can be obtained, and then the generated images are clustered to obtain the categories corresponding to the generated images respectively; respectively constructing class confrontation models respectively corresponding to each class, and training the class confrontation models corresponding to the corresponding classes based on the generated image of the same class to obtain the trained class confrontation models respectively corresponding to the classes; the method comprises the steps of obtaining random variable data, inputting the noise data and the random variable data into a second generator of each trained class countermeasure model in a superposition mode, obtaining class characteristic images corresponding to all classes respectively, further increasing different classes of images used for defect detection, greatly relieving the condition of insufficient sample amount, avoiding the over-fitting condition, building a defect detection model, training the defect detection model based on a plurality of class characteristic images, obtaining the trained defect detection model, and improving the generalization capability of the defect detection model.

In one embodiment, as shown in fig. 3, the feature enhancement model includes a first generator and a first discriminator, and the pre-training step of the feature enhancement model includes:

step S302, a source domain sample image and noise data are acquired.

The noise data is noise data carrying a plurality of task labels, for example, the noise data carries data of task labels such as incomplete etching, residual copper, needle prick, small burr on the line, small notch on the line, and the like.

Specifically, a computer device obtains source domain sample images and noisy data carrying a multitask tag. The source domain sample image may be a knowledge domain sample image based on the fields of semiconductors, panel displays, circuit boards, and the like.

And step S304, performing feature extraction on the current source domain sample image to obtain a current source domain feature map.

Specifically, the computer device performs feature extraction based on the acquired source domain sample image to obtain a current source domain feature map. For example, the computer device cuts the region to be extracted in the current source domain sample image corresponding to the panel display to obtain a region image to be extracted, and the computer device performs feature extraction on the region image to be extracted to obtain a current source domain feature map.

And step S306, generating a current source domain training generation image through the first generator based on the noise data and the hidden features of the source domain samples obtained in the previous training iteration.

Hidden features are understood to be hidden variables, which are variables that are not observable and which affect the state of the system and the observable output.

Specifically, the computer device obtains noise data carrying a plurality of task labels and hidden features of a source domain sample obtained in previous training iteration, inputs the noise data and the hidden features obtained in the previous training iteration into the first generator, and generates a current source domain training generation image through the first generator. For example, the computer device obtains noise data carrying task labels such as incomplete etching, residual copper, needle pricks, small burrs on lines and the like, wherein the noise data may be matrix data, obtains source domain sample hidden features corresponding to panel displays obtained in previous training iterations, and superposes and inputs the noise data and the source domain sample hidden features to the first generator, and generates a current source domain training generation image corresponding to the panel displays through the first generator, wherein the first generator may be constructed based on a deep convolutional network.

Step S308, inputting the current source domain feature map and the current source domain training generated image into a first discriminator to obtain a first discrimination result and the current source domain sample hidden feature.

The activation function of the first discriminator can be lreul (g) or sigmoid (d), fractional step convolution is adopted, and the first discriminator can avoid the problem of gradient disappearance or gradient explosion by normalizing each batch of data to realize batch normalization.

Specifically, the computer device obtains a current source domain feature map, generates a map by current source domain training, and inputs the current source domain feature map and the current source domain training generation map into a first discriminator together to obtain a first discrimination result and a current source domain sample hidden feature, where the first discrimination result represents the capability of the first discriminator to generate an image, that is, the probability of whether the generated image is real or not is judged. For example, the computer device inputs the current source domain feature map and the current source domain training generation map into a first discriminator to perform fractional step convolution operation, so as to obtain a first discrimination result and the current source domain sample hidden feature, if the first discrimination result is 1, the first generator generated image is characterized to be the same as the real image, and if the first discriminator is 0, the first generator generated image is characterized to be different from the real image, so that the first generator needs to be trained.

And S310, adjusting model parameters of the feature enhancement model based on the first judgment result, entering next cycle iteration, taking the hidden features of the current source domain sample as the hidden features of the source domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the current source domain sample image for feature extraction, and continuing to execute the step of obtaining the current source domain feature map until a first training stop condition is reached, so as to obtain the pre-trained feature enhancement model.

Specifically, the computer device obtains the first determination result, when the first determination result is not 0, the computer device adjusts the model parameters of the feature enhancement model based on the first determination result, enters the next iteration, uses the hidden features of the current source domain sample as the hidden features of the source domain sample obtained in the previous training iteration corresponding to the next training iteration, returns to the current source domain sample image for feature extraction, and continues to execute the step of obtaining the current source domain feature image until reaching a first training stop condition, so as to obtain a pre-trained feature enhancement model, where the first training stop condition may be that the iteration number reaches a first predetermined iteration number, or that the iteration time reaches a first predetermined iteration time.

For example, the computer device obtains the first determination result, and when the first determination result is not 0, the computer device adjusts parameters of the first generator and the first determination device based on the first determination result, sets a first predetermined iteration number, performs next loop iteration, uses the hidden feature of the current source domain sample as the source domain sample feature obtained by the previous training iteration corresponding to the next training iteration, returns to the current source domain sample image for feature extraction, and continues to execute the step of obtaining the current source domain feature image until the iteration number reaches the first predetermined iteration number, thereby obtaining a pre-trained feature enhancement model.

In the present embodiment, a source domain sample image and noise data are acquired; performing feature extraction on the current source domain sample image to obtain a current source domain feature map; generating a current source domain training generation image through the first generator based on the noise data and the hidden features of the source domain samples obtained in the previous training iteration; inputting the current source domain feature map and the current source domain training generated image into a first discriminator to obtain a first discrimination result and a current source domain sample hidden feature; adjusting model parameters of the feature enhancement model based on the first discrimination result, entering next cycle iteration, taking the hidden features of the current source domain sample as the hidden features of the source domain sample obtained by the previous training iteration corresponding to the next training iteration, returning the current source domain sample image to perform feature extraction, continuously executing the step of obtaining the current source domain feature map until reaching a first training stop condition, and obtaining the pre-trained feature enhancement model.

In one embodiment, as shown in fig. 4, the retraining step of the feature enhancement model includes:

step S402, acquiring a target domain sample image and noise data.

Specifically, a computer device obtains a target domain sample image and noisy data carrying a multitask label. The target domain sample image may be a knowledge domain sample image to be learned based on the fields of semiconductors, panel displays, circuit boards, and the like.

And S404, performing feature extraction on the current target domain sample image to obtain a current target domain feature map.

Specifically, the computer device performs feature extraction based on the acquired target domain sample image to obtain a current target domain feature map. For example, the computer device cuts the region to be extracted in the current target domain sample image corresponding to the circuit board to obtain a region image to be extracted, and the computer device performs feature extraction on the region image to be extracted to obtain a current target domain feature map.

Step S406, generating a current target domain training generation image through a first generator obtained by pre-training based on the noise data and the target domain sample hidden features obtained by the previous training iteration.

Specifically, the computer device obtains noise data carrying a plurality of task labels and hidden features of a target domain sample obtained by previous training iteration, the hidden features of the target domain sample are that a first discriminator in previous iteration training learns and distinguishes a source domain, a proper feature block and an improper synthesized feature block in a target domain, the domain invariant features of the previous training iteration are determined, the domain invariant features are the hidden features of the target domain sample obtained by the previous training iteration, the computer device inputs the noise data and the hidden features obtained by the previous training iteration into the first generator, the first generator generates a current target domain training generated image, and then tiny features can be further enhanced.

For example, the computer device obtains noise data carrying task labels such as incomplete etching, residual copper, pinpoint, small burr on line and the like, the noise data can be matrix data, and obtains a target domain sample hidden feature corresponding to a circuit board obtained by previous training iteration, the target domain sample hidden feature is a synthesized feature block which is obtained by a first discriminator in previous iteration training and is suitable for distinguishing a source domain from a target domain, and an unsuitable synthetic feature block, and determines a domain invariant feature of the previous training iteration, the computer device searches a cross-domain hidden feature space through data mapping and migrates a migration learning domain invariant feature based on migration, the domain invariant feature is the target domain sample hidden feature obtained by the previous training iteration, the computer device inputs the noise data and the hidden feature obtained by the previous training iteration to the first generator, and superposes the noise data and the target domain sample hidden feature and inputs the noise data to the first generator, and generating a current target domain training generation image corresponding to the circuit board through the first generator.

Step S408, inputting the current target domain feature map and the current target domain training generated image into a first discriminator obtained by pre-training to obtain a second discrimination result and the current target domain sample hidden feature.

The activation function of the first discriminator can be LReLU or Sigmoid, fractional step convolution is adopted, batch normalization is realized by normalizing each batch of data by the first discriminator, and the problems of gradient disappearance or gradient explosion can be avoided.

The computer equipment acquires a current target domain feature map, generates a map by current target domain training, and inputs the current target domain feature map and the current target domain training generated map into a first discriminator together to obtain a first discrimination result and a current target domain sample hidden feature, wherein the current target domain sample feature is an updated domain invariant feature, namely the current target domain sample hidden feature is obtained by updating the current target domain sample feature, and the first discrimination result represents the capability of the first discriminator for generating an image, namely the probability of judging whether the generated image is real or not. For example, the computer device inputs the current target domain feature map and the current target domain training generation map into a first discriminator to perform fractional step convolution operation, so as to obtain a first discrimination result and a current target domain sample hidden feature, if the first discrimination result is 1, the first generator generated image is represented to be the same as the real image, and if the first discriminator is 0, the first generator generated image is represented to be different from the real image, so that the first generator needs to be trained.

And S410, adjusting model parameters of the feature enhancement model obtained by pre-training based on the second judgment result, entering next cycle iteration, taking the hidden features of the current target domain sample as the hidden features of the target domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the current target domain sample image for feature extraction, and continuing to execute the step of obtaining the current target domain feature map until a second training stop condition is reached to obtain the trained feature enhancement model.

Specifically, the computer device obtains the first determination result, when the first determination result is not 0, the computer device adjusts the model parameters of the feature enhancement model obtained by the pre-training based on the first determination result, enters the next iteration, uses the hidden features of the current target domain sample as the hidden features of the target domain sample obtained by the previous training iteration corresponding to the next training iteration, returns to the current target domain sample image for feature extraction, and continues to execute the step of obtaining the current target domain feature image until reaching a second training stop condition, so as to obtain the trained feature enhancement model, where the second training stop condition may be that the iteration number reaches a second predetermined iteration number, or that the iteration time reaches a second predetermined iteration time.

For example, the computer device obtains the first determination result, and when the first determination result is not 0, the computer device adjusts parameters of the first generator and the first determination device based on the first determination result, sets a second predetermined iteration number, performs the next loop iteration, uses the hidden feature of the current target domain sample as the target domain sample feature obtained by the previous training iteration corresponding to the next training iteration, returns to the current target domain sample image for feature extraction, and continues to execute the step of obtaining the current target domain feature image until the iteration number reaches the second predetermined iteration number, thereby obtaining a trained feature enhancement model.

In the embodiment, the target domain sample image and the noise data are acquired, so that the generated features can be made to be more noise-resistant through the added noise data, and the feature extraction is performed on the current target domain sample image to obtain the current target domain feature map; generating a current target domain training generation image through a first generator obtained by pre-training based on the noise data and the target domain sample hidden features obtained by the previous training iteration; inputting the current target domain feature map and the current target domain training generated image into a first discriminator obtained by pre-training to obtain a second discrimination result and a current target domain sample hidden feature; and adjusting model parameters of the feature enhancement model obtained by pre-training based on the second judgment result, entering next loop iteration, taking the hidden features of the current target domain sample as the hidden features of the target domain sample obtained by the previous training iteration corresponding to the next training iteration, returning the current target domain sample image to perform feature extraction, continuously executing the step of obtaining the current target domain feature map until reaching a second training stop condition, and obtaining the trained feature enhancement model.

To facilitate a clearer understanding of the generation of the feature enhancement model, a more detailed embodiment is provided as shown in FIG. 5. The computer equipment obtains a target domain sample image and noise data carrying a plurality of task labels, the noise data is input into a first generator of a feature enhancement model constructed by a deep convolutional network to obtain a target domain training generated image, the computer equipment performs feature extraction on the target domain sample image to obtain a target feature map, the target feature map and the target domain training generated image are input into a first discriminator in the feature enhancement model, the first discriminator learns and distinguishes a source domain, a proper feature block and an improper synthesized feature block in the target domain to determine a domain invariant feature, the domain invariant feature is used as a target domain sample implicit feature to obtain a second discrimination result, the computer equipment inputs the target domain sample implicit feature and the noise data into the first generator to start the next iteration, and the target domain sample implicit feature is continuously updated through the iteration, and stopping until a second training stopping condition is reached to obtain the trained feature enhancement model.

In the embodiment, noise data carrying a plurality of task labels are processed by a first generator of a deep convolutional network to generate a target domain training generated image with finer features, based on the target domain training generated image and a target feature map obtained by extracting features of a target domain sample image, a domain invariant feature is determined by a first discriminator, the domain invariant feature is fed back to the first generator as a target domain sample hidden feature through transfer learning, and a trained feature enhancement model is finally obtained through iterative training, so that the finer features can be generated, and defect detection of a micron-scale image can be realized.

Specifically, the computer device obtains a current target domain image and determines a target area, the computer device cuts and extracts the target area in the current target domain sample image to obtain a target domain area image, the computer device extracts features of the target domain area image, converts high-dimensional pixel features into low-dimensional features, and obtains a current target domain feature map.

In this embodiment, a target area in the current target area sample image is extracted to obtain a target area image; the feature extraction is performed on the target area image to obtain a current target domain feature map, and thus, the current target domain feature map is input to the first discriminator to obtain more representative features.

In one embodiment, as shown in fig. 6, the class confrontation model includes a second generator and a second discriminator, the training of the class confrontation model with the corresponding class based on the generated image of the same class, and obtaining the trained class confrontation models respectively corresponding to the classes, including:

step S602, acquiring generated images of the same category, and performing feature extraction on the current generated image to obtain a current feature extraction map.

Specifically, the computer device acquires the category of each generated image obtained through the clustering process, acquires the generated image of the same category from a plurality of generated images, and performs feature extraction on the current generated image to obtain a current feature extraction map. For example, the computer device acquires a generated image of the type of the burr from a plurality of generated images, and performs feature extraction on each generated image of the type of the burr to obtain a feature extraction map of the current time.

Step S604, scene sample noise data and random variable data are obtained, and a current training class feature image is generated through the second generator based on the scene noise data, the random variable data and the semi-supervised sample obtained in the previous training iteration.

The scene sample noise data is noise data which is related to a scene and carries a plurality of task labels, the random variable data is data related to noise, the random variable data can be any random noise, and the semi-supervised sample is a hidden code c. Referring to the structure framework of the Info GAN, and considering that the implicit coding c of the Info GAN model and the image features have correlation, the model can be controlled to generate different types of pictures by controlling the difference of the implicit coding c, the second generator is a multilayer convolution network, a transposed convolution operation (G) and a convolution operation (D) are adopted in each layer of the convolution network, the activation functions adopt relu (G) and LeakyRelu (D), and the generators adopt the convolution operations in the output layer and tan (G) and Softmax (D) as the activation functions.

Specifically, the computer device obtains scene sample noise data and random variable data which are related to a scene and carry a plurality of task labels, obtains a semi-supervised sample obtained in a previous training iteration, and performs convolution operation through a multilayer convolution network of a second generator based on the scene noise data, the random variable data and the semi-supervised sample obtained in the previous training iteration to generate a training class feature image of the current time.

For example, the computer device obtains scene sample noise data and random variable data of a circuit board in a scene, and obtains a semi-supervised sample obtained in a previous training iteration, the semi-supervised sample is a semi-supervised sample obtained in a previous iteration training, a second discriminator in the previous iteration training performs impedance type domain adaptation learning, a domain discriminator is used for judging the difference between a target sample and a source domain sample, the computer device performs transfer learning according to the correlation and task covariance between cross-domain tasks to obtain the semi-supervised sample, and the computer device performs convolution operation through a multilayer convolution network of a second generator based on the scene noise data, the random variable data and the semi-supervised sample obtained in the previous training iteration to generate a training class feature image of the current time. The second discriminator is composed of multilayer convolution and multilayer full connection, the activation function adopts LReLU, and the second generator activation function is ReLU, so that the overfitting phenomenon can be effectively avoided.

Step S606, inputting the current feature extraction image and the current training class feature image into a second decision device to obtain a third decision result and a current semi-supervised sample.

The second discriminator is formed by multilayer convolution and multilayer full connection, the activation function adopts LReLU and Sigmoid, fractional stepping convolution is adopted, batch normalization is realized by normalizing each batch of data by the second discriminator, and the problem of gradient disappearance or gradient explosion can be avoided.

Specifically, the computer device obtains a current feature extraction diagram and a current training category feature image, and inputs the current feature extraction diagram and the current training category feature image into the second discriminator together to obtain a third discrimination result and a current semi-supervised sample, wherein the current semi-supervised sample is a semi-supervised sample obtained by updated current training iteration, and the third discrimination result represents the capability of the second generator to generate an image, namely, the probability that the generated image is close to a real image is judged. For example, the computer device inputs the feature extraction graph of the current time and the training class feature image of the current time into the second discriminator together, and obtains the second discrimination result and the semi-supervised sample of the current time through fractional stepping convolution operation based on the activation functions of lreul and Sigmoid.

And step S608, adjusting model parameters of the class countermeasure model based on the third judgment result, entering the next cycle iteration, taking the current semi-supervised sample as the semi-supervised sample obtained by the previous training iteration corresponding to the next training iteration, returning the current generated image to perform feature extraction, continuing to perform the step of obtaining the current feature extraction image until a third training stop condition is reached, and obtaining the class countermeasure model corresponding to the corresponding class after training.

Specifically, the computer device adjusts model parameters of the class of countermeasure models based on the third determination result, performs next loop iteration, uses the current semi-supervised sample as the semi-supervised sample obtained by the current training iteration corresponding to the next training iteration, returns the current generated image to perform feature extraction, continues to perform the step of obtaining the current feature extraction image until reaching a third training stop condition, obtains class countermeasure models corresponding to the corresponding classes after training, integrates the class countermeasure models to obtain a classification decision model associated with multiple classes, and the third training stop condition may be that the iteration number reaches a third predetermined iteration number, or that the iteration time reaches a third predetermined iteration time.

For example, the computer device obtains a third determination result, adjusts the model parameters of the class countermeasure model based on the third determination result, performs the next cycle iteration, uses the current semi-supervised sample as the semi-supervised sample obtained by the current training iteration corresponding to the next training iteration, returns the current generated image to perform the feature extraction, continues the step of obtaining the current feature extraction image until the iteration times reach a third preset iteration time, obtains the class countermeasure model corresponding to the corresponding class after training, if the class of each generated image is determined to be residual copper, burr and notch through clustering, obtains the trained residual copper class countermeasure model, the trained burr class countermeasure model and the trained notch class countermeasure model, and integrates the three class countermeasure models, and obtaining a classification decision model of 3 categories.

In this embodiment, generated images of the same category are acquired, and feature extraction is performed on the generated image of the current time to obtain a feature extraction map of the current time; scene sample noise data and random variable data are obtained, and a training class characteristic image of the current time is generated through the second generator on the basis of the scene noise data, the random variable data and a semi-supervised sample obtained in the previous training iteration; inputting the current feature extraction image and the current training class feature image into a second discriminator to obtain a third discrimination result and a current semi-supervised sample; and adjusting model parameters of the class countermeasure model based on the third judgment result, entering next cycle iteration, taking the current semi-supervised sample as the semi-supervised sample obtained by the previous training iteration corresponding to the next training iteration, returning the current generated image to perform feature extraction, continuously executing the step of obtaining the current feature extraction image until a third training stop condition is reached, and obtaining the trained class countermeasure model corresponding to the corresponding class.

In order to facilitate a clearer understanding of the generation process of the class countermeasure model corresponding to each class, taking the generation process of the class countermeasure model of class a as an example, as shown in fig. 7, a more detailed embodiment is provided for description. The computer equipment acquires an A-class generated image, scene sample noise data carrying a plurality of task labels and random variable data, inputs the scene sample noise data and the random variable data into a second generator of an A-class countermeasure model to obtain an A-class training class characteristic image, performs characteristic extraction on the A-class generated image to obtain an A-class characteristic extraction image, inputs the A-class characteristic extraction image and the A-class training class characteristic image into a second discriminator of the A-class countermeasure model to obtain a third discrimination result, performs fitting learning of a countermeasure domain by the second discriminator, judges the difference between a target sample and a source domain sample by a domain discriminator, and performs transfer learning according to the correlation and task covariance between cross-domain tasks to obtain a semi-supervised sample, and the computer equipment inputs the semi-supervised sample, the scene sample noise data and the random variable data into a second generator of the class countermeasure model of the class A to start the next iteration, continuously updates the semi-supervised sample through the iteration until a third training stop condition is reached, and stops to obtain the trained class countermeasure model of the class A.

In this embodiment, scene sample noise data carrying a plurality of task labels, random variable data, and a semi-supervised sample are input to a second generator in a class countermeasure model to obtain a training class feature image corresponding to a class, a feature extraction map is obtained based on a generated image corresponding to the class countermeasure model, the semi-supervised sample is determined based on the feature extraction map and the training class feature image, the semi-supervised sample is fed back to the second generator, and a trained class countermeasure model corresponding to the class is obtained through iterative training, so that a plurality of classes of class countermeasure models can be obtained, further, images of different classes for defect detection can be further increased, and the generalization capability of the defect detection model is improved.

In one embodiment, the obtaining scene sample noise data comprises:

Specifically, the computer device acquires a plurality of feature generation images, performs scene prejudgment on the plurality of feature generation images, determines a target scene corresponding to each feature generation image, and acquires scene sample noise data which is corresponding to the target scene and carries a plurality of task labels based on the target scene. For example, the computer device obtains a plurality of feature generation images, performs scene pre-judgment on the plurality of feature generation images based on a scene pre-judgment model, and determines a target scene corresponding to distribution of each feature generation image, for example, when the target scene is a circuit board scene, the computer device obtains circuit board sample noise carrying a plurality of task noises in the circuit board scene.

In the implementation, scene prejudgment is carried out on a plurality of characteristic generation images, and a target scene corresponding to each characteristic generation image is determined; scene sample noise data corresponding to the target scene is obtained, so that the noise data corresponding to the target scene can be rapidly determined, the precision of the category countermeasure model can be improved, and the detection efficiency and precision of the defect detection model can be improved.

In one embodiment, the inputting the current feature extraction graph and the current training class feature image into the second discriminator to obtain a third discrimination result and a current semi-supervised sample, includes:

Specifically, the computer device obtains a current feature extraction image and a current training class feature image, inputs the current feature extraction image and the current training class feature image into the second discriminator, obtains a third discrimination result and an intermediate semi-supervised sample, determines a preset threshold, for example, selects the preset threshold to be 0.75-0.85, extracts the semi-supervised sample according to the preset threshold, and takes the intermediate semi-supervised sample as the current semi-supervised sample when the intermediate semi-supervised sample meets a preset threshold range. The too high preset threshold easily causes overfitting after training of samples generated by the generator, the generalization capability is weak, and the too low preset threshold has little influence on the accuracy rate and causes the reduction of the calculation speed.

In this embodiment, the current feature extraction image and the current training class feature image are input into a second discriminator to obtain a third discrimination result and a middle semi-supervised sample; when the middle semi-supervised sample meets the preset threshold range, the middle semi-supervised sample is used as the current semi-supervised sample, so that the class confrontation model can be trained based on the more suitable semi-supervised sample, and the trained class confrontation model can be helpful for improving the generalization capability of the defect detection model.

To facilitate a clearer understanding of the training of the image defect detection model, a more detailed embodiment is provided as shown in fig. 8. Firstly, pre-training is carried out based on source domain sample images and noise data to obtain a trained pre-training feature enhancement model, then re-training is carried out based on target domain sample images and noise data, a trained feature enhancement model is obtained through transfer learning (namely knowledge transfer), then the noise data carrying a plurality of task labels are input into a first generator in the feature enhancement model to obtain a plurality of label-free generated images with enhanced features, clustering processing is carried out on the generated images to obtain categories respectively corresponding to the generated images, category countermeasure models respectively corresponding to each category are respectively constructed to obtain multi-category countermeasure models, the generated images of the same category are obtained for the category countermeasure models of each category, scene pre-judgment is carried out on the generated images to determine a target scene, and scene sample noise data is determined based on the target scene, the computer equipment inputs scene sample noise data (namely, a multi-task source) and random variable data carrying a plurality of task labels into a second generator in the class countermeasure model for training, obtains the trained class countermeasure models respectively corresponding to the classes through migration learning (multi-task recognition migration) of the classes, and integrates the class countermeasure models to obtain multi-class classification decision models, namely, multi-class countermeasure models. And feeding the generated semi-supervised samples back to the second generator in the training process of each class confrontation model so as to adjust the generated image characteristics. The computer equipment acquires random variable data, inputs the noise data and the random variable data into a second generator in each trained class countermeasure model, and obtains class characteristic images respectively corresponding to the classes; and constructing a defect detection model, and training the defect detection model by the computer equipment based on the plurality of class characteristic images to obtain the trained defect detection model. In addition, the detection process is guided based on the loss function, so that the result of the defect detection model is more accurate.

In this embodiment, features of a generated image can be greatly enhanced by a feature enhancement model, so that extraction of features with micron-scale defect separability and high migration adaptability is facilitated, class confrontation models corresponding to classes are trained on the basis of the feature enhanced generated image to obtain trained class confrontation models, different classes of images for defect detection can be further added on the basis of the trained class confrontation models, so that overfitting can be avoided, wherein scene sample noise data carrying a plurality of task labels is used as an input variable of a second generator, damage to overall information can be reduced, namely, a defect detection model is constructed after generation confrontation network and migration learning through two levels of the feature enhancement model for feature extraction and the class confrontation models for the classes, the multi-level and multi-task defect detection model is formed by connecting two levels of a feature enhancement model for feature extraction and a multi-class countermeasure model for each class and simultaneously by scene sample noise data (namely, a multi-task source) and transfer learning (multi-task recognition transfer) of each class. The defect detection model is trained based on the plurality of class characteristic images to obtain the trained defect detection model, so that a multi-class (multi-task) shared low-rank space can be formed, the generalization capability of the defect detection model is improved, and therefore, the defect detection is performed on the image to be detected in the target domain based on the trained defect detection model, and the accuracy and the recall rate of micron-scale defect detection can be greatly improved.

It should be understood that although the steps in the flowcharts of fig. 2 to 4 and 6 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 to 4 and 6 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternatively with other steps or at least some of the other steps or stages.

In one embodiment, as shown in fig. 9, there is provided a training apparatus for an image defect detection model, including: an obtaining module 902, a first generating module 904, a clustering module 906, a first training module 908, a second generating module 910, and a second training module 912, wherein:

an obtaining module 902, configured to obtain a first generator in the trained feature enhancement model; the feature enhancement model is obtained by pre-training based on source domain sample images and noise data and then re-training based on target domain sample images and noise data.

A first generation module 904 for inputting the noise data to the first generator resulting in a plurality of label-free generated images.

And a clustering module 906, configured to perform clustering processing on the generated images to obtain categories corresponding to the generated images respectively.

The first training module 908 is configured to respectively construct class confrontation models corresponding to each class, and train the class confrontation models corresponding to the corresponding classes based on the generated image of the same class, so as to obtain trained class confrontation models corresponding to the classes.

The second generating module 910 is configured to obtain random variable data, and superimpose and input the noise data and the random variable data to the second generator in each trained class countermeasure model to obtain class feature images respectively corresponding to the classes.

And a second training 912, configured to construct a defect detection model, and train the defect detection model based on the multiple category feature images to obtain a trained defect detection model, where the trained defect detection model is used to perform defect detection on an image to be detected in the target domain.

In one embodiment, an acquisition module 902 for acquiring source domain sample images and noise data; performing feature extraction on the current source domain sample image to obtain a current source domain feature map; generating a current source domain training generation image through the first generator based on the noise data and the hidden features of the source domain samples obtained in the previous training iteration; inputting the current source domain feature map and the current source domain training generated image into a first discriminator to obtain a first discrimination result and a current source domain sample hidden feature; and adjusting the model parameters of the feature enhancement model based on the first judgment result, entering next cycle iteration, taking the hidden features of the current source domain sample as the hidden features of the source domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the current source domain sample image for feature extraction, and continuing to execute the step of obtaining the current source domain feature map until a first training stop condition is reached, so as to obtain the pre-trained feature enhancement model.

In one embodiment, an acquisition module 902 for acquiring a target domain sample image and noise data; performing feature extraction on the current target domain sample image to obtain a current target domain feature map; generating a current target domain training generation image through a first generator obtained by pre-training based on the noise data and the target domain sample hidden features obtained by the previous training iteration; inputting the current target domain feature map and the current target domain training generated image into a first discriminator obtained by pre-training to obtain a second discrimination result and a current target domain sample hidden feature; and adjusting model parameters of the feature enhancement model obtained by pre-training based on the second judgment result, entering next cycle iteration, taking the hidden features of the current target domain sample as the hidden features of the target domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the current target domain sample image for feature extraction, and continuing to execute the step of obtaining the current target domain feature map until a second training stop condition is reached, so as to obtain the trained feature enhancement model.

In an embodiment, the obtaining module 902 is configured to extract a target area in a current target area sample image to obtain a target area image; and performing feature extraction on the target area image to obtain a current target area feature map.

In one embodiment, the first training module 908 is configured to obtain generated images of the same category, and perform feature extraction on the current generated image to obtain a current feature extraction map; scene sample noise data and random variable data are obtained, and a training class characteristic image of the current time is generated through the second generator on the basis of the scene noise data, the random variable data and a semi-supervised sample obtained in the previous training iteration; inputting the current feature extraction image and the current training class feature image into a second discriminator to obtain a third discrimination result and a current semi-supervised sample; and adjusting model parameters of the class countermeasure model based on the third judgment result, entering next cycle iteration, taking the current semi-supervised sample as the semi-supervised sample obtained by the previous training iteration corresponding to the next training iteration, returning the current generated image to perform feature extraction, continuously executing the step of obtaining the current feature extraction image until a third training stop condition is reached, and obtaining the trained class countermeasure model corresponding to the corresponding class.

In one embodiment, the first training module 908 is configured to perform scene prejudging on a plurality of feature generating images, and determine a target scene corresponding to each feature generating image; scene sample noise data corresponding to the target scene is acquired.

In one embodiment, the second training module 908 is configured to input the current feature extraction graph and the current training class feature image into a second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample; and when the intermediate semi-supervised sample meets the preset threshold range, taking the intermediate semi-supervised sample as the current semi-supervised sample.

For specific definition of the training apparatus for the image defect detection model, reference may be made to the above definition of the training method for the image defect detection model, and details are not described here. All or part of the modules in the training device of the image defect detection model can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing training data of the image defect detection model. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of training an image defect detection model.

Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the above-described method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A training method of an image defect detection model is characterized by comprising the following steps:

acquiring a first generator in the trained feature enhancement model; the feature enhancement model is obtained by pre-training based on source domain sample images and noise data and then re-training based on target domain sample images and noise data;

inputting the noise data to the first generator to obtain a plurality of label-free generated images;

clustering the generated images to obtain the categories corresponding to the generated images respectively;

respectively constructing class confrontation models respectively corresponding to each class, and training the class confrontation models corresponding to the corresponding classes based on the generated image of the same class to obtain the trained class confrontation models respectively corresponding to the classes;

acquiring random variable data, and superposing and inputting the noise data and the random variable data to a second generator in each trained class countermeasure model to obtain class characteristic images respectively corresponding to the classes;

and constructing a defect detection model, and training the defect detection model based on a plurality of class characteristic images to obtain a trained defect detection model, wherein the trained defect detection model is used for detecting the defects of the to-be-detected image of the target domain.

2. The method of claim 1, wherein the feature enhancement model comprises a first generator and a first discriminator, and wherein the pre-training of the feature enhancement model comprises:

acquiring source domain sample images and noise data;

performing feature extraction on the current source domain sample image to obtain a current source domain feature map;

generating a current source domain training generation image through the first generator based on the noise data and the source domain sample hidden features obtained by the previous training iteration;

inputting the current source domain feature map and the current source domain training generated image into a first discriminator to obtain a first discrimination result and a current source domain sample hidden feature;

and adjusting the model parameters of the feature enhancement model based on the first judgment result, entering next cycle iteration, taking the hidden features of the current source domain sample as the hidden features of the source domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the step of performing feature extraction on the current source domain sample image, continuously executing the step of obtaining the current source domain feature map, and stopping until a first training stop condition is reached to obtain the pre-trained feature enhancement model.

3. The method of claim 2, wherein the retraining of the feature-enhanced model comprises:

acquiring a target domain sample image and noise data;

performing feature extraction on the current target domain sample image to obtain a current target domain feature map;

generating a current target domain training generation image through a first generator obtained by pre-training based on the noise data and the target domain sample hidden features obtained by the previous training iteration;

inputting the current target domain feature map and the current target domain training generated image into a first discriminator obtained by pre-training to obtain a second discrimination result and a current target domain sample hidden feature;

and adjusting model parameters of the feature enhancement model obtained by pre-training based on the second judgment result, entering next cycle iteration, taking the hidden features of the current target domain sample as the hidden features of the target domain sample obtained by the previous training iteration corresponding to the next training iteration, returning to the step of performing feature extraction on the current target domain sample image, and continuing to execute the step of obtaining the current target domain feature map until a second training stop condition is reached, so as to obtain the trained feature enhancement model.

4. The method according to claim 3, wherein the performing feature extraction on the current target domain sample image to obtain a current target domain feature map comprises:

extracting a target area in the current target area sample image to obtain a target area image;

and performing feature extraction on the target area image to obtain a current target area feature map.

5. The method according to claim 1, wherein the class confrontation model comprises a second generator and a second discriminator, and the training of the class confrontation model corresponding to the corresponding class based on the generated image of the same class to obtain the trained class confrontation model corresponding to each class comprises:

acquiring generated images of the same category, and performing feature extraction on the current generated image to obtain a current feature extraction image;

scene sample noise data and random variable data are obtained, and a training class characteristic image of the current time is generated through the second generator on the basis of the scene noise data, the random variable data and a semi-supervised sample obtained in the previous training iteration;

inputting the current feature extraction image and the current training class feature image into a second discriminator to obtain a third discrimination result and a current semi-supervised sample;

and adjusting the model parameters of the class countermeasure model based on the third judgment result, entering the next cycle iteration, taking the current semi-supervised sample as the semi-supervised sample obtained by the previous training iteration corresponding to the next training iteration, returning to the step of performing feature extraction on the current generated image, continuously executing the step of obtaining the current feature extraction image until a third training stop condition is reached, and obtaining the trained class countermeasure model corresponding to the corresponding class.

6. The method of claim 5, wherein said obtaining scene sample noise data comprises:

performing scene prejudgment on a plurality of feature generation images, and determining a target scene corresponding to each feature generation image;

scene sample noise data corresponding to the target scene is obtained.

7. The method according to claim 5, wherein the inputting the current feature extraction map and the current training class feature image into a second discriminator to obtain a third discrimination result and a current semi-supervised sample comprises:

inputting the current feature extraction image and the current training class feature image into a second discriminator to obtain a third discrimination result and an intermediate semi-supervised sample;

and when the intermediate semi-supervised sample meets a preset threshold range, taking the intermediate semi-supervised sample as a current semi-supervised sample.

8. An apparatus for training an image defect inspection model, the apparatus comprising:

a first generation module for inputting the noise data to the first generator to obtain a plurality of label-free generated images;

and the second training module is used for constructing a defect detection model, training the defect detection model based on the plurality of class characteristic images to obtain a trained defect detection model, and the trained defect detection model is used for detecting the defects of the to-be-detected image of the target domain.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.