CN113284150A - Industrial quality inspection method and industrial quality inspection device based on unpaired industrial data - Google Patents
Industrial quality inspection method and industrial quality inspection device based on unpaired industrial data Download PDFInfo
- Publication number
- CN113284150A CN113284150A CN202110841393.9A CN202110841393A CN113284150A CN 113284150 A CN113284150 A CN 113284150A CN 202110841393 A CN202110841393 A CN 202110841393A CN 113284150 A CN113284150 A CN 113284150A
- Authority
- CN
- China
- Prior art keywords
- image
- main features
- source domain
- generated image
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides an industrial quality inspection method and an industrial quality inspection device based on unpaired industrial data, wherein the method comprises the following steps: inputting the source domain image into a generation countermeasure network to obtain a generated image, and judging whether the generated image is true or false; extracting main features of the source domain image and the generated image based on a VGG network and an SVD technology, and judging whether the main features of the source domain image and the generated image are approximately consistent; if not, optimizing the generation countermeasure network until the main characteristics of the two images are approximately consistent; and inputting the unpaired image to be detected into the optimized generation countermeasure network to generate a corresponding generated image and carry out industrial quality inspection. The method can realize real-time cross-domain conversion of the image, well balance information of a target domain and a source domain by adopting an SVD (singular value decomposition) technology, and has good real-time performance and visual effect based on generation of a countermeasure network, thereby efficiently carrying out large-scale industrial data generation tasks.
Description
Technical Field
The invention relates to the technical field of industrial quality inspection, in particular to an industrial quality inspection method based on unpaired industrial data and an industrial quality inspection device based on unpaired industrial data.
Background
In industrial quality inspection, quality inspection is mostly performed based on an image labeling model, however, in a real industrial scene, a large amount of data used for labeling model training needs expensive labeling cost, and a small amount of data is difficult to obtain, so that an image generation technology is widely applied in the field of industrial quality inspection.
At present, the image generation technology generally uses paired data sets to perform supervised training on a conditional deep neural network model or a simple regression model, however, the method is impractical in many application program scenes because paired data sets cannot be obtained in some scenes, for example, a photo of a real scene corresponding to starry sky drawn by Sanskrit cannot be obtained forever, and thus an unpaired image generation technology is derived. Unpaired image generation is an extremely important branch in the field of image generation, cross-domain synthesis of images can be realized under the condition of no paired data sets, and the unpaired image generation technology has wider application scenes compared with paired image generation technology.
In the related technology, the unpaired image generation technology mostly realizes cross-domain conversion of images in a coding hidden space mode, but the images generated by the method are not real enough and lack details.
Disclosure of Invention
In order to solve the above technical problems, a first object of the present invention is to provide an industrial quality inspection method based on unpaired industrial data, which can implement real-time cross-domain conversion of images, has a wider application range, can well balance information of a target domain and a source domain by using an SVD (Singular Value Decomposition) Decomposition technique, and has good real-time performance and visual effect based on generation of a countermeasure network, thereby being capable of efficiently performing a large-scale industrial data generation task.
A second object of the present invention is to provide an industrial quality inspection apparatus based on unpaired industrial data.
The technical scheme adopted by the invention is as follows:
an embodiment of the first aspect of the invention provides an industrial quality inspection method based on unpaired industrial data, which comprises the following steps: acquiring a source domain image, inputting the source domain image into a generation countermeasure network to acquire a generated image for mapping the source domain image to a target domain, and judging whether the generated image is true or false; extracting main features of the source domain image and main features of the generated image based on a VGG (Visual Geometry Group Network) Network and an SVD (singular value decomposition) technology, and judging whether the main features of the source domain image and the main features of the generated image are approximately consistent; if the main features of the source domain image and the generated image are not approximately consistent, optimizing the generation countermeasure network until the main features of the source domain image and the generated image are approximately consistent; inputting unpaired images to be detected into the optimized generation countermeasure network to generate corresponding generated images; and carrying out industrial quality inspection according to the corresponding generated image.
The industrial quality inspection method based on unpaired industrial data provided by the invention can also have the following additional technical characteristics:
according to one embodiment of the invention, extracting the main features of the source domain image and the main features of the generated image based on the VGG network and the SVD technology comprises the following steps: extracting features of the source domain image or the generated image through a pre-trained VGG network; and carrying out SVD on the extracted features, screening out main features according to the size of the singular value, and reconstructing the screened main features.
According to one embodiment of the invention, the generative countermeasure network is optimized with a loss function, the generative countermeasure network comprising a generator and an arbiter, wherein the loss function comprises: countermeasure loss, feature matching loss, and SVD reconstruction loss.
According to one embodiment of the present invention, the countermeasure loss is obtained according to the following formula (1):
LGAN(G，D，X，Y)= Ex~Pdata（x）[log(1-D(G(x) ) )]+ Ey~Pdata（y）[log(D(y))] （1），
wherein L isGANRepresenting the countermeasure loss, G representing the generator, D representing the discriminator, X representing the source domain image, Y representing the target domain image, Ex~Pdata（x）Expressing the expectation of a variable x subject to a random distribution, Ey~Pdata（y）Expressing expectation of variable y obeying random distribution, G (x) expressing a generated image, and D (y) expressing a discriminant probability;
obtaining the feature matching loss according to the following formula (2):
LFM(G，X，Y)= Ey~Pdata(y)||D(G(x))-D(y)||1 （2），
wherein L isFM(G, X, Y) represents a feature matching penalty;
obtaining the SVD reconstruction loss according to the following formula (3):
wherein L isSR(G, X) represents SVD reconstruction loss, ViRepresenting the ith activation layer of the VGG network, S representing the feature decomposition and the extraction of main features,c i 、h i andw i respectively representing the depth, height and width of the ith activation layer of the VGG network;
the loss function is obtained according to the following formula (4),
L(G，D，X，Y)=LGAN(G，D，X，Y)+ α LFM(G，X，Y)+ β LSR(G，X) （4）
where L (G, D, X, Y) represents a loss function, and α and β represent weight coefficients.
According to one embodiment of the invention, the generative countermeasure network is optimized by back propagation according to the loss function.
An embodiment of the second aspect of the present invention provides an industrial quality inspection apparatus based on unpaired industrial data, including: the generation module is used for acquiring a source domain image, inputting the source domain image into a generation countermeasure network to acquire a generated image for mapping the source domain image to a target domain, and judging whether the generated image is true or false; the feature extraction module is used for extracting the main features of the source domain image and the main features of the generated image based on a VGG network and an SVD technology and judging whether the main features of the source domain image and the main features of the generated image are approximately consistent or not; the optimization module is used for judging whether the main features of the source domain image are approximately consistent with the main features of the generated image or not, and optimizing the generation countermeasure network when the main features of the source domain image are not approximately consistent with the main features of the generated image until the main features of the source domain image are approximately consistent with the main features of the generated image; and the detection module is used for inputting the unpaired image to be detected into the optimized generation countermeasure network to generate a corresponding generated image and performing industrial quality inspection according to the corresponding generated image.
The industrial quality inspection device based on unpaired industrial data provided by the invention can also have the following additional technical characteristics:
according to one embodiment of the invention, the feature extraction module comprises: an extraction unit, configured to extract features of the source domain image or the generated image through a pre-trained VGG network; and the decomposition reconstruction unit is used for carrying out SVD on the extracted features, screening out main features according to the size of the singular value and reconstructing the screened main features.
According to one embodiment of the present invention, the optimization module optimizes the generative countermeasure network using a loss function, the generative countermeasure network including a generator and an arbiter, wherein the loss function includes: countermeasure loss, feature matching loss, and SVD reconstruction loss.
According to one embodiment of the invention, the optimization module obtains the antagonistic loss according to the following formula (1):
LGAN(G，D，X，Y)= Ex~Pdata（x）[log(1-D(G(x) ) )]+ Ey~Pdata（y）[log(D(y))] （1），
wherein L isGANRepresenting the countermeasure loss, G representing the generator, D representing the discriminator, X representing the source domain image, Y representing the target domain image, Ex~Pdata（x）Expressing the expectation of a variable x subject to a random distribution, Ey~Pdata（y）Expressing expectation of variable y obeying random distribution, G (x) expressing a generated image, D (y) expressing discriminant probability, and log expressing a logarithmic function;
obtaining the feature matching loss according to the following formula (2):
LFM(G，X，Y)= Ey~Pdata(y)||D(G(x))-D(y)||1 （2），
wherein L isFM(G, X, Y) represents a feature matching penalty;
obtaining the SVD reconstruction loss according to the following formula (3):
wherein L isSR(G, X) represents SVD reconstruction loss, ViRepresenting the ith activation layer of the VGG network, S representing the feature decomposition and the extraction of main features,c i 、h i andw i respectively representing the depth, height and width of the ith activation layer of the VGG network;
the loss function is obtained according to the following formula (4),
L(G，D，X，Y)=LGAN(G，D，X，Y)+ α LFM(G，X，Y)+ β LSR(G，X) （4）
where L (G, D, X, Y) represents a loss function, and α and β represent weight coefficients.
According to an embodiment of the present invention, the optimization module is specifically configured to: optimizing the generative countermeasure network by back propagation according to the loss function.
The invention has the beneficial effects that:
the method can realize real-time cross-domain conversion of the image, has wider application scenes, can well balance the information of a target domain and a source domain by adopting an SVD (singular value decomposition) technology, and has good real-time property and visual effect based on generation of a countermeasure network, thereby efficiently carrying out large-scale industrial data generation tasks.
Drawings
FIG. 1 is a flow diagram of an industrial quality inspection method based on unpaired industrial data according to one embodiment of the invention;
FIG. 2 is a schematic diagram of a method for industrial quality inspection based on unpaired industrial data according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of feature extraction according to one embodiment of the present invention;
FIG. 4 is a block diagram of an industrial quality inspection device based on unpaired industrial data according to one embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
FIG. 1 is a flow diagram of an industrial quality inspection method based on unpaired industrial data according to one embodiment of the invention. As shown in fig. 1, the method comprises the steps of:
s1, acquiring the source domain image, inputting the source domain image into the generation countermeasure network to acquire a generated image mapping the source domain image to the target domain, and judging whether the generated image is true or false.
Specifically, the source domain image is mapped to the target domain to obtain a generated image, i.e., a cross-domain image transformation, whose primary goal is to learn a function that maps images within the source and target domains. The content of the generated image should be similar to the content of the input source domain image, and the style of the generated image should be approximately consistent with that of the target domain. Cross-domain image conversion has been widely used in the fields of style conversion, image editing, super-resolution, colorization, and the like, and has received extensive attention from researchers in the fields of deep learning and computer vision.
And S2, extracting the main features of the source domain image and the generated image based on the VGG network and the SVD technology, and judging whether the main features of the source domain image and the generated image are approximately consistent.
Further, extracting main features of the source domain image and generating the main features of the image based on the VGG network and the SVD technology comprises: extracting source domain images and generating the characteristics of the images through a pre-trained VGG network; and carrying out SVD on the extracted features, screening out main features according to the size of the singular value, and reconstructing the screened main features.
Specifically, the source domain image and the generated image may be input into a pre-trained VGG network, respectively, and the intermediate feature layer extracted by the VGG (e.g., the feature extracted by the Relu4_ 2) serves as the feature of the source domain image and the generated image. Converting the extracted feature layers with the size of (h, w, c) into a matrix of (h x w, c), carrying out SVD on the matrix to obtain the feature values of the feature matrix and corresponding feature vectors, screening main features (feature vectors) according to the size of singular values, and finally reconstructing the screened features, wherein h represents height, w represents width, and c represents the number of channels.
And S3, if the main features of the source domain image and the generated image are not approximately consistent, optimizing the generation countermeasure network until the main features of the source domain image and the generated image are approximately consistent.
And S4, inputting the unpaired image to be detected into the optimized generation countermeasure network to generate a corresponding generated image, and performing industrial quality inspection according to the corresponding generated image.
Specifically, as shown in fig. 2, the generation of the countermeasure network may include a generator and a discriminator, the generator is composed of an end-to-end convolution network and is used for mapping an image from a source domain to a target domain, i.e. obtaining a generated image, the discriminator is used for receiving the generated image and performing a judgment, and in a training stage, the generation of the countermeasure network and the generation of the discriminator are performed, the reconstruction capability of the generator and the recognition capability of the discriminator are improved in the countermeasure, and the true or false of the generated image is judged through an adaptive threshold.
The source domain image is input into a generator, and the generator generates an image which has the content of the source domain image and accords with the style of the target domain, namely the generator generates a corresponding generated image. The discriminator judges the difference degree between the generated image and the target image, judges whether the generated image is true or false according to the difference degree, and outputs a judgment result. Thereby achieving cross-domain conversion of the image.
In the process of image cross-domain conversion, as shown in fig. 3, it is necessary to extract the main features of the source domain image and the main features of the generated image based on the VGG network and the SVD technology, extract the source domain image or the features of the generated image through the pre-trained VGG network, perform SVD decomposition on the extracted features, screen out the main features according to the magnitude of the singular value, and reconstruct the screened main features.
Then, whether the main features of the two images are approximately consistent or not is judged, specifically, whether the main features are approximately consistent or not can be judged according to the L2 loss (namely, the least square error) between the main features of the two images, and if the L2 loss between the main features of the two images approaches 0 and does not change any more, the main features of the two images are approximately consistent. And if the main characteristics of the source domain image and the main characteristics of the generated image are approximately consistent, optimizing the generation of the countermeasure network, and repeating until the main characteristics of the source domain image and the main characteristics of the generated image are approximately consistent, thereby completing the training of generating the countermeasure network. It can be understood that the VGG network can extract the features of the image, and then perform SVD decomposition on the features to further extract the main information of the features, i.e., the main features, so that the accurate extraction of the features can be completed, and the optimization of the production countermeasure network is performed according to the extracted main features, thereby well balancing the information of the target domain and the source domain.
In an actual industrial quality inspection application scene, an unpaired image to be detected is directly input into the optimized generation countermeasure network, so that cross-domain conversion of the image can be realized, and the converted image is directly used for industrial quality inspection. Therefore, real-time cross-domain conversion of images can be achieved, industrial quality inspection is achieved based on unpaired industrial data, a wider application scene is achieved, information of a target domain and information of a source domain can be well balanced by adopting the SVD technology, and good real-time performance and visual effect are achieved based on generation of a countermeasure network, so that a large-scale industrial data generation task can be efficiently carried out.
According to one embodiment of the invention, a generation countermeasure network is optimized by using a loss function, the generation countermeasure network comprises a generator and a discriminator, wherein the loss function comprises: countermeasure loss, feature matching loss, and SVD reconstruction loss.
Further, the countermeasure loss is obtained according to the following formula (1):
LGAN(G，D，X，Y)= Ex~Pdata（x）[log(1-D(G(x) ) )]+ Ey~Pdata（y）[log(D(y))] （1），
wherein L isGANRepresenting the countermeasure loss, G representing the generator, D representing the discriminator, X representing the source domain image, Y representing the target domain image, Ex~Pdata（x）Expressing the expectation of a variable x subject to a random distribution, Ey~Pdata(y)Expressing expectation of variable y obeying random distribution, G (x) expressing a generated image, D (y) expressing discriminant probability, and log expressing a logarithmic function;
the feature matching loss is obtained according to the following formula (2):
LFM(G，X，Y)= Ey~Pdata(y)||D(G(x))-D(y)||1 （2），
wherein L isFM(G, X, Y) represents a feature matching penalty;
obtaining the SVD reconstruction loss according to the following formula (3):
wherein L isSR(G, X) represents SVD reconstruction loss, ViRepresenting the ith activation layer of the VGG network, S representing the feature decomposition and the extraction of main features,c i 、h i andw i respectively representing the depth, height and width of the ith activation layer of the VGG network;
the loss function L (G, D, X, Y) is obtained according to the following equation (4):
L(G，D，X，Y)=LGAN(G，D，X，Y)+ α LFM(G，X，Y)+ β LSR(G，X) （4）
wherein, L (G, D, X, Y) represents a loss function, alpha and beta represent weight coefficients, and alpha and beta can be obtained according to relevant experiments.
The generation of the countermeasure network G may be represented in the form:
；
wherein the content of the first and second substances,
represents the optimization arbiter D such that
To the maximum extent that the number of the first,
the representation optimization generator G makes
The size of the particles is minimized and,
i.e. so that
Minimized generator G.
According to one embodiment of the invention, the countermeasure network is generated through back propagation optimization according to the loss function until the main characteristics of the extracted source domain image and the main characteristics of the generated image are approximately consistent, so that the generated image is closer to the target image.
Specifically, the back propagation is to derive the loss function, and the gradient information is reversely fed back to optimize the parameters, so that the loss function reaches an extremum.
In summary, according to the industrial quality inspection method based on unpaired industrial data of the embodiment of the present invention, a source domain image is acquired, the source domain image is input into a generation countermeasure network to acquire a generated image that maps the source domain image to a target domain, whether the generated image is true or false is judged, main features of the source domain image and main features of the generated image are extracted based on a VGG network and an SVD technique, whether the main features of the source domain image and the main features of the generated image are approximately consistent or not is judged, and if the main features of the source domain image and the main features of the generated image are not approximately consistent, the generation countermeasure network is optimized until the main features of the source domain image and the main features of the generated image are approximately consistent; inputting the unpaired image to be detected into the optimized generation countermeasure network to generate a corresponding generated image, and performing industrial quality inspection according to the corresponding generated image. Therefore, the method can realize real-time cross-domain conversion of the image, has wider application scenes, can well balance the information of the target domain and the source domain by adopting the SVD technology, and has good real-time performance and visual effect based on generation of the countermeasure network, thereby being capable of efficiently carrying out large-scale industrial data generation tasks.
Corresponding to the above-mentioned industrial quality inspection method based on unpaired industrial data, the present invention also provides an industrial quality inspection apparatus based on unpaired industrial data, and since the apparatus embodiment of the present invention corresponds to the above-mentioned method embodiment, details that are not disclosed in the apparatus embodiment can refer to the above-mentioned method embodiment, and are not described again in the present invention.
FIG. 4 is a block diagram of an industrial quality inspection apparatus based on unpaired industrial data according to one embodiment of the present invention, as shown in FIG. 4, the apparatus comprising: a generating module 1, a feature extracting module 2, an optimizing module 3 and a detecting module 4, wherein,
the generation module 1 is used for acquiring a source domain image, inputting the source domain image into a generation countermeasure network to acquire a generated image which maps the source domain image to a target domain, and judging whether the generated image is true or false; the feature extraction module 2 is used for extracting the main features of the source domain image and the main features of the generated image based on the VGG network and the SVD technology, and judging whether the main features of the source domain image and the main features of the generated image are approximately consistent; the optimization module 3 is used for judging whether the main features of the source domain image and the generated image are approximately consistent, and optimizing the generation countermeasure network when the main features of the source domain image and the generated image are not approximately consistent until the main features of the source domain image and the generated image are approximately consistent; the detection module 4 is used for inputting the unpaired image to be detected into the optimized generation countermeasure network to generate a corresponding generated image, and performing industrial quality inspection according to the corresponding generated image.
According to one embodiment of the present invention, the feature extraction module 2 includes: the device comprises an extraction unit and a decomposition reconstruction unit, wherein the extraction unit is used for extracting the source domain image or generating the characteristics of the image through a pre-trained VGG network; and the decomposition reconstruction unit is used for carrying out SVD on the extracted features, screening out main features according to the size of the singular value and reconstructing the screened main features.
According to one embodiment of the present invention, the optimization module 3 optimizes the countermeasure network using a loss function, the generation of the countermeasure network including a generator and a discriminator, wherein the loss function includes: countermeasure loss, feature matching loss, and SVD reconstruction loss.
According to one embodiment of the invention, the optimization module 3 obtains the opposing losses according to the following equation (1):
LGAN(G，D，X，Y)= Ex~Pdata（x）[log(1-D(G(x) ) )]+ Ey~Pdata（y）[log(D(y) )] （1），
wherein L isGANRepresenting the countermeasure loss, G representing the generator, D representing the discriminator, X representing the source domain image, Y representing the target domain image, Ex~Pdata（x）Expressing the expectation of a variable x subject to a random distribution, Ey~Pdata（y）Expressing expectation of variable y obeying random distribution, G (x) expressing a generated image, D (y) expressing discriminant probability, and log expressing a logarithmic function;
obtaining the feature matching loss according to the following formula (2):
LFM(G，X，Y)= Ey~Pdata(y)||D(G(x))-D(y)||1 （2），
wherein L isFM(G, X, Y) represents a feature matching penalty;
obtaining the SVD reconstruction loss according to the following formula (3):
wherein L isSR(G, X) represents SVD reconstruction loss, ViRepresenting the ith activation layer of the VGG network, S representing the feature decomposition and the extraction of main features,c i 、h i andw i respectively representing the depth, height and width of the ith activation layer of the VGG network;
the loss function is obtained according to the following formula (4),
L(G，D，X，Y)=LGAN(G，D，X，Y)+ α LFM(G，X，Y)+ β LSR(G，X) （4）
where L (G, D, X, Y) represents a loss function, and α and β represent weight coefficients.
According to an embodiment of the present invention, the optimization module 3 is specifically configured to: and generating the countermeasure network through back propagation optimization according to the loss function.
In summary, according to the industrial quality inspection apparatus based on unpaired industrial data of the embodiment of the present invention, the generation module acquires the source domain image, and inputs the source domain image into the generation countermeasure network to acquire the generated image mapping the source domain image to the target domain, and determines whether the generated image is true or false, the feature extraction module extracts the main features of the source domain image and the main features of the generated image based on the VGG network and the SVD technology, and determines whether the main features of the source domain image and the main features of the generated image are approximately consistent, the optimization module determines whether the main features of the source domain image and the main features of the generated image are approximately consistent, and optimizes the countermeasure network when the main features of the source domain image and the main features of the generated image are not approximately consistent, until the main features of the source domain image and the main features of the generated image are approximately consistent, the detection module inputs the unpaired image to be detected into the optimized generation countermeasure network, so as to generate a corresponding generated image and carry out industrial quality inspection according to the corresponding generated image. Therefore, the device can realize real-time cross-domain conversion of images, has wider application scenes, can well balance information of a target domain and a source domain by adopting the SVD technology, has good real-time performance and visual effect based on generation of a countermeasure network, and can efficiently perform large-scale industrial data generation tasks.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The meaning of "plurality" is two or more unless specifically limited otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. An industrial quality inspection method based on unpaired industrial data is characterized by comprising the following steps:
acquiring a source domain image, inputting the source domain image into a generation countermeasure network to acquire a generated image for mapping the source domain image to a target domain, and judging whether the generated image is true or false;
extracting the main features of the source domain image and the main features of the generated image based on a VGG network and an SVD technology, and judging whether the main features of the source domain image and the main features of the generated image are approximately consistent;
if the main features of the source domain image and the generated image are not approximately consistent, optimizing the generation countermeasure network until the main features of the source domain image and the generated image are approximately consistent;
inputting the unpaired image to be detected into the optimized generation countermeasure network to generate a corresponding generated image, and performing industrial quality inspection according to the corresponding generated image.
2. The unpaired industrial data based industrial quality inspection method according to claim 1, wherein extracting the main features of the source domain image and the main features of the generated image based on VGG network and SVD technology comprises:
extracting features of the source domain image and the generated image through a pre-trained VGG network;
and carrying out SVD on the extracted features, screening out main features according to the size of the singular value, and reconstructing the screened main features.
3. The unpaired industrial data based industrial quality inspection method according to claim 2, wherein the generative countermeasure network is optimized with a loss function, the generative countermeasure network comprising a generator and a discriminator, wherein the loss function comprises: countermeasure loss, feature matching loss, and SVD reconstruction loss.
4. The unpaired industrial data-based industrial quality inspection method of claim 3, wherein the countermeasure loss is obtained according to the following equation (1):
LGAN(G，D，X，Y)= Ex~Pdata（x）[log(1-D(G(x) ) )]+ Ey~Pdata（y）[log(D(y))] （1），
wherein L isGANRepresenting the countermeasure loss, G representing the generator, D representing the discriminator, X representing the source domain image, Y representing the target domain image, Ex~Pdata（x）Expressing the expectation of a variable x subject to a random distribution, Ey~Pdata（y）Expressing expectation of variable y obeying random distribution, G (x) expressing a generated image, and D (y) expressing a discriminant probability;
obtaining the feature matching loss according to the following formula (2):
LFM(G，X，Y)= Ey~Pdata(y)||D(G(x))-D(y)||1 （2），
wherein L isFM(G, X, Y) represents a feature matching penalty;
obtaining the SVD reconstruction loss according to the following formula (3):
wherein L isSR(G, X) represents SVD reconstruction loss, ViRepresenting the ith activation layer of the VGG network, S representing the feature decomposition and the extraction of main features,c i 、h i andw i respectively representing the depth, height and width of the ith activation layer of the VGG network;
the loss function is obtained according to the following formula (4),
L(G，D，X，Y)=LGAN(G，D，X，Y)+ α LFM(G，X，Y)+ β LSR(G，X) （4）
where L (G, D, X, Y) represents a loss function, and α and β represent weight coefficients.
5. The unpaired industrial data-based industrial quality inspection method of claim 4, wherein the generative countermeasure network is optimized by back propagation according to the loss function.
6. An industrial quality inspection apparatus based on unpaired industrial data, comprising:
the generation module is used for acquiring a source domain image, inputting the source domain image into a generation countermeasure network to acquire a generated image for mapping the source domain image to a target domain, and judging whether the generated image is true or false;
the feature extraction module is used for extracting the main features of the source domain image and the main features of the generated image based on a VGG network and an SVD technology and judging whether the main features of the source domain image and the main features of the generated image are approximately consistent or not;
the optimization module is used for judging whether the main features of the source domain image are approximately consistent with the main features of the generated image or not, and optimizing the generation countermeasure network when the main features of the source domain image are not approximately consistent with the main features of the generated image until the main features of the source domain image are approximately consistent with the main features of the generated image;
and the detection module is used for inputting the unpaired image to be detected into the optimized generation countermeasure network to generate a corresponding generated image and performing industrial quality inspection according to the corresponding generated image.
7. The unpaired industrial data-based industrial quality inspection device of claim 6, wherein the feature extraction module comprises:
an extraction unit, configured to extract features of the source domain image or the generated image through a pre-trained VGG network;
and the decomposition reconstruction unit is used for carrying out SVD on the extracted features, screening out main features according to the size of the singular value and reconstructing the screened main features.
8. The unpaired industrial data-based industrial quality inspection device of claim 7, wherein the optimization module optimizes the generative countermeasure network with a loss function, the generative countermeasure network comprising a generator and a discriminator, wherein the loss function comprises: countermeasure loss, feature matching loss, and SVD reconstruction loss.
9. The unpaired industrial data-based industrial quality inspection device of claim 8, wherein the optimization module obtains the countermeasure loss according to the following equation (1):
LGAN(G，D，X，Y)= Ex~Pdata（x）[log(1-D(G(x) ) )]+ Ey~Pdata（y）[log(D(y))] （1），
wherein L isGANRepresenting the countermeasure loss, G representing the generator, D representing the discriminator, X representing the source domain image, Y representing the target domain image, Ex~Pdata（x）Expressing the expectation of a variable x subject to a random distribution, Ey~Pdata（y）Expressing expectation of variable y obeying random distribution, G (x) expressing a generated image, D (y) expressing discriminant probability, and log expressing a logarithmic function;
obtaining the feature matching loss according to the following formula (2):
LFM(G，X，Y)= Ey~Pdata(y)||D(G(x))-D(y)||1 （2），
wherein L isFM(G, X, Y) represents a feature matching penalty;
obtaining the SVD reconstruction loss according to the following formula (3):
wherein L isSR(G, X) represents SVD reconstruction loss, ViRepresenting the ith activation layer of the VGG network, S representing the feature decomposition and the extraction of main features,c i 、h i andw i respectively representing the depth, height and width of the ith activation layer of the VGG network;
the loss function is obtained according to the following formula (4),
L(G，D，X，Y)=LGAN(G，D，X，Y)+ α LFM(G，X，Y)+ β LSR(G，X) （4）
where L (G, D, X, Y) represents a loss function, and α and β represent weight coefficients.
10. The unpaired industrial data-based industrial quality inspection device of claim 9, wherein the optimization module is specifically configured to: optimizing the generative countermeasure network by back propagation according to the loss function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110841393.9A CN113284150B (en) | 2021-07-26 | 2021-07-26 | Industrial quality inspection method and industrial quality inspection device based on unpaired industrial data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110841393.9A CN113284150B (en) | 2021-07-26 | 2021-07-26 | Industrial quality inspection method and industrial quality inspection device based on unpaired industrial data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113284150A true CN113284150A (en) | 2021-08-20 |
| CN113284150B CN113284150B (en) | 2021-10-19 |
Family
ID=77287250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110841393.9A Active CN113284150B (en) | 2021-07-26 | 2021-07-26 | Industrial quality inspection method and industrial quality inspection device based on unpaired industrial data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113284150B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793333A (en) * | 2021-11-15 | 2021-12-14 | 常州微亿智造科技有限公司 | Defect picture generation method and device applied to industrial quality inspection |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016002003A1 (en) * | 2014-07-01 | 2016-01-07 | 株式会社日立ハイテクノロジーズ | Substrate inspection apparatus and substrate inspection method |
| CN108090905A (en) * | 2018-01-05 | 2018-05-29 | 清华大学 | The determination methods and system of producing line exception |
| CN110442758A (en) * | 2019-07-23 | 2019-11-12 | 腾讯科技（深圳）有限公司 | A kind of figure alignment schemes, device and storage medium |
| CN110516435A (en) * | 2019-09-02 | 2019-11-29 | 国网电子商务有限公司 | Private key management method and device based on biological characteristics |
| CN111126433A (en) * | 2019-11-19 | 2020-05-08 | 佛山市南海区广工大数控装备协同创新研究院 | Positive and negative sample data balancing method in factory PCB defect detection |
-
2021
- 2021-07-26 CN CN202110841393.9A patent/CN113284150B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016002003A1 (en) * | 2014-07-01 | 2016-01-07 | 株式会社日立ハイテクノロジーズ | Substrate inspection apparatus and substrate inspection method |
| CN108090905A (en) * | 2018-01-05 | 2018-05-29 | 清华大学 | The determination methods and system of producing line exception |
| CN110442758A (en) * | 2019-07-23 | 2019-11-12 | 腾讯科技（深圳）有限公司 | A kind of figure alignment schemes, device and storage medium |
| CN110516435A (en) * | 2019-09-02 | 2019-11-29 | 国网电子商务有限公司 | Private key management method and device based on biological characteristics |
| CN111126433A (en) * | 2019-11-19 | 2020-05-08 | 佛山市南海区广工大数控装备协同创新研究院 | Positive and negative sample data balancing method in factory PCB defect detection |
Non-Patent Citations (2)
| Title |
|---|
| CHANGCHUNZHANG ET AL.: "Transferable attention networks for adversarial domain adaptation", 《INFORMATION SCIENCES》 * |
| 肖智恒: "基于生成对抗网络的不成对图像转换研究与应用", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793333A (en) * | 2021-11-15 | 2021-12-14 | 常州微亿智造科技有限公司 | Defect picture generation method and device applied to industrial quality inspection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113284150B (en) | 2021-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yang et al. | Infrared and visible image fusion via texture conditional generative adversarial network | |
| CN110706302B (en) | System and method for synthesizing images by text | |
| Lin et al. | Sequential dual attention network for rain streak removal in a single image | |
| Kavitha et al. | Multimodal medical image fusion based on integer wavelet transform and neuro-fuzzy | |
| Couturier et al. | Image denoising using a deep encoder-decoder network with skip connections | |
| CN113361646A (en) | Generalized zero sample image identification method and model based on semantic information retention | |
| Quan et al. | An intelligent system approach for probabilistic volume rendering using hierarchical 3D convolutional sparse coding | |
| CN113284150B (en) | Industrial quality inspection method and industrial quality inspection device based on unpaired industrial data | |
| Du et al. | Blind image denoising via dynamic dual learning | |
| Jiang et al. | A new nonlocal means based framework for mixed noise removal | |
| Hovhannisyan et al. | AED-Net: A single image dehazing | |
| Cherian et al. | A Novel AlphaSRGAN for Underwater Image Super Resolution. | |
| CN111489405A (en) | Face sketch synthesis system for generating confrontation network based on condition enhancement | |
| CN112785540B (en) | Diffusion weighted image generation system and method | |
| CN114494387A (en) | Data set network generation model and fog map generation method | |
| Avcı et al. | MFIF-DWT-CNN: Multi-focus ımage fusion based on discrete wavelet transform with deep convolutional neural network | |
| Wang et al. | MDD-ShipNet: Math-Data Integrated Defogging for Fog-Occlusion Ship Detection | |
| CN113158970A (en) | Action identification method and system based on fast and slow dual-flow graph convolutional neural network | |
| CN117217997A (en) | Remote sensing image super-resolution method based on context perception edge enhancement | |
| Kumar et al. | Underwater image enhancement using deep learning | |
| Whiteley et al. | Direct image reconstruction from raw measurement data using an encoding transform refinement-and-scaling neural network | |
| CN115688234A (en) | Building layout generation method, device and medium based on conditional convolution | |
| CN114612618A (en) | Image generation method, device, equipment, storage medium and computer program product | |
| Zhao et al. | Unpaired sonar image denoising with simultaneous contrastive learning | |
| CN113327221A (en) | Image synthesis method and device fusing ROI (region of interest), electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |