CN111126446B - Method for amplifying defect image data of robot vision industrial product - Google Patents
Method for amplifying defect image data of robot vision industrial product Download PDFInfo
- Publication number
- CN111126446B CN111126446B CN201911198833.2A CN201911198833A CN111126446B CN 111126446 B CN111126446 B CN 111126446B CN 201911198833 A CN201911198833 A CN 201911198833A CN 111126446 B CN111126446 B CN 111126446B
- Authority
- CN
- China
- Prior art keywords
- feature map
- image
- discriminator
- label
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
- G06F18/2113—Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a robot vision industrial product defect image data augmentation method, which comprises the steps of numbering original images according to categories, and using the numbers as real labels; dividing an original image input data loader into a plurality of batches and then training to obtain corresponding random noise and random labels; inputting random noise and random labels into a generator to obtain a generated image, inputting the generated image and the random labels into a discriminator, calculating generator loss according to a discrimination result and updating generator parameters; inputting the original image and the real label into a discriminator, calculating the discriminator according to the discrimination result to discriminate the loss of the original image, inputting the generated image and the random label into the discriminator, calculating the discriminator according to the discrimination result to discriminate the loss of the generated image, calculating the discriminator loss and updating the discriminator parameters, and obtaining the defect image data augmentation model after iteration. The problem of small sample data amplification is solved, and the industrial product defect detection effect is improved.
Description
Technical Field
The invention belongs to the technical field of defect detection methods, and relates to a method for augmenting defect image data of a robot vision industrial product.
Background
The manufacturing industry is the basic industry of national economy, and with the development of science and technology, intelligent manufacturing has become a breakthrough and a necessary way for the transformation from a large manufacturing country to a strong manufacturing country. Industrial robots have become an important mark for measuring the state of manufacturing and technology as an irreplaceable important equipment and means in intelligent manufacturing. The robot vision is intelligentized, which is an important way for solving the problems of cost rise and environmental restriction of the manufacturing industry. In the intelligent manufacturing industry, the industrial robot vision technology is adopted to detect industrial products, so that the technical problems in the production process can be found, the production technical problems can be improved in time, and the product quality can be improved.
In recent years, with the development of artificial intelligence and deep learning, more and more manufacturing enterprises begin to introduce deep learning technology to solve the detection problem that the traditional machine vision is difficult to solve. In order to realize accurate identification of parts in deep learning, a large amount of training data, namely part pictures, is needed, otherwise, problems such as under-fitting and the like can occur, and the detection effect is influenced. However, in practical situations, the number of pictures taken manually is limited, and even in some complex industrial conditions, only a few picture samples can be taken. In order to realize deep learning detection under the condition of small samples, a small sample picture data set needs to be expanded, the traditional processing method is to perform operations such as rotation, scaling, clipping, noise adding and the like on an original picture, and the methods do not essentially complete the expansion of the picture data set, cannot fully learn image characteristics and even cannot obviously improve the deep learning detection effect.
The GAN generation countermeasure network is an unsupervised deep learning model and comprises a generator and a discriminator, wherein the generator is used for generating images, and the discriminator is used for distinguishing original images from generated pictures and continuously confronts until the generator can generate vivid pictures. When the existing GAN generation countermeasure network generates various types of images, the situation of disorder of the types can occur, the effect of image data broadening is influenced, further, the deep learning training of defect detection is influenced, the defect detection accuracy is low, and the situations of missing detection or false detection are easy to occur.
Disclosure of Invention
The invention aims to provide a method for augmenting defective image data of a robot vision industrial product, which solves the problem of disordered image types generated by the conventional method for augmenting the defective image data.
The technical scheme adopted by the invention is that the method for amplifying the defect image data of the robot vision industrial product comprises the following steps:
numbering the original images according to categories, and taking the numbers as real labels; dividing an original image input data loader into a plurality of batches and then performing iterative training to obtain corresponding random noise and random labels; inputting each batch of random noise and random labels into a generator to obtain a generated image, inputting the generated image and the random labels into a discriminator, calculating the loss of the generator according to the discrimination result and updating the parameters of the generator; inputting the original image and the real label into a discriminator, calculating the loss of the discriminator to discriminate the loss of the original image according to the discrimination result, inputting the generated image and the random label into the discriminator, calculating the loss of the discriminator to discriminate the generated image according to the discrimination result, calculating the loss of the discriminator, updating parameters of the discriminator, and obtaining a defect image data augmentation model after iteration.
The invention is also characterized in that:
the method specifically comprises the following steps:
step 1, generating an original image set by the original image according to categories, and taking the number of each category of original image as a corresponding real label; inputting original images into a data loader, dividing an original image set into a plurality of batches according to the batch size, and determining a preset training round number;
step 2, traversing the data loader to obtain an original image and a real label and generate random noise and random labels of corresponding batches of large and small groups;
step 3, inputting the random noise and the random label into a generator module, and after deconvoluting the random label to obtain a feature map, superposing the feature map with the random noise and deconvoluting to obtain a generated image;
step 4, inputting the random label and the generated image into a discriminator, and after a feature map obtained by deconvolution of the random label is obtained, overlapping the feature map with a corresponding feature map of the generated image and convolving the feature map to obtain a discrimination result;
step 5, obtaining the loss of the generator after processing the judgment result, and updating the parameters of the generator by using the loss of the generator;
step 6, inputting the original image and the corresponding real label into a discriminator module, and after a characteristic image is obtained by deconvolution of the real label, superposing the characteristic image with the corresponding characteristic image of the original image and convolving to obtain a discrimination result;
step 7, processing the discrimination result to obtain the loss of the original image discriminated by the discriminator;
step 8, inputting the random label and the generated image into a discriminator module for convolution to obtain a discrimination result, processing the discrimination result to obtain the loss of the discriminator for discriminating the generated image, obtaining the loss of the discriminator by combining the loss of the original image, and performing iterative updating on the parameters of the discriminator by using the loss of the discriminator;
step 9, processing the judgment result to obtain the loss of the judgment generated image of the discriminator, processing the loss of the judgment original image and the loss of the judgment generated image, and updating the parameters of the discriminator;
step 10, returning to the step 2, and performing next iteration until all the original image sets are trained;
and 11, returning to the step 10, and performing the next round of training until all the preset rounds of training are completed to obtain the defect image data augmentation model.
The step 3 specifically comprises:
3.1, expanding the random label into a multi-dimensional random label through an Embedding function of a Pythrch, and sequentially deconvoluting the multi-dimensional random label by utilizing a label data deconvolution network of a generator to obtain a first characteristic diagram of A, B, \8230Aand N, wherein the sizes of the first characteristic diagram A-N are sequentially increased;
step 3.2, deconvoluting the input random noise into a second feature map A with the same size as the first feature map A by using a tag data deconvolution network of the generator, adding the second feature map A and the first feature map A, and then deconvoluting by using a noise data deconvolution network to obtain a second feature map B with the same size as the second feature map B; and in the same way, after the second feature map N is added with the first feature map N, deconvolution is carried out through a noise data deconvolution network to obtain a generated image.
The step 4 specifically comprises the following steps:
step 4.1, expanding the random label into a multi-dimensional random label, and deconvoluting the multi-dimensional random label into a third characteristic diagram of A, B, \8230nby using a label data deconvolution network of a discriminator, wherein the sizes of A-N of the third characteristic diagram are increased in sequence;
step 4.2, convolving the generated image into a fourth feature map N by using an image convolution network of a discriminator; adding the fourth feature map N and the third feature map N, and performing convolution through an image convolution network to obtain a fourth feature map N-1; and by analogy, after the fourth feature map A and the third feature map A are added, the convolution is carried out through an image convolution network to obtain a 0-1 judgment result.
The step 5 specifically comprises the following steps:
and 5, comparing the judgment result with the judgment result 1, performing error calculation through an MSE mean square error function to serve as the loss of the generator, performing back propagation, and updating the parameters of the generator through an Adam optimization algorithm.
The step 6 specifically comprises the following steps:
step 6.1, expanding the real label into a multi-dimensional real label, and deconvolving the multi-dimensional real label into a fifth characteristic diagram of A, B, \8230andN by using a label data deconvolution network of a discriminator, wherein the sizes of A-N of the fifth characteristic diagram are increased in sequence;
step 6.2, convolving the original image into a sixth feature map N by using an image convolution network of a discriminator; adding the sixth feature map N and the fifth feature map N, and then carrying out convolution through an image convolution network to obtain a sixth feature map N-1; and by analogy, after the fourth feature map A and the third feature map A are added, the convolution is carried out through an image convolution network to obtain a 0-1 judgment result.
The step 7 specifically comprises the following steps:
and comparing the discrimination result with 1, and performing error calculation through an MSE mean square error function to be used as a discriminator for discriminating the loss of the original image.
The step 8 specifically comprises:
8.1, expanding the random label into a multi-dimensional random label, and deconvoluting the multi-dimensional random label into a seventh characteristic diagram of A, B, \8230nby using a label data deconvolution network of a discriminator, wherein the sizes of the seventh characteristic diagram A-N are sequentially increased;
8.2, convolving the generated image into an eighth feature map N by using an image convolution network of the discriminator; adding the eighth characteristic diagram N and the seventh characteristic diagram N, and then carrying out convolution through an image convolution network to obtain an eighth characteristic diagram N-1; and by analogy, after the eighth characteristic diagram A and the seventh characteristic diagram A are added, the convolution is carried out through an image convolution network to obtain a 0-1 judgment result.
The step 9 specifically comprises:
comparing the discrimination result with 0, and performing error calculation through an MSE mean square error function to be used as a discriminator for discriminating the loss of the generated image; and averaging the loss of the original image and the loss of the generated image to be used as the loss of the discriminator, reversely propagating, and updating the parameters of the discriminator through an Adam optimization algorithm.
The invention has the beneficial effects that:
according to the method for augmenting the defect image data of the robot vision industrial product, the tag data deconvolution network is added into the generator and the discriminator and is respectively superposed with the generator and the discriminator to obtain the augmentation model, so that the influence of the tag data on the GAN network can be enhanced, the classification of the generated defect image is more definite, and the problem of disordered image categories generated by the existing image data augmentation method is effectively solved; different types of industrial product defect images are generated according to needs, the problem of small sample data amplification is solved, the industrial product defect detection effect is improved, a new method is provided for industrial product defect detection, and the quality detection cost of enterprises is reduced.
Drawings
FIG. 1 is a flow chart of a method for augmenting defective image data of a robotic visual industrial product according to the present invention;
FIG. 2 is a schematic diagram of a generator in the method for augmenting defect image data of a robot vision industrial product according to the present invention;
FIG. 3 is a schematic diagram of a structure of an arbiter in the method for augmenting defect image data of a robot vision industrial product according to the present invention;
FIG. 4 is an original image of an embodiment of a method for augmenting defect image data for a robotic vision industrial product of the present invention;
FIG. 5a is a defect image generated by 100 rounds of training using a conventional CDCGAN network;
fig. 5b is a defect image generated by 500 rounds of training using a conventional CDCGAN network;
FIG. 5c is a defect image generated by 1000 rounds of training using a conventional CDCGAN network;
FIG. 5d is a defect image generated by training 100 rounds using a robot vision industrial product defect image data augmentation method of the present invention;
FIG. 5e is a defect image generated by 500 rounds of training using a robot vision industrial product defect image data augmentation method of the present invention;
FIG. 5f is a defect image generated by 1000 training rounds using the method for augmenting defect image data of a robot vision industrial product according to the present invention.
Detailed Description
The invention is described in detail below with reference to the drawings and the detailed description.
The invention discloses a method for augmenting defect image data of a robot vision industrial product, which comprises the following steps of:
numbering the original images according to categories, and taking the numbers as real labels; dividing an original image input data loader into a plurality of batches and then performing iterative training to obtain random noise and random labels; inputting random noise and random labels of each batch of original images into a generator to obtain a generated image, inputting the generated image and the random labels into a discriminator, calculating the loss of the generator according to the discrimination result and updating the parameters of the generator; inputting the original image and the real label into a discriminator, calculating the loss of the discriminator to discriminate the loss of the original image according to the discrimination result, inputting the generated image and the random label into the discriminator, calculating the loss of the discriminator to discriminate the generated image according to the discrimination result, calculating the loss of the discriminator, updating parameters of the discriminator, and obtaining a defect image data augmentation model after iteration.
Step 1, generating an original image set by the original image according to categories, and taking the number of each category of original image as a corresponding real label; inputting the original images into a data loader, dividing the original image set into a plurality of batches according to the batch size, and determining the number of preset training rounds.
Specifically, the number of classes of the original image is n _ classes, and the original defect image is placed in a folder with the number of 0 to n _ classes (excluding n _ classes) according to the class, the number of the folder is used as a corresponding real label, and finally a total data set folder is placed. In this embodiment, 3 types of representative strip steel defect pictures are selected and respectively placed into three folders with the numbers of 0, 1 and 2, each folder contains 300 pictures, and each picture is a gray scale image with the resolution of 64x 64. The method comprises the steps of using a Pythrch to construct a data loader, inputting an original defect image data set, dividing a trained image data set into a plurality of batches according to the batch size, wherein each batch contains original images and corresponding real labels which may have incomplete equal division, and the number of the images of the last batch is possibly smaller than the preset batch size, so that training is not influenced.
And 2, traversing the data loader to obtain the original image and the corresponding real label, generating groups of random noises corresponding to the batch size according to the batch size of the batch of samples, wherein each group of random noises comprises 100 random decimal numbers from 0 to 1, the class number of the defect image to be trained is n _ classes, and then generating random integers corresponding to the batch size from 0 to n _ classes (not comprising n _ classes) as random labels.
Step 3, inputting the random noise and the random label into a generator module, and after deconvoluting the random label to obtain a feature map, superposing the feature map with the random noise and deconvoluting to obtain a generated image;
3.1, expanding the random label into a multi-dimensional random label through an Embedding function of a Pythrch, and sequentially deconvoluting the multi-dimensional random label by utilizing a label data deconvolution network of a generator to obtain a first characteristic diagram of A, B, \8230Aand N, wherein the sizes of the first characteristic diagram A-N are sequentially increased;
specifically, as shown in fig. 2, the random label is expanded from 1 dimension to an n _ classes dimension random label, and the n _ classes dimension is deconvoluted into first feature maps of 4x4, 8x8, 16x16, and 32x32 sizes in sequence by using a label data deconvolution network of a generator, which are called a conditional Mask C-Mask;
step 3.2, deconvoluting the input random noise into a second feature map A with the same size as the first feature map A by using a tag data deconvolution network of the generator, adding the second feature map A with the first feature map A, and then deconvoluting by using a noise data deconvolution network to obtain a second feature map B with the same size as the second feature map B; and in the same way, after the second feature map N is added with the first feature map N, deconvolution is carried out through a noise data deconvolution network to obtain a generated image.
Specifically, deconvolving the input random noise into a 4x4 second feature map by using a tag data deconvolution network of a generator, adding the 4x4 second feature map and the 4x4 first feature map, and deconvoluting by using a noise data deconvolution network to obtain an 8x8 second feature map;
adding the 8x8 second feature map and the 8x8 first feature map, and then performing deconvolution through a noise data deconvolution network to obtain a 16x16 second feature map;
adding the 16x16 second feature map and the 16x16 first feature map, and then performing deconvolution through a noise data deconvolution network to obtain a 32x32 second feature map;
and adding the 32x32 second feature map and the 32x32 first feature map, and then performing deconvolution through a noise data deconvolution network to obtain a 64x64 generated image.
Step 4, inputting the random label and the generated image into a discriminator, and superposing and convolving a feature map obtained by deconvolution of the random label and a feature map corresponding to the generated image to obtain a discrimination result;
step 4.1, expanding the random label into a multi-dimensional random label through an Embedding function of the Pythrch, and deconvoluting the multi-dimensional random label into a third characteristic diagram of A, B, \8230Aand N in sequence by utilizing a label data deconvolution network of a discriminator, wherein the sizes of the third characteristic diagram A-N are increased in sequence;
specifically, as shown in fig. 3, the random label is expanded into a multi-dimensional random label, and the multi-dimensional random label is sequentially deconvolved into a third feature map with a size of 4x4, 8x8, 16x16, or 32x32 by using a label data deconvolution network of a discriminator;
step 4.2, convolving the generated image into a fourth feature map N by using an image convolution network of a discriminator; adding the fourth feature map N and the third feature map N, and performing convolution through an image convolution network to obtain a fourth feature map N-1; and by analogy, after the fourth feature map A and the third feature map A are added, the convolution is carried out through an image convolution network to obtain a 0-1 judgment result.
Specifically, the 64 × 64 generated image is convolved into a 32 × 32 fourth feature map by an image convolution network of a discriminator;
adding the 32x32 fourth feature map and the 32x32 third feature map, and performing convolution through an image convolution network to obtain a 16x16 second feature map B;
adding the 16x16 fourth feature map and the 16x16 third feature map, and then performing convolution through an image convolution network to obtain an 8x8 second feature map B;
adding the 8x8 fourth feature map and the 8x8 third feature map, and performing convolution through an image convolution network to obtain a 4x4 second feature map B;
and adding the fourth feature map of 4x4 and the third feature map of 4x4, and then performing convolution through an image convolution network to obtain a 0-1 judgment result, wherein the closer to 0, the more false the picture is, and the closer to 1, the more true the picture is.
And 5, comparing the judgment result with the judgment result 1, performing error calculation through an MSE mean square error function to serve as the loss of the generator, performing back propagation, and updating the parameters of the generator through an Adam optimization algorithm.
Step 6, inputting the original image and the corresponding real label into a discriminator module, and after a characteristic image is obtained by deconvolution of the real label, superposing the characteristic image with the corresponding characteristic image of the original image and convolving to obtain a discrimination result;
6.1, expanding the real label into a multi-dimensional real label through an Embedding function of a Pythrch, and deconvoluting the multi-dimensional real label into a fifth characteristic diagram of A, B, \8230Aand N in sequence by utilizing a label data deconvolution network of a discriminator, wherein the sizes of the fifth characteristic diagram A-N are increased in sequence;
specifically, the real label is expanded into a multi-dimensional real label, and the multi-dimensional real label is deconvoluted into a fifth feature map with the size of 4x4, 8x8, 16x16, 32x32 by using a label data deconvolution network of a discriminator in sequence;
6.2, convolving the original image into a sixth feature map N by using an image convolution network of a discriminator; adding the sixth feature map N and the fifth feature map N, and then carrying out convolution through an image convolution network to obtain a sixth feature map N-1; and by analogy, after the fourth feature diagram A and the third feature diagram A are added, the convolution is carried out through the image convolution network to obtain a 0-1 judgment result.
Specifically, the original image is convolved into a fourth feature map a of 32 × 32 by using an image convolution network of a discriminator;
adding the 32x32 sixth feature map and the 32x32 fifth feature map, and performing convolution through an image convolution network to obtain a 16x16 sixth feature map;
adding the sixth feature map of 16x16 and the fifth feature map of 16x16, and performing convolution through an image convolution network to obtain a sixth feature map of 8x 8;
adding the sixth feature map of 8x8 and the fifth feature map of 8x8, and performing convolution through an image convolution network to obtain a sixth feature map of 4x 4;
and adding the sixth feature map of 4x4 and the fifth feature map of 4x4, and then performing convolution through an image convolution network to obtain a 0-1 judgment result.
Step 7, comparing the judgment result with the judgment result 1, and performing error calculation through an MSE mean square error function to be used as a discriminator for judging the loss of the original image;
and 8, inputting the random label and the generated image into a discriminator module for convolution to obtain a discrimination result, processing the discrimination result to obtain the loss of the discriminator for discriminating the generated image, obtaining the loss of the discriminator by combining the loss of the original image, and performing iterative updating on the parameters of the discriminator by using the loss.
8.1, expanding the random label into a multi-dimensional random label through an Embedding function of the Pythrch, and deconvoluting the multi-dimensional random label into a seventh characteristic diagram of A, B, \8230Aand N in sequence by utilizing a label data deconvolution network of a discriminator, wherein the sizes of the seventh characteristic diagram A-N are increased in sequence;
specifically, the random label is expanded into a multi-dimensional real label, and the multi-dimensional random label is deconvoluted into a seventh feature map with the size of 4x4, 8x8, 16x16 and 32x32 in sequence by using a label data deconvolution network of a discriminator;
8.2, convolving the generated image into an eighth feature map N by using an image convolution network of the discriminator; adding the eighth characteristic diagram N and the seventh characteristic diagram N, and performing convolution through an image convolution network to obtain an eighth characteristic diagram N-1; and by analogy, after the eighth feature map A and the seventh feature map A are added, the convolution is carried out through an image convolution network to obtain a 0-1 judgment result.
Specifically, the generated 64 × 64 image is convolved into an eighth feature map of 32 × 32 by an image convolution network of the discriminator;
adding the eighth feature map of 32x32 and the seventh feature map of 32x32, and then performing convolution through an image convolution network to obtain an eighth feature map of 16x 16;
adding the 16x16 eighth feature map and the 16x16 seventh feature map, and performing convolution through an image convolution network to obtain an 8x8 eighth feature map;
adding the 8x8 eighth feature map and the 8x8 seventh feature map, and performing convolution through an image convolution network to obtain a 4x4 eighth feature map;
and adding the eighth feature map of 4x4 and the seventh feature map of 4x4, and performing convolution through an image convolution network to obtain a 0-1 judgment result.
Step 9, comparing the judgment result with 0, and performing error calculation through an MSE mean square error function to be used as a discriminator for judging the loss of the generated image; averaging the loss of the original image and the loss of the image generated by discrimination to be used as the loss of the discriminator, reversely propagating, and updating the parameters of the discriminator through an Adam optimization algorithm;
step 10, returning to the step 2, and performing next iteration until all batches of original images are trained;
and 11, returning to the step 10, and performing the next round of training until all the training rounds with the preset number of training rounds are completed to obtain a defect image data augmentation model (Mask-CDCGAN model). Experiments prove that after 2000 rounds of training, the GAN generated confrontation network model completes convergence, and a vivid defect image can be generated.
Through the mode, according to the robot vision industrial product defect image data augmentation method, the tag data deconvolution network is added into the generator and the discriminator, and the tag data deconvolution network is superposed with the generator network and the discriminator network respectively to obtain the augmentation model, so that the influence of tag data on a GAN network can be enhanced, the generated defect images are classified more clearly, and the problem of disorder of image types generated by the existing image data augmentation method is effectively solved; different types of industrial product defect images are generated according to needs, the problem of small sample data amplification is solved, the industrial product defect detection effect is improved, a new method is provided for industrial product defect detection, and the quality detection cost of enterprises is reduced.
The invention carries out comparison experiment on the traditional CDCGAN model and the Mask-CDCGAN model of the patent, the original image is shown as figure 4, the comparison result is shown as figures 5a-f, each column in the figure is the defect of the same category:
fig. 5a, fig. 5b, and fig. 5c are defect images generated by 100, 500, and 1000 rounds of training in the conventional method, and fig. 5d, fig. 5e, and fig. 5f are defect images generated by 100, 500, and 1000 rounds of training in the Mask-CDCGAN model according to the present invention. When a traditional CDCGAN network generates multi-class images, label data and random noise are directly mixed and input into a generator and a discriminator, and the problem of class confusion easily occurs during training. The Mask-CDCGAN provided by the invention well solves the problem, and the pictures generated at the early stage of training have obvious classification.
In the experiment, firstly, an original image data set with a small sample not being expanded is expanded, then, three image data expansion methods of rotating scaling cutting, CDCGAN and Mask-CDCGAN are respectively adopted to expand the original image data set, a classical Yolo algorithm is respectively adopted to train the four data sets, and the identification accuracy is tested, and the accuracy is shown in Table 1. The result shows that the data set after the Mask-CDCGAN model is expanded has certain improvement on the identification accuracy, the effect is superior to the methods such as rotating, zooming and cutting, CDCGAN and the like, and the accuracy is greatly improved compared with the accuracy of a small sample without expansion.
TABLE 1 comparison of the impact of different image data augmentation methods on recognition accuracy
| Image data broadening method | Rate of identification accuracy |
| Small sample is not expanded | 86.3% |
| Rotational zoom cropping | 92.5% |
| CDCGAN | 94.2% |
| Mask-CDCGAN | 95.7% |
Claims (8)
1. A robot vision industrial product defect image data augmentation method is characterized by specifically comprising the following steps:
step 1, generating an original image set by an original image according to categories, and taking the number of each category of the original image as a corresponding real label; inputting the original images into a data loader, dividing an original image set into a plurality of batches according to the batch size, and determining a preset training round number;
step 2, traversing a data loader to acquire the original image and the real label and generate random noise and random labels of corresponding batch size groups;
step 3, inputting random noise and a random label into a generator module, and after a feature map obtained by deconvolution of the random label is obtained, superposing the feature map with the corresponding feature map of the random noise and deconvoluting to obtain a generated image;
step 4, inputting the random label and the generated image into a discriminator, and after a feature map obtained by deconvolution of the random label is obtained, overlapping the feature map with a corresponding feature map of the generated image and convolving the feature map to obtain a discrimination result;
step 5, obtaining the loss of the generator after processing the judgment result, and updating the parameters of the generator by using the loss of the generator;
step 6, inputting the original image and the corresponding real label into a discriminator module, and after a characteristic image is obtained by deconvolution of the real label, superposing the characteristic image with the corresponding characteristic image of the original image and convolving to obtain a discrimination result;
step 7, processing the discrimination result to obtain the loss of the discrimination original image of the discriminator;
step 8, inputting the random label and the generated image into a discriminator module for convolution to obtain a discrimination result, processing the discrimination result to obtain the loss of the discriminator for discriminating the generated image, obtaining the loss of the discriminator by combining the loss of the original image, and performing iterative updating on the parameters of the discriminator by using the loss of the discriminator;
step 9, processing the discrimination result to obtain the loss of the discrimination generation image of the discriminator, processing the loss of the discrimination original image and the loss of the discrimination generation image, and updating the parameters of the discriminator;
step 10, returning to the step 2, and performing the next iteration until all the original image sets are trained;
and step 11, returning to the step 10, and performing the next round of training until all the training rounds with the preset number of training rounds are completed to obtain the defect image data augmentation model.
2. The method for augmenting the defective image data of the robot vision industrial product according to claim 1, wherein the step 3 specifically comprises:
3.1, expanding the random label into a multi-dimensional random label through an Embedding function of a Pythrch, and sequentially deconvoluting the multi-dimensional random label by utilizing a label data deconvolution network of a generator to obtain a first feature map of A, B, \ 8230N, wherein the sizes of A-N of the first feature map are sequentially increased;
step 3.2, deconvoluting the input random noise into a second feature map A with the same size as the first feature map A by using a tag data deconvolution network of the generator, adding the second feature map A and the first feature map A, and then deconvoluting by using a noise data deconvolution network to obtain a second feature map B with the same size as the second feature map B; and in the same way, after the second feature map N is added with the first feature map N, deconvolution is carried out through a noise data deconvolution network to obtain a generated image.
3. The method for augmenting the defective image data of the robot vision industrial product according to claim 1, wherein the step 4 specifically comprises:
step 4.1, expanding the random label into a multi-dimensional random label, and deconvoluting the multi-dimensional random label into a third characteristic diagram of A, B, \8230andN in sequence by using a label data deconvolution network of a discriminator, wherein the sizes of A-N of the third characteristic diagram are increased in sequence;
step 4.2, convolving the generated image into a fourth feature map N by using an image convolution network of a discriminator; adding the fourth feature map N and the third feature map N, and performing convolution through an image convolution network to obtain a fourth feature map N-1; and by analogy, after the fourth feature map A and the third feature map A are added, performing convolution through an image convolution network to obtain a 0-1 judgment result.
4. The method for augmenting the image data of the defects of the robot vision industrial product according to claim 1 or 2, wherein the step 5 specifically comprises:
and 5, comparing the judgment result with the judgment result 1, performing error calculation through an MSE mean square error function to serve as the loss of the generator, performing back propagation, and updating the parameters of the generator through an Adam optimization algorithm.
5. The method for augmenting the defective image data of the robot vision industrial product according to claim 3, wherein the step 6 specifically comprises:
step 6.1, expanding the real label into a multi-dimensional real label, and sequentially deconvolving the multi-dimensional real label into a fifth feature map of A, B, \8230andN by using a label data deconvolution network of a discriminator, wherein the sizes of A-N of the fifth feature map are sequentially increased;
6.2, convolving the original image into a sixth feature map N by using an image convolution network of a discriminator; adding the sixth feature map N and the fifth feature map N, and performing convolution through an image convolution network to obtain a sixth feature map N-1; and by analogy, after the fourth feature map A and the third feature map A are added, the convolution is carried out through an image convolution network to obtain a 0-1 judgment result.
6. The method for augmenting the defective image data of the robot vision industrial product according to claim 1, wherein the step 7 specifically comprises:
and comparing the judgment result with 1, and performing error calculation through an MSE mean square error function to be used as a discriminator for judging the loss of the original image.
7. The method for augmenting image data of defects in robotic vision industrial products according to claim 1, wherein step 8 specifically comprises:
8.1, expanding the random label into a multi-dimensional random label, and deconvoluting the multi-dimensional random label into a seventh characteristic diagram of A, B, \8230andN in sequence by utilizing a label data deconvolution network of a discriminator, wherein the sizes of A-N of the seventh characteristic diagram are increased in sequence;
8.2, convolving the generated image into an eighth feature map N by using an image convolution network of a discriminator; adding the eighth characteristic diagram N and the seventh characteristic diagram N, and performing convolution through an image convolution network to obtain an eighth characteristic diagram N-1; and by analogy, after the eighth characteristic diagram A and the seventh characteristic diagram A are added, the convolution is carried out through an image convolution network to obtain a 0-1 judgment result.
8. The method for augmenting the defective image data of the robot vision industrial product according to claim 1, wherein the step 9 specifically comprises:
comparing the discrimination result with 0, and performing error calculation through an MSE mean square error function to be used as a discriminator for discriminating the loss of the generated image; and averaging the loss of the judging original image and the loss of the judging generated image to be used as the loss of the discriminator, reversely propagating, and updating the parameters of the discriminator through an Adam optimization algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911198833.2A CN111126446B (en) | 2019-11-29 | 2019-11-29 | Method for amplifying defect image data of robot vision industrial product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911198833.2A CN111126446B (en) | 2019-11-29 | 2019-11-29 | Method for amplifying defect image data of robot vision industrial product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111126446A CN111126446A (en) | 2020-05-08 |
| CN111126446B true CN111126446B (en) | 2023-04-07 |
Family
ID=70497094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911198833.2A Active CN111126446B (en) | 2019-11-29 | 2019-11-29 | Method for amplifying defect image data of robot vision industrial product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111126446B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112949706B (en) * | 2021-02-25 | 2024-01-05 | 平安科技（深圳）有限公司 | OCR training data generation method, device, computer equipment and storage medium |
| CN113129231B (en) * | 2021-04-07 | 2023-05-30 | 中国科学院计算技术研究所 | Method and system for generating high-definition image based on countermeasure generation network |
| CN113361443A (en) * | 2021-06-21 | 2021-09-07 | 广东电网有限责任公司 | Method and system for power transmission line image sample counterstudy augmentation |
| CN113298190B (en) * | 2021-07-05 | 2023-04-07 | 四川大学 | Weld image recognition and classification algorithm based on large-size unbalanced samples |
| CN115526891B (en) * | 2022-11-28 | 2023-04-07 | 深圳精智达技术股份有限公司 | Training method and related device for defect data set generation model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110097543A (en) * | 2019-04-25 | 2019-08-06 | 东北大学 | Surfaces of Hot Rolled Strip defect inspection method based on production confrontation network |
| WO2019180868A1 (en) * | 2018-03-22 | 2019-09-26 | 日本電気株式会社 | Image generation device, image generation method, and image generation program |
| CN110363122A (en) * | 2019-07-03 | 2019-10-22 | 昆明理工大学 | A kind of cross-domain object detection method based on multilayer feature alignment |
-
2019
- 2019-11-29 CN CN201911198833.2A patent/CN111126446B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019180868A1 (en) * | 2018-03-22 | 2019-09-26 | 日本電気株式会社 | Image generation device, image generation method, and image generation program |
| CN110097543A (en) * | 2019-04-25 | 2019-08-06 | 东北大学 | Surfaces of Hot Rolled Strip defect inspection method based on production confrontation network |
| CN110363122A (en) * | 2019-07-03 | 2019-10-22 | 昆明理工大学 | A kind of cross-domain object detection method based on multilayer feature alignment |
Non-Patent Citations (1)
| Title |
|---|
| 何剑华 ; 龙法宁 ; 朱晓姝 ; .基于改进的CycleGAN模型非配对的图像到图像转换.玉林师范学院学报.2018,(02),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111126446A (en) | 2020-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111126446B (en) | Method for amplifying defect image data of robot vision industrial product | |
| CN112329588B (en) | Pipeline fault detection method based on Faster R-CNN | |
| CN104331712B (en) | A kind of alga cells classification of images method | |
| CN111080622A (en) | Neural network training method, workpiece surface defect classification and detection method and device | |
| CN112734691A (en) | Industrial product defect detection method and device, terminal equipment and storage medium | |
| CN112766136B (en) | Space parking space detection method based on deep learning | |
| CN110705630A (en) | Semi-supervised learning type target detection neural network training method, device and application | |
| CN111401358B (en) | Instrument dial correction method based on neural network | |
| CN111738367B (en) | Part classification method based on image recognition | |
| CN111161213B (en) | Industrial product defect image classification method based on knowledge graph | |
| CN113392930B (en) | Traffic sign target detection method based on multi-level division control network | |
| CN113506239B (en) | Strip steel surface defect detection method based on cross-stage local network | |
| CN112651989A (en) | SEM image molecular sieve particle size statistical method and system based on Mask RCNN example segmentation | |
| CN114863464B (en) | Second-order identification method for PID drawing picture information | |
| CN112819748A (en) | Training method and device for strip steel surface defect recognition model | |
| CN113096085A (en) | Container surface damage detection method based on two-stage convolutional neural network | |
| CN113362277A (en) | Workpiece surface defect detection and segmentation method based on deep learning | |
| CN116205876A (en) | Unsupervised notebook appearance defect detection method based on multi-scale standardized flow | |
| CN116029979A (en) | Cloth flaw visual detection method based on improved Yolov4 | |
| CN114187247A (en) | Ampoule bottle printing character defect detection method based on image registration | |
| CN111738310B (en) | Material classification method, device, electronic equipment and storage medium | |
| CN114065798A (en) | Visual identification method and device based on machine identification | |
| CN114861801A (en) | Automobile part detection method, system, equipment and storage medium based on AI identification | |
| CN113901947A (en) | Intelligent identification method for tire surface flaws under small sample | |
| CN105023016A (en) | Target sensing method based on compressed sensing classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |