CN110992334A

CN110992334A - Quality evaluation method for DCGAN network generated image

Info

Publication number: CN110992334A
Application number: CN201911200153.XA
Authority: CN
Inventors: 李潇
Original assignee: Shenzhen Yijiaen Technology Co ltd
Current assignee: Homwee Technology Co ltd
Priority date: 2019-11-29
Filing date: 2019-11-29
Publication date: 2020-04-10
Anticipated expiration: 2039-11-29
Also published as: CN110992334B

Abstract

The invention relates to the field of image processing, and discloses a quality evaluation method for DCGAN network generated images, which is used for improving the accuracy of the quality evaluation of the DCGAN network generated images. According to the method, the pictures generated by the DCGAN network are used as the input of the DCGAN network for repeated iteration, the pictures are stored for one time in an equal interval manner in the iteration process, and a part of pictures with good quality are selected from the pictures stored for each time; then, labeling the pictures, simultaneously taking out a part of the pictures from the original pictures and labeling the pictures, and mixing the pictures together in equal proportion; then, obtaining a qualified classifier through mixed picture set training; inputting the mixed picture set into a DCGAN network, generating a certain number of pictures x, and putting the pictures x into a classifier to classify the pictures x, thereby obtaining a multi-dimensional vector y and a probability p (y) thereof; and finally, obtaining a quality evaluation result of the DCGAN network generated image based on the probability p (y). The method is suitable for the quality evaluation of the image generated by the DCGAN network.

Description

Quality evaluation method for DCGAN network generated image

Technical Field

The invention relates to the field of image processing, in particular to a quality evaluation method for DCGAN network generated images.

Background

GAN is collectively referred to as a Generative adaptive Networks, meaning a Generative countermeasure network. The original GAN is an unsupervised learning method, which skillfully utilizes the thought of 'antagonism' to learn a generative model, and can generate a brand new data sample once training is completed. DCGAN expands the concept of GAN into a convolutional neural network, and can generate picture samples with higher quality.

The generation type confrontation network is the most popular image generation method at present, various GAN networks are also in endless, the quality of generated pictures is higher and higher, but currently, the judgment methods for the quality of the pictures generated by the confrontation network are not many, people usually judge whether a confrontation network is generated or not according to the quality of the finally generated pictures, but most of the confrontation networks are qualitatively and subjectively judged by a visual observation method to judge the difference between the generated pictures and real pictures. The most popular quantitative evaluation method at present IS an IS and FID discrimination method, the IS can only discriminate the diversity of the generated image, the generated sample and the real sample are not compared, and the method has certain defects, while the FID value also depends on the inclusion Net of ImageNet and training, the inclusion activation value between the real image and the generated image IS compared, and the comparison makes the activation values of the real image and the generated image approximate to gaussian distribution, and the improvement of detail change cannot be clearly explained.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: a quality evaluation method for DCGAN network generated images is provided to improve the accuracy of the quality evaluation of the DCGAN network generated images.

In order to solve the problems, the invention adopts the technical scheme that: the quality evaluation method for the DCGAN network generated image comprises the following steps:

step 1: repeatedly iterating the picture generated by the DCGAN network as the input of the DCGAN network until the iteration times reach a threshold value M times, wherein M is more than 1, and the initial input of the DCGAN network is an original picture prepared by a user; in the iteration process, outputting and storing a picture after each iteration is carried out for N times, wherein M is an integral multiple of N; after iteration is completed, a part of pictures with better quality are selected from the pictures stored each time and are used for subsequent picture mixing, wherein the pictures stored each time can be sorted from high to low according to quality during selection, and the part of the pictures sorted in the front is selected, such as the front 1/4 and the front 1/3;

step 2: respectively labeling the selected pictures in the step 1 with different labels, simultaneously taking out a part of the pictures from the original pictures and labeling, and then mixing the labeled pictures together in equal proportion to obtain a mixed picture set;

and step 3: inputting a part of the mixed picture set as a training set into a classifier to train the classifier, testing the classification precision of the classifier by using the rest part of the mixed picture set, and entering the step 4 when the classification precision meets the requirement;

and 4, step 4: inputting the mixed picture set into a DCGAN network, enabling the mixed picture set to generate a certain number of pictures which are called x, then putting x into the classifier obtained in the step 3 to be classified, thereby obtaining a multidimensional vector y and the probability p (y) of the vector y, wherein the value of each dimension of the vector y corresponds to the probability p (y | x) that x belongs to various pictures, and obtaining the quality evaluation result of the images generated by the DCGAN network based on the probability p (y | x) and the probability p (y).

Further, after the probabilities p (y) and p (y | x) are obtained in step 4, the quality evaluation result of the DCGAN network generated image can be obtained by calculating the divergence of p (y) and p (y | x), wherein the smaller the divergence is, the better the quality evaluation result of the DCGAN network generated image is.

Further, a preferred allocation manner of step 3 is: and inputting 90% of the mixed picture set into a classifier as a training set to train the classifier, and testing the classification precision of the classifier by using the remaining 10% of the mixed picture set.

Further, the last layer of the activation function of the generator of the DCGAN network preferably uses the tanh function. The reason for using the tanh function is that the image is output from the last layer, and the pixel value of the image has a range of values, such as 0 to 255. The output of ReLU function may be very large, while the output of tanh function is between-1 and 1, so long as the pixel values of 0 to 255 can be obtained by adding 1 to the output of tanh function and multiplying by 127.5.

The invention has the beneficial effects that: the invention adopts the idea of iteration and the idea in the IS method, and can well reflect the fact that the characteristic situation of loss of a GAN network under the iteration of a generation and a generation at the end on another level, and indirectly judge the quality of DCGAN from the other aspect. In addition, the present example is able to identify the over-fit problem well through iteration.

Drawings

FIG. 1 is a flow chart of an embodiment of the present invention.

Detailed Description

The existing evaluation means analyzes a network model too severely and analyzes results weakly, but a generative confrontation network is used, so that a better result can be obtained, the quality of the confrontation network can be judged, and the result can be started, so that the evaluation method can be used for evaluating the DCGAN, and the evaluation method is generated based on convolution characteristic extraction according to the DCGAN network, has similar principle and heredity, and can analyze the relation coefficient of the original data and the generated data to just become a judgment standard. The IS and FID values are based on the fact that the method of inclusion cannot distinguish the relationship between image quality and image diversity, in other words, the two methods only know whether the quality of the final picture IS good or bad, and do not know whether the reason for influencing the quality of the final picture IS related to the original data. When the factors influencing the DCGAN network are analyzed, the factors probably analyzing long parameters still cannot find the reason, and finally, the factors are found to be the factors with poor original training data. Our approach can be very good at avoiding this problem. Our method solves this problem with new methods of image classification based on accuracy, improves the IS method, and demonstrates the significant difference between the real image and the generated image. And the overfitting problem can be better distinguished aiming at the overfitting problem of DCGAN training. The inspiration of the invention comes from finding the son according to the appearance of the couple, and can also distinguish the parents according to the appearance of the son, because the picture has the pixel characteristics which can be reflected in the aspect.

The scheme of the invention is explained in detail below with reference to the accompanying drawings, and the specific scheme of the invention is as follows:

(1) the method mainly comprises the steps that the DCGAN selects a proper generator, a discriminator, a data set, an optimized loss function and training method, and a classifier parameter adjusting and optimizing mode to find the best classification effect. And the implementation is to implement DCGAN on a Pytorch.

(2) The structure of a generated model in the DCGAN is as follows: the generator first generates a 100-dimensional noise, which can be regarded as a 100 × 1 picture, and since the training data set is a 3 × 96 picture, it is said that the resolution of the final generated picture by the generator should also be 3 × 96. After five convolution layers, pictures with resolution of 1024 × 4 — >256 × 8 — >64 × 16 — >64 × 32 — >3 × 96 are output, and the fully-connected layers of the first four layers are all subjected to BatchNorm2d (ndf), and batch normalization is performed to calculate the mean and standard deviation of each dimension input in each small batch (mini-batch) data. gamma and beta are learnable size-C parameter vectors (C is the input size). During training, the layer calculates the mean and variance of each input and performs a moving average. The input standardization solidification learning process is carried out, the training efficiency can be improved, and the influence caused by poor initialization is reduced. The moving average defaults to a momentum value of 0.1. The activation function of the first four layers is activated by adopting a ReLU, and the output layer is activated by adopting a tanh function. The reason for using the tanh function is that the image is output from the last layer, and the pixel value of the image has a range of values, such as 0 to 255. The output of ReLU function may be very large, while the output of tanh function is between-1 and 1, so long as the pixel values of 0 to 255 can be obtained by adding 1 to the output of tanh function and multiplying by 127.5.

(3) The network structure of the inventive discriminator and the network of the generator are very similar, basically a symmetrical process, and all have five layers of networks, the first four layers are reduced in size by convolution of Conv2d two-dimensional convolution layer functions, the scale of the discriminator is the reverse process of the generator, and pictures with 3 × 96 ^ 64 × 32 ^ 64 × 16 ^ 256 × 8 ^ 1024 × 4 are respectively output through five convolution layers. And then, normalization operation is carried out, the first four layers are all activated by an activation function LeakyReLU, the last layer of output layer has no activation function, and the last layer of output layer is normalized to a number between 0 and 1 through a Sigmoid () function, and the probability that the picture is true is also shown.

(4) The method can adopt the cartoon head portrait data set as experimental data, and applies a deep learning algorithm Convolutional Neural Network (CNN) to extract the characteristics of the cartoon head portrait, and predicts and classifies the characteristic samples extracted by the CNN through a machine learning algorithm. The parameter adjusting and optimizing experiment of the SVM classifier is mainly completed in a machine learning classification algorithm, and 20 combinations of the kernel, the C and the gamma are compared and optimized respectively. And other machine learning classification methods are used as comparison experiments, which comprise: k nearest neighbor classification (KNN), Gaussian naive Bayes classification (GNB), extreme random tree classification (ET), random forest classification (RF), multi-layer perceptron classification (MLP), linear discriminant analysis classification (LDA), self-training increment net and the like. The comprehensive comparative evaluation of the classification efficiency and precision of each classifier is carried out in the experiment by using evaluation standards such as a t-SNE characteristic diagram, a confusion matrix, accuracy, recall ratio, F1 value and the like. The final classification accuracy is averaged to obtain a value p (w).

(5) As shown in fig. 1, the trained DCGAN network is debugged, 3 ten thousand pictures are also generated by training 3 ten thousand original cartoon characters (the number of the pictures can be adjusted according to the user requirement), and then the generated 3 ten thousand pictures are used as training data to enter the DCGAN network to generate 3 ten thousand pictures, so that the iteration is continuously performed. After every 5 iteration times (the iteration times can be adjusted according to the needs of users), storing and outputting the pictures once, iterating for 25 times, outputting 5 times of pictures totally, outputting 3 ten thousand pictures each time, manually picking out 1 ten thousand pictures with good quality, sorting the stored pictures each time according to the quality from high to low when picking the pictures, and picking the 1 ten thousand pictures with the top sorting.

(6) And then labeling the pictures output each time, taking out ten thousand pictures from the original data and labeling the pictures, and taking the pictures as six types. And then taking out 90% of the mixed six classes of sixty thousand pictures according to the proportion, using the 90% of the mixed pictures as a training set, inputting the training set into a classifier, using the remaining 10% of the training set as test data, testing the classification precision p (z) of the classifier, and entering the step (7) after the classification precision meets the requirement.

(7) Then, the mixed sixty thousand pictures are continuously input into a DCGAN network, 6000 pictures are generated and called x, the 6000 pictures are put into a classifier to be classified, a 6-dimensional vector y and the probability of the vector y are obtained, the value of each dimension of the vector y corresponds to the probability p (y | x) that the pictures belong to the original pictures and the 5 th, 10 th, 15 th, 20 th and 25 th iterations generate the pictures, and then the divergence between p (y) and p (y | x) IS calculated just like IS, of course, the smaller the divergence between the p (y) and the p (y | x), the better the divergence IS not the IS. But the final result does not have to be done with kl divergence, only two values of p (y) and p (y | x) are already very problematic, they are even somewhat mutually exclusive, i.e. one is good and the other is certainly bad.

(8) The present example can also find out the overfitting state better, because once overfitting occurs, the number of iterations is no matter how many, and finally the value of p (y) related to the original data is very close to 1, so the overfitting problem can be reflected well.

Claims

1. The quality evaluation method for the DCGAN network generated image is characterized by comprising the following steps:

step 1: repeatedly iterating the picture generated by the DCGAN network as the input of the DCGAN network until the iteration times reach a threshold value M times, wherein M is more than 1, and the initial input of the DCGAN network is an original picture prepared by a user; in the iteration process, outputting and storing a picture after each iteration is carried out for N times, wherein M is an integral multiple of N; after iteration is finished, selecting a part of pictures with better quality from the pictures stored each time for mixing subsequent pictures;

2. The method of claim 1, wherein the quality assessment of the DCGAN network generated image is obtained by calculating divergence of p (y) and p (y | x).

3. The method of claim 1, wherein the step 3 inputs 90% of the mixed image set as a training set into a classifier to train the classifier, and then tests the classification accuracy of the classifier using the remaining 10% of the mixed image set.

4. The quality assessment method for DCGAN network generated images of claim 1, wherein the activation function last layer of the generator of the DCGAN network uses a tanh function.