KR102261869B1 - Artificial Intelligence rapid image generation method using Meta-learning Generative adversarial network - Google Patents

Abstract

The present invention relates to a method for generating an image of artificial intelligence using a meta-learned generative adversarial network. The present invention consists of a generator that learns and generates the image and a discriminator that distinguishes the image generated by the generator, and a GAN that repeatedly updates the generator and the discriminator and the updated GAN It includes an mtGAN made by meta-updating learning, and iteratively training the learned mtGAN to generate a desired image.

Description

Artificial Intelligence rapid image generation method using Meta-learning Generative adversarial network}

The present invention relates to a rapid image generation method of artificial intelligence using meta-learned generative adversarial networks (neural networks).

Today, the development of artificial intelligence (AI) is endless, and recently, continuous technology is being developed for research called Generative Adversarial Network (GAN), or generative adversarial neural network (or network) in Korean.

As the name of the generative adversarial neural network (or network) above, it learns and produces results through the confrontation between two neural network models and the resulting competition. The two models have opposing purposes, called 'Generator' and 'Discriminator'.

The generator learns the real data and generates false data based on it. Ultimately, the goal is to generate false data close to the real one.

And, the discriminator learns to determine whether the data provided by the generator is real or false. The purpose is not to play with the false data of the constructor.

The creator of GAN, Ian Goodfellow, compared the generator to a counterfeiter and the discriminator to the police. For example, the generator learns data that has not been deceived by the discriminator, and the discriminator receives and learns data deceived by the generator. As this process is repeated, it is possible to create false data that is closer to the real thing, just as counterfeit bills become more sophisticated.

1A is a diagram showing the principle of the learning process of the GAN, and FIG. 1B is a diagram schematically illustrating the concept of the GAN. As shown in Figures 1a and 1b, while G (Generator) tries to create a more realistic image that can fool D (Discriminator), D is constantly trying to distinguish the image generated by G from the real image. to adjust the parameters. From a game theory point of view, these networks compete with each other in a zero-sum game.

Since the first GAN paper was published in 2014, various follow-up studies have been published. The reason why GAN is attracting attention as a next-generation deep learning algorithm in academia is that it has broken away from the existing supervised learning method and laid the foundation for unsupervised learning. Most AI research is supervised learning, in which a person tells the right answer. It is necessary to process the data in a way that AI can learn, such as tagging whether the image is a cat or a dog. Behind the world of AI, human manual labor of labeling data one by one is taking place.

As such, the limitations of the supervised learning method are that large amounts of data cannot be processed without a purification process, and human intervention is required in this process. On the other hand, GANs learn by themselves in the process of competition even if humans do not tell them the correct answer. The ripple effect is greater because AI learns large amounts of data by itself, and it is significantly different from other supervised learning algorithms in that it directly creates images or voices through generative models.

Korean Patent Publication No. 2001-0087974 Korean Patent Publication No. 2007-0067484 Korean Patent Publication No. 2018-0130511

An object of the present invention is to provide a fast image generation method of artificial intelligence using a meta-learned generative adversarial network in which a gradient-based meta-learned generative adversarial network (GAN) is proposed to achieve arbitrary image generation.

In addition, it is intended to provide a fast image generation method of artificial intelligence using a meta-learned generative adversarial network to implement image generation, which is a method of generating good samples with a small number of training.

In order to achieve this object, the present invention provides a method for generating an image of artificial intelligence using a meta-learned generative adversarial network, the steps of training a GAN as a model for changing an image, and repeated training for updating the trained GAN again and meta-updating the iteratively trained GAN in the above step.

In addition, the GAN consists of a generator that learns and generates an image and a discriminator that distinguishes the image generated by the generator, and the generator and the discriminator are each repeatedly updated. will be..

로 정의되며,

는 i 작업동안 변수에 대한 손실이며,

는 각 작업동안 k-스텝 업데이트된 변수이며,

는 업데이트 되는 메타-변수의 스텝 크기이며, T는 작업의 수인 것을 특징으로 한다.In addition, the learning equation of meta-update in the above step is

is defined as

is the loss for the variable during i operation,

is the k-step updated variable during each operation,

is the step size of the meta-variable being updated, and T is the number of operations.

In addition, the training is characterized in that the variable is adjusted after the input and the result is compared to the desired result when the result is different, and when the desired error is reached while repeating the comparison, the training is terminated.

Therefore, the rapid image generation method of artificial intelligence using the meta-learned generative adversarial network proposed by the present invention allows the meta-learned GAN to gradually change to an image of a desired color and size, as well as convert various types of images. It has the effect of being able to work quickly and accurately.

1A is a diagram showing the principle of the learning process of GAN.
1B is a diagram schematically illustrating the concept of a GAN.
2 is a diagram of the proposed MAML-styled mtGAN model.
3 is an algorithm for implementing a meta-learned GAN (mtGAN).
4 is an algorithm for implementing a meta-learned GAN (mtGAN).
5 is a flowchart of a generating method according to the present invention.
6 is a photograph showing a circle trained as a yellow circle through mtGAN on a meta-trained circle.
7 is a photograph showing the results of training to generate a dark-haired person with blonde hair color through mtGAN.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. First, in adding reference numerals to the components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated on different drawings.

In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

First, before describing the present invention, a generative adversarial network (GAN) that appears frequently in the specification will be described in detail.

Generative adversarial networks (GANs) are one of the most popular models in machine learning as a whole, as well as in generative model networks.

The generative adversarial network (GAN) is a kind of artificial intelligence (AI) algorithm having a deep neural network structure. The GAN technology, a deep learning technology, looks very realistic to the human eye and generates an image with realistic characteristics. help you do

Images that are nearly identical to photographs are now used in a variety of industrial design fields, such as shoes, bags, or computer game scenes. The GAN models motion patterns of moving images, reconstructs 3D models of objects from images, and creates astronomical images.

Also, GAN returns an image by first taking a random number. Images are composed of information derived from real data sets and the images are sent to a discriminator. The discriminator takes both the real image and the fake image and calculates the probability.

The deep neural network, a technology applied to the neural network that is the basis of GAN, is a technology that succeeds the genealogy of artificial neural networks, and the deep neural network used in most machine learning has various examples. Recently, the development of a new learning technology is being proposed to solve the problem of high cost and time-consuming GNA technology.

Hereinafter, with reference to the drawings, an artificial intelligence rapid image generation method using the meta-learned generative adversarial network of the present invention will be described.

2 is a diagram of the proposed MAML-styled mtGAN model, FIG. 3 is an algorithm for implementing a meta-learned GAN (mtGAN), FIG. 4 is an algorithm for implementing a meta-learned GAN (mtGAN), and FIG. 5 is a flowchart of an image generation method according to the present invention, FIG. 6 is a photograph showing a circle trained as a yellow circle through mtGAN in a meta-learned circle, and FIG. 7 is a black-haired person with blonde hair color through mtGAN It is a picture showing the result of training to generate .

For reference, it should be clarified in advance that the terms 'learning' or 'learning' appearing in this specification are terms that refer to performing machine learning through computing according to a procedure.

The core of the present invention is to combine two different GANs with each other. To explain this, the GAN (Generative Adversarial Network) is a generator (G: Generator) that learns and generates an image and an image generated by the generator. It consists of a discriminator (D: Discriminator) that distinguishes The generator G includes an encoder including a plurality of convolution layers and a decoder including a plurality of deconvolution layers, and the discriminator D includes a plurality of deconvolution layers. It is characterized in that it consists of an encoder including a deconvolution layer and a sigmoid.

That is, a generator (G) that learns a model that converts an arbitrary image into another image, and a classifier (D) that learns a model that distinguishes an image generated by the generator and a sample image representing the generated image will be.

를 n개의 클래스 중 하나로 분류하는 것이고, 이미지의 생성은 신속히 샘플을 생성하는 문제이다. 상기 식은

으로 정의된다. After all, it consists of the classification of an image and the creation of an image, in general, the classification of the image is generally

is to classify into one of n classes, and the generation of images is a matter of generating samples quickly. the above formula

is defined as

In the process of the generation, the GAN must be trained for an extensive training phase. Therefore, there is a need for image generation, which is a method of generating plausible samples with a small number of training examples.

Since the number of training examples required for meta-parameters determined by gradation-based meta-training to achieve the image can be reduced, a meta-learning framework as shown in FIG. 2 is proposed and used for image generation. Therefore, the present invention proposes a GAN as a meta-learned GAN (mtGAN) for quick adaptation.

FIG. 2 is a diagram of a proposed Model-agnostic meta learning (MAML)-styled mtGAN model, in which the GAN is trained during each task and stopped after k-step (the number of steps in G to be updated in FIG. 2).

를 최소화하기 위해 discriminator(D)와 generator(G)를 업데이트 한다. Then, the predicted GAN loss

We update the discriminator(D) and generator(G) to minimize

As shown, after updating the discriminator (D) and the generator (G) so that the loss of the discriminator (D) and the generator (G) is minimized in individual blocks within each step, the overall loss is minimized through meta-update before proceeding to the next step. The update will be executed again so that

In the above process, gradient-based meta-learning finds the meta-parameter optimization loss for the task as shown in the following equation. That is, a desired image is generated by repeatedly training the learned mtGAN.

는 i 작업동안 변수에 대한 손실이며,

는 각 작업동안 k-스텝 업데이트된 변수이며,

는 업데이트 되는 메타-변수의 스텝 크기이며, T는 작업의 수이다.here

is the loss for the variable during i operation,

is the k-step updated variable during each operation,

is the step size of the meta-variable being updated, and T is the number of operations.

3 is an algorithm for implementing a meta-learned GAN (mtGAN). The algorithm was configured to implement the block diagram of FIG. 2 .

4 is an algorithm for implementing a meta-learned GAN (mtGAN). The algorithm was configured to implement the Model-agnostic meta learning (MAML)-styled mtGAN of FIG. 3 in a simple manner. The meta-learning method of the mtGAN is characterized in that it is MAML (model agnostic meta learning)-STYLED.

Hereinafter, with reference to FIG. 5, a method for rapidly generating an artificial intelligence image using a meta-learned generative adversarial network according to the present invention will be described step by step.

First, we train the GAN during each task as a transforming model that transforms the image of A into the image of B. (Step 1)

The images A and B in the first step are described with reference to the drawings. In the case of FIG. 6, the image of A called the green circle means the image of B called the yellow circle, and in the case of FIG. 7, it is the black hair in the photo. It means changing image A to image B of blonde. In order to achieve this objective, the generator G and the discriminator D constituting the GAN are repeatedly trained alternately. As shown in Fig. 2, during each operation, the discriminator (D) and the generator (G), which are components of the GAN, are trained. And, the meaning of the model refers to G of a Generative Adversarial Network (GAN), which means that a new image is created.

Repeat the update of the trained GAN. (Step 2)

In the first step of the update, the discriminators D are updated in a direction to minimize the loss of the discriminators D, and in the next step, the total generators G determined by the updated discriminators D are updated. By minimizing the loss, the generators G are updated. That is, at first, the generator G and the discriminator D are updated N times, and the process of generating the N+1th update to be satisfied by the user is repeated N times, so that the 2Nth update is optimally adopted.

The next step is to meta-update the GAN trained repeatedly in the second step. (Step 3)

(

는 i 작업동안 변수에 대한 손실이며,

는 각 작업동안 k-스텝 업데이트된 변수이며,

는 업데이트 되는 메타-변수의 스텝 크기이며, T는 작업의 수)으로 정의된다. I've explained it before. The learning formula of the meta-update

(

is the loss for the variable during i operation,

is the k-step updated variable during each operation,

is the step size of the meta-variable being updated, and T is defined as the number of operations).

등)를 변경하여 조정하는 과정을 의미하는 것으로, 입력되어 변경시킨 변수가 결과 값이 나왔을 때 원하는 결과와 어느 정도 차이가 있는지 비교하며, 상기 비교를 반복하면서 원하는 오차(원하는 이미지에 가장 근접하게 접근할 때를 의미)가 되면 종료하는 것이다.In the third step, training is performed by inputting several data (ie, images) and internal variables (ie,

It refers to the process of adjusting by changing and adjusting the inputted and changed variables, and comparing how much difference there is from the desired result when the inputted and changed variable comes out, and repeating the comparison, while repeating the comparison, the desired error (the closest approach to the desired image) When it is time to do it), it ends.

That is, in the process of learning by GAN, G and D update variables in order to achieve their respective goals. After sufficient learning, G creates an image similar to the real one, and discriminates by meta-variables. It has been found that the rate theoretically converges to almost zero. In this way, by improving the GAN, it is possible to generate images of various classes (types) corresponding to one image.

Explaining this with reference to the drawings, FIG. 6 is circles trained in yellow through mtGAN in meta-learned Circles, and the number at the bottom of FIG. 6 represents the number of trained steps (steps: step), and progressive You can see that it is displayed in the desired color and in various sizes of circles.

7 is a photograph showing the results of training to generate a dark-haired person with blonde hair color through meta-learned mtGAN. The number at the bottom of FIG. 7 indicates the trained stage. As described above, the training is repeated over and over again until the result of the desired image is almost identical to the result of the training.

That is, in the case of FIG. 6, the size and color from a green circle to a yellow circle, and in the case of FIG. 7, from black hair to blonde hair, which is a desired color, when generated gradually approaching, training (training) is performed. it will end

Therefore, the image generating method according to the present invention requires a lot of learning and experience (deep learning) in order for the AI (artificial intelligence) to change and newly create an image, etc., but due to the above-described learning method, sufficient training ( training), so it has the advantage of being able to create almost similarly and diversely in the form you want to transform the image creatively with this repeated training and learning.

The image generating method of artificial intelligence using the meta-learned generative adversarial network of the present invention described as described above can also be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored.

That is, the computer-readable recording medium includes a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.) and an optical readable medium (eg, a CD-ROM, a DVD, etc.). In addition, the computer-readable recording medium may be distributed in a network-connected computer system to store and execute computer-readable codes in a distributed manner.

As described above, the above-described contents are merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains may vary within the scope not departing from the essential characteristics of the present invention. Modifications, changes and substitutions may be made.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are for explaining, not limiting, the technical spirit of the present invention, and the scope of the technical spirit of the present invention is not limited by these embodiments and the accompanying drawings. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims (4)

The generative adversarial network (GAN) consists of a generator (G) that learns and generates an image and a distinguisher (D) that distinguishes the image generated by the generator, and the generator (G) includes a plurality of convolutions. ) consists of an encoder including a layer and a decoder including a plurality of deconvolution layers, and the discriminator D includes a plurality of deconvolution layers and a sigmoid. In the image generation method of artificial intelligence using a meta-learned generative adversarial network (GAN) consisting of an encoder comprising:

a first step of training the GAN as a model for changing the image;
a second step of repeatedly training for re-updating the GAN trained in the first step;
It includes a third step of meta-updating the GAN trained repeatedly in the second step,
In the first step, the generator (G) and the discriminator (D) constituting the GAN are repeatedly trained alternately and stop after k-steps (the number of steps of the generator G to be updated in FIG. 2),
In the second step, the GAN loss predicted in the first step

Update the discriminator D and the generator G to minimize the discriminator D, but at first the discriminator D is updated in a direction to minimize the loss of the discriminator D, and in the next step, the updated discrimination The generators G are updated by minimizing the loss of the total generators G determined by the generators D, and first, the generator G and the discriminator D are updated N times, and N By repeating the process of generating the + 1st update N times, performing the 2Nth update to optimally adopt the GAN,
The learning equation of meta-update for meta-parameter optimization loss calculation in the third step

is defined as

is the loss for the variable during i operation,

is the k-step updated variable during each operation,

is the step size of the meta-variable being updated, T is the number of operations,
In the third step, training is performed by inputting several data (that is, images) to generate the desired result by the artificial intelligence module (not shown).

) to change and adjust, the input and changed variable compares the result value with the desired result (the image closest to the preset image), and repeats the comparison with the desired error (closest to the desired image). An image creation method of artificial intelligence using a meta-learned generative adversarial network, characterized in that it terminates when it is approached delete delete delete

Images