KR20200058297A - Method and device for explainable few-shot image classification - Google Patents

Abstract

Disclosed are a method and a device for classifying explainable few-shot images. According to the present invention, the device for classifying explainable few-shot images comprises: a generative adversarial network that is composed of a generator including a latent space generator generating latent variables for an actual image by using the actual image as input and an image generator generating a virtual artificial image by using a random latent variable as input, and a discriminator calculating and outputting a probability that the artificial image belongs to the actual image; and a few-shot training part that classifies whether a first image and a second image different from each other have the same entity by using the trained latent space generator, and generates an image for qualitative evaluation of few-shot training by combining latent variables for each of the first image and the second image.

Description

Method and device for explainable few-shot image classification}

The present invention relates to a method and apparatus for classifying a small number of shot images.

Few-shot learning is a model learning method that can solve a given problem in an environment where only a small amount of training data can be used, and has high difficulty in the field of artificial intelligence.

The problems that minor shot learning attempts to solve are very diverse, but recently, studies to classify images have been actively conducted.

Image classification is to classify objects (Classes) of images, such as objects, animals, people, and medical images, and the difficulty of the problem changes according to the quality, quantity, and number of objects.

Recently, image classification using machine learning showed a remarkable improvement in performance, but minor shot training has difficulty in improving performance because only a small amount of training data can be utilized.

Accordingly, it is possible to consider a method of generating a latent space projecting characteristics of data and hidden information and using it for learning a small number of shots.

However, studies for generating latent space in a small-shot learning environment are still incomplete, and latent space generated with a small number of data is likely to overfit.

During recent research, hostile adversarial networks (GANs) can generate more generalized latent spaces in the data seen first.

Accordingly, there is a need for research on a method of generating a more generalized latent space using a method such as a hostile generation neural network to increase the performance of learning a small number of shots.

Non-patent Document 1: [Li, F., Fergus, R., Perona, P., “Few-shot learning of object categories,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 28 , Issue 4, April 2006], learning in an environment in which a small number of learning data per class of an image to be classified exists exists is defined as minority shot learning. However, the number of data of objects that are not classified is not limited. For example, in a one-shot learning classifying cat images and dog images, one cat image, one dog image, and several other images may be used as learning data.

In order to overcome a very extreme number of data, Korean Patent Publication No. 10-2018-0120478 proposed a method for artificially generating data using hostile adversarial networks (GAN). However, the above document suggested only a data generation method and is not suitable for solving problems such as image classification.

1. [Li, F., Fergus, R., & Perona, P., “Few-shot learning of object categories,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 28, Issue 4, April 2006] 2. [Dumoulin, V., Belghazi, I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., & Courville, A., “Adversarially learned inference,” ICLR, 2017]

In order to solve the above-mentioned problems of the prior art, the present invention creates a latent space in which hidden information of data is projected, improves quantitative performance of learning a small number of shots through the generated latent space, and latent space Describes a method and apparatus for classifying a fractional shot image that can be explained and qualitative evaluation of fractional shot learning by decoding.

In order to achieve the above object, according to an embodiment of the present invention, as a descriptive fractional shot image classification apparatus, a potential space generator and a random potential generating a potential variable for the real image by using the real image as an input An adversary generating neural network consisting of a generator including an image generator for generating a virtual artificial image by taking a variable as input, and a discriminator for calculating and outputting a probability that the artificial image belongs to the real image; And classifying whether different first and second images have the same entity using the latent spatial generator where learning has been completed, and combining latent variables for each of the first and second images to synthesize a small number of shots. An explainable fractional shot image classification apparatus including a fractional shot learning unit generating an image for qualitative evaluation is provided.

The fractional shot learning unit may include a first potential space generator generating the first potential variable by using the first image as an input; A second potential space generator that generates the second potential variable by using the second image as an input; And a synthesizer that synthesizes the first latent variable and the second latent variable and outputs it as vector data.

In the synthesizer, the vector type data may be a third latent variable generated by synthesizing the first latent variable and the second latent variable while some elements are used for classification of the individual.

The fractional shot synthesizing unit may include an image generator that completes learning in the hostile neural network and generates an image using the third potential variable as an input.

According to another aspect of the present invention, a method for classifying a small number of shot images of a device including a processor and a memory, wherein the potential spatial generator of the hostile generating neural network inputs a real image to generate a latent variable for the real image ; Generating a virtual artificial image by inputting a random latent variable as an image generator of the hostile generating neural network; A discriminator of a hostile generating neural network calculating a probability that the artificial image belongs to the real image; Classifying whether different first images and second images have the same entity using the latent spatial generator having completed learning; And generating an image for qualitative evaluation of the learning of the minority shot by synthesizing potential variables for each of the first image and the second image.

According to the present invention, an approach to learning a small number of shots is presented to explainability, which is one of the biggest drawbacks of artificial intelligence.

The present invention has an advantage in that it is possible to perform qualitative evaluation that cannot be performed in general image classification by utilizing the latent space.

1 is a diagram illustrating a system for classifying a small number of shots according to a preferred embodiment of the present invention.
2 is a diagram showing the detailed configuration of the hostile generating neural network of the present invention.
3 is a diagram showing a detailed configuration of the generator according to the present embodiment.
4 is a diagram showing a detailed configuration of a minority shot learning unit according to the present embodiment.
5 is a diagram showing a detailed configuration of the classifier according to the present embodiment.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The present invention generates a latent space in which hidden information of data is projected, and through this, learns a small number of shots in image classification.

Specifically, a potential space is generated with hostile generating neural networks (GAN), and an entity variable and a hidden variable are respectively extracted from the generated potential space, and finally, a small number shot as an object variable It improves the quantitative performance of learning and improves the explanatory and qualitative performance of the model with hidden variables.

1 is a diagram illustrating a system for classifying a small number of shots according to a preferred embodiment of the present invention.

와

를 출력하는 적대적 생성 신경망(100) 및 학습이 완료된 적대적 생성 신경망(100)의 생성기를 이용하여 잠재 변수를 입력 받고 동일한 영상인지 분류하고 영상을 합성하는 소수샷 학습부(102)를 포함할 수 있다. Referring to FIG. 1, the system according to the present embodiment receives images X and Y and each latent variable in a potential space

Wow

It may include a small number shot learning unit 102 that receives the potential variable using the generator of the hostile generating neural network 100 outputting the output and the generator of the hostile generating neural network 100 that has completed learning, classifies whether it is the same image and synthesizes the image. .

2 is a diagram showing the detailed configuration of a hostile generating neural network of the present invention, and FIG. 3 is a diagram showing the detailed configuration of a generator according to the present embodiment.

Referring to FIG. 2, the hostile generating neural network may include a generator 200 and a discriminator 202.

Referring to FIG. 3, the generator 200 includes a latent space generator 300 and an image generator 302, and the latent space generator 300 receives a real image and generates latent variables corresponding to the real image, The image generator 302 may generate a virtual artificial image by receiving random latent variables .

Here, the potential space generator 300 and the image generator 302 share the same potential space . This is possible by using a method of making an image as a latent variable when learning each generator (300,302), and creating an image again with the corresponding latent variable.

That is, the generator 200 has a cycle consistency.

The discriminator 202 receives a virtual artificial image and a real image generated by the generator 200 and determines true / false of the input image.

At this time, the generator 200 is trained to generate a variable or image close to the real image through the latent space, and the discriminator 202 is trained to distinguish the artificial image generated close to the real image from the real image.

The discriminator 202 according to the present embodiment inputs data consisting of pairs of artificial images and random potential variables corresponding thereto and data consisting of pairs of potential variables of actual images output to the real image and potential space generator 300. To calculate the probability that the artificial image belongs to the real image.

4 is a diagram showing a detailed configuration of a minority shot learning unit according to the present embodiment.

As shown in FIG. 4, the fractional shot learning unit 102 according to the present embodiment may include a first latent space generator 400, a second latent space generator 402 and a classifier 404.

Here, the first latent spatial generator 400 and the second latent spatial generator 402 are included in the generator 200 of the hostile generating neural network 100 and are learning neural network networks.

를 생성하고, 제2 잠재 공간 생성기(402)는 영상 Y를 입력으로 하여 잠재 변수

를 생성한다. The first potential space generator 400 uses the image X as an input, and the latent variable

And the second potential space generator 402 inputs the image Y as a potential variable

Produces

및

는 분류기(404)로 입력된다. Latent variable

And

Is input to the classifier 404.

를 생성한다. The classifier 404 classifies whether the two inputted latent variables have the same entity, synthesizes the latent variables, and synthesizes new latent variables.

Produces

5 is a diagram showing a detailed configuration of the classifier according to the present embodiment.

와

를 입력 받고, 벡터형 데이터를 출력한다. Referring to Figure 5, the classifier 404 includes a synthesizer 500, specifically, the synthesizer 500 is a potential variable

Wow

Receives and outputs vector type data.

이다 .Here, some elements of vector data are used for classification of objects (equal / different), others are synthesized latent variables

to be .

를 적대적 생성 신경망(100)에서 학습이 완료된 영상 생성기(302)에 입력하면 새로운 영상 X'가 출력된다. Newly created latent variables

When input to the image generator 302 where learning is completed in the hostile generation neural network 100, a new image X 'is output.

Image X 'is defined as a composite image of image X and image Y, and is an image that visually checks whether images X and Y have the same object.

Specifically, it can be interpreted that there is a possibility of explanation because qualitative evaluation of small number shot learning is possible according to the quality and the amount of change of the image X '.

In addition, the weights of the latent spatial generator 300 and the image generator 302 that are learned in the hostile generating neural network during the minor shot training are maintained at a fixed value so as not to influence the learning.

The descriptive fractional shot image classification apparatus according to the present embodiment may include a processor and a memory. The processor may include a central processing unit (CPU) capable of executing computer programs or other virtual machines.

The memory may include non-volatile storage devices such as fixed hard drives or removable storage devices. The removable storage device may include a compact flash unit, a USB memory stick, and the like. The memory may also include volatile memory such as various random access memories.

According to an embodiment of the present invention, a potential image for the real image is generated by inputting a real image through the hostile generation neural network model in the memory, and a virtual artificial image is generated by using a random latent variable as the input. Calculate the probability that the artificial image belongs to the real image, classify whether the different first image and the second image have the same entity by using the latent spatial generator of the hostile generation neural network after learning, and classify the first image and the second image. Program commands for generating an image for qualitative evaluation of fractional shot learning by combining potential variables for each of the 2 images may be stored.

The above-described embodiments of the present invention have been disclosed for purposes of illustration, and those skilled in the art having various knowledge of the present invention will be able to make various modifications, changes, and additions within the spirit and scope of the present invention. It should be regarded as belonging to the following claims.

Claims (5)

Describeable fractional shot video classification device,
A generator including a potential space generator for generating a potential variable for the real image by using a real image and an image generator for generating a virtual artificial image by inputting a random latent variable, and the artificial image belongs to the real image. A hostile generating neural network consisting of a discriminator for calculating and outputting a probability; And
Using the latent spatial generator where learning is completed, classifying whether different first and second images have the same entity, and combining potential variables for each of the first and second images to determine the fractional shot learning Explainable fractional shot image classification device including a fractional shot learning unit that generates an image for evaluation. According to claim 1,
The minority shot learning unit,
A first potential space generator that generates the first potential variable by using the first image as an input;
A second potential space generator that generates the second potential variable by using the second image as an input; And
An explainable fractional shot image classification device including a synthesizer that synthesizes the first latent variable and the second latent variable and outputs the vector type data. According to claim 2,
The synthesizer,
The vector type data is a descriptive fractional shot image classification device in which some elements are used for classification of an entity, and others are third potential variables generated by synthesizing the first potential variable and the second potential variable. According to claim 3,
The fractional shot synthesis unit,
A descriptive fractional shot image classification device comprising an image generator that completes learning in the hostile neural network and generates an image using the third potential variable as an input. A method of classifying a small number of shot images of a device including a processor and a memory,
A potential spatial generator of a hostile generating neural network generating a potential variable for the real image by using the real image as an input;
Generating a virtual artificial image by inputting a random latent variable as an image generator of the hostile generating neural network;
A discriminator of a hostile generating neural network calculating a probability that the artificial image belongs to the real image;
Classifying whether different first images and second images have the same entity using the latent spatial generator having completed learning; And
And generating a video for qualitative evaluation of learning of a minority shot by synthesizing potential variables for each of the first and second images.

Images