KR102580131B1 - Cycle generative adversarial network conditional gen erative adversarial network - Google Patents

Abstract

A cyclically generated adversarial network conditional generated adversarial network technology is disclosed. According to one embodiment, an unsupervised meta-learning method performed by an unsupervised meta-learning system includes randomly selecting a plurality of images from image samples; And it may include performing unsupervised meta-learning using UMTRA-based CycleGAN for each of the selected plurality of images.

Description

CYCLE GENERATIVE ADVERSARIAL NETWORK CONDITIONAL GEN ERATIVE ADVERSARIAL NETWORK}

The explanation below is about unsupervised meta-learning techniques.

Unsupervised meta-learning was first proposed in 2019. Meta-learning is about how machines learn. In meta-learning, both the training set and the test set are tasks, not data. Images are both training and testing tasks for supervised meta-learning. For each small task, the training set is called the support set, and the test set is called the query set. This is an essential thing in meta-learning.

The characteristics of unsupervised meta-learning are close to supervised meta-learning. UMTRA simply expands the image, using the original image as the support set and the expanded image as the query set. While supervised meta-learning uses two completely different images of the same type as the support set and query set, UMTRA simply extends the images for training.

However, because the difference between the two images is not large, image recognition errors increase and a lot of training is needed to reduce the errors. As such, simple image augmentation is no longer satisfactory for small-sample unsupervised meta-learning and testing.

An unsupervised meta-learning method and system using UMTRA-based Cycle GAN can be provided.

An unsupervised meta-learning method performed by an unsupervised meta-learning system includes randomly selecting a plurality of images from image samples; And it may include performing unsupervised meta-learning using UMTRA-based CycleGAN for each of the selected plurality of images.

The augmenting step may include performing unsupervised meta learning for few-shot image classification using UMTRA.

The augmenting step may include augmenting the image by enlarging the original image with a gap between the original images using CycleGAN.

The augmenting step may include executing MAML for unsupervised meta learning of the augmented image and the original image.

The augmentation step classifies the selected image into categories, augments the image using CycleGAN for each image classified by category, uses the augmented image as a meta-learning query, and uses the original image as a support set. It may include steps to use.

The unsupervised meta-learning system includes a sampling unit that randomly selects a plurality of images from image samples; And it may include a meta-learning unit that performs unsupervised meta-learning using UMTRA-based CycleGAN for each of the selected plurality of images.

Unsupervised meta-learning samples that perform image augmentation using UMTRA-based CycleGAN can be provided.

By using UMTRA-based CycleGAN, important features of images and accuracy when images are recognized can be improved. Additionally, since the image has larger gaps than the original image while maintaining important features, it can be more accurately identified through machine learning.

Figure 1 is a diagram for explaining UMTRA operation, according to one embodiment.
Figure 2 is an example to explain the operation of enlarging an image using CycleGAN in one embodiment.
Figure 3 is a block diagram for explaining the configuration of an unsupervised meta-learning system according to an embodiment.
Figure 4 is a flowchart illustrating an unsupervised meta-learning method in an unsupervised meta-learning system according to an embodiment.
Figure 5 is a flowchart for explaining an unsupervised meta-learning operation in an unsupervised meta-learning system according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

Figure 1 is a diagram for explaining UMTRA operation, according to one embodiment.

The difference between unsupervised and supervised learning is that the data set used in supervised meta-learning has labels, whereas the data used in unsupervised meta-learning does not have labels. The purpose of unsupervised meta-learning is to make the prediction result closer to the supervised meta-learning result. For this purpose, UMTRA (Unsupervised Meta-Learning for Few-Shot Image Classification) was proposed.

UMTRA's meta-learning process can be performed from unlabeled images. These images can be grouped into a set of different classes, assumed to be related to the target task, but no explicit information about the classes or labels is required. UMTRA can use random sampling and augmentation to generate synthetic training tasks for meta-learning processes. Labels are only needed during the final target task learning process and can be as small as one sample per class.

UMTRA can be started from an unlabeled data set. UMTRA must be drawn from the same distribution as the objects classified in the target task, and must have a class set that is much larger than the number of classes in the final classifier. Starting from an unlabeled data set, you can use statistical diversity features and domain-specific augmentation to generate training and validation data for a suite of synthetic tasks. The generated training data and validation data can then be used in a meta-learning process based on a modified classification variant of the MAML algorithm. We need to create tasks from unsupervised data that play the same role as meta-learning tasks in MAML algorithms. Therefore, both training sets and validation data must be created.

In UMTRA, training data can be generated by randomly selecting N samples from a data set. Validation data can be generated as an augmentation function is applied to each sample of training data. For both MAML and UMTRA, artificial temporary labels (1, 2,??, N) can be used. If two data points have the same label, they should have the same artificial temporary label, while if they have different labels, they should have different artificial temporary labels.

UMTRA always performs one-shot learning with a specific value (e.g., K=1) during the meta-learning process. During the learning process, certain values may be set different from 1. We will now explain how the structure of the training data satisfies the class classification conditions. The first condition is met because there is only one sample for each label. The second condition can be satisfied statistically by the fact that the total number of classes in the data set is If the number of samples is significantly smaller than the number of classes, it is likely that all samples are drawn from different classes. UMTRA must generate appropriate validation data for the synthetic work. The minimum requirement for validation data is that it be properly labeled in the given context. This means that the synthetic numeric label must map to the same class in the unlabeled dataset in both cases.

UMTRA can augment samples used in training data and generate augmented samples using an augmentation function, which is a hyperparameter. In UMTRA, an augmentation function can be constructed that verifies properties for a given data set based on what is known about the domain described by the data set. These augmentation functions must maintain class membership and can apply types of domain-specific changes to images or videos. For example, you can set some pixel values to 0 in an image, or convert pixels in a training image to a specific range (between -6 and 6).

The unsupervised meta-learning system can randomly select a plurality of images from image samples and augment the image data using UMTRA-based CycleGAN for each of the selected plurality of images.

In the embodiment, CycleGAN can be used for image augmentation. CycleGAN is a model in which a condition is added to maintain the shape of the original image to the GAN structure in which a generator and a classifier compete and learn. Enables image conversion even if two different images do not match. In other words, it has the feature of being able to convert an image even without a set of data forming a perfect pair.

CycleGAN can be used for image augmentation based on enlarging the original images with large gaps between them. Figure 2 is an example to explain the operation of image augmentation using CycleGAN in one embodiment. In embodiments, the image enlargement method may include increasing the size of the image as well as increasing the amount of the image. For example, as shown in Figure 2, if there are images of apples and oranges, commonalities between the two images can be found and converted by applying a style to another image.

Originally, small-sample unsupervised meta-learning used UMTRA, which converts an image into two images through simple augmentation of the image, using one image for training and the other image for testing. In the embodiment, the image augmentation function, which is more advanced than simple image augmentation, is replaced with CycleGAN, resulting in an image with a larger gap than the original image while maintaining important features, allowing for more accurate discrimination through machine learning.

FIG. 3 is a block diagram for explaining the configuration of an unsupervised meta-learning system according to an embodiment, and FIG. 4 is a flowchart for explaining an unsupervised meta-learning method in an unsupervised meta-learning system according to an embodiment.

The processor of the unsupervised meta-learning system 100 may include a sampling unit 310 and a meta-learning unit 320. These processor components may be expressions of different functions performed by the processor according to control instructions provided by program codes stored in the unsupervised meta-learning system. The processor and its components may control the unsupervised meta-learning system to perform steps 410 to 420 included in the unsupervised meta-learning method of FIG. 4. At this time, the processor and its components may be implemented to execute instructions according to the code of an operating system included in the memory and the code of at least one program.

The processor may load the program code stored in the file of the program for the unsupervised meta-learning method into memory. For example, when a program is executed in an unsupervised meta-learning system, the processor can control the unsupervised meta-learning system to load program code from the program file into memory under the control of the operating system. At this time, the sampling unit 310 and the meta learning unit 320 are each different functional expressions of the processor for executing the subsequent steps 410 to 420 by executing instructions of the corresponding portion of the program code loaded in the memory. You can.

In step 410, the sampling unit 310 may randomly select a plurality of images from image samples.

In step 420, the meta learning unit 320 may perform unsupervised meta learning using UMTRA-based CycleGAN for each of the plurality of selected images. The meta learning unit 320 can perform unsupervised meta learning for few-shot image classification using UMTRA. The meta learning unit 320 can augment the image by enlarging the original image with a large gap between the original images using CycleGAN. The meta learning unit 320 may execute MAML for unsupervised meta learning of the augmented image and the original image. The meta learning unit 320 classifies the selected image into categories, augments the image using CycleGAN for each image classified by category, uses the augmented image as a meta learning query, and uses the original image as a support set. there is.

Figure 5 is a flowchart for explaining an unsupervised meta-learning operation in an unsupervised meta-learning system according to an embodiment.

The unsupervised meta-learning system can randomly select multiple images (510). By randomly sampling a plurality of images, each image can be classified 520 into individual categories, which is an N-way-1-shot task. It is called an n-way-one-shot task depending on the number of classes to be classified (n) when evaluating after learning. For example, if there are 9 classes to classify in an image, it becomes 9-way-one-shot-task. After the network training is completed, testing can be conducted by creating a test image. Only one image of the same class is included in the support set, and images of different classes are included in the rest.

The unsupervised meta-learning system can perform image augmentation using CycleGAN for each image (530). An unsupervised meta-learning system can generate a new image from a selected image through an image enlargement method using CycleGAN. When using CycleGAN, key points can be better captured during machine learning because the change between before and after images is greater than with typical augmentation methods.

An unsupervised meta-learning system can use the augmented image as a query set 541 for meta-learning, and use the original image with n-way-1-shot and N labels as a support set 540. In this way, n-way-1-shot support + query data and N labels may be included.

The unsupervised meta-learning system can execute the MAML algorithm for learning (550). MAML means learning meta parameters to learn from few shots. Parameters can be updated based on external losses.

In this way, the unsupervised meta-learning system can make UMTRA closer to supervised meta-learning by making the difference between the image generated through CycleGAN and the original image clear.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. It can be embodied in . Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (6)

In the unsupervised meta-learning method performed by an unsupervised meta-learning system,
randomly selecting a plurality of images from the image sample; and
Performing unsupervised meta learning using UMTRA-based CycleGAN for each of the selected plurality of images
Including,
The step of performing the unsupervised meta learning is,
Each of the selected plurality of images is classified into categories, and for each image classified by category, the images are augmented by enlarging the original images with intervals between the original images using CycleGAN, and the augmented images are performing unsupervised meta-learning for few-shot image classification via UMTRA using as a meta-learning query and the original image as a support set, thereby bringing the UMTRA closer to supervised meta-learning.
Including,
The CycleGAN finds commonalities between the plurality of selected images and creates a new image by applying a different style to the original image.
An unsupervised meta-learning method including. delete delete According to paragraph 1,
The step of performing the unsupervised meta learning is,
Executing MAML for unsupervised meta learning of the augmented image and the original image
An unsupervised meta-learning method including. delete In an unsupervised meta-learning system,
a sampling unit that randomly selects a plurality of images from image samples; and
A meta-learning unit that performs unsupervised meta-learning using UMTRA-based CycleGAN for each of the selected plurality of images.
Including,
The meta learning department,
Each of the selected plurality of images is classified into categories, and for each image classified by category, the images are augmented by enlarging the original images with intervals between the original images using CycleGAN, and the augmented images are performing unsupervised meta-learning for few-shot image classification over UMTRA using ,
The CycleGAN finds commonalities between the plurality of selected images and creates a new image by applying a different style to the original image.
Unsupervised meta-learning system.