WO2020170803A1

WO2020170803A1 - Augmentation device, augmentation method, and augmentation program

Info

Publication number: WO2020170803A1
Application number: PCT/JP2020/004113
Authority: WO
Inventors: 真弥山口; 毅晴江田; 沙那恵村松
Original assignee: 日本電信電話株式会社
Priority date: 2019-02-20
Filing date: 2020-02-04
Publication date: 2020-08-27
Also published as: JP2020135485A

Abstract

In the present invention, an augmentation device selects a prescribed dataset from an external dataset on the basis of a matrix calculated from the degree of similarity between a target dataset and the external dataset and from the degree of uniformity of the external dataset. The augmentation device performs learning of a generation model so that when a target class is designated for a generator, the generator generates data that is easily identified by an identifier as being genuine and that is calculated by a classifier so that the probability of belonging to a plurality of external classes is not zero and is uniform. The augmentation device designates a target class for the generator of the learned generation model, and generates data. The augmentation device extracts, from the generated data and as augmentation data, data identified as being fake by the identifier. The augmentation device affixes a target label to the extracted augmentation data.

Description

Expansion device, expansion method, and expansion program

The present invention relates to an expansion device, an expansion method, and an expansion program.

Preparation of learning data in the deep learning model requires a large cost. The preparation of learning data includes not only the collection of learning data but also the addition of annotations such as labels to the learning data.

Conventionally, rule-based data extension (Data Augmentation) has been known as a technique for reducing the cost of maintaining learning data. For example, a method is known in which another learning data is generated by adding changes to an image used as learning data according to a specific rule such as inversion, scaling, noise addition, rotation, etc. See

Patent Document

1 or 2. Also, when the learning data is voice or text, similar rule-based data expansion may be performed.

However, the conventional technology has a problem that it may be difficult to efficiently obtain high-quality learning data that greatly improves the accuracy of the model by data expansion. Specifically, with conventional rule-based data expansion, it is difficult to increase variations in the attributes of learning data, which limits the improvement of model accuracy. For example, in the rule-based data expansion described in Non-Patent

Documents

1 and 2, an image in which the attributes of “front of the window”, “cat”, and “front” of the image of a cat facing the front of the window are changed is generated. Is difficult to do.

It is often difficult to prepare a large amount of target data that will be the basis of expansion. On the other hand, it is desirable to obtain data that greatly contributes to the accuracy improvement of the model from a limited amount of target data, but such a method is not known.

In order to solve the above-mentioned problems and achieve the object, the expansion device includes a first data set, which is a set of data belonging to a first class, and a second data, which is a set of data belonging to a second class. Belongs to a third class from the second data set based on a metric calculated from the similarity between the second data set and the degree of uniformity of data included in the second data set. When the first class is designated by the selecting unit that selects the third data set and the generator that generates data based on the designated class, the generator causes the first class to be determined by the classifier. The generator, the identifying so as to generate data that is likely to be identified as data of a data set and is calculated by the classifier such that the probabilities of belonging to the plurality of third classes are non-zero and equal. Generating a data by specifying the first class to a learning unit that learns a generative model including a generator and the classifier, and to the generator of the generative model that has been learned by the learning unit And an extraction unit that extracts, from the data generated by the generation unit, data identified as data of the first data set by the identifier as extension data, and an extraction unit that extracts the data. And a assigning unit that assigns a label indicating that the extension data belongs to the first class.

According to the present invention, high-quality learning data that greatly improves the accuracy of the model can be efficiently obtained by data expansion.

FIG. 1 is a diagram for explaining the learning system according to the first embodiment. FIG. 2 is a diagram illustrating a configuration example of the expansion device according to the first embodiment. FIG. 3 is a diagram for explaining a classifier that also has a function as a classifier. FIG. 4 is a diagram for explaining a learning process for external data. FIG. 5 is a diagram for explaining the learning process for the target data. FIG. 6 is a diagram for explaining extraction of extension data. FIG. 7 is a diagram for explaining the expanded data set. FIG. 8 is a flowchart showing a processing flow of the expansion device according to the first embodiment. FIG. 9 is a flowchart showing a flow of parameter update processing of the expansion device according to the first embodiment. FIG. 10 is a diagram showing data used in the experiment. FIG. 11 is a diagram showing the results of the experiment. FIG. 12 is a diagram showing the results of the experiment. FIG. 13 is a diagram showing the results of the experiment. FIG. 14 is a diagram showing conditions of the additional experiment. FIG. 15 is a diagram showing the result of the additional experiment. FIG. 16 is a diagram illustrating an example of a computer that executes the extension program.

Hereinafter, embodiments of the expansion device, the expansion method, and the expansion program according to the present application will be described in detail with reference to the drawings. The present invention is not limited to the embodiments described below.

[Configuration of First Embodiment]
First, the learning system will be described with reference to FIG. FIG. 1 is a diagram for explaining the learning system according to the first embodiment. As shown in FIG. 1, the learning system 1 includes an expansion device 10 and a learning device 20.

The expansion device 10 performs data expansion of the target data set S _T using the external data set S _O and outputs the expanded data set S′ _gen . The learning device 20 also learns the target model 21 using the expanded data set S′ _gen . The target model 21 may be a known model that performs machine learning. For example, the objective model 21 is a class classifier such as Resnet-152.

Moreover, each data set in FIG. 1 is labeled data used in the target model 21. That is, each data set is a combination of data and a label indicating the class to which the data belongs. For example, when the target model 21 is a model for image recognition, each data set is a combination of image data and a label. The target model 21 may be a voice recognition model or a natural language recognition model. In that case, each data set is voice data with a label or text data with a label.

Here, the target data set S _T is assumed to be a combination of the target data X _T and the target label y _T attached to the target data X _T. The external data set S _O is assumed to be a combination of the external data X _O and the external label y _O attached to the external data X _O.

The target label y _T is a label to be learned by the target model 21. For example, when the target model 21 is a model for recognizing a person in the image, the target label y _T is an ID for identifying the person shown in the image of the target data. Further, for example, when the target model 21 is a model in which text is recognized from voice, the target label y _T is a text in which the voice of the target data is transcribed.

The external data set S _O is a data set for extending the target data set S _T. The external dataset S _O may be a dataset in a domain different from that of the target dataset S _T. Here, the domain is a characteristic peculiar to the data set, and is represented by the data, the label, and the generation distribution. For example, a domain of a data set whose data is X ₀ and whose label is y ₀ is represented as (X ₀ , y ₀ , P(X ₀ , y ₀ )).

Here, as an example, it is assumed that the target model 21 is an image recognition model, and the learning device 20 learns the target model 21 so that the image of the person whose ID is “0002” can be recognized from the image. In this case, the target data set S _T is a combination of the label “ID:0002” and an image in which the person is known to be reflected. The external data set S _O is a combination of a label indicating an ID other than “0002” and an image in which it is known that the person corresponding to the ID is shown.

The expansion device 10 outputs an expanded data set S′ _gen in which an attribute that the data of the target data set S _T does not have is taken in from the external data set S _O. This makes it possible to obtain variation data that could not be obtained from the target data set S _T alone. For example, according to the expansion device 10, even when the target data set S _T includes only the image of the back of a certain person, it is possible to obtain the image of the front of the person. Become.

Hereafter, an example when each data set is a combination of image data and label is explained. In the following description, data representing an image in a computer processable format is referred to as image data or simply an image.

The generation model 121 is a model based on GAN (Generative Adversarial Networks). For example, both the generator G and the discriminator D are neural networks. When the target class label y _T or the outer class label y _O is specified together with the noise z, the generator G generates an image based on the specified label. The purpose class is a class to which the purpose data X _T belongs. The external class is a class to which the external data set X _O belongs.

The discriminator D discriminates whether the image generated by the generator G is a real product (Real) or a fake product (Fake). For example, the discriminator D receives the image X _gen generated by the generator G and the image _XT _+O of the target data set and the external data, and which of the two images is X _gen (fake), And which is _XT+O (genuine: Real).

A plurality of external data sets having different labels may be input to the expansion device 10. That is, the expansion device 10 may be input with an external data set S _kO (where kO=1, 2,..., K _O ) that is a combination of the external data X _kO and the label y _kO .

Furthermore, in the first embodiment, the discriminator D also has a function as a classifier. That is, the classifier D can calculate the probability that the image belongs to each class. Details of the function of the discriminator D as a classifier will be described later.

The expansion device 10 learns the generative model 121 using a predetermined data set selected from the external data set S _O. Further, the expansion device 10 reduces the error (Adversarial Loss) when the discriminator D discriminates whether the image is a genuine image or a fake image, and the error (OLSR Loss) regarding the function as the classifier, Learning is performed by updating the parameters of the discriminator D. Note that updating of each parameter in learning is performed by, for example, an error back propagation method (Backpropagation).

Further, when the expansion device 10 actually outputs the expanded data set S′ _gen , the expansion data is not the generation data X _gen generated by the generator G based on the label y _T of the target data set S _T , but the generated data X _gen. include _X'gen extracted from the X _gen to the extended set of data _S'gen.

The configuration of the expansion device 10 will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the expansion device according to the first embodiment. As shown in FIG. 2, the expansion device 10 includes an input/output unit 11, a storage unit 12, and a control unit 13. The input/output unit 11 is an interface for receiving data input and outputting data. The input/output unit 11 may be a communication module that communicates data with an external device.

The storage unit 12 is a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), and optical disk. The storage unit 12 may be a rewritable semiconductor memory such as RAM (Random Access Memory), flash memory, NVSRAM (Non Volatile Static Random Access Memory). The storage unit 12 stores an OS (Operating System) and various programs executed by the expansion device 10. Further, the storage unit 12 stores various information used in executing the program. The storage unit 12 also stores the generation model 121. Specifically, the storage unit 12 stores parameters used in each process by the generative model 121.

The control unit 13 controls the entire expansion device 10. The control unit 13 includes, for example, electronic circuits such as CPU (Central Processing Unit), MPU (Micro Processing Unit), and GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), and the like. It is an integrated circuit. Further, the control unit 13 has an internal memory for storing programs and control data defining various processing procedures, and executes each process using the internal memory. Further, the control unit 13 functions as various processing units by operating various programs. For example, the control unit 13 has a selection unit 131, a learning unit 132, a generation unit 133, an extraction unit 134, and an addition unit 135.

The selection unit 131 determines the similarity between the target data set, which is a set of data belonging to the target class, and the external data set, which is a set of data belonging to the external class, and the uniformity of the data included in the external data set. A predetermined data set is selected from the external data sets based on a metric calculated from the degree. The data set selected by the selection unit 131 is an example of the third data set. The label of the data set selected by the selection unit 131 is an example of the third label.

The selection unit 131 has selected from the external data sets based on the metric calculated from the degree of similarity and the degree of uniformity calculated using the statistical value of the brightness value of the image data included in each data set. External datasets can be selected. For example, the selection unit 131 calculates the metric M by the equation (1). At this time, the selection unit 131 selects a data set whose calculated metric M is equal to or greater than the threshold value.

Further, the FID of the expression (1) is the degree of similarity and is represented by the expression (2) (reference: Non-Patent Document 9). μ _XT and μ _XO are average values of the luminance values of the image included in the target data set and the image included in the external data set. Further, Σ _XT and Σ _XT are matrices representing the brightness values of the image of the target data set and the image of the external data set. The FID decreases as the degree of similarity between the target data set and the external data set increases.

Further, the MS-SSIM of the formula (1) is the degree of uniformity and is represented by the formula (3) (reference: Non-Patent Document 10). I _M (x ₁ , x ₂ ) is a function that increases as the difference in brightness between the images x ₁ and x ₂ increases. c _M (x ₁ , x ₂ ) is a function that increases as the difference in contrast between the images x ₁ and x ₂ increases. s _M (x ₁ , x ₂ ) is a function that increases as the difference in structure between the images x ₁ and x ₂ increases.

Next, the learning unit 132 will be described. Note that the external data set and the external label in the description of the learning unit 132 mean the external data set and the label of the external data selected by the selection unit 131.

The learning unit 132 easily identifies the generator G as the data of the target data set by the classifier D when the target class is specified in the generator G that generates data based on the specified class. To learn.

Here, as described above, as shown in FIG. 3, the discriminator D also has a function as a classifier (reference: Non-Patent Document 17). FIG. 3 is a diagram for explaining a classifier that also has a function as a classifier. In the following description, when the classifier D functions as a classifier, the classifier D may be referred to as the classifier C. L _Adv. Is the error of the discrimination result of the discriminator D. On the other hand, L _OLSR in FIG. 3 is an error of the classification result of the classifier C. OLSR is an abbreviation for Outer Label Smoothing Regularization.

The learning unit 132 includes a generator G, a classifier D, and a classifier C so as to generate data calculated by the classifier C such that the probabilities of belonging to a plurality of outer classes are non-zero and even. Train the model.

The learning process by the learning unit 132 will be described with reference to FIGS. 4 and 5. FIG. 4 is a diagram for explaining a learning process for external data. Further, FIG. 5 is a diagram for explaining the learning process for the target data.

First, the learning process for an external data set will be described. As shown in FIG. 4, the generator G generates the image data X _gen when the external label y _O is specified together with the noise z. In addition, the external data set S _O includes image data X _O. The discriminator D discriminates whether the image data X _gen and the image data X _O are genuine or counterfeit. Then, the learning unit 132 updates the parameter of the discriminator D so that the binary discrimination error of whether it is a genuine article or a fake one becomes small, and conversely, the parameter of the generator G so that the binary discrimination error becomes large. To update.

Further, the classifier C calculates the probability that the external data X _O belongs to each label y _kO (k _O =1, 2,..., K _O ) included in the external label y _O. At this time, the learning unit 132 calculates the error according to the equation (4) where Z is 0.

For example, in the example of FIG. 4, k _O =1 represents the Retriever class, k _O =2 represents the Dachshund class, and k _O =K _O represents the Terrier class. Further, it is assumed that the image X _O is an image of the Terrier class (k _O =K _O ). At this time, the classifier C calculates the probability that the image X _O is the Retriever class (k _O =1) is 0.11, the probability that the image X _O is the Dachshund class (k _O =2) is 0.05, and the image X _The probability that _O is the Terrier class (k _O =K _O ) is calculated as 0.70. Then, the learning unit 132 includes, for example, the generation model 121 including the generator G so that the probability of being the Terrier class (k _O =K _O ) is calculated to be larger when the next image X _O is input. Update the whole parameter. That is, according to the learning process performed by the learning unit 132, the classifier C calculates the data of the external data set so that the probability of belonging to the class to which the data belongs is 1.

Next, the learning process for the target data set will be described. As shown in FIG. 5, when the target label y _T is specified together with the noise z, the generator G generates the image data X _gen . Further, the target data set S _T includes the image data X _T. The discriminator D discriminates whether the image data X _gen and the image data X _T are genuine or counterfeit. Then, the learning unit 132 updates the parameter of the discriminator D so that the binary discrimination error between the genuine and the fake is small, and conversely updates the generator G so that the binary discrimination error is large. To do.

Further, the classifier C calculates the probability that the external data X _T belongs to each label y _kO (k _O =1, 2,..., K _O ) included in the external label y _O. At this time, the learning unit 132 calculates the error according to the equation (4) where Z is 1.

For example, in the example of FIG. 5, k _O =1 represents the Retriever class, k _O =2 represents the Dachshund class, and k _O =K _O represents the Terrier class. Further, the image X _T does not belong to any class from k _O =1 to K _O. In this case, the classifier C, the image _{X T} is Retriever class _(k O = 1) 0.2 the probability is, the image _{X T} is the probability that Dachshund class _(k O = 2) 0.14, image X _The probability that _T is the Terrier class (k _O =K _O ) is calculated as 0.11.

Here, since the image X _T does not belong to any class from k _O =1 to K _O , it is considered that a general classifier performs learning so that the calculated probability becomes 0. .. On the other hand, the learning unit 132 calculates, for example, that the probability that each class is from k _O =1 to K _O is 1/K _O when the next image X _T is input, The parameters of the entire generative model 121 including the generator G are updated. That is, according to the learning process performed by the learning unit 132, the classifier C calculates the data of the target data set so that the probability of belonging to each outer class is the reciprocal of the number of classes included in the outer class. Like

Here, the label specified for the generator G is a unique expression of each class. On the other hand, noise is a common expression common to all classes. For example, when the label represents the type of animal, it is considered that the type of animal shown in the image is determined by the label, and the background and posture of the image are determined by noise. Therefore, for example, if noise is common between the case of specifying a label indicating "dog" and the case of specifying a label indicating "cat", in each case, the generator G determines the image of the dog and the dog. It is conceivable to generate an image of a cat having the same background and posture as the image of.

However, in reality, even if the generator G has common noise, for example, “an image of a dog sitting outdoors facing left” and “an image of a cat sitting indoors facing left””. May be generated. This is because the information that determines whether the background of the image is outdoors or indoors is included not only in the noise but also in the label itself. Therefore, the learning unit 132 performs the learning of the classifier C as described above in order to obtain useful information for improving the quality of generated data even from a label different from the target label.

As a result, when the label “dog” is designated while changing the noise, the generator G can generate “an image of a dog sitting facing left indoors” that could not be generated conventionally. Is possible. As a result, the generator G can generate various kinds of data, so that the quality of the extension data is improved.

The generation unit 133 generates data by designating the target class in the generator G of the generation model learned by the learning unit 132. At this time, the generation unit 133 specifies noise together with the target class. For example, the noise may be a value generated according to a normal distribution of N(0,1).

As illustrated in FIG. 6, the extraction unit 134 sets, as the extension data X′ _gen , the data identified by the identifier D as the data (Good) of the target data set from the data generated by the generation unit 133. Extract. FIG. 6 is a diagram for explaining extraction of extension data. It is conceivable that the generator G may generate data that is discriminated as an imitation (Bad) by the discriminator D even if it has been learned. Therefore, the extraction unit 134 extracts the extension data by using, for example, DRS (Discriminator Rejection Sampling) (reference: Non-Patent Document 13).

Further, the extraction unit 134 further excludes, from the extension data, data whose Euclidean distance from the target data set is a predetermined value or more. For example, when the Euclidean distance between the feature vector of the extracted image of the extension data and the feature vector of the image of the target data set is equal to or greater than the threshold value, the extraction unit 134 excludes the extension data.

The assigning unit 135 assigns a label indicating that it belongs to the target class to the extension data extracted by the extracting unit 134. FIG. 7 is a diagram for explaining the expanded data set. As shown in FIG. 7, the expanded data set S′ _T is obtained by adding a target label y _T to the target data X _T and the expansion data X′ _gen .

[Processing of First Embodiment]
The process flow of the expansion device 10 will be described with reference to FIG. FIG. 8 is a flowchart showing a processing flow of the expansion device according to the first embodiment. As shown in FIG. 8, first, the expansion device 10 receives an input of a target data set and an external data set (step S11). Next, the expansion device 10 evaluates the external data set by a metric based on the similarity between the data sets and the degree of uniformity within the external data set, and selects a predetermined data set (step S12).

Then, the expansion device 10 uses the generation model to generate an image from the target data set and the external data set (step S13). At this time, the expansion device 10 specifies the target label and the external label together with noise in the generation model. Here, the expansion device 10 updates the parameters of the generation model based on the generated image (step S14). The parameter updating process (learning process) in step S14 will be described later with reference to FIG.

After that, the expansion device 10 specifies the label of the target data set in the generation model (step S15), and generates an image based on the specified label (step S16). The expansion device 10 extracts an image for expansion from the generated image (step S17). At this time, the expansion device 10 can extract the image for expansion according to the identification criterion of the identifier of the generation model.

Then, the expansion device 10 integrates the image of the target data set and the image for expansion and gives the label of the target data set (step S18). Further, the expansion device 10 outputs the expanded data set to which the target label has been added (step S19) and transfers it to the learning device 20.

The parameter updating process in step 14 will be described with reference to FIG. FIG. 9 is a flowchart showing a flow of parameter update processing of the expansion device according to the first embodiment. As shown in FIG. 9, the expansion device 10 first updates the parameters of the generative model 121 based on the binary determination error (step S141).

Next, the expansion device 10 selects an unselected label (step S142). When the selected label is the target label (step S143, target label), the expansion device 10 updates the parameter so that the label is smoothed (step S144). That is, the expansion device 10 is a generation model such that the probability calculated by the classifier C and the probability that the target data is classified into each outer class is 1/K (K is the number of outer classes). The parameters of 121 are updated (step S144).

On the other hand, when the selected label is the external label (step S143, external label), the parameters of the generated model 121 are updated so that the external class is correctly classified (step S145).

If there is an unselected label (step S146, Yes), the expansion device 10 returns to step S142 and repeats the processing. On the other hand, when there is no unselected label (step S146, No), the expansion device 10 ends the parameter updating process.

[Effects of First Embodiment]
As described above, the extension device 10 determines the similarity between the target data set, which is a set of data belonging to the target class, and the external data set, which is a set of data belonging to the external class, and the external data set. A predetermined data set is selected from the external data sets based on a metric calculated from the degree of uniformity of the included data. The extension device 10 easily identifies the generator G as the data of the target data set by the discriminator D when the target class is designated in the generator G that generates data based on the designated class. Learning a generation model including a generator G, a classifier D, and a classifier C so as to generate data calculated by the classifier C such that the probabilities of belonging to a plurality of selected outer classes are non-zero and equal. I do. The expansion device 10 generates data by designating a target class to the generator G of the learned generation model. The expansion device 10 extracts, from the generated data, the data identified as the data of the target data set by the identifier D as the extension data. The extension device 10 adds a label indicating that it belongs to the target class to the extracted extension data. In this way, the expansion device 10 preselects an input external data set according to a predetermined criterion, performs learning so that useful information obtained from the external data set is not lost, and generates the generated data of the discriminator. Extract by standard. Therefore, according to the first embodiment, it is possible to efficiently obtain high-quality learning data that greatly improves the accuracy of the model by data expansion.

The expansion device 10 has already selected from the external data sets based on the metric calculated from the degree of similarity and the uniformity calculated using the statistic of the brightness value of the image data included in each data set. Select an external dataset. As a result, data that does not contribute to improving the quality of learning data can be excluded in advance, and the quality of learning data can be improved.

The expansion device 10 performs a calculation so that the classifier C, with respect to the data of the target data set, has the probability of belonging to each of the outer classes of the selected outer data set to be the reciprocal of the number of classes included in the selected outer class. Then, the data of the external data set is calculated so that the probability of belonging to the class to which the data belongs is 1. This makes it possible to generate learning data having useful information included in the label.

The extension device 10 further excludes, from the extension data, data whose Euclidean distance from the target data set is a predetermined value or more. As a result, outliers can be excluded from the extension data, and the quality of the learning data can be improved.

[Experimental result]
Here, an experiment conducted to compare the conventional technique with the embodiment will be described. In the experiment, as the external data set, data obtained by collecting 5000 points from each of the data sets shown in FIG. 10 was used. FIG. 10 is a diagram showing data used in the experiment. Then, the selection unit 131 changed the threshold value of the metric for selecting the external data set, and compared the accuracy (Top-1 Accuracy) of the target model 21 with the FID of the expanded data set.

As a result, as shown in FIG. 11, the FID of the expanded data set decreased as the threshold value of the metrics on the horizontal axis increased. FIG. 11 is a diagram showing the results of the experiment. In other words, the similarity between the expanded data set and the target data set increased as the metric threshold increased. Therefore, it can be said that the selection of data by the metrics improves the quality of the learning data.

Also, as shown in FIG. 12, the accuracy (Acc.) of the target model 21 increased as the threshold value of the metrics shown on the horizontal axis increased. FIG. 12 is a diagram showing the results of the experiment. Therefore, it can be said that the selection of the data by the metrics improves the quality of the learning data and, as a result, the accuracy of the target model 21.

As shown in FIG. 13, the learning unit 132 smoothes the probabilities belonging to each outer class calculated by the classifier C (OLSR), and the extraction unit 134 extracts data using the criterion of the discriminator D (DRS). ), the accuracy of the target model 21 and the FID of the expanded data set improved. FIG. 13 is a diagram showing the results of the experiment. Baseline is a conventional rule-based data extension method.

Further, as an additional experiment, a comparison was made between the case where the target data was reduced to 1/10 and the method of Baseline and the first embodiment was applied, and the case where the conventional method was applied without reducing the target data. FIG. 14 is a diagram showing conditions of the additional experiment.

Further, FIG. 15 is a diagram showing the result of the additional experiment. As shown in FIG. 15, in the method of the first embodiment, even if the target data is reduced to 1/10, the accuracy close to that of the conventional method when the target data is not reduced is obtained.

[Other Embodiments]
In the above embodiment, the learning function of the objective model 21 is provided in the learning device 20 different from the expansion device 10. On the other hand, the extension device 10 may be provided with an objective model learning unit that causes the objective model 21 to learn the extended data set S′ _gen . As a result, the expansion device 10 can suppress resource consumption due to data transfer between the devices, and efficiently execute data expansion and learning of the target model as a series of processes.

[System configuration, etc.]
In addition, each constituent element of each illustrated device is functionally conceptual, and does not necessarily have to be physically configured as illustrated. That is, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of the device may be functionally or physically distributed in arbitrary units according to various loads or usage conditions, or It can be integrated and configured. Furthermore, all or arbitrary parts of the processing functions performed by each device may be realized by a CPU and a program that is analyzed and executed by the CPU, or may be realized as hardware by a wired logic.

Further, among the respective processes described in the present embodiment, all or part of the processes described as being automatically performed may be manually performed, or the processes described as being manually performed may be performed. All or part of the process can be automatically performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above-mentioned documents and drawings can be arbitrarily changed unless otherwise specified.

[program]
As an embodiment, the expansion device 10 can be implemented by installing an expansion program for executing the above data expansion as package software or online software in a desired computer. For example, the information processing apparatus can be caused to function as the expansion apparatus 10 by causing the information processing apparatus to execute the above-described expansion program. The information processing device includes a desktop or notebook personal computer. Further, in addition to the above, the information processing apparatus includes in its category a mobile communication terminal such as a smartphone, a mobile phone or a PHS (Personal Handyphone System), and a slate terminal such as a PDA (Personal Digital Assistant).

Also, the expansion device 10 can be implemented as a terminal device used by a user as a client and as an expansion server device that provides the client with the service related to the above data expansion. For example, the extended server device is implemented as a server device that provides an extended service in which target data is input and extended data is output. In this case, the expansion server device may be implemented as a Web server, or may be implemented as a cloud that provides the above-mentioned service related to data expansion by outsourcing.

FIG. 16 is a diagram illustrating an example of a computer that executes an extension program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to the display 1130, for example.

The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process of the expansion device 10 is implemented as the program module 1093 in which the code executable by the computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing the same processing as the functional configuration of the expansion device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by SSD.

Further, the setting data used in the processing of the above-described embodiment is stored as the program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

10 expansion device 11 input/output unit 12 storage unit 13 control unit 20 learning device 21 target model 121 generation model 131 selection unit 132 learning unit 133 generation unit 134 extraction unit 135 addition unit

Claims

The similarity between a first data set, which is a set of data belonging to the first class, and a second data set, which is a set of data belonging to the second class, and the second data set are included. A selection unit that selects a third data set belonging to a third class from the second data set based on a metric calculated from the uniformity of the data;
When the first class is specified to the generator that generates data based on the specified class, the generator is likely to be identified as the data of the first data set by the classifier, and is classified. Training a generative model including the generator, the classifier, and the classifier so as to generate data in which the probability that the probability of belonging to the plurality of third classes is non-zero and equal A learning department to do,
A generator that generates data by designating the first class to the generator of the generative model that has been learned by the learning unit;
An extraction unit that extracts, from the data generated by the generation unit, data identified as data of the first data set by the identifier as extension data;
An assigning unit that assigns a label indicating that the extension data extracted by the extracting unit belongs to the first class,
An expansion device comprising:
The selection unit may include the second data set based on a metric calculated from the degree of similarity and the degree of uniformity calculated using a statistic of luminance values of image data included in each data set. The expansion device according to claim 1, wherein the third data set is selected from among the following.
The learning unit is
For the data of the first data set, the classifier performs a calculation such that the probability of belonging to each of the third classes is the reciprocal of the number of classes included in the third class, For the data of the data set of 3, the calculation is performed so that the probability of belonging to the class to which the data belongs is 1.
The expansion device according to claim 1, wherein the generative model is learned as described above.
The expansion device according to claim 1, wherein the extraction unit further excludes, from the expansion data, data whose Euclidean distance from the first data set is a predetermined value or more.
An expansion method performed by a computer,
The similarity between a first data set, which is a set of data belonging to the first class, and a second data set, which is a set of data belonging to the second class, and the second data set are included. A selection step of selecting a third data set belonging to a third class from the second data set based on a metric calculated from the uniformity of the data;
When the first class is specified to the generator that generates data based on the specified class, the generator is likely to be identified as the data of the first data set by the classifier, and is classified. Training a generative model including the generator, the classifier, and the classifier so as to generate data calculated so that the probabilities belonging to the plurality of third classes are non-zero and equal by the classifier. The learning process to perform,
A generation step of generating data by designating the first class to the generator of the generation model that has been learned by the learning step;
An extracting step of extracting, from the data generated by the generating step, the data identified as the data of the first data set by the identifier as extension data;
An assigning step of assigning a label indicating that it belongs to the first class to the extension data extracted by the extracting step;
An expansion method comprising:
An expansion program for causing a computer to function as the expansion device according to any one of claims 1 to 4.