KR20230110138A - Self-Diagnosing Generative Adversarial Networks to Diagnose and Emphasize Underrepresented Samples - Google Patents

Abstract

A technology for generating data based on a Generative Adversarial Network (GAN) is disclosed. The data generation method includes measuring an underrepresentation score indicating an underrepresentation degree of each training data in a learning process of a generative adversarial network; and additionally learning the generative adversarial neural network through weighted sampling based on the underrepresentation score.

Description

Design of generative adversarial networks that self-diagnose and highlight underrepresented data {Self-Diagnosing Generative Adversarial Networks to Diagnose and Emphasize Underrepresented Samples}

The following description relates to a technique for generating various data using an artificial neural network model.

Artificial intelligence technology is used in various industrial fields and performs various tasks such as image classification, image generation, and language generation.

Among artificial intelligence technologies, generative models aim to create virtual generated data similar to the learning data by learning the distribution of the data when given learning data such as images and language.

This generative model can be utilized in various industries.

A large amount of data is required for learning artificial intelligence models, and a method of generating virtual data through a generative model can be effectively used to cope with the problem of lack of data.

In addition, the generative model can be used to create virtual portraits or voices to be used in virtual reality in an industrial situation where virtual reality is emerging.

Among generative models, generative adversarial networks (GANs) are used in various industries due to their effective performance capable of generating virtual data of the same quality as real ones.

However, the problem of the generative adversarial neural network is that the variety of generated virtual data is limited because it expresses only a part of the training data, not the whole, despite the high quality of the generated data.

In particular, it is known that when a specific training data sample has a relatively small number of features distributed within the entire data, the generation performance of the corresponding data is significantly lower than that of other data.

1. Goodfellow et al., Generative Adversarial Networks, 2014

2. Korea Patent Publication No. 10-2021-0048100 (Method and apparatus for generating condition monitoring data using generative adversarial neural network, publication date: May 03, 2021)

The present invention measures the degree of underrepresentation of each training data sample in the conventional generative adversarial neural network learning and proposes an improved generative adversarial network with increased data representation ability and diversity through a technique of detecting and emphasizing the underrepresented data based on the score obtained from the measurement.

A data generation method performed in a computer system, wherein the computer system includes at least one processor configured to execute computer readable instructions included in a memory, and the data generation method comprises: measuring an underrepresentation score indicating a degree of underrepresentation of each training data in a learning process of a generative adversarial network (GAN) by the at least one processor; and additionally learning, by the at least one processor, the generative adversarial neural network through weighted sampling based on the underrepresentation score.

According to one aspect, the measuring may include measuring the underrepresentation score by calculating a discrepancy score between a data distribution and a model distribution in the training data.

According to another aspect, the generative adversarial network includes a generator neural network and a discriminator neural network, and in the measuring step, the underrepresentation score may be measured based on a log density ratio (LDR) based on an output value of the discriminator neural network.

According to another aspect, in the measuring step, the underrepresentation score may be measured using the mean and variance of the LDR calculated in the learning process of the generative adversarial network.

According to another aspect, the generative adversarial neural network includes a generator neural network and a discriminator neural network, and in the measuring step, the underrepresentation score may be measured using an average and a variance of output values of the discriminator calculated in a learning process of the generative adversarial neural network.

According to another aspect, a non-saturating loss function or a hinge loss function may be used when learning the generative adversarial network.

According to another aspect, the additional learning may include adjusting a sampling probability of the learning data in proportion to the underrepresentation score.

According to another aspect, the adjusting may include adjusting the sampling probability of the training data within a range in which a ratio between a maximum sampling probability and a minimum sampling probability does not exceed a predetermined value.

According to another aspect, the generative adversarial neural network includes a generator neural network and a discriminator neural network, and the method of generating data may further include modifying, by the at least one processor, a model distribution of the generative adversarial neural network through rejection sampling using an output value of the discriminator neural network.

A computer program stored in a computer readable recording medium to execute a data generation method on a computer, the data generation method comprising: measuring an underrepresentation score indicating a degree of underrepresentation of each training data in a learning process of a generative adversarial neural network; additionally learning the generative adversarial neural network through weighted sampling based on the underrepresentation score; and modifying a model distribution of the generative adversarial neural network through reject sampling using an output value of the discriminator neural network.

A computer system, comprising at least one processor configured to execute computer readable instructions included in a memory, wherein the at least one processor includes: measuring an underrepresentation score indicating a degree of underrepresentation of each training data in a learning process of a generative adversarial neural network; and a computer system processing a process of additionally learning the generative adversarial neural network through weighted sampling based on the underrepresentation score.

According to embodiments of the present invention, in the process of learning a generative adversarial network, by identifying data that is not well represented by the model and emphasizing the corresponding data, the model can more evenly represent each training data well, thereby improving the diversity of generated data and the performance of data generation.

1 is a block diagram for explaining an example of an internal configuration of a computer system according to an embodiment of the present invention.
2 is a diagram showing an example of components that may be included in a processor of a computer system according to an embodiment of the present invention.
3 is a flowchart illustrating an example of a data generation method that can be performed by a computer system according to an embodiment of the present invention.
4 and 5 are exemplary diagrams for explaining a process of identifying underrepresented data and emphasizing the corresponding data in the learning process of a generative adversarial network in one embodiment of the present invention.
6 shows an example of a sample generated by a generative adversarial neural network and a partial recall analysis result of the sample in one embodiment of the present invention.
7 shows an example of a training image of a generative adversarial network according to an embodiment of the present invention.
8 illustrates a result of pixel intensity histogram comparison for a training image according to an embodiment of the present invention.
9 shows an example of a detailed algorithm for a self-diagnosing generative adversarial neural network in one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to techniques for generating data using generative adversarial networks.

Embodiments including those specifically disclosed in this specification identify data that is not well represented by the model in the learning process of the generative adversarial neural network and emphasize the data, thereby improving the diversity of generated data and the performance of generating data of the generative adversarial neural network.

1 is a block diagram illustrating an example of a computer system according to one embodiment of the present invention. For example, a data generation system according to embodiments of the present invention may be implemented by the computer system 100 shown in FIG. 1 .

As shown in FIG. 1, a computer system 100 is a component for executing a data generation method according to embodiments of the present invention, and may include a memory 110, a processor 120, a communication interface 130, and an input/output interface 140.

The memory 110 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, a non-perishable mass storage device such as a ROM and a disk drive may be included in the computer system 100 as a separate permanent storage device separate from the memory 110 . Also, an operating system and at least one program code may be stored in the memory 110 . These software components may be loaded into the memory 110 from a recording medium readable by a separate computer from the memory 110 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, software components may be loaded into the memory 110 through the communication interface 130 rather than a computer-readable recording medium. For example, software components may be loaded into memory 110 of computer system 100 based on a computer program installed by files received over network 160 .

The processor 120 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 120 by memory 110 or communication interface 130 . For example, processor 120 may be configured to execute received instructions according to program codes stored in a recording device such as memory 110 .

Communication interface 130 may provide functionality for computer system 100 to communicate with other devices via network 160 . For example, a request, command, data, file, etc. generated by the processor 120 of the computer system 100 according to a program code stored in a recording device such as the memory 110 is controlled by the communication interface 130. It can be transmitted to other devices through the network 160. Conversely, signals, commands, data, files, etc. from other devices may be received into the computer system 100 via the communication interface 130 of the computer system 100 via the network 160 . Signals, commands, data, etc. received through the communication interface 130 may be transmitted to the processor 120 or the memory 110, and files, etc. may be further included in the computer system 100. It may be stored in a storage medium (the permanent storage device described above).

The communication method is not limited, and a short-distance wired / wireless communication between devices as well as a communication method utilizing a communication network (eg, a mobile communication network, wired Internet, wireless Internet, and broadcasting network) that the network 160 may include may also be included. For example, the network 160 may include any one or more of networks such as a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), and the Internet. In addition, the network 160 may include any one or more of network topologies including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, and the like.

The input/output interface 140 may be a means for interface with the input/output device 150 . For example, the input device may include devices such as a microphone, keyboard, camera, or mouse, and the output device may include devices such as a display and a speaker. As another example, the input/output interface 140 may be a means for interface with a device in which functions for input and output are integrated into one, such as a touch screen. The input/output device 150 may be configured as one device with the computer system 100 .

Also, in other embodiments, computer system 100 may include fewer or more elements than those of FIG. 1 . However, there is no need to clearly show most of the prior art components. For example, the computer system 100 may be implemented to include at least some of the aforementioned input/output devices 150 or may further include other components such as a transceiver, a camera, various sensors, and a database.

Hereinafter, a specific embodiment of a technology for generating data based on a generative adversarial network that self-diagnoses and emphasizes underrepresented data will be described.

2 is a diagram showing an example of components that may be included in a processor of a computer system according to an embodiment of the present invention, and FIG. 3 is a data generation method that can be performed by a computer system according to an embodiment of the present invention. It is a flow chart showing an example.

As shown in FIG. 2 , the processor 120 may include a learning unit 210 , a score measurement unit 220 , an additional learning unit 230 , and a model correction unit 240 . Components of the processor 120 may represent different functions performed by the processor 120 according to a control command provided by at least one program code. For example, the learner 210 may be used as a functional representation operating to control the computer system 100 so that the processor 120 learns a generative adversarial network (GAN).

The processor 120 and components of the processor 120 may perform steps S310 to S340 included in the data generation method of FIG. 3 . For example, the processor 120 and components of the processor 120 may be implemented to execute an instruction according to an operating system code included in the memory 110 and at least one program code described above. Here, at least one program code may correspond to a code of a program implemented to process a data generation method.

The data generation method may not occur in the order shown, and some of the steps may be omitted or additional processes may be further included.

The processor 120 may load a program code stored in a program file for a data generation method into the memory 110 . For example, a program file for a data generation method may be stored in a permanent storage device separate from the memory 110, and the processor 120 may control the computer system 100 to load a program code into the memory 110 from a program file stored in the permanent storage device through a bus. At this time, each of the processor 120, the learning unit 210, and the fine-tuning unit 220 included in the processor 120 may be different functional representations of the processor 120 for executing subsequent steps S310 to S340 by executing a command of a corresponding part among the program codes loaded into the memory 110. For the execution of steps S310 to S340, the processor 120 and components of the processor 120 may directly process an operation according to a control command or control the computer system 100.

Referring to FIG. 3 , in step S310, the learning unit 210 may learn a generative adversarial neural network, which is a kind of artificial neural network. As a method of learning a generative adversarial network, a known technique may be applied. The generative adversarial neural network includes a generator neural network (G) and a discriminator neural network (D), and a loss function used in learning both neural networks is as shown in Equation 1.

[Equation 1]

In step S320, the score measurement unit 220 may measure the degree of underrepresentation of the learning data. The score measuring unit 220 uses the discriminator neural network D of the generative adversarial neural network learned in the learning unit 210 to measure a score representing the degree of underrepresentation of each training data sample x (hereinafter referred to as 'underrepresentation score'). The underrepresentation score is higher when the training data is underrepresented and lower when it is overrepresented. The LDR (log density ratio) score, which is the standard when measuring the underrepresentation score, is as shown in Equation 2.

[Equation 2]

The score measurement unit 220 may calculate an LDR score through Equation 2 for any data x. At this time, D(x) is the output value produced when the discriminator receives x. The LDR score is calculated using the output of the discriminator as log(p _data (x)/p _g (x)), that is, a score defined to estimate the difference between the true data distribution p _data and the generator distribution p _g at each data value x.

After measuring the LDR score in each learning process, it goes through several steps and calculates the mean and variance of the LDR score to obtain the LDRM score and the LDRV score. In this embodiment, the underrepresentation score may be defined as in Equation 3.

[Equation 3]

At this time, k represents a constant, which can be adjusted according to the type of learning data.

When training a generative adversarial network, a non-saturating loss or a hinge loss can be used as a loss function.

If the loss function used in learning the generative adversarial network is a hinge loss rather than a desaturation loss, the underrepresentation score can be obtained by calculating the mean and variance of D _h (x), which is the result of the discriminator, instead of calculating the mean and variance of the LDR scores.

In step S330, the additional learning unit 230 may additionally learn the generative adversarial network in a direction that promotes the generation of underrepresented data based on the underrepresentation score. The additional learning unit 230 may additionally learn the generative adversarial network by emphasizing data based on the underrepresentation score measured by the score measuring unit 220 for all training data. The additional learning unit 230 may adjust the probability of selecting data x (ie, sampling probability) in proportion to the underrepresentation score of the data x in the learning process of the generative adversarial network. In this case, the additional learning unit 230 may stably sample data by limiting the score gap to a range where the ratio between the maximum sampling probability and the minimum sampling probability does not exceed a predetermined value.

After calculating the training data sampling probability proportional to the underrepresentation score, the additional learning unit 230 may further perform learning of the generative adversarial network based on the calculated training data sampling probability. Depending on the embodiment, the training data sampling probability may be calculated in proportion to the LDRM score or the LDRV score.

In step S340, the model modifying unit 240 may modify the final model distribution of the generative adversarial neural network. The model correction unit 240 may finally modify the model distribution of the generative adversarial neural network through reject sampling using discriminator values after the model distribution with more emphasis on diversity is created through the additional learning unit 230.

Therefore, in the present embodiment, by identifying data that is not well represented by the model in the learning process of the generative adversarial network and emphasizing the corresponding data, it is possible to implement a model that more evenly represents the training data well.

A method for diagnosing and highlighting underrepresented samples during training of a generative adversarial network will be described in detail.

General generative adversarial networks can effectively generate fake data by mimicking the distribution of training data. However, as shown in FIG. 4, the generative adversarial network does not fully imitate the distribution of learning data, and as it imitates some data, it often fails to represent some data with minority characteristics.

Accordingly, the present invention is a method for expressing learning data evenly and well. After scoring the degree of underrepresentation of the learning data (underrepresentation score), as shown in FIG.

In particular, the processor 120 may detect and emphasize underrepresented real samples instead of fake samples.

The processor 120 uses the disparity statistics between the data distribution and the model distribution at each data instance, and can emphasize the underrepresented sample during training of the generative adversarial network based on the observation that the average disparity or variability of the underrepresented sample is high.

The processor 120 can simply calculate in the discriminator to detect underrepresented samples by diagnosing the learning of the generative adversarial network, and can effectively emphasize underrepresented data through score-based weighted sampling during learning of the generative adversarial network.

Processor 120 may use the empirical mean and variance over several epochs to detect minor group samples that are underrepresented, and may use the mean as well as the variance of the discrepancy estimate to detect underrepresented samples.

The metric for detecting underrepresented samples during training of a generative adversarial network is as follows.

Generative adversarial neural network training aims to train a generator with an implicit model distribution p _g (x) that closely matches the data distribution p _data (x). The discrepancy between p _data (x) and pg(x) can be measured as log(p _data (x)/p _g (x)), but cannot be directly computed in a generative adversarial network because p _g (x) is unknown and p _g (x) is implicit.

Instead, in this embodiment, the discriminator output can be used to define an estimate for the LDR. A generative adversarial network solves the min-max optimization min _G max _D V(D,G) for the loss function (Equation 1). For a generator G, the optimal discriminator D is It is calculated as, which can define the LDR through Equation 2.

When D(x) = D ^* (x), LDR(x) is equal to log(p _data (x) = p _g (x)). When LDR(x)>0, the data point x is underrepresented in the model. That is, it corresponds to p _data (x) > p _g (x). When LDR(x)<0, the data is overrepresented, corresponding to p _data (x)<p _g (x). Thus, the LDR(x) value of each instance x can be utilized to provide feedback to improve the generator if the estimate is valid.

The present invention can use the statistic of LDR(x), which is a much more reliable and informative metric, to detect underrepresented data regions during learning. It is to use the learning dynamics (behavior of the model as the learning progresses) to diagnose the learning behavior of each sample. Metrics for diagnosing learning dynamics are not clear, as there is no explicit reference to measure the model's accuracy when training a generative model. In this embodiment, we define metrics for estimating the mean and variance of the generative adversarial network, LDRM (LDR mean), LDRV (LDR variance) mismatch at each sample x over the learning phase T={t _s , ,t _e }.

[Equation 4]

[Equation 5]

Here, LDR(x) _k is the LDR estimate (Equation 2) recorded in the kth learning step. LDRM(x) measures how close to p _g (x) is for learning at sample point x, while LDRV(x) measures how that disparity varies across learning.

An intuitively well-learned and generalized sample is Afterwards, since LDR(x) is continuously small, LDRM and LDRV appear low, whereas samples that are difficult to learn show high LDRM or LDRV values.

LDRV (LDR Variance) is effective in detecting from the minor group.

Generative adversarial networks are known to have difficulty modeling minor samples.

Figure 6 shows the analysis results for generative adversarial networks trained with (1) single-mode Gaussian, (2) a mixture of MNIST (major) and FMNIST (minor), and (3) red (major) and green (minor) colored MNIST samples. (a) to (d) show examples of samples generated with major/minor features, and (e) shows partial recall of major samples (dotted lines) and minor samples (solid lines) of each data set.

Referring to (e) of FIG. 6, it can be seen that both sample quality and partial recall are higher in the major group, and the recall gap between the major group and the minor group is large and the gap gets worse as the minority levels increase. Given that the minor group suffers from poor quality as well as low coverage, the need for a technique for detecting and emphasizing minor samples can be felt.

LDRV can be used to apply heuristic arguments that can detect samples with minor features, that is, features of a minor group. In particular, minor samples tend to have higher LDRV values. First, the discriminator can be viewed as a logistic regression model, and for each input x _i the discriminator is a feature vector The actual score (probability that sample x _i is real (y _i =1)) is calculated using the dot product between the weight vector of the last layer and the weight vector of the last layer (Equation 6).

[Equation 6]

From a Bayesian point of view, the prior distribution of θ is Assuming that , the posterior distribution for θ is is given as To get a Gaussian approximation to the posterior distribution, first define the Gaussian mean Find the maximum posterior estimate θ _MAP that maximizes .

The covariance is given by the second derivative inverse matrix of the negative log likelihood taking the form of Equation 7.

[Equation 7]

Finally, LDRV, an approximation of D(x _i ;θ)) in Equation 6 by Taylor expansion in θ=θ _MAP , can be expressed as Equation 8.

[Equation 8]

This reveals an important aspect regarding LDRV and minor features. Equation 8 shows that the more the feature vector Φ _i is correlated with the principal component of S _n (the eigenvector with the largest eigenvalue), the larger the LDRV. Each eigenvalue of S _n is Since it is the eigenvalue and the reciprocal of , the minimum eigenvalue of Equation 9 Consider the characteristics of the eigenvector v of

[Equation 9]

Equation 9 shows that y tends to have smaller eigenvalues if it is not aligned (or orthogonal) to most of the feature vectors {Φ _i } where D(1-D)>0. Since the minor feature vector Φ _j may have a small component in the eigenspace formed by the majority having D(1-D)>0, substituting y = Φ _j into Equation 9 results in a small sum. This indicates that the minor feature vector Φ _j It can be seen that it will have a higher LDRV since it is correlated with the minimum eigenvector v of .

LDRM (LDR average) is effective in detecting missing modes.

From the definition of LDRM (Equation 4), high LDRM samples x tend to be less than p _g (x) through training and are therefore underrepresented. During training, record the LDR(x) of the training sample and calculate the LDRM value with the window size ITI=50. During training, we examine the average LDRM value for samples of each mode. The fact that samples of underrepresented modes have higher average LDRM values means that mode recovery can be detected by examining the average of LDRM values. To further investigate the mode recovery of the generated samples, we assign each generated sample to the closest mode and consider it a high-quality sample if it is within 4 standard deviations from the assigned mode. Then, we count the number of high-quality samples of each mode among the samples and analyze the correlation between the number of high-quality samples and the LDRM distribution. Modes with fewer high-quality samples tend to have higher LDRMs. This indicates that the LDRM of data instances can be used to detect regions of the data manifold not yet covered by the model, without looking at the generated samples.

In the present invention, as an algorithm for emphasizing underrepresented samples, a Stochastic Gradient Descent (SGD) algorithm sampled by inconsistency may be used. As an example, we can simply modify the training procedure of generative adversarial networks by using score-based weighted sampling for mini-batch SGD to highlight underrepresented samples.

Set D={x _i ] as the training dataset. A mini-batch of size B on the training dataset is , i.e. each sample x _i ∈ D is sampled with a certain probability P _s (i). The goal of the present invention is to design a sampling frequency P _s (i) that can highlight underrepresented samples. A mismatch score reflecting the underrepresentation of each sample, that is, the underrepresentation score s(x _i ;T) can be defined as in Equation 10.

[Equation 10]

where T is the set of steps used to compute the discordance score and k is a hyperparameter that modulates the contribution of each statistic. Equation 10 can be interpreted as an upper bound of the confidence interval of the LDR estimate, and can be interpreted as an upper bound of the square root of the LDRV controlled by the weight of the LDRM and k. Clip the minimum value of s(x _i ) to ε = 0.01 (min_clip) and the maximum value to a maximum-minimum ratio of 50, that is, max s (x) / min s (x) = 50 (max_clip) so that all data is sampled with at least some probability. For the clipped score s'(x _i ;T)=max_clip (min_clip(s(x _i ;T))), the final weighted sampling frequency is Same as

Analyze samples with the lowest or highest discordance score to ensure that the discordance score (underrepresentation score) really captures the underrepresentation sample. For example, train SNGAN on CIFAR-10 for 40k steps and measure the discrepancy score of each sample. Referring to FIG. 7, the image with the lowest discrepancy score (image in FIG. 7 (a)) and the image with the highest discrepancy score (image in (b) in FIG. 7) among the training images are presented and compared with the generated sample image (image in (c) in FIG. 7). Images with high discrepancies scores have characteristics that distinguish them from the generated samples (e.g., unusual backgrounds or shapes), whereas images with low discrepancies scores contain features that are also available in the generated samples. Comparing the pixel intensity histogram for the image of FIG. 7 , it can be seen that the difference in sample properties is more pronounced as shown in FIG. 8 . Images with a low discrepancy score show a similar intensity distribution to the generated sample, while images with a high discrepancy score show a very different trend. In addition, considering the partial Frechet Inception Distance (FID) (FID measured with a subset of the training samples) of the lowest/highest scoring groups, the partial FID scores of the group with high discordance scores are higher than those of the group with low discordance scores. A large gap between the two groups indicates that the generator is not generating samples similar to the high scoring group. These results imply that the discordance score successfully identifies underrepresented data that may need emphasis for further learning.

In the present invention, post-processing for discriminator rejection sampling (DRS) may be performed using an auxiliary discriminator.

Discordance score-based weighted sampling biases against under-represented samples during training. Although effective in improving diversity, this results in a modified data distribution p' _data (x)=f(x)p _data (x). Here, f(·) is the normalized sampling frequency. Therefore, the learned model distribution p _g may differ from the original data distribution p _data . To solve this, we utilize DRS to correct the bias after training. DRS is the probability for a constant M > 0 Accept samples generated by Performing DRS requires an estimate of p _data (x)/p _g (x) calculated based on the discriminator output. In the present invention, since the discriminator is learned with biased p' _data (x), an auxiliary discriminator is added during the weighted sampling procedure to obtain the LDR estimate (Equation 2) for the DRS, and uniform sampling (i.e., without applying the sampling technique). It can be learned and used for the DRS of the generated sample.

9 shows an example of a detailed algorithm for a self-diagnosing generative adversarial neural network in one embodiment of the present invention.

In summary, referring to FIG. 9, the generative adversarial network-based data generation method includes a learning and diagnosis phase (phase 1) in which the generative adversarial network is trained and a discord score (i.e., an underrepresentation score) is evaluated for each data instance, a score-based weighted sampling step (phase 2) in which the generative adversarial network learns an underrepresented region of the data manifold through discord score-based weighted sampling (phase 2), and model distribution p _g (x) is modified through DRS after training of the generative adversarial network. It may include a DRS phase (phase 3) of

As described above, according to the embodiments of the present invention, by identifying data that is not well represented by the model in the learning process of the generative adversarial neural network and emphasizing the corresponding data, the model can more evenly represent each training data, and through this, the diversity of generated data and the performance of data generation can be improved.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general purpose or special purpose computers, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will recognize that the processing device may include a plurality of processing elements and/or multiple types of processing elements. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired, or may independently or collectively direct a processing device. The software and/or data may be embodied in any tangible machine, component, physical device, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. In this case, the medium may continuously store a program executable by a computer or temporarily store the program for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or combined hardware, but is not limited to a medium directly connected to a certain computer system, and may be distributed on a network. Examples of media may include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media include recording media or storage media managed by an app store that distributes applications, a site that supplies or distributes various other software, and a server.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, even if the described techniques are performed in an order different from the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or replaced or substituted by other components or equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (15)

In the data generation method performed in a computer system,
The computer system includes at least one processor configured to execute computer readable instructions contained in a memory;
The data generation method,
measuring, by the at least one processor, an underrepresentation score representing an underrepresentation degree of each training data in a learning process of a generative adversarial network (GAN); and
Further learning, by the at least one processor, the generative adversarial network through weighted sampling based on the underrepresentation score.
Data generation method comprising a. According to claim 1,
The measuring step is
Measuring the underrepresentation score by calculating a discrepancy score between a data distribution and a model distribution in the training data.
Characterized by a data generation method. According to claim 1,
The generative adversarial neural network includes a generator neural network and a discriminator neural network,
The measuring step is
Measuring the underrepresentation score based on a log density ratio (LDR) based on an output value of the discriminator neural network
Characterized by a data generation method. According to claim 3,
The measuring step is
Measuring the underrepresentation score using the mean and variance of the LDR calculated in the learning process of the generative adversarial neural network
Characterized by a data generation method. According to claim 1,
The generative adversarial neural network includes a generator neural network and a discriminator neural network,
The measuring step is
Measuring the underrepresentation score using the mean and variance of output values of the discriminator neural network calculated in the learning process of the generative adversarial neural network
Characterized by a data generation method. According to claim 1,
Using a non-saturating loss function or a hinge loss function when learning the generative adversarial network
Characterized by a data generation method. According to claim 1,
The additional learning step is,
Adjusting the sampling probability of the training data in proportion to the underrepresentation score
Data generation method comprising a. According to claim 7,
The adjustment step is
Adjusting the sampling probability of the training data within a range where the ratio between the maximum sampling probability and the minimum sampling probability does not exceed a certain value
Characterized by a data generation method. According to claim 1,
The generative adversarial neural network includes a generator neural network and a discriminator neural network,
The data generation method,
Modifying, by the at least one processor, a model distribution of the generative adversarial neural network through rejection sampling using an output value of the discriminator neural network.
Data generation method further comprising a. In a computer program stored in a computer readable recording medium to execute a data generating method on a computer,
The data generation method,
measuring an underrepresentation score representing an underrepresentation degree of each training data in a learning process of a generative adversarial network;
additionally learning the generative adversarial neural network through weighted sampling based on the underrepresentation score; and
Modifying the model distribution of the generative adversarial neural network through reject sampling using an output value of the discriminator neural network.
Including, a computer program. In a computer system,
at least one processor configured to execute computer readable instructions contained in memory;
including,
The at least one processor,
A process of measuring an underrepresentation score indicating the degree of underrepresentation of each training data in the learning process of a generative adversarial network; and
A process of additionally learning the generative adversarial network through weighted sampling based on the underrepresentation score
A computer system that processes According to claim 11,
The at least one processor,
Measuring the underrepresentation score by calculating a discrepancy score between a data distribution and a model distribution in the training data.
Characterized by a computer system. According to claim 11,
The generative adversarial neural network includes a generator neural network and a discriminator neural network,
The at least one processor,
Measuring the underrepresentation score based on an LDR based on an output value of the discriminator neural network.
Characterized by a computer system. According to claim 11,
The at least one processor,
Adjusting the sampling probability of the training data in proportion to the underrepresentation score within a range where the ratio between the maximum sampling probability and the minimum sampling probability does not exceed a certain value
Characterized by a computer system. According to claim 13,
The at least one processor,
Modifying the model distribution of the generative adversarial neural network through reject sampling using the output value of the discriminator neural network.
Characterized by a computer system.

Images