KR102093090B1 - System and method for classifying base on generative adversarial network using labeled data - Google Patents

Abstract

Disclosed are a generative adversarial network based classification system using labeled data and a method thereof. The generative adversarial network based classification system can be trained even by using a labeled data set having missing data and an unbalanced data set by using missing data generated by a generative adversarial network (GAN). The generative adversarial network based classification system comprises a generator (100), a discriminator (200), an actor (400), and a weight function part (500).

Description

A system and method for generating hostile neural networks based on label data {SYSTEM AND METHOD FOR CLASSIFYING BASE ON GENERATIVE ADVERSARIAL NETWORK USING LABELED DATA}

The present invention relates to a system and method for generating hostile neural networks based classification systems using label data, and more specifically, missing data using missing alternative values generated by a generative adversarial network (GAN) The present invention relates to a system and method for generating hostile neural network based classification using label data that can be learned from labeling data sets and unbalanced data sets.

Machine learning is an application of artificial intelligence that allows complex systems to learn and improve automatically from experience without being explicitly programmed.

The accuracy and effectiveness of machine learning models can depend in part on the data used to train them.

For example, machine learning classifiers can be trained using a set of labeled (or labeled data), where samples of data to learn to be recognized by the classifier are classified into samples. ) Is provided to the classifier with one or more labels identifying it.

Here, the labeled data means that the answer to the data is given (or evaluated).

However, in a decision-making system, it often happens that the following problems arise.

One is bad data processing that includes information such as missing data, and the missing data lowers the overall quality of the data set, and there is a problem of distorting the predicted results from the decision system.

The other is the imbalance of the data set, the difference between these imbalances is very severe, and a small number of classes occupy only a very small part of the data, but as a result, samples of those classes are almost never in the process of updating the decision-making system. There is a problem that is not queried.

Korean Patent Application Publication No. 10-2019-0117969 (Invention name: Semi-supervised reinforcement learning method using labeled data and unlabeled data and apparatus using the same)

In order to solve this problem, the present invention uses a missing substitution value generated by a generative adversarial network (GAN) to label data that has missing data and can learn from unbalanced data sets. An object of the present invention is to provide a classification system and method based on a productive hostile neural network.

In order to achieve the above object, an embodiment of the present invention is a classification system based on a hostile neural network using label data, which generates a missing value for a missing part of a state from a labeled data set; A discriminator discriminating between the missing data and the original data generated by the generator; An actor predicting an action through a policy with a missing substitution value generated by the constructor; And a weight function unit that generates a weight of a reward based on the state replaced by the missing substitution value, the predicted action, and the label of the labeled data set, wherein the weight function unit includes a relatively small frequency label. In order to increase the weight of the reward, and to the label with a relatively high frequency, the weight of the reward is lowered so that the balance between the labels is operated, and the actor sets the weight of the reward generated in the predicted action and weight function unit. Reflect to learn the above policy to optimize the policy loss function,

Learning the above policy is

는 스테이트, 액션 및 레이블에 대한 리워드의 가중치 임 - 을 이용하는 것을 특징으로 한다.Where y is the label of the state, a is the action predicted by policy π for a given state,

Is characterized by using the weight of the reward for states, actions and labels.

In addition, the classification method based on the generative hostile neural network using label data according to an embodiment of the present invention is a label using a generative adversarial network (GAN) composed of a generator, a discriminator, an actor and a weight function unit. Using the data, a) generating a missing substitution value for the missing portion of the state from the labeled data set by the creator; b) actor predicting an action through a policy with a missing substitution value generated by the actor; c) generating a weight value of a reward based on a state in which the weight function unit is replaced with the missing substitution value, the predicted action, and the label of the labeled data set; And d) the actor learning the policy such that the policy loss function is optimized by reflecting the predicted action and the weight of the reward generated by the weight function unit.

Learning the above policy is

는 스테이트, 액션 및 레이블에 대한 리워드의 가중치 임 - 을 이용하고,Where y is the label of the state, a is the action predicted by policy π for a given state,

Is the weight of the rewards for states, actions, and labels-

In step c), the weight function unit operates such that the weight of the reward is increased for a label with a relatively low frequency, and the weight of the reward is lowered for a label with a relatively large frequency, thereby balancing the labels. do.

The present invention has an advantage in that it is possible to learn from a labeling data set and an unbalanced data set in which missing data is present using a missing substitution value generated by a generative hostile neural network (GAN).

1 is a block diagram showing the construction of a classification system based on a productive hostile neural network using label data according to an embodiment of the present invention.
2 is a flowchart illustrating a method for generating a hostile neural network based classification using label data according to an embodiment of the present invention.
FIG. 3 is a flow chart showing a process of learning missing data of a classification method based on a productive hostile neural network using the label data according to FIG. 2.
4 is a flowchart illustrating a classification learning process using weights of a classification method based on a productive hostile neural network using the label data according to FIG. 2.
FIG. 5 is a flowchart illustrating a weight estimation process of the classification method based on the productive hostile neural network using the label data according to FIG. 2.

Hereinafter, a preferred embodiment of a system and method for generating hostile neural networks based classification using label data according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

In the present specification, the expression that a part “includes” a certain component does not exclude other components, but means that other components may be further included.

In addition, terms such as "‥ unit", "‥ group", "‥ module" mean a unit that processes at least one function or operation, which can be divided into hardware or software, or a combination of the two.

In addition, throughout the detailed description and claims of the present invention, 'learning' or 'learning' is a term that refers to performing machine learning through computing according to a procedure in a computer system. It is not intended to refer to the same mental action, and training is used in a generally accepted sense of machine learning.

In addition, the computing device includes a communication device and a processor, and can communicate directly or indirectly with an external computing device through the communication device.

Specifically, a computing device includes typical computer hardware (eg, a device that may include components of a computer processor, memory, storage, input and output devices, and other existing computing devices; electronic communication devices such as routers, switches, etc.); Electronic information storage systems such as network-attached storage (NAS) and storage area network (SAN) and computer software (i.e., instructions that cause a computing device to function in a particular way). It may be that the desired system performance is achieved using a combination.

The communication device of such a computing device can transmit and receive requests and responses with other computing devices to be interlocked. As an example, such requests and responses may be made by the same Transmission Control Protocol (TCP) session. It is not limited, for example, it may be transmitted and received as a user datagram protocol (UDP) datagram.

In addition, in a broad sense, the communication device may include a keyboard, a mouse, other external input devices, a printer, a display, and other external output devices for receiving commands or instructions.

In addition, the processor of the computing device is a micro processing unit (MPU), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU) or a tensor processing unit (TPU), cache memory, data It may include a hardware configuration such as a bus (data bus).

1 is a block diagram showing the configuration of a classification system based on a productive hostile neural network using label data according to an embodiment of the present invention, and FIG. 2 is a productive hostile neural network using label data according to an embodiment of the present invention Fig. 3 is a flow chart showing a process of missing data learning of a productive hostile neural network based classification method using the label data according to Fig. 2, and Fig. 4 is a generation using the label data according to Fig. 2 It is a flowchart showing a classification learning process using weights of the hostile neural network based classification method, and FIG. 5 is a flowchart showing a weight estimation process of the productive hostile neural network based classification method using the label data according to FIG. 2.

Referring to FIGS. 1 to 5, a classification system based on a productive hostile neural network using label data includes a constructor 100, a discriminator 200, an actor 400, and a weight function unit 500 It is composed.

The generator 100 and the discriminator 200 use the generative adversarial network (GAN), which is a network in a competitive structure, so that the generator 100 looks at the distribution of the original data and deceives the discriminator 200. The learning for the generation of the replacement value is performed, and the discriminator 200 performs learning to discern which data is the data generated by the constructor 100.

In addition, the constructor 100 performs learning for generating a missing replacement value that deceives the discriminator 200 by referring to the distribution of the original data.

In addition, the generator 100 is a generative hostile neural network based classification system using labeled data S _L , and generates a missing replacement value from the labeled data set 10.

In addition, the generator 100 may perform a learning process for generating a missing substitute value from an unlabeled data set rather than a labeled data set as a pre-processing process.

In addition, the constructor 100 is an input for generating a missing substitution value, n states (states) from the data set 10, and n missing indicators 20 indicating whether elements of states corresponding to n states are missing. m _L ).

Here, S _L means that each state is a labeled data set, and m _L means a labeled missing indicator.

In addition, the labeled data set 10 includes n states consisting of S ₁ , S ₂ , S ₃ , ..., S _n ∈ R ^d , where d is a state feature.

Further, the j-th element of state i is expressed as s _i ^j , where j is a constant up to d, and s _i ^j has a scalar or missing value.

Further, the data set 10 may be composed of at least one of labeled data and unlabeled data.

In addition, the missing indicator 20 is an indicator for indicating whether the element of the state is missing, m ₁ , m ₂ , m ₃ , ‥, m _n ∈ R ^d is used, where m _i ^j is s _i ^j If there is missing data, the missing indicator value 22 is displayed as '0', otherwise the missing indicator value 21 is displayed as '1'.

)을 입력 받아 계산한다.In addition, the constructor 100 is a preset value to a missing element 12 randomly (randomly) selected for any element 11 among n states, for example, an equality between '0' and '1'. Missing substitution values replaced by random noise 'Z' from the distribution (

) To calculate.

)은 하기식을 통해 입력으로 받는다. At this time, the missing value

) Is received as input through the following equation.

Here, m is a vector of the missing indicator corresponding to the state s, z is a vector of noise randomly selected from the uniform distribution between '0' and '1', and can be expressed as an element-by-element product.

)을 이용하여 생성된 원소들의 벡터로 이루어진 스테이트(

)를 출력한다.In addition, the constructor 100 is the missing replacement value (

State consisting of vectors of elements created using)

).

)로 대체된 결측 대체값(

)을 생성하여 출력한다.In addition, the constructor 100 is a state (

Missing substitution value replaced by)

) And print it out.

)에 해당하는 데이터가 판별자(200)의 학습을 위해 사용될 수 있도록 한다. At this time, the missing substitution value (following the following equation) through the output of the constructor (100

) So that the data corresponding to) can be used for the learning of the discriminator 200.

Here, m is a vector of the missing indicator corresponding to state s.

)과 원본 데이터를 구분하는 구성으로서, 판별자(200)가 출력한 것의 각 원소들이 결측(fake)인지 아닌지(real)를 구분하고, 결과적으로 m은

을 위한 레이블로 사용될 수 있다.* The discriminator 200 is a missing substitution value generated by the constructor 100 (

) And the original data, each element of what is output by the discriminator 200 distinguishes whether it is missing or not, and consequently m is

Can be used as a label for.

의 i번째 원소가 결측 데이터가 아닐 확률에 해당하는 판별자(200)의 i번째 출력을 S →[0, 1]^d로나타낼 수 있다.In addition, the discriminator 200 is a state through a function

The i-th output of the discriminator 200 corresponding to the probability that the i-th element of is not the missing data may be represented by S → [0, 1] ^d .

Also, the discriminator 200 may represent the output as D ₁ , D ₂ , D ₃ , ..., D _d through the discriminator output indicator 30.

Meanwhile, the generator 100 and the discriminator 200 may be learned through a loss function, and the constructor loss function for learning the constructor 100 may be as follows.

Here, the generator loss function may be composed of two terms, and the first term is to maximize the probability D ⁱ for the missing data.

In addition, the second term is a reconstruction loss (40) that converts the missing data generated in the generator 100 to be close to the original data using the original data distribution, where λ is a scale factor. .

In addition, the discriminator loss function for learning the discriminator 200 may be as follows.

Discriminator loss function is the i th element missing data, can be configured to learn a direction to maximize the probability of D and ^i, or learning in a direction to minimize the probability of D ⁱ contrary.

The actor 400 predicts a probability of performing an action from a labeled data set using a policy of a vector of missing replacement values generated by the constructor 100.

In addition, the actor 400 may be a component of the 'Actor-critic' architecture, which is a well-known decision-making framework in reinforcement learning.

In addition, the actor 400 receives a state as an input, outputs a probability of performing a given action, and uses 'Actor-critic' to learn Policy π, a policy loss function, 41) may be defined as follows.

는 주어진 스테이트에서 예측된 액션이 좋은지 또는 나쁜지를 결정하는 크리틱(Critic)으로부터 평가되는 함수이다.here,

Is a function evaluated from critic that determines whether the predicted action in a given state is good or bad.

는 'total discounted reward', 'action-value function' 또는 'TD-error'와 같은 형태를 가질 수도 있다.Also,

May have the form of 'total discounted reward', 'action-value function' or 'TD-error'.

The above-described policy loss function 41 is a general form in which an action is not determined, and the actor 400 must be learned from both correct and incorrect actions.

의 추정치가 나쁜 경우, 그 정책 손실 함수는 잘못된 방향으로 최적화를 하게 되고, 그 결과, 천천히 수렴하거나 또는 발산하게 될 수 있다.But,

If the estimate of is poor, the policy loss function is optimized in the wrong direction, and as a result, it can converge slowly or diverge.

Accordingly, the actor 400 according to an embodiment of the present invention may be defined in the following manner so that the policy loss function 41 is omitted from learning from an incorrect action, and only the correct label given is used.

는 스테이트, 액션 및 레이블에 대한 리워드의 가중치이다.Where y is the label of the state, a is the action predicted by policy π for a given state,

Is the weight of the rewards for states, actions and labels.

를 가중치 함수(Weighted Function) W로 대체한다.That is, the predicted action is replaced with the correct label, and the function

Replace W with a Weighted Function W.

로부터 얻은 분류 손실 가중치(Classification loss weighted)이다.Therefore, supervised policy loss L _L is a weight function

Classification loss weighted from.

In addition, when the weight function is '1' for all states, actions, and labels, L _L is completely equal to the classification loss weight.

In addition, the actor 400 may learn the map policy using the weight value of the reward generated by the weight function unit 500 by the policy loss function 41 for map classification.

로부터 가져올 수 있는 리워드의 가중치로서, 레이블이 있는 데이터 세트로부터 레이블의 빈도수에 기반하여 스테이트, 액션 및 레이블에 대한 리워드의 가중치를 생성한다.The weight function unit 500 is a state

As a weight of a reward that can be obtained from, a reward weight for states, actions, and labels is generated based on the frequency of the label from the labeled data set.

Here, assuming that the weight function unit 500 has a data set S _L with K labels (k = 0, 1, ..., K-1), the frequency of the K-th label is approximated by the following equation. Can be.

는 (0, 1)의 범위 안에 있다.Where n _k is the number of samples in the k-th label,

Is in the range of (0, 1).

Further, the weighting coefficient ω _k can be estimated by the following equation for each label.

Here, b is based on the log (b = e, 10, ...).

Therefore, by giving a weight of high rewards to a small number of labels (minority lable) having a relatively small frequency, and by giving a weight of lower rewards to a plurality of labels (high) having a relatively high frequency of labels. , It can be created to balance the labels.

Also, the weight function unit 500 may define a weight function, that is, a weight of rewards for states, actions, and labels, using the following equation.

는 스테이트

로부터 가져올 수 있는 리워드이고, a는 주어진 스테이트에 대한 정책 π가 예측한 액션이며, y는 스테이트의 레이블이고, ω_y와 ω_a는

(b 는 로그에 기초한 e, 10 …)에 기반한 가중 계수이다.here,

The state

Is a reward that can be fetched from, a is the action predicted by policy π for a given state, y is the label of the state, and ω _y and ω _a are

(b is a weighting coefficient based on e, 10…) based on a logarithm.

The following describes a classification and learning method based on a productive hostile neural network using label data according to an embodiment of the present invention.

The learning procedure can be divided into two steps: generating a missing value (S100) and generating a learning policy (S200).

And, each of the steps S100 and S200 can be updated while iterating through various epochs of a labeled data set, and turning a data set once is called 1 epoch.

In addition, a generative adversarial network (GAN) composed of a generator 100, a discriminator 200, an actor 400, and a weight function unit 500 may be used.

First, in step S100 of generating a missing substitution value, the generator 100 and the discriminator 200 are learned, and n states are randomly (randomly) from the data set to be input to the generator 100 in each iteration. A step (S110) of selecting and n missing indicators (m) indicating whether an element of the state corresponding to the state is missing is performed (S120).

At this time, steps S110 and S120 may be provided from an external terminal or may be provided from a preset data set.

Further, in steps S110 and S120, the data set may be a data set consisting of at least one of labeled data and unlabeled data.

)과, 스테이트(

)와, 결측 대체값(

)을 계산(S140)한다.Constructor by selecting (S130) a vector that is replaced with random noise 'Z' (here Z ∈ [0, 1]) from a uniform distribution between '0' and '1' in a predetermined value for n states. When input as (100), the generator 100 is the missing replacement value (

) And state (

) And the missing value (

) Is calculated (S140).

는 노이즈 'Z'로 대체된 결측 대체값이고,

는 생성자(100)에 의해 생성된 스테이트를 나타내며,

는 생성자에 의해 생성된 값으로 대체된 결측 대체값이다.here,

Is the missing replacement value replaced by the noise 'Z',

Indicates a state generated by the constructor 100,

Is the missing replacement value, replaced by the value generated by the constructor.

)으로 이루어진 벡터를 입력받아 계산하는데, 하기식을 통해 입력으로 받는다. In step S140, the generator 100 is a missing substitution value (replaced by random noise 'Z')

) Is calculated by receiving the input vector.

= G(

)를 통해

∈ R^d를 계산하여 스테이트(

)를 생성한다.Also, the constructor 100

= G (

)Through the

∈ Calculate the state of R ^d

).

)로 대체된 결측 대체값으로 이루어진 벡터인 결측 대체값(

)을 계산하는데, 하기식을 통해 계산될 수 있다.In addition, the constructor 100 is generated state (

A missing replacement value, which is a vector of missing replacement values replaced by)

), It can be calculated through the following equation.

)은 판별자(200)로 제공되고, 판별자 손실 함수를 이용하여 판별자(200)가 학습(S150)되도록 한다.Also, the missing replacement value generated by the constructor 100 (

) Is provided to the discriminator 200, and uses the discriminator loss function to allow the discriminator 200 to learn (S150).

)은 생성자 손실 함수를 이용하여 생성자(100)가 학습(S160)되도록 한다.Also, the missing replacement value generated by the constructor 100 (

) Allows the constructor 100 to learn (S160) using the generator loss function.

Meanwhile, in order to learn all components, an 'Adam optimizer' that optimally adjusts an update rate for each parameter may be used.

Step (S200) of generating a learning policy is n (State) randomly (random) from the labeled data set (S _L ) at each iteration, and n indicating whether the element of the state corresponding to the state is missing. Dog missing indicator (m _L ) is selected (S210).

_L)과, 스테이트(

_L)와, 결측 대체값(

_L)을 계산(S230)한다.Subsequently, a vector substituted with random noise 'Z' (here Z ∈ [0, 1]) is selected from the uniform distribution between '0' and '1', which is a preset value for n states (S220). ) And input to the constructor 100, the generator 100 is missing missing replacement value (

_L ) and state (

_L ) and the missing value (

_L ) is calculated (S230).

_L은 노이즈 'Z'로 대체된 결측 대체값이고,

_L은 생성자(100)에 의해 생성된 스테이트를 나타내며,

_L은 생성자에 의해 생성된 값으로 대체된 결측 대체값이다.here,

_L is the missing replacement value replaced by the noise 'Z',

_L represents the state generated by the constructor 100,

_L is a missing substitution value that is replaced by the value generated by the constructor.

_L)으로 이루어진 벡터를 입력받아 계산하는데, 하기식을 통해 입력으로 받는다. In step S230, the generator 100 is the missing substitution value replaced by random noise 'Z' (

Calculate by taking a vector consisting of _L ), and receive it through the following formula.

_L = G(

_L)를 통해

_L ∈ R^d를 계산하여 스테이트(

_L)를 생성한다.Also, the constructor 100

_L = G (

Through _L )

Calculate the state of _L ∈ R ^d (

_L ).

_L)로 대체된 결측 대체값으로 이루어진 벡터인 결측 대체값(

_L)을 계산하는데, 하기식을 통해 계산될 수 있다.In addition, the constructor 100 is generated state (

A missing replacement value, which is a vector of missing replacement values replaced by _L )

_L ), which can be calculated through the following equation.

_L)이 정책

을 통해 액션을 수행할 확률값을 예측(S240)한다.Subsequently, the actor 400 generates the missing missing replacement value (

_L ) this policy

Predict the probability value to perform the action through (S240).

At this time, the weight function unit 500 uses the weight function to generate weights of rewards for states, actions, and labels through the following equation (S250).

In addition, in step S250, the weight function unit 500 is a weight of a reward that can be obtained from the state, and can be reflected as a weight of the reward for states, actions, and labels based on the frequency of the label from the labeled data set. .

At this time, the label frequency may be approximated through the following equation.

Subsequently, the weight generated in step S250 is learned through the map policy loss function 41 using the following equation (S260).

는 스테이트, 액션 및 레이블에 대한 리워드의 가중치이다.Where y is the label of the state, a is the action predicted by policy π for a given state,

Is the weight of the rewards for states, actions and labels.

Meanwhile, in order to learn all components, an 'Adam optimizer' that optimally adjusts an update rate for each parameter may be used.

Therefore, it is possible to learn from the labeling data set and the unbalanced data set in which missing data is present by using the missing replacement value generated by the generative hostile neural network (GAN).

As described above, although described with reference to preferred embodiments of the present invention, those skilled in the art variously modify and change the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You can understand that you can.

In addition, the drawing numbers described in the claims of the present invention are merely for clarity and convenience of description, and are not limited thereto. In the course of explaining the embodiment, the thickness of the lines or the size of components shown in the drawings, etc. May be exaggerated for clarity and convenience.

In addition, the above-mentioned terms are terms defined in consideration of functions in the present invention, which may vary depending on the intention or custom of the user or operator, and thus interpretation of these terms should be made based on the contents throughout the present specification. .

In addition, even if not explicitly shown or described, a person having ordinary knowledge in the technical field to which the present invention pertains can make various modifications including the technical spirit according to the present invention from the description of the present invention. Obviously, it is still within the scope of the present invention.

In addition, the above-described embodiments described with reference to the accompanying drawings are described for the purpose of illustrating the present invention and the scope of the present invention is not limited to these embodiments.

10: data set
11: element
12: missing element
20: missing indicator
21, 22: missing indicator value
30: discriminator output indicator
40: loss function
41: policy loss function
100: constructor
200: discriminator
400: actor
500: weight function unit

Claims (2)

A generator 100 for generating a missing value for the missing portion of the state from the labeled data set 10;
A discriminator 200 for distinguishing the missing data and the original data generated by the constructor 100;
An actor 400 predicting an action through a policy with a missing substitution value generated by the constructor 100; And
It includes; a weight function unit 500 for generating a weight of a reward based on the state replaced by the missing substitution value, the predicted action, and the label of the labeled data set;
The weight function unit 500 operates so that the weight of the reward increases for a label with a relatively low frequency, and the weight of the reward decreases for a label with a relatively large frequency, thereby balancing the labels.
The actor 400 learns the policy to optimize the policy loss function 41 by reflecting the predicted action and the weight of the reward generated by the weight function unit 500,
Learning the above policy is

Where y is the label of the state, a is the action predicted by policy π for a given state,

Is a constructive hostile neural network based classification system using label data, characterized by using weights of rewards for states, actions, and labels. A classification method using label data using a generative adversarial network (GAN) composed of a generator 100, a discriminator 200, an actor 400, and a weight function unit 500,
a) the generator 100 generating a missing replacement value for the missing portion of the state from the labeled data set 10;
b) actor 400 predicting an action through a policy with a missing substitution value generated by the constructor 100;
c) the weight function unit 500 generating a weight value of the reward based on the state replaced by the missing substitution value, the predicted action, and the label of the labeled data set; And
d) the actor 400 reflecting the predicted action and the weight of the reward generated by the weight function unit 500, comprising the step of learning the policy so that the policy loss function 41 is optimized,
Learning the above policy is

Where y is the label of the state, a is the action predicted by policy π for a given state,

Is the weight of the rewards for states, actions, and labels-
In step c), the weight function unit 500 operates so that the weight of the reward increases for a label with a relatively low frequency, and the weight of the reward decreases for a label with a relatively large frequency, thereby balancing the labels. Genetic hostile neural network based classification method using label data, characterized in that.

Images