KR20200049273A - A method and apparatus of data configuring learning data set for machine learning - Google Patents

Abstract

The present invention relates to a method for constructing a training data set labeled for machine learning. The method for constructing a training data set labeled for machine learning comprises the steps of: calculating, by a utility module, a ratio (L1 / L2) of the total number (L1) of first labeled data and the total number (L2) of second labeled data that are included in a training data set, wherein L1 > L2; and generating, by the utility module, the calculated N number of subsets; and constructing, by the utility module, each generated subset to include the same number of the first labeled data and the second labeled data.

Description

{A method and apparatus of data configuring learning data set for machine learning}

The present invention relates to a method and apparatus for configuring a learning data set for machine learning.

Machine learning is a method of learning a computer using data, and includes supervised learning, unsupervised learning, and reinforcement learning.

Of these, supervised learning is a method of learning a computer with a label "explicit correct answer" for data, and using a set of labeled learning data.

When learning with the learning data set configured in this way, the computer can determine what result value the label has for the new data.

In order to use the learned computer in various environments, it is necessary to make correct judgments about normal and abnormal situations by learning both correct and incorrect answers in the course of learning.

If the amount of each data set for the correct and incorrect answers is not equal, that is, the amount of data set on one side is significantly higher than the data set on the other side, the computer must learn to be biased. . It is difficult to trust the result of the computer learning. Therefore, it is necessary to properly configure the training data set so that the computer can proceed with unbiased learning.

Registered Patent Publication No. 10-1590896, 2016.02.02.

The present invention for solving the above-described problems may provide a method of constructing a learning data set for machine learning that solves a data imbalance problem in constructing a learning data set for machine learning.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A method of configuring a learning data set for machine learning according to an embodiment of the present invention for solving the above-described problem includes: a total number (L1) and a second number of first label data included in the learning data set by a utility module Calculating N, which is a ratio (L1 / L2) of the total number of label data L2 (L1> L2); And the utility module generating a subset of the number corresponding to the calculated N; And configuring the utility module to include the same number of the first label data and the second label data in each of the generated subsets.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

The data classification program of learning data for machine learning according to another embodiment of the present invention is combined with a computer that is hardware to execute the above-mentioned method of constructing a learning data set for machine learning, and is stored in a medium.

According to the present invention as described above, it is possible to solve the data imbalance problem in the training data set and machine learning to balance the computer through the training data set, so that the computer can more accurately determine when new data is input. It has the effect.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flow chart for the steps that are generally performed in machine learning.
2 is a flowchart of a method of configuring a learning data set for machine learning according to an embodiment of the present invention.
3 is a diagram illustrating learning data received from a pre-processing module according to an embodiment of the present invention.
4 is a diagram illustrating that the ratio of the first label data to the second label data of the learning data according to an embodiment of the present invention is matched.
5 is a diagram illustrating that three subsets are generated according to an embodiment of the present invention.
6 is a diagram illustrating that a 5-fold verification set is generated.
7 is a diagram illustrating that a 3-fold verification set for cross verification according to an embodiment of the present invention is generated.
8 is a diagram illustrating a learning process performed in machine learning.
9 is a diagram illustrating performing a prediction step by applying a learning model generated through an embodiment of the present invention.
10 is a block diagram of an apparatus for configuring a learning data set for machine learning according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and are common in the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, “comprises” and / or “comprising” does not exclude the presence or addition of one or more other components other than the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and “and / or” includes each and every combination of one or more of the components mentioned. Although "first", "second", etc. are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless explicitly defined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Prior to the description, the meaning of terms used in the present specification will be briefly described. However, it should be noted that the description of terms is intended to aid understanding of the present specification, and is not used to limit the technical spirit of the present invention unless explicitly described as a limiting matter of the present invention.

Classifier module 150: serves to divide what category the data corresponds to.

Feature selection module 140: Selects and extracts features of the input and collected data.

1 is a flow chart for the steps that are generally performed in machine learning.

Referring to FIG. 1, steps generally performed in machine learning include an available data verification and problem definition step, a data search and preprocessing step, a learning algorithm selection step, a model verification and conclusion step.

First, you can check available data and conduct a collaborative interview. Then, the hypothesis will be materialized, and the direction of the future will be established. At this time, a DB schema and a blueprint may be required in advance.

Next, the propulsion direction established in all stages will be reviewed.

After confirming the actual data, the variables for each scale are identified, and after data is searched, data pre-processing is performed through scaling, anomaly missing value processing, and the like.

Next, as a learning algorithm selection step, the directionality reviewed in the previous step is verified.

Data labeling tasks and features are extracted, algorithms are selected, and a single model verification technique is performed.

Next, in the model verification and conclusion stage, the performance of the entire model is verified, and the model is finally selected. In addition, it draws various solutions and draws consensus to draw conclusions.

The above-described steps are flowcharts of steps performed in machine learning, and the embodiments of the present invention to be described below are contents corresponding to the step of selecting a learning algorithm, and additionally, a model verification and conclusion step will also be covered.

2 is a flowchart of a method of configuring a learning data set for machine learning according to an embodiment of the present invention, and FIG. 3 is a diagram illustrating learning data received from the pre-processing module 130 according to an embodiment of the present invention, 4 is a diagram illustrating that the ratio of the first label data to the second label data of the learning data according to an embodiment of the present invention is matched, and FIG. 5 shows that three subsets according to an embodiment of the present invention are generated It is an illustrated drawing.

With reference to FIGS. 2 to 5, a method of configuring a learning data set ( sub-set configuration ) for machine learning according to an embodiment of the present invention will be described.

At this time, it is assumed that the pre-processed learning data is received from the pre-processing module 130 as described in FIG. 1.

As an example of this, FIG. 3 shows learning data input from the pre-processing module 130.

In addition, in describing an embodiment of the present invention, the label (Label) means that the data is labeled. For example, the first label means an explicit correct answer, and the second label means an explicit incorrect answer, meaning that machine learning is performed for the correct answer and incorrect answer, but the first label and the second label are different feature values of the data. Can correspond to

First, the utility module 110 calculates the ratio (L1 / L2) N of the total number (L1) of the first label data and the total number (L2) of the second label data included in the learning data set (however, L1 > L2). (Step S510)

The existing learning method was a method in which machine learning is performed by learning the learning data received from the pre-processing module 130 as it is.

Machine learning itself could be performed using this method, but there was a problem in that the ratio of the first label data to the second label data was not correct, so that learning was performed to be biased due to a data imbalance problem.

In other words, if the classifier is trained with unbalanced data on the label value, the deflected value is predicted when applied in the actual field, which is a major cause of various errors and accidents.

Here, it is assumed that L1> L2, and a ratio of the total number of first label data to the total number of second label data is calculated to calculate the natural number N.

In some embodiments, two types of label data, not two types of label data, may be included in the learning data set, and the learning data set includes L1 first label data to Ln nth label data (however, L1 > L2>?> Ln), where the first label data has the largest number of data and the second label data has the lowest number of data.

Next to step S510, the utility module 110 generates a subset of the number corresponding to N calculated in step S510. (Step S520)

As shown in FIG. 5, it can be seen that a first sub-set, a second sub-set, a third sub-set, and a total of three sub-sets were generated.

In addition, it can be seen that (3 + 2 + 5) / (1 + 1 + 1) = 3 is calculated by applying L1 / L2 = N as the number of subsets.

Next, in step S520, the utility module 110 is configured to include the same number of first label data and the second number of label data in each of the generated subsets. (Step S530)

As shown in FIG. 4, it can be seen that one first label data is added to the cases of FIGS. 3 and 2, and one first label data is deleted to the three cases. Then, it can be seen that the ratio of the overall first label data to the second label data is set to 3: 1.

Then, the utility module 110 classifies the first label data and the second label data set of each case into the generated subset as shown in FIG. 5. At this time, the set of the first label data and the second label data means that the first label data and the second label data are paired together as shown in FIG. 5.

And, the utility module 110 is characterized by including L1 / N first label data and second label data set in each subset.

More specifically, the utility module 110 includes a set of L1 / N first label data and second label data from one sub-set to an N-sub set, but the first is included in a subset in which the second label data is insufficient. And copying the second label data of the subset.

That is, the utility module 110 divides the first label data included in the learning data set into N equal parts and includes them in each subset, and each subset includes learning to include all of the second label data included in the learning data set. And copying the second label data included in the data set.

In some embodiments, two types of label data, not two types of label data, may be included in the learning data set, and the learning data set includes L1 first label data to Ln nth label data (however, L1 > L2>?> Ln), where the first label data has the largest number of data and the second label data has the smallest number of data, the utility module 110 has the same number in each generated subset. It can be configured to include the first label data from the n-th label data. To this end, at least a portion of the second label data to the n-th label data may be copied and used.

In some embodiments, in the step of calculating N (S510), the utility module 110 rounds up to calculate N when N is not a natural number, and in the configuration step (S530), the utility module 110 displays the first label data. It is also possible to copy the first label data of the first sub-set to a subset that does not reach.

For example, when the total number (L1) of first label data included in the learning data set received from the pre-processing module 130 is 13 and the total number (L2) of second label data is 3, N is 4.333 Is calculated as In this case, it means that N is not set to 4, but is rounded up to 5.

In addition, when N is set to 5, 5 subsets are generated to classify and copy the first label data and the second label data, so a subset may occur in the first label data. It means that the first label data of the subset is copied to maintain the ratio of the first label data to the second label data.

Through the above steps S510, S520, and S530, the steps of configuring the subset are completed.

Hereinafter, steps for performing cross-validation will be described.

2 is a flowchart of a method of configuring a learning data set for machine learning according to an embodiment of the present invention, FIG. 6 is a diagram illustrating that a 5-fold verification set is generated, and FIG. 7 is an embodiment of the present invention It is a diagram illustrating that a 3-fold verification set for cross verification according to is generated.

When separating the training data into training and verification data, the verification results vary slightly depending on how they are separated. For this reason, the data is separated in several ways to perform training and verification, and the model performance is judged / evaluated with the average and variance of the verification results.

Cross validation is one of the methods to separate data and perform learning and verification multiple times. In FIG. 6, validation data to which five iterations are applied in k-Fold Cross Validation is illustrated.

In the embodiment of the present invention, as shown in FIG. 7, first, the pipeline module 120 generates N verification sets, and the first label data and the second label data of the subset are added to each verification set at the same ratio, and N Place learning group data. (Step S540)

At this time, N means N (= L1 / L2) calculated in step S510.

Next, in step S540, the pipeline module 120 places the verification data in one of the N learning group data of each verification set, but arranges the order of verification data differently in different verification sets, and places the verification data in N verification sets. Do not overlap the order of verified data. (Step S550)

As described above, since the method of arranging the verification data of the cross-validation method is applied to the verification set using the subset generated in the sub-configuration steps, more reliable verification can be performed.

Next, in step S550, the pipeline module 120 performs N times of cross validation using the generated N verification sets. (Step S560)

Through step S560, the pipeline module 120 performs substantial cross-verification using the verification set generated in step S550.

Next to step S560, the pipeline module 120 checks the verified result through the cross-validation in step S560 to extract the error value. (Step S570)

By performing the verification process as described above, extracting errors and error values is to select the optimal (best) verification set by tuning, correcting, and correcting it in consideration of the subsequent steps.

Next to step S570, the pipeline module 120 performs tuning for each verification set through the error value extracted in step S560, and stores the verification set that satisfies the optimal condition among the N verification sets. (Step S580)

At this time, the verification set that satisfies the optimal condition means selecting one or more verification sets that are judged to have the best (appropriate, conforming) verification result from among the verification sets that have performed verification.

8 is a diagram illustrating a learning process performed in machine learning.

FIG. 8 is a diagram for briefly explaining the above-described items with reference to FIGS. 2 to 7, in which the pre-processing module 130 performs pre-processing, and the feature selection module 140 selects features to store a feature selection model and classifier. It is explained that the module 150 trains the classifier to proceed with the learning process as a flow for storing the classifier model.

The pre-processing module 130 performs pre-processing and verifies to store the pre-processing model, the feature selection module 140 selects and verifies the feature to store the feature selection model, and the classifier module 150 proceeds to classifier learning and verification It explains how to store the classifier model and perform the evaluation.

By performing such processes, the data classification device 10 according to an embodiment of the present invention repeats the learning process to find an appropriate hyperparameter, repeats the learning process while changing the hyperparameters of the classifier, and has good evaluation results. The parameters are selected.

At this time, the hyper parameter means a parameter that determines the characteristics of the machine learning model.

9 is a diagram illustrating performing a prediction step by applying a learning model generated through an embodiment of the present invention.

Referring to FIG. 9, the prediction step is performed in a manner similar to the verification step, and is performed using a model generated / learned through configurations according to an embodiment of the present invention. It cannot be done.

10 is a block diagram of a configuration device 10 of a learning data set for machine learning according to an embodiment of the present invention.

The apparatus 10 for constructing a learning data set for machine learning according to an embodiment of the present invention includes a utility module 110, a verification module, a pre-processing module 130, a feature selection module 140, and a classifier module 150 do.

However, in some embodiments, the data classification device 10 may include fewer components or more components than those illustrated in FIG. 10.

The utility module 110 calculates the ratio (L1 / L2) N of the total number (L1) of the first label data and the total number (L2) of the second label data included in the learning data set, and the calculated N The number of subsets corresponding to is generated, and the set of the first label data and the second label data are configured in the same number and ratio in each of the generated subsets.

In addition, the utility module 110 is characterized by classifying a set of L1 / N first label data and second label data into each subset during the copying process.

More specifically, the utility module 110 separately includes a set of L1 / N first label data and second label data from one sub-set to N sub-sets, but the second label data is less than a subset It is characterized in that the second label data of one sub-set is copied.

In addition, when calculating N, when the N is not a natural number, the utility module 110 determines N by raising N from the decimal point, and sets the first sub in a subset in which the first label data is not reached in the process of classifying and copying. It characterized in that the first label data of the copy.

The pipeline module 120 generates N verification sets, and the N learning group data is arranged by adding the first label data and the second label data of the subseng at the same ratio to each verification set.

In addition, the pipeline module 120 arranges verification data in one of the N learning group data of each verification set, but the order of verification data is differently arranged in different verification sets, and verifications are arranged in the N verification sets. Avoid ordering data.

In one embodiment, the pipeline module 120 performs N times of cross validation using the generated N verification sets. Then, the pipeline module 120 extracts an error value by checking the result verified through the cross-validation.

Then, the pipeline module 120 performs tuning for each verification set through the extracted error value, and stores a verification set that satisfies an optimal condition among the N verification sets.

Other detailed descriptions of configurations or configurations are omitted because only the method of constructing the learning data set for machine learning described above with reference to FIGS. 1 to 9 and the category of the invention are different.

The method of configuring the learning data set for machine learning according to the above-described embodiment of the present invention is implemented as a data program (or application) of learning data for machine learning in order to be executed in combination with a server that is hardware. Can be stored in.

The above-described program is C, C ++, JAVA, machine language, etc., in which a processor (CPU) of the computer can be read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program. It may include a code (Code) coded in the computer language of. Such code may include functional code related to a function or the like that defines necessary functions for executing the above methods, and control code related to an execution procedure necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include a memory reference-related code as to which location (address address) of the internal or external memory of the computer should be referred to additional information or media necessary for the computer's processor to perform the functions. have. Also, when the processor of the computer needs to communicate with any other computer or server in the remote to execute the functions, the code can be used to communicate with any other computer or server in the remote using the communication module of the computer. It may further include a communication-related code for whether to communicate, what information or media to transmit and receive during communication, and the like.

The storage medium refers to a medium that stores data semi-permanently and that can be read by a device, rather than a medium that stores data for a short time, such as registers, caches, and memory. Specifically, examples of the medium to be stored include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers that the computer can access or various recording media on the computer of the user. In addition, the medium is distributed in a computer system connected by a network, so that the computer readable code may be stored in a distributed manner.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, a software module executed by hardware, or a combination thereof. The software modules may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer readable recording medium well known in the art.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but a person skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive.

10: Configuration device
110: utility module
120: pipeline module
130: pre-processing module
140: feature selection module
150: classifier module

Claims (10)

In the method of constructing a set of labeled learning data for machine learning,
Calculating, as a ratio (L1 / L2) N of the total number of first label data (L1) and the total number of second label data (L2) included in the learning data set (L1> L2) ); And
Generating, by the utility module, a subset of the number corresponding to the calculated N;
The utility module is configured to include the same number of the first label data and the second label data in each of the generated subsets.
How to configure a learning data set for machine learning.
According to claim 1,
In the configuration step,
The utility module is characterized in that the first label data included in the learning data set is divided into N equal parts and included in each subset.
How to configure a learning data set for machine learning.
According to claim 2,
In the configuration step,
The utility module is characterized in that the second label data included in the learning data set is copied so that all of the second label data included in the learning data set are included in each subset.
How to configure a learning data set for machine learning.
According to claim 1,
The learning data set includes L1 first label data to Ln nth label data (L1>L2>?> Ln),
In the configuration step,
The utility module is configured to include the same number of the first label data to the n-th label data in each of the generated subsets,
How to configure a learning data set for machine learning.
According to claim 1,
A pipeline module generating N verification sets, and placing N learning group data in each verification set by adding the first label data and the second label data of the subset at the same ratio;
The pipeline module places verification data in one of the N learning group data of each verification set, but the order of verification data arranged in the N verification sets is differently arranged by disposing the order of verification data in different verification sets. The step of arranging the verification data to prevent overlapping, further comprising
How to configure a learning data set for machine learning.
The method of claim 5,
The pipeline module performing N cross validations using the generated N verification sets;
The pipeline module extracts an error value by checking a result verified through the cross-validation.
How to configure a learning data set for machine learning.
The method of claim 6,
Further comprising, performing tuning for each verification set through the extracted error value, and storing a verification set that satisfies an optimal condition among the N verification sets.
How to configure a learning data set for machine learning.
In the configuration device of the learning data set constituting the learning data set labeled for machine learning,
The ratio (L1 / L2) N of the total number (L1) of the first label data and the total number (L2) of the second label data included in the learning data set is calculated (L1> L2), and the calculated And a utility module configured to generate a subset of the number corresponding to N and to include the same number of the first label data and the second label data in each of the generated subsets,
Device for configuring learning data sets for machine learning.
The method of claim 8,
The utility module,
Characterized in that the first label data included in the learning data set is divided into N equal parts and included in each subset,
Device for configuring learning data sets for machine learning.
The method of claim 9,
The utility module,
The utility module is characterized in that the second label data included in the learning data set is copied so that all of the second label data included in the learning data set are included in each subset.
Device for configuring learning data sets for machine learning.

Images