KR102189362B1 - Method and Device for Machine Learning able to automatically-label - Google Patents

Abstract

In the automatic labeling method of unlabeled data according to an embodiment of the present invention, the step of collecting unlabeled data (S31), and classifying the unlabeled data into a plurality of groups by a clustering technique Step (S34), selecting some data from each of the plurality of groups (S35), assigning a label of the selected partial data (S36), and assigning a label assigned to the partial data to all belonging to the corresponding group It characterized in that it comprises a step (S37) to give to the data. According to an embodiment of the present invention, since labels are assigned to only some training data and then labels are automatically assigned to all other training data, learning of a function trained by machine learning can be performed in a faster time, and It is possible to drastically reduce the labor force to be given.

Description

Machine learning method and device capable of automatic labeling {Method and Device for Machine Learning able to automatically-label}

The present invention relates to a machine learning method and apparatus capable of automatic labeling, and more specifically, labeling only a part of training data for machine learning and then automatically labeling the rest. It relates to a method and apparatus.

Machine Learning, as a field of artificial intelligence, refers to the field of developing algorithms and technologies that enable computers to learn. For example, it can be trained to identify whether an email received through machine learning is spam or not.

Machine learning is largely divided into supervised learning and unsupervised learning. Supervised learning is one of the machine learning methods for training a function using training data. The training data generally contains the properties of the input object in the form of a vector, and the desired result for each vector is displayed. have. Among the trained functions, outputting continuous values is called regression, and marking what kind of values a given input vector is called classification. On the other hand, unsupervised learning, unlike supervised learning, does not give a target value for an input value.

Supervised learning aims to find a fixed answer through an algorithm. Therefore, supervised learning is a method of inferring a function that can best obtain an output value from an input value from training data given an input value and a target value.

In supervised learning, labeled data, that is, data containing target value information is used to train a function. In the supervised learning method, the actual output value when the data is input and the target value are compared according to a predetermined learning algorithm based on the labeled data, and if there is an error, the task of modifying the function based on this result is repeated. .

Such supervised learning requires humans to directly label individual training data, which is time consuming and expensive.

Meanwhile, Korean Patent Laid-Open Publication No. 10-2017-0083419 discloses a method of training a deep learning model using a plurality of unlabeled training data, but this has a disadvantage of inferior accuracy.

Korean Patent Publication No. 10-2017-0083419 (registered on July 18, 2017)

The present invention has been proposed to solve the above problems, and an object of the present invention is to provide a machine learning method and apparatus capable of automatically labeling a plurality of data that are not labeled.

In the automatic labeling method of unlabeled data according to an embodiment of the present invention, the step of collecting unlabeled data (S31), and classifying the unlabeled data into a plurality of groups by a clustering technique Step (S34), selecting some data from each of the plurality of groups (S35), assigning a label of the selected partial data (S36), and assigning a label assigned to the partial data to all belonging to the corresponding group It characterized in that it comprises a step (S37) to give to the data.

For example, the labels of some of the selected data may be assigned by a predetermined algorithm. The type and number of output values of data that the user wants to classify may be determined in advance. Therefore, the user can pre-configure the algorithm and determine and assign the label of the selected partial data by the predetermined algorithm. When the machine learning device receives the label of the selected partial data, all data belonging to the group including the selected partial data have the same label.

Preferably, the label of the selected partial data may be given by user input. The user can determine the correct output value of the data by making a very simple selection and input it into the machine learning device. When the machine learning device receives a user input, the label of the selected partial data is accurately determined, and all data belonging to the group including the selected partial data have the same label.

The automatic labeling method according to an embodiment of the present invention may further include the step of signal processing the collected data (S32) and the step of performing dimensionality reduction on the signal-processed data (S33), and the step (S34) In, the dimensionally reduced and unlabeled data can be classified into a plurality of groups by a clustering technique.

In the machine learning method capable of automatic labeling according to an embodiment of the present invention, the step of determining a hyperparameter combination to be used for learning (S10), a function including a parameter by a machine learning method as labeled data. The first learning step of training (S20), the automatic labeling step of unlabeled data (S30), the second learning of learning the function by machine learning with the data automatically labeled in the automatic labeling step (S30) Including step (S40), calculating an accuracy index of the function (S50), and changing the hyperparameter combination (S60), and repeating the steps S20 to S60, but when the highest accuracy index is calculated It is characterized by selecting the used hyperparameter combination and parameters.

For example, in the automatic labeling step (S30), the step of collecting unlabeled data (S31), the step of classifying the unlabeled data into a plurality of groups by a clustering technique (S34), the plurality of Selecting some data from each of the groups (S35), giving a label of the selected partial data (S36); And assigning a label assigned to the partial data to all data belonging to a corresponding group (S37).

The machine learning apparatus capable of automatically labeling according to an embodiment of the present invention includes an input unit for collecting unlabeled data, a labeling unit for automatically labeling the unlabeled data, and a machine with labeled data. A learning unit for learning a function including parameters by a running method, a calculation unit for calculating an accuracy index of the function, the input unit, a labeling unit, a learning unit, and a control unit for controlling the calculation unit, and the labeling unit After classifying the unassigned data into a plurality of groups by a clustering technique, some data is selected from each of the plurality of groups, a label is assigned to the selected partial data, and the label assigned to the partial data belongs to the corresponding group. It is characterized by giving it to all data.

For example, the control unit may determine a combination of hyperparameters used for learning, and the accuracy index generated each time the function is trained by machine learning with automatically labeled data by applying the hyperparameter combination is the highest. You can select hyperparameter combinations and parameters of.

According to an embodiment of the present invention, since labels are assigned to only some training data and then labels are automatically assigned to all other training data, learning of a function trained by machine learning can be performed in a faster time, and It is possible to drastically reduce the labor force to be given.

1 is a conceptual diagram illustrating a machine learning method and apparatus capable of automatic labeling related to the present invention.
2 is a diagram illustrating a machine learning method capable of automatic labeling according to an embodiment of the present invention.
3 is a diagram illustrating the automatic labeling step S30 of FIG. 2.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all modifications included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

1 is a conceptual diagram illustrating a machine learning method and apparatus capable of automatic labeling related to the present invention.

When ambient sound occurs, it is sensed in real time through an input unit 10 such as a microphone and stored as data. The ambient sound may be a silent sound 11 with no meaning, a sound that the user does not care about, that is, a noise 12, or a sound of interest 13 that the user wants to classify or analyze. In some cases, the sound of interest 13 may be a patient's groan 131, a baby's cry 132, or an adult's voice 133. However, the sound of interest 13 is not limited to the above three examples, and may be any sound such as a traffic accident collision sound, vehicle operation sound, and animal sound.

For example, when the sound of interest 13 is an adult's voice 133, the baby crying sound 132 may be classified as a noise 12. For example, when the sound of interest 13 is an animal sound, a patient's groan 131, a baby's cry 132, an adult's voice 133, and a traffic accident collision sound may be classified as noise 12. .

Classification of such data types may be performed by the function F in the machine learning apparatus 1. However, in the above classification task to classify types, in order to utilize a vast amount of data as training data for machine learning, labeling that attaches what type of each data is, and labeling the entire data is done by humans. It takes a lot of time to do it.

Supervised Learning is one of the machine learning methods for training a function using training data. Training data generally contains the properties of the input object in the form of a vector, and the desired result for each vector is It is marked what it is. Among the trained functions, outputting continuous values is called regression, and marking what kind of values a given input vector is called classification. On the other hand, unsupervised learning, unlike supervised learning, does not give a target value for an input value.

Preferably, in an embodiment of the present invention, the learning unit 30 may use a semi-supervised learning method having an intermediate characteristic between supervised learning and unsupervised learning. The quasi-supervised learning refers to using both data with a target value indicated and data not indicated for training. In most cases, the training data used in this method has few data with a target value displayed and a lot of data without a target value. According to an embodiment of the present invention, when the quasi-supervised learning is applied, time and cost for labeling can be greatly saved. For example, if a function is trained after labeling only a part of a plurality of data, and the remaining data without a label is grouped into several groups by a clustering technique and the label of the sample extracted for each group is determined, the labeling unit 20 A label is determined according to a predetermined condition so that all data in the group have the one label. This data is called automatic labeling data. After that, it is obvious that the learning unit 30 can learn the function using the automatic labeling data in a supervised learning method.

The task of displaying the target value is labeling. For example, if ambient sound is generated and the sound data is input, the labeling operation is to indicate whether the type of sound is silent (11), noise (12), or sound of interest (13). In other words, labeling is a basic operation for pre-marking an example of an output in data and learning a function using a machine learning algorithm.

What a person directly displays is supervised learning, and what does not display is unsupervised learning, and some is directly marked by a person, and others are not marked as semi-supervised learning.

In an embodiment of the present invention, the machine learning apparatus 1 may perform an auto-labeling operation. Label means the result values that the function should output. For example, the labels are result values of silence, noise, baby crying, and baby sounds excluding crying sounds. The automatic labeling may be performed in the following order. The automatic labeling may be performed by the learning unit 30, for example.

First, a person intervenes and displays a label for a certain number of data (eg, 100). Specifically, sound data is collected through the input unit 10. The labeling unit 20 performs appropriate signal processing on the collected sound data and then undergoes a dimension reduction operation. After that, a clustering technique that classifies a group with homogeneity is used to group a plurality of data classified as a homogeneity into a single data group. In this case, the clustering technique is classified based on a predetermined hyperparameter, but the hyperparameter may be changed according to the learning accuracy performed in the future.

Next, when a plurality of data groups is formed, the labeling unit 20 randomly selects only a predetermined number (eg, 4 pieces of data) for each data group and determines which element has a characteristic. For example, if three or more of the four data selected from the first data group are found to correspond to noise, all the first data groups are regarded as noise, and all data in the first data group are labeled as noise. . If there are two or less baby crying sounds among the four data selected from the second data group, all data in the second data group are labeled as noise.

Next, the labeling unit 20 performs labeling with this predetermined algorithm, and the learning unit 30 uses the automatically labeled data as training data. In this case, the calculation unit 40 calculates an accuracy index, and the control unit 50 finally selects a hyperparameter combination and a parameter when the accuracy index is the highest.

2 is a diagram illustrating a machine learning method capable of automatic labeling according to an embodiment of the present invention.

In machine learning, a parameter is a variable that can be checked inside a learning model, and is a value that can be calculated through data. In other words, parameters are learned from data by machine learning. Weights in artificial neural networks, support vectors in a support vector machine (SVM), and coefficients of determination in regression analysis are examples of parameters. In short, if the function to be obtained by machine learning has the form of a linear equation (y=ax+b), the slope (a) and the intercept (b) correspond to the parameters. In this case, by selecting the slope (a) and the intercept (b) used for the optimized function by machine learning and analyzing the data later, the correct output value can be accessed.

In machine learning, hyperparameters are elements outside of the learning model and are not values that can be calculated through data. That is, it is a factor that is controlled by the user or the administrator. In neural network learning, the learning rate, the cost (C) in the support vector machine (SVM), and the number of Ks in the K-Nearest Neighbor (KNN) are examples of hyperparameters. Hyperparameters can be determined by the algorithm user, and can be changed or adjusted if problems with the algorithm are found.

Referring to FIG. 2, in order to perform machine learning capable of automatic labeling, a hyperparameter combination to be used for learning is first determined. The hyperparameter combination may be a plurality of hyperparameters.

Next, supervised learning is performed. In other words, the function is trained by machine learning with labeled data. The function includes parameters. When the parameters are optimized, the function can perform at its best.

Next, automatic labeling is performed. The input unit 10 collects unlabeled data, and the labeling unit 20 classifies the unlabeled data into a plurality of groups using a clustering technique. The labeling unit 20 selects some data from each of the plurality of groups and assigns a label to the selected data. Labeling some data selected here may be performed mechanically or by a program, but it is preferable to receive a user's input.

When the user inputs the first label of the selected partial data, the first label is assigned to all data of the data group to which the partial data belongs. The user can input the labels of some data selected in each data group, and all data in all data groups have a label.

Then, the learning unit 30 can again learn the function by machine learning with the automatically labeled data. In this case, a labeling operation can be performed much faster than a person manually labeling all data.

Accuracy metrics can be pre-determined using an accuracy measure such as the F1 measure (F1 measure or F1 score). Therefore, when the first hyperparameter combination is used, the higher the calculated accuracy index, the more optimized the function can be understood.

For example, assuming that the nth hyperparameter combination is used from the first hyperparameter combination, it is preferable to select the hyperparameter combination when the accuracy index is the highest among n learning processes, and then select the calculated parameter. It is desirable to do.

As described above, according to the machine learning method capable of automatic labeling according to an embodiment of the present invention, a human label is assigned to a small amount of data to ensure the accuracy of learning, and the remaining data is automatically labeled to speed up learning. Can be secured. This has the advantage that the accuracy at the initial learning may be slightly lower than in the case of supervised learning, in which the entire data is labeled and then trained, but it has the advantage that machine learning can be carried out very quickly because the procedure that previously used to label all data is omitted. Have. From the perspective of the whole learning process, it is possible to perform more efficient machine learning than before by securing a medium level of accuracy and a high level of speed. Accordingly, labor costs can be reduced, and the learning speed by machine learning can be very fast.

3 is a diagram illustrating the automatic labeling step S30 of FIG. 2.

Collect data such as voice data. The data is collected unlabeled. The collected data may be subjected to signal processing for machine learning. The signal processing may include pre-processing and feature vector extraction. After that, you can perform dimensional reduction on the signal-processed data. The dimension reduction may be, for example, a principal component analysis (PCA) method. The amount of data can be reduced by the dimension reduction. This is to extract features that can properly express the meaning of data.

The dimensionally reduced data can be divided into a plurality of groups by a clustering technique. The learning unit 30 selects some data from each of the plurality of groups and gives a label to the selected data.

Preferably, a label is entered by the user for some selected data. It is desirable that labels are assigned to all data in the data group even if the user enters only 3 or 4 labels.

1: machine learning device
10: input
11: silent
12: noise
13: sounds of interest
131: patient moaning
132: baby cry
133: adult voice
20: labeling unit
30: Learning Department
40: calculation unit
50: control unit

Claims (10)

delete delete delete delete Determining, by the control unit, a hyperparameter combination to be used for learning (S10);
In order to ensure the accuracy of training, a first learning step (S20) of learning a function for classifying sounds, including parameters, by a machine learning method using speech data labeled with a small amount of speech data;
As an automatic labeling step (S30) of voice data that is not labeled,
Collecting, by the input unit, unlabeled voice data (S31);
Classifying, by a labeling unit, the voice data to which the label is not assigned into a plurality of groups by a clustering technique (S34);
Selecting some voice data from each of the plurality of groups by the labeling unit (S35);
Labeling some voice data selected by a predetermined algorithm by the labeling unit (S36); And
An automatic labeling step (S30) including the step (S37) of assigning, by the labeling unit, a label assigned to the partial voice data to all voice data belonging to a corresponding group;
A second learning step (S40) of learning a function for classifying the sound by a machine learning method using speech data automatically labeled in the automatic labeling step (S30) by a learning unit in order to secure the speed of training;
Calculating an accuracy index of the function by a calculation unit (S50); And
Including the step (S60) of changing the hyperparameter combination by the control unit,
Repeating the steps S20 to S60, but selecting the hyperparameter combination and parameters used when the highest accuracy index is calculated by the control unit,
A machine learning method that efficiently trains poorly accurate models by automatically labeling entire unlabeled speech data. The method of claim 5,
The calculation of the accuracy indicator is determined using the F1 measurement method,
Machine learning method to efficiently train poorly accurate models by automatically labeling entire unlabeled speech data. delete delete An input unit for collecting unlabeled data;
A labeling unit for automatically assigning a label to the unlabeled data;
A learning unit that learns a function for classifying sounds, including parameters by machine learning, with labeled data;
A calculation unit for calculating an accuracy index of the function;
And a control unit for controlling the input unit, a labeling unit, a learning unit, and a calculation unit,
The labeling unit classifies the unlabeled data into a plurality of groups by a clustering technique, selects some data from each of the plurality of groups, assigns a label to the selected partial data, and then assigns a label to the partial data. Assign to all data belonging to the corresponding group,
The control unit may determine a hyperparameter combination used for learning, and the hyperparameter when the accuracy index generated every time the function is trained by machine learning with automatically labeled data by applying the hyperparameter combination is the highest Select combinations and parameters,
The learning unit learns a function including parameters by machine learning with a small amount of labeled speech data in order to ensure the accuracy of learning, and to secure the speed of learning, machine learning with automatically labeled speech data Learning the function by way of,
Machine learning device that efficiently trains poorly accurate models by automatically labeling entire unlabeled speech data. The method of claim 9,
The calculation of the accuracy indicator is determined using the F1 measurement method,
Machine learning device that efficiently trains poorly accurate models by automatically labeling entire unlabeled speech data.

Images