KR102201198B1 - Apparatus and method for classifying data by using machine learning and ensemble method - Google Patents

Abstract

A device for classifying data using a machine learning and ensemble method may comprise: a collection unit collecting learning data; a generation unit generating an ensemble network for each of the plurality of layers and generating a classification model including an ensemble network generated for each of the plurality of layers; a learning unit learning the classification model to classify learning data by inputting the learning data into the classification model; and a classification unit classifying the classification data by inputting the classification data into the learned classification model. Therefore, the accuracy for automatic classification and classification prediction can be improved.

Description

Device and method for classifying data using machine learning and ensemble techniques {APPARATUS AND METHOD FOR CLASSIFYING DATA BY USING MACHINE LEARNING AND ENSEMBLE METHOD}

The present invention relates to an apparatus and method for classifying data using an ensemble technique and machine learning.

Machine learning technology refers to a technology for creating a model that can solve a specific problem from the generalization process for input data. In order to create a model with excellent performance, good quality training data and a learning algorithm for the generalization process are required.

As a technique for improving the performance of a model, there is an ensemble technique. The ensemble technique is a technique that combines a number of weak learners to create one strong learner.

These ensemble techniques include a bagging method using a voting method, a boosting method using a weighted voting method, and stacking using predicted values obtained from a single model as training data. There is a technique.

Referring to FIG. 1A, the bagging technique samples training data and constructs N sub-learning data with the same size, then inputs N sub-learning data one-to-one to N models to train N models, and a voting method. This is a technique that finally classifies (or predicts) the result. This bagging technique is fast because individual models can independently and simultaneously learn sub-learning data, and increases the performance of the model by increasing the variance of individual models.

Referring to FIG. 1B, unlike the bagging technique, the boosting technique does not treat a plurality of generated models equally, and a weight assigned to each model is reflected in a vote for final classification. That is, after a first model in which all of the training data is learned in an initial stage is generated, the training data is classified through the generated first model. A weight to be assigned to the classified training data is determined based on the classification result of the training data classified through the first model. The weights are adjusted by assigning a small weight to the training data that is well classified by the first model and a high weight to the training data that is incorrectly classified. A new model is continuously generated through the random extraction process of the training data whose weight is adjusted, and as a result, classification is performed using a weighted voting method using models with different reliability. This boosting technique improves the model's performance by adjusting the bias.

Referring to FIG. 1C, the stacking technique is a technique of generating a single model using two or more learning algorithms, and generating a meta model (or meta classifier) by using a prediction value obtained from the generated single model as training data.

That is, in the stacking technique, the prediction result of a single model trained with input data is used as training data and classified through a meta model (Meta Mdoel). The structure of the stacking technique has two layers, a first layer composed of a single model part and a second layer composed of a meta model part. Since this stacking technique assumes that each individual model is independent, it has a high ability to respond to outliers and has a value smaller than the misclassification rate of the single model, and thus has excellent performance.

Korean Patent Publication No. 10-1731626 (registered on April 24, 2017)

An object of the present invention is to generate a classification model including an ensemble network generated for each of a plurality of layers, and to learn classification of data through the generated classification model.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an apparatus for classifying data using machine learning and ensemble techniques according to a first aspect of the present invention includes: a collection unit for collecting learning data; A generator for generating an ensemble network for each of a plurality of layers and generating a classification model including an ensemble network generated for each of the plurality of layers; A learning unit for learning the classification model to classify the training data by inputting the training data into the classification model; And a classification unit for classifying the classification data by inputting classification data into the learned classification model.

A method of classifying data using machine learning and ensemble techniques performed by a data classification apparatus according to a second aspect of the present invention includes the steps of: collecting learning data; Generating an ensemble network for each of a plurality of layers, and generating a classification model including an ensemble network generated for each of the plurality of layers; Learning the classification model to classify the training data by inputting the training data into the classification model; And classifying the classification data by inputting the classification data into the learned classification model.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, the present invention generates a classification model including an ensemble network generated for each of a plurality of layers, and automatically classifies and classifies data by learning classification of data through the generated classification model. The accuracy of prediction can be improved.

Through this, it is possible to provide stable automatic classification performance by improving the performance limitation problem of the conventional stacking ensemble method composed of two layers, and thereby can be used in a big data analysis platform of various domains, thereby expanding marketability.

1A to 1D are diagrams for explaining a conventional data classification method.
2 is a block diagram of an apparatus for classifying data using machine learning and an ensemble technique according to an embodiment of the present invention.
3A to 3C are diagrams for explaining a method of generating a classification model using machine learning and ensemble techniques according to an embodiment of the present invention.
4 is a flowchart illustrating a method of classifying data using machine learning and ensemble techniques according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware.

In the present specification, some of the operations or functions described as being performed by the terminal or device may be performed instead by a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal or device connected to the server.

Hereinafter, with reference to the accompanying configuration diagram or processing flow chart, specific details for the implementation of the present invention will be described.

The present invention generates a classification model having a multi-stacking ensemble technique that constructs each layer by using ensemble networks learned by a stacking technique having a form of a deep neural network (DNN).

The present invention can improve the classification performance of the classification model by increasing the number of layers of the classification model similar to the form of a DNN having two or more hidden layers of an artificial neural network architecture (see FIG. 1D).

In other words, it is possible to increase the existing stacking ensemble technique having two layers to N layers and build each layer with ensemble networks learned by the stacking ensemble technique.

Accordingly, the process of learning the ensemble network included in the next layer is repeated by using the learned output values (classification values) of the ensemble network included in the current layer as input data of the ensemble network included in the next layer when creating N layers. can do.

2 is a block diagram of a data classification apparatus 100 using machine learning and an ensemble technique according to an embodiment of the present invention.

Referring to FIG. 2, the data classification apparatus 100 may include a collection unit 200, a generation unit 210, a learning unit 220, an evaluation unit 230, and a classification unit 240. However, the data classification apparatus 100 shown in FIG. 2 is only an example of an implementation of the present invention, and various modifications are possible based on the components shown in FIG. 2.

Hereinafter, FIG. 2 will be described with reference to FIGS. 3A to 3C.

The collection unit 200 may collect training data for classification learning of a classification model.

The generator 210 may generate an ensemble network for each of a plurality of layers, and may generate a classification model including an ensemble network generated for each of the plurality of layers.

In the case of a classification model, the performance of the classification model may vary depending on the number of layers in which an ensemble network is built and the type and number of classification algorithms used in the ensemble network to be built in each layer. That is, a difference occurs in the classification performance of the classification model according to the number of layers constituting the classification model and the type and number of classification algorithms used for each layer.

For this reason, the generation unit 210 may determine the number of layers in which the ensemble network is to be built and the number of classification algorithms included in each layer.

Also, the generator 210 may generate a classification model based on the determined number of layers and the determined number of classification algorithms included in each layer.

The generation unit 210 generates an ensemble network including a plurality of classification algorithms for ensemble learning for each of a plurality of layers based on the determined number of layers and the number of classification algorithms included in each layer, A classification model including the generated ensemble network can be generated.

Here, ensemble learning is a machine learning technique that combines several decision trees to achieve better performance than one decision tree. The core of ensemble learning is to create a strong classifier by combining several weak classifiers, so the accuracy of learning can be improved.

At this time, the plurality of classification algorithms are, for example, SVM (Support Vector Machine) algorithm, Decision Tree algorithm, Random Forest algorithm, K-Nearest Neighbor (KNN) algorithm. , Naive Bayes (Naive Bayes) may include at least one of the algorithm. In this case, the SVM algorithm may include a linear kernal algorithm and an RBF kernel algorithm.

The generator 210 may select k classification algorithms from among ensemble networks including m classification algorithms, and may construct a plurality of layers while generating another ensemble network including the selected k classification algorithms.

이다. 이 때, 생성부(210)는 생성 가능한 모든 분류 알고리즘의 조합에서 임의로 하나의 조합을 선택하여 앙상블 네트워크를 생성하고, 생성될 앙상블 네트워크가 위치할 레이어를 결정할 수 있다. Here, since k classification algorithms are arbitrarily selected among m classification algorithms, the number of possible combinations of classification algorithms is

to be. In this case, the generator 210 may generate an ensemble network by randomly selecting a combination of all possible classification algorithm combinations, and may determine a layer in which the generated ensemble network is located.

3A shows a classification model 30 in which the number of layers is three.

Referring to FIG. 3A, the generator 210 may build a first ensemble network 30-1 including a plurality of classification algorithms (eg, six different classification algorithms) in the first layer.

Subsequently, the generation unit 210 randomly selects at least one classification algorithm (eg, four classification algorithms) among a plurality of classification algorithms included in the first ensemble network 30-1, and classifies at least one selected A second ensemble network 30-3 including an algorithm may be created and built into the second layer. In this case, each of the plurality of classification algorithms included in the second ensemble network 30-2 may be a different algorithm.

Finally, the generation unit 210 randomly selects at least one classification algorithm (eg, five classification algorithms) among a plurality of classification algorithms included in the first ensemble network 30-1, and selects at least one or more A third ensemble network 30-5 including a classification algorithm may be created and built into the third layer.

That is, the present invention may generate an ensemble network to be built in the next layer using k classification algorithms arbitrarily selected among m classification algorithms, and repeat the stacking process by adjusting the number of built layers.

In another embodiment, the generator 210 may select m-1 from m classification algorithms and then construct a plurality of layers while generating another ensemble network including the selected m-1 classification algorithms.

이고, 결과적으로 m 개의 앙상블 네트워크로 구성된 m개의 레이어가 생성될 수 있다. 따라서, 앙상블 네트워크는 분류 알고리즘의 수와 같고 최대로 생성 가능한 레이어의 개수 또한 분류 알고리즘의 수와 같게 된다. If this process is repeated, the number of combinations of classification algorithms that can be generated is

As a result, m layers composed of m ensemble networks may be generated. Accordingly, the ensemble network is equal to the number of classification algorithms, and the number of maximum layers that can be generated is also equal to the number of classification algorithms.

The learning unit 220 may learn a sub classification model included in the classification model while the generation unit 210 generates a classification model. Here, the sub-classification model may include at least one layer.

For example, the learning unit 220 may learn a first sub-classification model including only the first layer. Subsequently, when the generator 210 generates the second layer, the learning unit 220 may learn a second sub-classification model including the first layer and the second layer.

After the generation unit 210 generates the classification model, the learning unit 220 may train the classification model to classify the training data by inputting the training data into the classification model.

Referring to FIG. 3A, the learning unit 220 receives a first output value (classified learning data) output from each of a plurality of algorithms included in the first ensemble network 30-1 into which the training data 301 is input. It is possible to train the second ensemble network 30-3 to classify the first output value through at least one classification algorithm included in the second ensemble network 30-3 by inputting it into the second ensemble network 30-3. have.

The learning unit 220 inputs a second output value (reclassified learning data) output from at least one classification algorithm included in the second ensemble network 30-3 to the third ensemble network 30-5. The third ensemble network 30-5 may be trained to classify the second output value through at least one classification algorithm included in the third ensemble network 30-5.

That is, the first ensemble network 30-1 receiving the training data 301 may output the first output value as a result value classified through each of a plurality of algorithms included in the first ensemble network 30-1. have. The first output value output at this time is input data of the second ensemble network 30-3 and is input to the second ensemble network 30-3.

The second ensemble network 30-3 may output a second output value as a result value classified through at least one classification algorithm included in the second ensemble network 30-3. At this time, the outputted second output value is input to the third ensemble network 30-5 as input data of the third ensemble network 30-5.

The third ensemble network 30-5 may output a final output value 303 that is finally classified through at least one classification algorithm included in the third ensemble network 30-5.

Returning to FIG. 2 again, the evaluation unit 230 may evaluate the classification performance of each of the ensemble networks generated for each of a plurality of layers through logistic regression analysis.

3B, the evaluation unit 230 includes a plurality of classification algorithms (SVM_Linear algorithm, SVM_ RBF algorithm, Decision Tree algorithm, Random Forest algorithm, KNN algorithm and Naive Bayes) included in the first ensemble network built in the first layer. Algorithm) A logistic regression analysis may be performed on output values (first to sixth output values) output from each, and classification performance of the first ensemble network may be evaluated based on the result of logistic regression analysis on the output values.

In addition, the evaluation unit 230 performs logistic regression analysis on the output values output from each of at least one classification algorithm included in the second ensemble network built in the second layer (not shown), and logistic regression analysis on the output values. It is possible to evaluate the classification performance of the second ensemble network based on the result of.

In addition, the evaluation unit 230 performs logistic regression analysis on the output values output from each of at least one classification algorithm included in the third ensemble network built in the third layer (not shown), and logistic regression analysis on the output values. The classification performance of the third ensemble network may be evaluated based on the result of.

According to an embodiment of the present invention, as shown in FIG. 3C, the classification model includes m layers, and each layer includes m classification algorithms. In this case, when a classification model is generated while selecting m-1 from m classification algorithms, the performance of the classification model is confirmed to be high.

For example, among six classification algorithms (SVM_Linear algorithm, SVM_ RBF algorithm, Decision Tree algorithm, Random Forest algorithm, KNN algorithm, and Naive Bayes algorithm) included in the first ensemble network built in the first layer, 5 excluding the SVM_Linear algorithm. A second ensemble network including six classification algorithms (at least one classification algorithm may be included) through a combination of three classification algorithms is constructed in the second layer, and among the six classification algorithms included in the first ensemble network When repeating the layer creation process in which a third ensemble network including six classification algorithms (at least one classification algorithm may be included) through a combination of five classification algorithms excluding the SVM_ RBF algorithm is constructed in the third layer. , A classification model including an ensemble network including 6 classification algorithms in every 6 layers is generated.

The classification unit 240 may classify the classification data by inputting the classification data into the learned classification model.

As described above, the present invention can provide a multiple stacking ensemble learning technique that improves the existing stacking technique. Such a learning structure of the multi-stacking ensemble technique can form an ensemble network for each layer, and as a result, take a form similar to a deep learning structure, and can provide superior classification performance compared to the existing stacking technique.

On the other hand, for those skilled in the art, each of the collection unit 200, the generation unit 210, the learning unit 220, the evaluation unit 230, and the classification unit 240 may be implemented separately, or one or more of them may be integrated and implemented. You will fully understand that you can.

4 is a flowchart illustrating a method of classifying data using machine learning and ensemble techniques according to an embodiment of the present invention.

Referring to FIG. 4, in step S401, the data classification apparatus 100 may collect training data.

In step S403, the data classification apparatus 100 may generate an ensemble network for each of a plurality of layers, and may generate a classification model including an ensemble network generated for each of the plurality of layers.

In step S405, the data classification apparatus 100 may train the classification model to classify the training data by inputting the training data into the classification model.

In step S407, the data classification apparatus 100 may classify the classification data by inputting the classification data into the learned classification model.

In the above description, steps S401 to S407 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed.

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. .

100: data classification device
200: collection unit
210: generation unit
220: Learning Department
230: evaluation unit
240: classification unit

Claims (12)

In an apparatus for classifying data using machine learning and ensemble techniques,
A collection unit for collecting learning data;
A generator for generating an ensemble network for each of a plurality of layers and for generating a classification model including an ensemble network for each of the plurality of layers;
A learning unit for learning the classification model to classify the training data by inputting the training data into the classification model; And
Classification unit for classifying the classification data by inputting classification data into the learned classification model
Including,
The generator generates the ensemble network including a plurality of classification algorithms for ensemble learning for each of the plurality of layers,
The generator constructs a first ensemble network including the plurality of classification algorithms on a first layer, and a second including at least one classification algorithm randomly selected from among a plurality of classification algorithms included in the first ensemble network Build the ensemble network on the second layer,
The first ensemble network includes m classification algorithms, and the second ensemble network includes m-1 classification algorithms.
delete The method of claim 1,
Wherein the generation unit determines the number of the plurality of layers and the number of classification algorithms included in each layer.
The method of claim 3,
The generation unit generates the classification model based on the determined number of layers and the number of classification algorithms included in each of the determined layers.
delete The method of claim 1,
Each of the plurality of classification algorithms included in the first ensemble network is a different algorithm.
The method of claim 1,
The learning unit inputs a first output value output from each of a plurality of classification algorithms included in the first ensemble network into which the training data is input to the second ensemble network, and at least one classification algorithm included in the second ensemble network To learn the second ensemble network through the data classification apparatus.
The method of claim 1,
The generation unit
The data classification apparatus, wherein a third ensemble network including at least one randomly selected classification algorithm among a plurality of classification algorithms included in the first ensemble network is constructed in a third layer.
The method of claim 8,
The learning unit inputs a second output value output from at least one classification algorithm included in the second ensemble network to the third ensemble network, and the third ensemble through at least one classification algorithm included in the third ensemble network To train the network, data classification device.
In a method of classifying data using machine learning and an ensemble technique performed by a data classification device,
Collecting learning data;
Generating an ensemble network for each of a plurality of layers, and generating a classification model including an ensemble network generated for each of the plurality of layers;
Learning the classification model to classify the training data by inputting the training data into the classification model; And
Classifying the classification data by inputting the classification data into the learned classification model,
The generating of the classification model includes generating the ensemble network including a plurality of classification algorithms for ensemble learning for each of the plurality of layers,
Generating the ensemble network
Building a first ensemble network including the plurality of classification algorithms in a first layer; And
Constructing a second ensemble network including at least one randomly selected classification algorithm among a plurality of classification algorithms included in the first ensemble network in a second layer,
The first ensemble network includes m classification algorithms, and the second ensemble network includes m-1 classification algorithms.
The method of claim 10,
The step of generating the classification model
And determining the number of the plurality of layers and the number of classification algorithms included in each layer.
The method of claim 11,
The step of generating the classification model
And generating the classification model based on the determined number of layers and the number of classification algorithms included in each of the determined layers.

Images