KR20210050362A - Ensemble pruning method, ensemble model generation method for identifying programmable nucleases and apparatus for the same - Google Patents

Abstract

Provided is an ensemble pruning method using a positive determination result which comprises the steps of: allowing a computer device to receive training data; allowing the computer device to generate learning models for constructing an ensemble model using the training data; and allowing the computer device to prune a plurality of learning models among the learning models so that the ensemble model has an area under the ROC curve (AUC) which is greater than or equal to a reference value based on a determination result obtained by inputting data or other training data into the learning models.

Description

Ensemble model pruning method, ensemble model generation method and apparatus for detecting genetic scissors {ENSEMBLE PRUNING METHOD, ENSEMBLE MODEL GENERATION METHOD FOR IDENTIFYING PROGRAMMABLE NUCLEASES AND APPARATUS FOR THE SAME}

The technique described below relates to a technique for generating an ensemble model. In particular, the technique described below relates to an ensemble pruning technique.

Machine learning models are being used in various fields. A classifier is a learning model that performs positive or negative judgment on input data. The classifier can be implemented with various techniques such as artificial neural networks and ensembles.

Ensemble is a generic term for a technique that uses a plurality of learning algorithms in machine learning. Representatively, the ensemble technique includes a bagging technique including a random forest or a boosting technique.

Korean Patent Publication No. 10-2019-0022431

Classifiers basically classify input data into one of various types. Depending on the type of application using the machine learning model, there are cases where the number of negative data is larger than the number of positive data, and the accuracy of classification for positive data is low. This problem is called an unbalanced classification problem. In this case, a model with higher classification accuracy for positive data than for negative data may be required.

The technique described below is intended to provide a technique for generating an effective ensemble model for positive judgment.

The ensemble pruning method includes: receiving training data by a computer device, generating training models for constructing an ensemble model by the computer device using the training data, and the computer device receiving the training data or other training data. And pruning a plurality of learning models among the learning models so that the ensemble model has an area under the ROC curve (AUC) greater than or equal to a reference value based on a determination result displayed by inputting the learning models.

The device for generating the ensemble model generates an input device for receiving training data, a storage device for storing the training data and a program for generating the ensemble model, and a learning model for constructing the ensemble model using the training data using the program. And, based on the determination result displayed by inputting the training data or other training data into the learning models, a plurality of learning models are selected among the learning models so that the ensemble model has an area under the ROC curve (AUC) equal to or greater than a reference value It includes a computing device that performs pruning.

The technique described below is an ensemble pruning technique to select a group of learning models that are effective in determining positive. The technique described below can generate an ensemble model for selecting effective genetic scissors for a specific sequence.

1 is an example of a service system using a learning model.
2 is an example of a process of operating an ensemble model.
3 is an example of a process of generating an enhanced ensemble model.
4 is an example of an enhanced ensemble model generation device.
5 is an example of verifying the effect on the reinforced ensemble model.
6 is another example of verifying the effect on the reinforced ensemble model.

The technology to be described below may be modified in various ways and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology to be described below with respect to a specific embodiment, and it should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the technology to be described below.

Terms such as 1st, 2nd, A, B, etc. may be used to describe various components, but the components are not limited by the above terms, and only for the purpose of distinguishing one component from other components. Is only used. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component without departing from the scope of the rights of the technology described below. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In terms of the terms used in the present specification, expressions in the singular should be understood as including plural expressions unless clearly interpreted differently in context, and terms such as "includes" are specified features, numbers, steps, actions, and components. It is to be understood that the presence or addition of one or more other features or numbers, step-acting components, parts or combinations thereof is not meant to imply the presence of, parts, or combinations thereof.

Prior to the detailed description of the drawings, it is intended to clarify that the division of the constituent parts in the present specification is merely divided by the main function that each constituent part is responsible for. That is, two or more constituent parts to be described below may be combined into one constituent part, or one constituent part may be divided into two or more for each more subdivided function. In addition, each of the constituent units to be described below may additionally perform some or all of the functions of other constituent units in addition to its own main function, and some of the main functions of each constituent unit are different. It goes without saying that it can also be performed exclusively by.

In addition, in performing the method or operation method, each of the processes constituting the method may occur differently from the specified order unless a specific order is clearly stated in the context. That is, each of the processes may occur in the same order as the specified order, may be performed substantially simultaneously, or may be performed in the reverse order.

Hereinafter, terms used in the description will be described.

Machine learning is a field of artificial intelligence, which refers to the field in which algorithms are developed so that computers can learn. Machine learning model or learning model means a model developed so that a computer can learn. There are various models of learning models, such as artificial neural networks and decision trees, depending on the approach method.

Ensemble is a generic term for a technique that uses a plurality of learning algorithms in machine learning. Representatively, the ensemble technique includes a bagging technique including a random forest or a boosting technique.

Random Forest is a kind of bagging algorithm composed of a combination of CART's decision tree. The random forest consists of a plurality of decision trees. Each of the plurality of decision trees is trained in advance by randomly selecting some of the training data and feature variables. In the random forest, each tree individually determines the target variable and then aggregates the decisions of all trees to make a final decision.

Hereinafter, it will be described focusing on the ensemble model. The plurality of learning models constituting the ensemble model may be a decision tree. Furthermore, the plurality of learning models constituting the ensemble model may be a model such as an artificial neural network.

The indicators for evaluating the classification model will be described.

Accuracy is the proportion of correctly classified data among the total data. Accuracy refers to how accurately the classifier classifies data.

Sensitivity is the percentage of actual positive data that the classifier classified as positive. TPR (true positive rate) is the same concept. Sensitivity indicates how accurately the model classifies positive data.

FPR (false positive rate) is the percentage of actual negative data that the classifier classified as positive.

The area under the ROC curve (AUC) is the probability that the predicted value for positive data of the classifier is higher than the predicted value for negative data. This can also be obtained as the area of the ROC (receiver operating characteristics) curve. The ROC curve is expressed through the true and false positive rates of the classifier.

The technique described below is an ensemble pruning technique. The technique described below is a technique of pruning the ensemble model based on the AUC rather than pruning the ensemble model based on the accuracy. In the ensemble model described below, the model is trained or pruned by considering only the positive judgment without considering the true negative judgment. In the following description, a new type of model generated through ensemble pruning is referred to as a reinforced learning model (ensemble model). It is different from reinforcement learning in the field of machine learning.

In the technique described below, the types of individual members (learning models) constituting the ensemble model may vary. For example, the ensemble model may be composed of a plurality of decision trees. Alternatively, the ensemble model may be composed of a plurality of neural network models. Furthermore, the ensemble model includes a plurality of trees, and each node of the tree may be a learning model. Hereinafter, for convenience of description, a description will be given centering on a random forest. However, the ensemble pruning technique described below is not applied only to a specific type of model.

1 is an example of a service system 100 using a learning model. 1 is an example of a service system that predicts the effect of genetic scissors designed by a researcher. 1 is an example in which an analysis device 130, 140, 150 analyzes a gene sequence and selects an effective gene scissors for a target sequence.

In FIG. 1, the analysis device is shown in the form of a server 130 and computer terminals 140 and 150. The server 130 may provide a service for analyzing gene sequences on a network. The computer terminal 140 is connected to a network to receive a gene sequence, and analyzes the gene sequence using an installed application. The computer terminal 150 receives input data from a medium (eg, USB, SD card, etc.) in which the gene sequence is stored, and analyzes the gene sequence using an installed application. The analysis devices 130, 140, 150 may be implemented in various forms.

The analysis devices 130, 140, 150 analyze the effect of the corresponding genetic scissors sequence using the genetic scissors sequence. The sequence to which the analysis devices 130, 140, and 150 are input may be the entire sequence of the genetic scissors and all or part of the guide RNA sequence.

The designer terminal 110 designs a genetic scissors that is expected to be suitable for a specific target. That is, the designer terminal 110 designs the RNA sequence constituting the genetic scissors. The designer terminal 110 may store the generated genome data in a separate DB 120.

Users (10, 20, 30) can check the suitability of the scissors for a specific sequence. The user 10 may access the server 130 through a user terminal (PC, smartphone, etc.) and check the analysis result performed by the server 130. The user 20 can check the suitability of the genetic scissors through the computer terminal 140 used by the user 20. The user 30 can check the suitability of the genetic scissors through the computer terminal 150 used by the user. Users 10, 20, 30 may be researchers who study genetic scissors.

The analysis devices 130 and 140 analyze the gene sequence using the learning model. The analysis devices 130 and 140 predict the genetic scissors effect by inputting input data into a learning model prepared in advance. The analysis devices 130 and 140 may use various learning models. For example, the analysis devices 130 and 140 may analyze the suitability of the genetic scissors using an ensemble technique.

Individual models constituting the ensemble model can receive the gene sequence itself. In addition, individual models may receive information on the secondary or tertiary structure of the gene sequence. For the secondary or tertiary structure, various tools (RNAfold Web server, http://rna.tbi.univie.ac.at/cgi-bin/RNAWebSuite/RNAfold.cgi, etc.) are used to predict the structure by receiving gene sequences. Can be created using Alternatively, individual models can be analyzed by receiving the secondary structure or the tertiary structure of the gene sequence in the form of an image. In this case, the individual model may be an artificial neural network such as a convolutional neural network (CNN).

An ensemble is a model that uses a plurality of learning models and performs more accurate prediction by combining the prediction results of the learning models. Ensemble techniques include bagging, boosting, random forest, and stacking. Criteria or tools for model effect analysis include misclassification rates, AUC, ROC curves, and lift charts.

2 is an example of a process 200 of operating an ensemble model. The analysis device inputs the input data into the pre-learned ensemble model (210). The analysis device analyzes the input data using the ensemble model. The ensemble model 220 inputs input data into an individual model and outputs a result of analyzing the input data by the individual model.

2 shows a random forest composed of N decision trees as an example. Each decision tree constituting a random forest makes a decision starting with input data and outputs a final decision result. Referring to FIG. 2, decision tree A outputs a result of high (eg, high fitness) for an analysis object, and decision tree B outputs a result of low. The random forest considers all of the output results of each decision tree to make a final decision. For example, a random forest can decide the final conclusion according to the principle of majority vote.

The analysis device predicts the analysis result (classification result) of the input data based on the information output from the random forest (230). For example, the analysis device may determine that the input gene sequence has a high degree of suitability of the gene scissors to the target sequence.

3 is an example of a process 400 of generating an enhanced ensemble model. It is assumed that the computer device generates an ensemble model. A computer device refers to a device capable of processing and calculating data, and may have various physical forms. Developers can create ensemble models using computing devices such as PCs, servers, and smart devices. Alternatively, an ensemble model to be used by the analysis device described in FIG. 1 may be generated.

Training data should be prepared in advance. The training data is a plurality of data sets. The training data may be the following set D. D = ((x _i , y _i )| i = 1....,n}. x is the input value, and y is the classification value for the input value. If x is a candidate sequence for the genetic scissors, then x _i ∈ {A,C,G,U} may be ^20. may be y _i ∈ {0,1}. 1 means true and 0 means false.

Individual models constituting the ensemble model may be a model that predicts the effect of shearing genes. In this case, the input value may be the scissors sequence or the structural information of the scissors. The computer device may generate information on the secondary or tertiary structure of the RNA by using a program that predicts the RNA structure based on the scissor sequence. The output value may be an effect score of the genetic scissors. The effect score can be calculated using a known solution. For example, a solution such as GenomeCRISPR_full05112017 (https://www.dkfz.de/signaling/crispr-downloads/GENOMECRISPR/) can be used.

The computer device generates an initial ensemble model using the training data (310). The computer device uses the training data to create an individual learning model constituting the ensemble model.

The computer device trains each learning model by randomly selecting some of the data sets included in the training data. Taking a random forest as an example, the computer device randomly selects training data for a plurality of decision trees, and randomly selects and trains feature variables. In FIG. 3, the computer device selects D ₁ , D ₂ ,..., D _{m - 1} ,D _m from the training data set D, and each training model h ₁ (x), h ₂ (x),..., h _{m -} a _{1 (x), h m (} x) was produced. The result of classifying the input data by each learning model may be as shown in Table 1 below. Table 1 below assumes that there are 5 individual learning models.

i y _i h ₁ (x) h ₂ (x) h ₃ (x) h ₄ (x) h ₅ (x)

One One One 0 One One 0 One 2 One 0 0 One 0 One 0 3 0 0 0 0 0 One 0 4 0 0 0 One 0 0 0 5 0 One 0 0 0 0 0

는 입력데이터의 실제 분류값이고,

는 학습모델이 예측한 값이다.

는 다수결 원칙에 따라 결정될 수 있다. 즉,

는 아래의 수학식 1과 같이 표현될 수 있다.

Is the actual classification value of the input data,

Is the predicted value by the learning model.

May be determined according to the principle of majority vote. In other words,

Can be expressed as Equation 1 below.

Referring to Table 1, the ensemble model accurately classified input data 1, but made a false negative for input data 2. If only true positives are important to the data for which the application is true, then it is necessary to prun the learning models that make up the ensemble. Referring to Table 1, _{an ensemble composed of only h 3} (x), h ₄ (x) and h ₅ (x) can be constructed. Ensemble pruning to increase AUC will be described. The model built with such ensemble pruning was described above as a reinforced ensemble model and an inherently reinforced ensemble model.

Ensemble pruning techniques include ordering-based pruning, clustering-based pruning, and optimization-based pruning. Hereinafter, a pruning technique for generating an enhanced ensemble model will be described.

The computer unit selects an individual model that can minimize the value of a given objective function. Conversely, the computer device eliminates models that do not meet the criteria. The technique below corresponds to optimization-based pruning.

In addition, the computer device may construct a set of classifiers constituting the ensemble model by using a genetic algorithm. Briefly describing the genetic algorithm, it consists of the following processes. (i) The initialization step is the process of creating a gene group for problem solving. (ii) The selection step is the process of selecting successful genes close to problem solving for the next generation. (iii) The genetic computation step is a process of recombining the genes selected in the selection step. (iv) The termination step is a process of verifying whether the termination condition is satisfied. If the conditions are not satisfied, the process for generation replacement is repeated.

Explaining based on the ensemble model, the initialization step is a process of generating an initial ensemble model using a training data set. The selection step is a process of selecting only a specific learning model using a pruning technique. The genetic computation step can be said to be a process of constructing an ensemble model with only the learning model selected by pruning. The final step can be said to be a process of verifying whether the ensemble model shows a certain objective function value.

Explain about pruning for next generation selection. The computer device may select an individual model capable of minimizing a given objective function value, as shown in Equation 2 below.

z denotes whether to select an individual learning model (classifier) constituting the ensemble. If the i-th classifier is selected, the z value is 1, and if not, it has a value of 0. g ₁ is the value of the AUC (Area Under Curve) of the ROC (Receiver Operating Characteristics) curve, which is one of the performance indicators of the imbalance classification problem. g ₂ is a value about the number of learning models that are finally selected among the learning models that make up the ensemble. g ₃ is a measure of the diversity of the final selected learning model. The individual items constituting the objective function will be described.

및

는 각각 아래의 수학식 4와 수학식 5로 표현할 수 있다.g ₁ is 1-AUC. The objective function is _{to ensure that g 1} + g ₂ + g ₃ is the minimum, so in the end, g ₁ corresponds to the condition in which the AUC is the maximum. here,

And

Can be expressed by Equation 4 and Equation 5 below, respectively.

는 입력데이터에서 참인 데이터를 의미한다.

는 선택된 학습모델 z이 참인 데이터를 제대로 분류하는 척도(점수)라고 할 수 있다.

Means the data that is true in the input data.

Can be said to be a measure (score) for properly classifying data for which the selected learning model z is true.

는 입력데이터에서 거짓인 데이터를 의미한다.

는 선택된 학습모델 z이 거짓인 데이터를 제대로 분류하는 척도(점수)라고 할 수 있다.

Means data that is false in the input data.

Can be said to be a measure (score) for properly classifying data for which the selected learning model z is false.

g ₂ is a measure of the number of learning models that are finally selected to form an ensemble.

g ₃ is a measure of the diversity of the last selected classification period. Various values can be used for various scales. Equation 7 is an example related to Q-statistic.

이다.

와 관련된 변수는 각각 아래의 수학식 8 및 수학식 9와 같다.L is the total number of individual learning models that make up the ensemble. In other words,

to be.

Variables related to are as in Equation 8 and Equation 9 below, respectively.

는 학습모델 분류기 u와 v가 모두 정분류한 객체의 수이다.

는 분류기 u와 v가 모두 오분류한 객체의 수이다.

는 분류기 u는 정분류하고, v는 오분류한 객체의 수이다.

는 분류기 u는 오분류하고, v는 정분류한 객체의 수이다.

Is the number of objects correctly classified by both the learning model classifiers u and v.

Is the number of objects misclassified by both the classifiers u and v.

Is the classifier u is the correct classification and v is the number of misclassified objects.

Is the classifier u is misclassified, and v is the number of correctly classified objects.

,...,

로 구성된다.The computer device may prun the initial ensemble model using the above-described optimization technique or genetic algorithm (220). 3 shows a learning model selected by pruning. The final ensemble model is

,...,

It consists of

The computer device can perform pruning by evaluating an individual learning model using training data that was initially input. Furthermore, the computer device may perform pruning by evaluating an individual learning model using a separate individual training data set, not the initial training data.

The computer device may classify the input data using the final ensemble model (230). The computer device can derive the final analysis result by combining the classification results of the selected individual learning models constituting the ensemble model. The final analysis result can be calculated according to the principle of majority vote as shown in Equation 10 below.

x _new means the input data to be analyzed.

4 is an example of an enhanced ensemble model generating apparatus 400. The model generation device 400 generates an ensemble model enhanced by the pruning technique described in FIG. 3. The model generating device 400 may be physically implemented in various forms. For example, the model generating device 400 may have a form such as a PC, a server of a network, a chipset dedicated to image processing, and a smart device.

The model generation device 400 includes a storage device 410, a memory 420, an operation device 430, an interface device 440, a communication device 450, and an output device 460.

The storage device 410 may store a training data set for generating an ensemble model. The storage device 410 may store a program for generating an ensemble model. Furthermore, the storage device 410 may store other programs or source codes required for data processing. The storage device 410 stores the generated ensemble model.

The memory 420 may store data and information generated in a process of analyzing the data received by the model generating device 400.

The interface device 440 is a device that receives certain commands and data from the outside. The interface device 440 may receive dielectric data from an input device physically connected or an external storage device. The interface device 440 may receive a code or a program for generating an individual learning model constituting an ensemble. The interface device 440 may receive a code or a program for ensemble pruning. The interface device 440 may receive training data, information, and parameter values for training a learning model.

The communication device 450 refers to a configuration that receives and transmits certain information through a wired or wireless network. The communication device 450 may receive training data, input data, and the like from an external object. The communication device 450 may receive a program or code for generating an ensemble. The communication device 450 may transmit the generated ensemble model to an external object.

The communication device 450 to the interface device 440 are devices that receive certain data or commands from the outside. The communication device 450 to the interface device 440 may be referred to as an input device.

The output device 460 is a device that outputs certain information. The output device 460 may output an interface required for a data processing process, an analysis result, and the like.

The computing device 430 may generate an ensemble model using a program in the storage device 410. The computing device 430 may perform ensemble pruning using a program in the storage device 410. The computing device 430 may be a device such as a processor, an AP, or a chip in which a program is embedded that processes data and processes certain operations.

Furthermore, the computing device 430 may analyze the input data using the generated ensemble model. In this case, the data generating device 400 corresponds to the analysis devices 130, 140, and 150 in FIG. 1.

Hereinafter, a result of verifying the effect on the reinforced ensemble model will be described.

5 is an example of verifying the effect on the reinforced ensemble model. 5 is an experiment result using a data set provided from the UCI machine learning repository. This is the result of using 115 unbalanced data sets. Performance estimation was repeated 30 times using 70% of training data and 30% of other test data.

The comparison target is the AdaBoost+CART model. In AdaBoost, boosting was repeated 500 times, and CART constructed a tree by randomly selecting 1-10 depths.

The enhanced ensemble model was constructed using the aforementioned genetic algorithm. The initial production was 1,000 using 300 chromosomes. Selection was determined by linear rank. The crossover was set to a single point with a probability of 0.8, and a uniform random with probability of 0.1 was used for the mutation.

In FIG. 5, the x-axis represents the AUC value for each data of AdaBoost+CART, and the y-axis represents the AUC value for the enhanced ensemble model. It refers to an ensemble model with enhanced AUCP. Of the 115 data sets, 73 had a higher mean of AUC values after pruning. In other words, it was analyzed that the performance of the enhanced ensemble model is better than the AdaBoost+CART technique. The results of Welch's t-test (significance level of 0.01) showed that the improvement for 68 data sets had statistically significant results.

6 is another example of verifying the effect on the reinforced ensemble model. 6 is an example of using data on genetic scissors. In other words, a model that identifies whether a specific genetic scissor sequence is effective for a specific sequence was verified.

The data set was CRISPR cas9. The positive class was 5,759 and the negative class was 1,672. As for the feature, 20 characters (sequence) were used. Performance estimation was repeated 30 times using 70% of training data and 30% of other test data.

The enhanced ensemble model was constructed using the aforementioned genetic algorithm. The initial production was 1,000 using 300 chromosomes. Selection was determined by linear rank. The crossover was set to a single point with a probability of 0.8, and a uniform random with probability of 0.1 was used for the mutation. The selection was elitism, which preserved the top 5%.

The comparison target is the AdaBoost+CART model. In AdaBoost, boosting was repeated 500 times, and CART constructed a tree by randomly selecting 1-10 depths.

Referring to FIG. 6, it can be seen that the AUC value of the model (AUCP) using enhanced ensemble pruning is remarkably higher than that of the AdaBoost+CART model.

In addition, the classification model generation method, the ensemble generation method, or the ensemble pruning method as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be provided by being stored in a non-transitory computer readable medium.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short moment, such as a register, cache, and memory. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, or the like.

The present embodiment and the accompanying drawings are merely illustrative of some of the technical ideas included in the above-described technology, and those skilled in the art can easily It will be apparent that all of the modified examples and specific embodiments that can be inferred are included in the scope of the rights of the above technology.

Claims (12)

Receiving, by the computer device, training data;
Generating, by the computer device, learning models for constructing an ensemble model by using the training data; And
A plurality of learning models among the learning models so that the ensemble model has an area under the ROC curve (AUC) greater than or equal to a reference value based on a determination result displayed by the computer device inputting the training data or other training data into the learning models. Comprising the step of pruning,
The learning models are an ensemble model generation method for detecting genetic scissors, which is a model for classifying genetic scissors specific to a specific sequence. The method of claim 1,
The computer device is an ensemble model generation method for detecting genetic scissors for selecting the plurality of learning models such that the ensemble model has a maximum AUC value according to optimization-based pruning. The method of claim 1,
The computer device is a method for generating an ensemble model for detecting genetic scissors that selects the plurality of learning models so that an area under curve (AUC) of a receiver operating characteristic (ROC) curve for the true positive rate and the false positive rate is maximized. The method of claim 1,
The computer device selects the plurality of learning models using a genetic algorithm, and an ensemble for detecting genetic scissors that selects the plurality of learning models such that the _{objective function g 1} + g ₂ + g _{3 of the genetic algorithm is minimized.} How to create a model.
(g ₁ is 1-AUC, g ₂ is the number of selected learning models, g ₃ is the diversity of the plurality of learning models, and AUC is the AUC (Area) of the ROC (Receiver Operating Characteristics) curve for the true positive rate and false positive rate. Under Curve)) The method of claim 1,
The training data is a method for generating an ensemble model for detecting a genetic scissors including information on a secondary structure or a tertiary structure for a gene sequence constituting the genetic scissors acting on the specific sequence. Receiving, by the computer device, training data;
Generating, by the computer device, learning models for constructing an ensemble model by using the training data; And
A plurality of learning models among the learning models so that the ensemble model has an area under the ROC curve (AUC) greater than or equal to a reference value based on a determination result displayed by the computer device inputting the training data or other training data into the learning models. An ensemble pruning method comprising the step of pruning. The method of claim 6,
The computer device is an ensemble pruning method for selecting the plurality of learning models such that an area under curve (AUC) of a receiver operating characteristic (ROC) curve for the true positive rate and the false positive rate is maximized.
The value predicted by the learning model, 1 is positive, -1 is negative, θ is the threshold value) The method of claim 6,
The computer device is an ensemble pruning method for selecting the plurality of learning models such that an area under curve (AUC) of a receiver operating characteristic (ROC) curve for the true positive rate and the false positive rate is maximized. The method of claim 6,
The computer device selects the plurality of learning models using a genetic algorithm, and the ensemble pruning method selects the plurality of learning models such that the _{objective function g 1} + g ₂ + g _{3 of the genetic algorithm is minimized.}
(g ₁ is 1-AUC, g ₂ is the number of selected learning models, g ₃ is the diversity of the plurality of learning models, and AUC is the AUC (Area) of the ROC (Receiver Operating Characteristics) curve for the true positive rate and false positive rate. Under Curve)) An input device for receiving training data;
A storage device for storing the training data and a program for generating an ensemble model; And
Using the program, learning models for constructing an ensemble model from the training data are generated, and the ensemble model is an AUC ( area under the ROC curve). The method of claim 10,
The computing device generates an ensemble model for selecting the plurality of learning models such that an area under curve (AUC) of a receiver operating characteristic (ROC) curve for the true positive rate and the false positive rate is maximized. The method of claim 10,
The computing device selects the plurality of learning models using a genetic algorithm, and generates an ensemble model for selecting the plurality of learning models such that the _{objective function g 1} + g ₂ + g _{3 of the genetic algorithm is minimized.} .
(g ₁ is 1-AUC, g ₂ is the number of selected learning models, g ₃ is the diversity of the plurality of learning models, and AUC is the AUC (Area) of the ROC (Receiver Operating Characteristics) curve for the true positive rate and false positive rate. Under Curve))

Images