JP2014241060A - Tree model learning device, tree model learning method, and tree model learning program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quickly obtain a tree model having a size suitable for generalization performance without a pruning test set.SOLUTION: The tree model learning device includes: split evaluation value calculation means 51 that calculates an evaluation value for when a node is split at each split point in each dimension of an explanatory valuable; splitless evaluation value calculation means 53 that calculates an evaluation value for when the node is not split; and split determination means 54 that determines whether the node is split or not at a dimension having the best evaluation value on the basis of the resulting evaluation values. Each of the split evaluation value calculation means and splitless evaluation value calculation means calculates the evaluation value by using an evaluation function including at least two types of terms that are in a trade-off relationship with respect to tree growth, one of the terms expressing fitting to data, and the other being a penalty term serving to reduce the evaluation value according to the number of data having fallen into a child node.

Description

  The present invention relates to a tree model learning device, a tree model learning method, and a tree model learning program for learning a model that represents a division of a data space such as a regression tree and a decision tree by a tree.

  There are models of trees called regression trees and decision trees. A regression tree is a model of a tree that performs regression, and a decision tree is a model of a tree that performs classification. There are other tree models that can be used for data clustering.

  These tree models divide the data space so that something similar at a certain index falls on the same leaf.

  More specifically, in the model of the tree, a condition concerning an explanatory variable is associated with a branch extending from each node of the tree, and the tree model is traced from the root node to the leaf node according to the condition, and associated with the arrived leaf node. A value (a real value for a regression tree, a category value for a decision tree) is output as a predicted value. Using such a tree model, it is possible to make predictions on certain unknown data samples.

  Tree model learning, such as regression trees and decision trees, has the advantage that it operates at high speed because the learning method is simple, and that the resulting model is easy for humans to interpret. On the other hand, in terms of accuracy, it is generally worse than a support vector machine. However, it is known that high accuracy can be achieved by performing boosting by learning and combining a plurality of trees in model learning of trees. Regression trees and decision trees are still one of the most commonly used data mining methods.

  However, even if the learning method is a simple tree model, it takes time to learn when the amount of data is large, and thus speeding up is necessary. With the recent development of distributed and parallel processing infrastructure, the speed of learning algorithms has been increased by distributed parallelization (see, for example, Non-Patent Document 1). Further, as a method of speeding up on a different axis from distributed parallelism, there is a tree algorithm called a Hoeffing tree (see Non-Patent Document 2, for example).

  The Hoeffing tree is a decision tree learning algorithm for a data stream. However, the Hoeffing tree is not limited to a case where the object is a data stream, and is an effective method for speeding up batch learning of a regression tree or a decision tree. The idea of speeding up with Hoeffing trees is that statistical processing is used to ensure accuracy after processing a portion of the data, rather than splitting the tree after processing all the data, i.e. growing the tree. If the data is sufficient, the tree is split at that time. When there is a large amount of data, a tree can be constructed while processing only the minimum data required for accuracy assurance, so that model learning of the tree can be performed at high speed.

Panda Biswanath, Herbach J.S., Basu Sugato, Bayarde R.J., "PLANET: massively parallel learning of tree ensembles with MapReduce", VLDB Endowment, 2009. Domingos Pedro, Hulten Geoff, "Mining high-speed data streams", PCM SIGKDD, 2000.

  However, the Hoeffing tree algorithm has the following problems. That is, the Hoeffing tree algorithm has a problem that it cannot be determined where to stop the tree growth from the split evaluation function. In the Hoeffing tree algorithm, since the information gain and Gini coefficient are used for the evaluation function of split, the result of the evaluation should always be split. If nodes are split based on such an evaluation function, the tree grows too much and overlearns, and generalization performance is lost. To avoid such problems, a test data set was needed for pruning overgrown trees.

  Apart from pruning, there is a method to prevent overlearning by setting the depth, the number of leaves, and the number of data per leaf by setting, but without limiting the data to be predicted, It is difficult to set an appropriate value in advance. If an appropriate value is not set, a model with inaccurate accuracy is generated due to insufficient growth of the tree, or generalization performance is lost due to excessive growth of the tree.

  In view of the above-described problems, the present invention provides a tree model learning apparatus, a tree model learning method, and a tree that can quickly obtain a tree model having an appropriate size in terms of generalization performance without using a test set for pruning. The purpose is to provide a model learning program.

  A tree model learning apparatus according to the present invention uses a specified dimension of an explanatory variable, split evaluation value calculating means for calculating an evaluation value when a specified node is split at each split point of the dimension, split evaluation Based on the evaluation value obtained by the value calculation means, the split point search means for searching the best split point for splitting the specified node for each dimension of the explanatory variable and the evaluation value when the node is not split are calculated. Split the node in the dimension with the best evaluation value based on the evaluation value calculation means without splitting, the evaluation value at the best split point for each dimension of the explanatory variable, and the evaluation value when not splitting, or Split evaluation means for determining whether or not to split a node, and a split evaluation value The calculation means are two types of terms that are in a trade-off relationship with the growth of the tree, and the number of passing data or the number of passing data with respect to the term representing the fitting to the data by the good evaluation value and the branch node and the leaf node, respectively The evaluation value is calculated using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value in accordance with the number of falling data. There are two types of terms that are in a trade-off relationship: a term that represents fitting to data with good evaluation value, and a penalty term that works in the direction of worsening the evaluation value according to the number of data drops for leaf nodes. An evaluation value is calculated using an evaluation function including at least a kind of term.

  In addition, the tree model learning method according to the present invention provides, for each node in order from the root node, an evaluation value for splitting the node at each split point of the dimension for each dimension of the explanatory variable. The two types of terms that are in a trade-off relationship with respect to the term that represents the fitting to the data with good evaluation values, and the branch node and the leaf node are evaluated according to the number of passing data or the number of falling data, respectively. The best split point for splitting a node for each dimension of explanatory variables based on the evaluation value calculated using an evaluation function that includes at least two types of penalty terms that work in the direction of worsening the value. And the evaluation value when the node is not split is two types of terms that are in a trade-off relationship with the growth of the tree, It is calculated using an evaluation function that includes at least two types of terms: a term that expresses goodness of the evaluation value, and a penalty term that works in the direction of worsening the evaluation value according to the number of data that falls for leaf nodes. Based on the evaluation value at the best split point for each dimension and the evaluation value when not splitting, it is determined whether to split the node at the dimension with the best evaluation value or not to split the node. To do.

  Also, the tree model learning program according to the present invention uses a specified dimension of an explanatory variable in a computer to calculate an evaluation value when splitting a specified node at each split point of the dimension. Split point search processing to search the best split point for splitting the specified node for each dimension of the explanatory variable based on the evaluation value obtained by the processing, split evaluation value calculation processing, evaluation when the node is not split Splits the node in the dimension with the best evaluation value based on the evaluation value calculation process without split and the evaluation value at the best split point for each dimension of the explanatory variable and the evaluation value when not splitting Split decision process to determine whether or not to split the node In the split evaluation value calculation process, there are two types of terms that are in a trade-off relationship with the growth of the tree, a term that represents the fitting to the data with good evaluation values, a branch node, and a leaf node For the evaluation value calculation process without splitting, the evaluation value is calculated by using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value according to the number of passing data or the number of falling data, respectively. Two types of terms that have a trade-off relationship with the growth of the tree, the term representing the fitting to the data by the goodness of the evaluation value, and the leaf node, the evaluation value is deteriorated according to the number of data falling The evaluation value is calculated using an evaluation function including at least two types of penalty terms acting in the direction.

  According to the present invention, a tree model having an appropriate size in terms of generalization performance can be obtained at high speed without a test set for pruning.

It is a block diagram which shows the structural example of the tree model learning system of 1st Embodiment. FIG. 3 is a block diagram illustrating a hardware configuration example of the tree model learning device 10. It is explanatory drawing which shows an example of the learning data stored in the data storage part. 3 is an explanatory diagram schematically showing an example of a tree model stored in a tree model storage unit 102. FIG. It is explanatory drawing which shows an example of the bin divided as a result of the calculation of a binning point. It is a flowchart which shows an example of operation | movement of the tree model learning system of 1st Embodiment. It is a block diagram which shows the structural example of the tree model learning system of 2nd Embodiment. It is a flowchart which shows an example of operation | movement of the tree model learning system of 2nd Embodiment. It is a block diagram which shows the minimum structural example of the tree model learning apparatus of this invention. It is a block diagram which shows the other structural example of the tree model learning apparatus of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, the case where the tree model is a regression tree will be described as an example. The technical concepts of the split evaluation method and the confidence interval calculation method used in the tree model learning method according to the present invention can be commonly used in model learning of other trees. For example, in the case of decision tree learning, the technique disclosed in the literature “Bouchard Guillaume,“ Efficient Bounds for the softmax Function, Applications to Inference in Hybrid Models ”, NIPS 2007” etc. Can result in processing. Further, even a tree model other than a regression tree can be reduced to a process similar to learning of a regression tree if the branch of the branch corresponds to the division of the data space.

Embodiment 1. FIG.
FIG. 1 is a block diagram illustrating a configuration example of a tree model learning system according to the first embodiment of this invention. As shown in FIG. 1, the tree model learning system of this embodiment includes a tree model learning device 10. The tree model learning device 10 includes a data storage unit 101, a tree model storage unit 102, a split point calculation unit 103, a no split calculation unit 104, a binning point calculation unit 105, a split determination unit 106, Tree node generation means 107. FIG. 1 shows an example in which one tree model learning apparatus 10 includes these storage units and processing means. However, these storage units and processing means are separately implemented in a plurality of devices. Also good.

  FIG. 1 shows prediction means 20 that performs prediction using a learned tree model in combination with the tree model learning apparatus 10, but the tree model learning apparatus 10 includes the prediction means 20. Also good.

  FIG. 2 is a block diagram illustrating a hardware configuration example of the tree model learning device 10. As shown in FIG. 2, the tree model learning device 10 may include a calculation unit 11, a storage unit 12, and an input / output unit 13. Such an apparatus is realized by an information processing apparatus such as a personal computer that operates according to a program, for example. In this case, the calculation unit 11, the storage unit 12, and the input / output unit 13 are realized by a CPU, a memory, and various input / output devices (for example, a keyboard, a mouse, a network interface unit, and the like), respectively.

  In the present embodiment, the data storage unit 101 and the tree model storage unit 102 are realized by a storage device (for example, the storage unit 12 illustrated in FIG. 2). The split point calculation unit 103, the no split calculation unit 104, the binning point calculation unit 105, the split determination unit 106, and the tree node generation unit 107 are, for example, a CPU (for example, the calculation unit 11 shown in FIG. 2) that operates according to a program. It is realized by.

  The data storage unit 101 stores data used when learning a tree model (hereinafter referred to as learning data). FIG. 3 is an explanatory diagram illustrating an example of learning data stored in the data storage unit 101.

  In the example shown in FIG. 3, one line represents one learning data. The column is matrix data representing the attribute value of each learning data. FIG. 3 shows an example of learning data for regression for predicting an objective variable from a three-dimensional explanatory variable.

The tree model storage unit 102 stores a learned tree model. FIG. 4 is an explanatory diagram schematically illustrating an example of a tree model stored in the tree model storage unit 102. The tree model storage unit 102 associates each node with information indicating the tree structure of the generated tree model (for example, how many nodes exist and how these nodes are connected to other nodes, etc.). , A predicted value, an ID list of learning data falling on the node, an explanatory variable condition corresponding to an edge to the child node if a child node exists, and the like. In FIG. 4, “135” is stored as the predicted value in association with the child node identified by ID = i, and “60.0 <X ₂ ” is stored as the condition of the explanatory variable corresponding to the edge to the child node. An example is shown. The condition of the explanatory variable corresponding to the edge to the child node may be information represented by the split dimension (second dimension in the example of Node i) and the split value information as described above. The data storage format of the tree model is not limited to this, and is not particularly limited as long as it is configured so that the learned tree model can be specifically grasped.

For each data dimension, the binning point calculation means 105 calculates a binning point, which is a breakpoint for dividing the data into bins in the dimension. For example, the binning point calculation unit 105 may obtain a breakpoint that divides bins from the possible values for each dimension of the explanatory variables of the learning data as shown in FIG. As a binning point calculation method, for example, there are a method of dividing with equal widths, a method of obtaining binning points so that the number of data included in each bin is substantially equal, and the like. FIG. 5 is an explanatory diagram illustrating an example of bins divided as a result of calculating binning points for a certain dimension. FIG. 5 shows an example of _n bins (Bin _{1 to} Bin _n ) divided by a method of dividing the range of the first-dimensional data of the explanatory variables with equal widths.

  The split point calculation means 103 is the best split point (boundary condition) for splitting a node (hereinafter referred to as a target node) for which to determine whether or not to split for each dimension of the explanatory variable. And a binning means 1031, a split evaluation value calculation means 1032, a split evaluation value confidence interval calculation means 1033, and a split point search means 1034.

The binning means 1031 distributes the learning data falling on the target node to the bins based on the binning points obtained by the binning point calculation means 105 for the specified dimension, calculates the statistics for each bin, and stores them. . The necessary statistics are calculated according to the evaluation value calculation formula. For example, the binning unit 1031 may calculate and store three types of statistics such as Σy ² , Σy, and Σ1 for the learning data included in each bin. The binning means 1031 of this example calculates the statistics for each bin and also calculates and stores the statistics accumulated for all bins. In the above example, the values of Σy ² , Σy, and Σ1 accumulated for all bins may be calculated.

  The split evaluation value calculation means 1032 calculates an evaluation value when the split is performed at each candidate point of the split at the target node using a predetermined evaluation function for the designated dimension. Hereinafter, the evaluation value obtained by the split evaluation value calculation unit 1032 may be referred to as a “split evaluation value”. The split evaluation value calculation means 1032 does not calculate the split evaluation value using all the points that can be split as candidate points, but only between the bins separated by the binning points calculated by the binning point calculation means 105. A split evaluation value may be calculated as a split candidate point.

  The split evaluation value calculation means 1032 may calculate the split evaluation value using, for example, an evaluation function represented by the following formula (1).

  The evaluation function of the above equation (1) includes a penalty term associated with the complexity of the tree model in addition to the term representing the fitting to the data. Specifically, the penalty term is a term that works in the direction of worsening the evaluation value in accordance with the number of data falling in the child nodes. In Equation (1), the second term is a penalty term for the left leaf node, the fourth term is a penalty term for the right leaf node, and the sixth term is a penalty term for the branch node. In these terms, a penalty is applied according to the number of data (data passing in the case of a branch node) falling on each node.

In Expression (1), N represents the total number of data falling on the target node. _{R L} and r _R represent the ratio of data falling on the left and right leaf nodes of the target node, respectively. That is, r _L and r _R are r _L = n when the number of data falling on the left and right leaf nodes is n _L and n _R , respectively, when n data is viewed with respect to the target node in a certain dimension. _L / _n, a random variable represented by _r R = 1-r _L. The squares of σ _L and σ _R with hats represent the variance of the objective variable of the data that has fallen on the left and right leaf nodes, respectively. Expression (1) is an example of an evaluation function for obtaining a split evaluation value, and is not limited to this.

  The evaluation function used in the Hoeffing tree algorithm included only a term representing data fitting. For this reason, the evaluation value was calculated only by looking at the degree of data fitting without considering how complicated the tree after splitting was. Such an evaluation value results in a determination result that it is always better to split even when compared with the no-split evaluation value calculated by the no-split calculation means 104 described later.

  On the other hand, the evaluation function used by the split evaluation value calculation unit 1032 of this embodiment includes two types of terms that have a trade-off relationship with respect to tree growth. That is, it includes at least a term that increases in value when growing a tree and a term that decreases in value when growing a tree. In the example of the above evaluation function, a term that expresses the degree of fit to the data in the leaf by the good evaluation value (right and left in the case of a binary tree) and a penalty term that makes the evaluation value worse by the complexity of the tree (branch node and Leaf nodes), and these are in a trade-off relationship. By including two terms that have such a trade-off relationship, a value that compares the merit of data fitting when splitting at each point and a penalty that complicates the tree appears as a split evaluation value. I have to.

  If such an evaluation function is used, for example, if the merit of fitting data to a split is larger than the penalty that the tree becomes complicated by splitting, the split at that point is permitted, while the fitting to the data by splitting is allowed. If the advantage is smaller than the penalty that the tree becomes complicated by splitting, it can be determined that splitting at that point is not permitted.

  The split evaluation value confidence interval calculation unit 1033 obtains a confidence interval of the split evaluation value calculated by the split evaluation value calculation unit 1032. When the speed-up method in the Hoeffding tree algorithm is applied to the tree model learning, since a part of the learning data is read and the split evaluation value is calculated, an accurate value cannot be obtained. In such a case, the split evaluation value confidence interval calculation unit 1033 obtains a confidence interval of the calculated split evaluation value, thereby enabling determination in consideration of an error included in the split evaluation value. The confidence interval is an interval in which an accurate evaluation value exists with a high probability 1-δ when only n pieces of data are read. Note that δ represents the error probability of the confidence interval.

  By the way, when the evaluation function for obtaining the split evaluation value is divided into a fitting term for data and a penalty term due to the complexity of the tree, it is expected that a logarithm is used for the penalty term. In such a case, it is not possible to obtain a confidence interval by regarding the entire evaluation function as a random variable. In such a case, the split evaluation value confidence interval calculation unit 1033 only includes the portion corresponding to the random variable in the evaluation function (in the above example, the ratio of the data falling on the left / right lobe, the data on the left / right lobe) Focusing on the confidence intervals focusing on the square error), the confidence intervals of the entire evaluation function are calculated using those confidence intervals.

  For example, the split evaluation value confidence interval calculation unit 1033 maximizes and minimizes the above-described equation (1) using the following equations (2) to (7), so that the confidence interval of the split evaluation value is obtained. May be calculated. Equations (2) to (7) are examples of conditional expressions for deriving the confidence interval of the split evaluation value based on the assumption that the square error of the data falling on the leaf follows the chi-square distribution. It is not limited to this.

  The split point search means 1034 searches for the best split point for each dimension of the explanatory variable. The split point search unit 1034 causes the split evaluation value calculation unit 1032 to calculate the split evaluation value at each candidate point of the split in the target node for each dimension of the explanatory variable, and for each dimension of the explanatory variable based on the result. Find the best split point. For example, the split point search unit 1034 may set the candidate point that has obtained the best split evaluation value in each dimension as the best split point in the dimension.

  In this way, the split point calculation means 103 calculates the split condition, the split evaluation value, and the confidence interval as the information of the split point that is best split at the target node for each dimension of the explanatory variable. Get by.

  The no split calculation means 104 is a means for calculating an evaluation value and its confidence interval when the target node is not split, and includes a no split evaluation value calculation means 1041 and a no split evaluation value confidence interval calculation means 1042.

  The evaluation value calculation means 1041 without splitting uses a predetermined evaluation function to calculate an evaluation value when the target node is not split. Hereinafter, the evaluation value calculated by the evaluation value calculation unit 1041 without split may be referred to as “evaluation value without split”. The evaluation value calculation unit 1041 without split may calculate the evaluation value without split using, for example, an evaluation function represented by the following formula (8).

  In Expression (8), N represents the total number of data falling on the target node. Further, the square of σ with a hat represents the variance of the objective variable of the data that has fallen on the target node. Note that the learning data is the same as the learning data when the split evaluation value is obtained. Specifically, the statistics calculated by the binning means 1031 may be used.

  The evaluation function used by the split-less evaluation value calculation unit 1041 only needs to include two types of terms that have a trade-off relationship with respect to tree growth at the target node. In the example of the evaluation function, a term that represents the degree of fit in the leaf by the good evaluation value and a penalty term that deteriorates the evaluation value due to the complexity of the tree are included, and these are in a trade-off relationship. Specifically, in Equation (8), the second term is a penalty term for the leaf node, and a penalty corresponding to the number of data falling on the leaf node is applied. Equation (8) is one example of an evaluation function for obtaining an evaluation value without splitting, and is not limited to this.

  The evaluation value confidence interval calculation means 1042 without split obtains a confidence interval of the evaluation value without split calculated by the evaluation value calculation means 1041 without split. The evaluation value confidence interval calculation means 1042 without splitting calculates, for example, the confidence interval of the evaluation value without splitting by maximizing and minimizing the above equation (8) using the following equation (9). May be.

  Expression (9) is an example of a conditional expression for deriving the confidence interval of the evaluation value without split based on the assumption that the square error of the data falling on the leaf follows the chi-square distribution, and is not limited thereto. .

  In this way, the non-split calculating means 104 obtains the non-split evaluation value, which is an evaluation value when the target node is not split, and its confidence interval by calculation.

  The split determination unit 106 determines whether or not to split based on the evaluation value obtained by the split point calculation unit 103 and the non-split calculation unit 104 and its confidence interval. For example, if the best evaluation value among the split evaluation value at the best split point and the non-split evaluation value for each dimension in the target node is the split evaluation value, the split determination unit 106 obtains the split evaluation value. On the other hand, if the best evaluation value is the non-split evaluation value, it may be determined that the target node does not split.

  In addition, the split determination unit 106 may perform determination based on the following three options in addition to the confidence interval of each evaluation value. That is, in the determination, the split determination unit 106 may determine whether to split, not split, or add learning data for the target node. When the addition of learning data is determined, the determination of whether or not the target node is split is suspended. The split determination unit 106 extracts, for example, the first and second highest evaluations from the split evaluation value at the best split point and the non-split evaluation value for each dimension in the target node. If there is an overlap between the confidence intervals of the first and second evaluation values, it is assumed that the number of processed data is not sufficient to determine the presence or absence of splitting, and further learning data is read and the evaluation value is recalculated. Also good. On the other hand, if there is no overlap in the confidence intervals, it is sufficient to determine whether or not to split based on whether the first evaluation value is a split evaluation value or a non-split evaluation value, as in the case described above. Here, the case where there is an overlap in the confidence interval of the evaluation value specifically means a case where the lower limit of the confidence interval of the first evaluation value is smaller than the upper limit of the confidence interval of the second evaluation value.

  If the result of determination by the split determination unit 106 is that the target node is determined to be split, the tree node generation unit 107 splits the target node with the specified split dimension and split value. Specifically, a child node is added to the target node, the target node is set as a parent node, information on the predicted value, split dimension, and split value is associated with the parent node, and the ID of data that falls on the child node is associated with the child node Associate. These pieces of information are stored in the tree model storage unit 102.

  When an unknown data sample is input, the prediction unit 20 performs prediction using a tree model stored in the tree model storage unit 102. For example, the prediction unit 20 traces the tree model from the root based on the value of each dimension of the explanatory variable obtained from the input data sample, and returns a predicted value associated with the finally reached leaf node. Can be done.

  Although not shown in the figure, the tree model learning device 10 performs overall control related to tree model learning, such as reading learning data or activating each processing means to give necessary work instructions. Instruction means may be included. The learning instruction means is realized by, for example, a CPU (for example, the calculation unit 11 shown in FIG. 2) that operates according to a program.

  Next, the operation of this embodiment will be described.

  FIG. 6 is a flowchart showing an example of the operation of the tree model learning system of this embodiment. In the example illustrated in FIG. 6, first, the tree model learning device 10 (more specifically, the learning instruction unit) reads the learning data from the data storage unit 101 and activates the binning point calculation unit 105. The activated binning point calculation means 105 calculates binning points for each dimension of the explanatory variable based on the read learning data (step S101). Below, the example which divides into 100 equally as binning is shown. For example, when the dimension of an explanatory variable takes a value from 0 to 1000, a binning point is placed in increments of 10 values.

  Next, the tree model learning device 10 adds a root node to the node queue (hereinafter abbreviated as NQ) (step S102). NQ is a FIFO-structured buffer that can extract nodes in the order of addition. It is assumed that the tree model storage unit 102 stores information in which all learning data is specified as the ID of data falling on the node as information on the root node.

  Next, the tree model learning device 10 checks whether or not NQ is empty (step S103), and if not empty, extracts one node from the NQ (No in step S103, S104). Here, the root node is first taken out. Hereinafter, the node extracted in step S104 is set as a split target node, and the processes in steps S105 to S115 are performed.

  On the other hand, if NQ is empty, the tree model learning is terminated (Yes in step S103).

  In step S105, for the target node extracted from the NQ, the tree model learning device 10 refers to the ID set of data falling on the target node and reads n pieces of learning data falling on the target node.

  Next, the tree model learning device 10 initializes a variable D representing the dimension of the explanatory variable to be calculated (step S106). Here, D = 1 is set to represent the first dimension.

  When the learning data reading and the initialization of the variable D are completed for the target node, the tree model learning device 10 activates the split point calculation unit 103, and for the dimension indicated by the variable D, the best split in the target node. Start the process of finding points.

  When the split point calculation unit 103 is activated, the binning unit 1031 first performs binning processing for the dimension indicated by the variable D (step S107). Based on the binning points calculated by the binning point calculation unit 105, the binning unit 1031 processes the n pieces of learning data read in step S105, and calculates the statistics for each bin. For example, when a certain bin is in the range of [10, 20], the binning means 1031 uses the sum of squares of the objective variable and the objective variable as the statistics of the bin for the data in which the D-dimensional explanatory variable is in this range. And the number of data is calculated and stored as bin statistical information. As a result of determination processing described later, in the step processing after learning data is added to the same node and the same dimension, bin statistics are accumulated with respect to the stored bin statistical information. .

  Next, the split point search unit 1034 searches for the best split point in the dimension indicated by the variable D (step S108). In step S108, the split point search unit 1034 first activates the split evaluation value calculation unit 1032 to calculate the split evaluation value of each candidate point of the split according to the dimension indicated by the variable D. Based on the bin statistical information created in step S107, the split evaluation value calculation unit 1032 uses, for example, the split evaluation value when splitting between each bin that is a candidate point for split of the designated dimension. Calculate using equation (1).

  Next, the split evaluation value confidence interval calculation unit 1033 calculates the confidence interval of the split evaluation value obtained in step S108 (step S109). The split evaluation value confidence interval calculation unit 1033 calculates the confidence interval of the split evaluation value obtained in step S108 using, for example, the equations (2) to (7).

  Next, the split point search unit 1034 determines whether D <number of dimensions, and if so, increments D and returns to step S107 (Yes in step S110, S111). On the other hand, if D <number of dimensions is not satisfied, the process is completed for all dimensions, and the process proceeds to step S112 (No in step S110).

  With the above processing, the split point that maximizes the evaluation function, the split evaluation value, and the confidence interval are obtained for all dimensions of the evaluation variable.

  In step S112, the tree model learning device 10 activates the no split calculation unit 104 to calculate an evaluation value (non-split evaluation value) when the target node is not split and its confidence interval. In step S112, first, the evaluation value calculation unit 1041 without split calculates an evaluation value without split. The evaluation value calculation unit 1041 without splitting calculates the evaluation value when the target node does not split using, for example, the equation (8).

  Next, the evaluation value confidence interval calculation means 1042 without split calculates the confidence interval of the evaluation value without split obtained in step S112 (step S113). The evaluation value confidence interval calculation means 1042 without split calculates the confidence interval of the evaluation value without split obtained in step S112 using, for example, the equations (8) and (9).

  When the split evaluation value at each dimension split point and its confidence interval, the non-split evaluation value and its confidence interval are found in this way for the target node, the split determination means 106 determines whether to split the target node. It is determined whether or not to add learning data (step S114).

  In step S114, for example, the split determination means 106 obtains the best first and second ranks among these evaluation values, and determines as follows based on the result.

  In the example shown below, Case 1 is “a slit evaluation value of a dimension having the first evaluation value, and the upper limit of the confidence interval of the second evaluation value is smaller than the lower limit of the confidence interval of the first evaluation value. And Case 2 is a case where “the first evaluation value is an evaluation value without splitting and the upper limit of the confidence interval of the second evaluation value is smaller than the lower limit of the confidence interval of the first evaluation value”. Case 3 is a case where “the upper limit of the confidence interval of the second-ranked evaluation value is larger than the lower limit of the confidence interval of the first-ranked evaluation value, whatever the evaluation value of the first-ranked evaluation value”. The split determination means 106 determines which of the three evaluation values corresponds to the first and second evaluation values, and in case 1, the split is performed at the split candidate point that obtained the first evaluation value. Make a decision to that effect. In case 2, it is decided not to split. Also, in case 3, it is determined that the learning data is to be added because the number of processed data is not sufficient to determine the presence / absence of split.

  As a result of the determination in step S114, if it is determined to split, the tree node generation unit 107 splits the target node at the split point of the designated dimension (step S115). Specifically, the tree node generation unit 107 newly adds information on a child node whose parent node is the target node to the tree model storage unit 102, and adds the child node added to the target node that is the parent node. In addition to associating the condition of the explanatory variable corresponding to the edge, the predicted value and the ID list of the learning data falling on the child node are associated with each other and held. In addition, when adding a node, the tree node generation unit 107 adds the added child node to the NQ. As a result, the child node added to the determination target node for the next and subsequent splits is added.

  On the other hand, if it is determined in step S114 that splitting is not to be performed, the tree model learning device 10 ends the process for the target node without splitting the target node, and determines whether the next node is split. The process returns to step S103 for determination. In step S103 after the return, if there is a child node added to NQ in step S115 as a result of the split determination at a certain node, those child nodes are sequentially extracted and set as the target node. In this way, the tree grows in the process of repeatedly performing split determination while changing the target node until NQ becomes empty.

  On the other hand, if it is determined as a result of the determination in step S114 that learning data is to be added, the target node returns to step S105 as it is. When the Hoeffing tree algorithm is adopted for speeding up the tree model learning, the obtained evaluation value includes an error. Therefore, in the above determination, if the first-ranked evaluation value is higher than the second-ranked evaluation value even if the error is included, the presence / absence of split is determined based on the evaluation value. Otherwise, the addition of learning data is determined. I am doing so. Returning to step S105, n pieces of learning data are added, and the evaluation value is recalculated in steps S108 and S112. Such addition of learning data is repeated until, for example, the accuracy at which the presence / absence of split in the target node can be determined by the determination in step S114.

  As described above, according to the present embodiment, an appropriately sized tree model can be learned in terms of generalization performance without preparing a test data set for pruning. The reason for this is that, for each node, a means for calculating a split evaluation value that determines the quality of a plurality of split candidates using an evaluation function including two terms that are in a trade-off relationship with the growth of the tree; This is because there is provided means for calculating an evaluation value in the case of not splitting using an evaluation function including two terms that are in a trade-off relationship with respect to growth. This is because it is possible to determine whether to split at each node or to stop using an evaluation value that represents the degree of fitting to the data and the degree of complexity of the tree by comparative consideration. This is because by using such an evaluation value to determine whether or not there is a split at each node, it is possible to stop the tree growth with an optimum size in terms of generalization performance.

  Further, according to the present embodiment, it is possible to perform model learning of a tree having an optimum size in terms of such generalization performance at high speed. The reason is that a means for calculating a confidence interval of the evaluation value calculated from a part of the data is provided. This is because it is possible to determine the presence or absence of splitting with only a part of statistically sufficient data necessary for ensuring accuracy without processing all data.

Embodiment 2. FIG.
FIG. 7 is a block diagram illustrating a configuration example of the tree model learning system according to the second embodiment of this invention. The tree model learning system of this embodiment is different from the tree model learning system shown in FIG. 1 in that the split point calculation unit 103 is a split point parallel calculation unit 203 and a calculation unnecessary dimension deletion unit 108 is newly added. It has different points.

  In the present embodiment, the split point parallel calculation unit 203 performs the processing performed by the split point calculation unit 103 for each dimension in the first embodiment in parallel for each dimension. The split point parallel calculation means 203 is provided for the number of dimensions of explanatory variables, for example.

  The calculation unnecessary dimension deletion unit 108 determines whether or not there is a split dimension that can be excluded from the recalculation of the evaluation value after adding the learning data when the split determination unit 106 determines to add the learning data for a certain target node. If there is, the split dimension is deleted from the recalculation target.

  For example, the unnecessary calculation dimension deleting unit 108 has an evaluation in which the upper limit of the confidence interval is smaller than the lower limit of the confidence interval of the first evaluation value among the evaluation values obtained when the split determination unit 106 performs the determination. If there is a value, the dimension from which the evaluation value has been obtained may be subjected to a process of excluding it from the target of subsequent recalculation of the evaluation value at the node. In other words, if the upper limit of the confidence interval of the evaluation value at the best split point of a dimension is smaller than the lower limit of the confidence interval of the first evaluation value, the dimension is excluded from the recalculation of the subsequent evaluation value. The

  The calculation unnecessary dimension deleting unit 108 may be provided with a flag or the like indicating that the split point parallel calculating unit 203 associated with the excluded dimension is not activated at the time of recalculation.

  FIG. 8 is a flowchart showing an example of the operation of the tree model learning system of this embodiment. As shown in FIG. 8, in the operation of this embodiment, compared to the operation of the first embodiment shown in FIG. 6, the process related to the variable D indicating the dimension is deleted, and the processes in steps S107 to S109 are executed in parallel. And the point that a calculation unnecessary dimension deletion step (step S201) is added is different.

  In parallel execution of steps S107 to S109, for example, when reading of learning data is completed for the target node, the tree model learning device 10 activates each split point parallel calculation unit 203 and performs processing for the corresponding dimension. What is necessary is just to start the process which calculates | requires the best split point in a node.

  Further, in step S201, the calculation-unnecessary dimension deletion unit 108 that has received the decision to add learning data determines whether or not there is a split dimension that can be excluded from the recalculation of the evaluation value after adding learning data. If there is, the process of deleting the split dimension from the recalculation target is performed. The calculation unnecessary dimension deletion unit 108 compares the evaluation value of the first rank out of the evaluation values used for determination with the evaluation value of the arbitrary rank i, and determines the evaluation value of the rank i from the lower limit of the confidence interval of the first evaluation value. When the upper limit of the confidence interval is smaller, a process of excluding the split dimension from which the evaluation value of the ranking i is obtained from the calculation target of the subsequent split point for the node is performed.

  As described above, according to the present embodiment, the calculation part for obtaining each evaluation value of the split candidate, which becomes a bottleneck of processing, is performed in parallel, and further includes a means for deleting a calculation unnecessary dimension. As a result, it is possible to omit recalculation processing of split points of dimensions that do not require calculation, and the processing speed is further increased.

  Next, the minimum configuration of the tree model learning apparatus according to the present invention will be described. FIG. 9 is a block diagram showing a minimum configuration example of the tree model learning apparatus according to the present invention. As shown in FIG. 9, the tree model learning apparatus according to the present invention includes, as the minimum components, a split evaluation value calculation means 51, a split point search means 52, a non-split evaluation value calculation means 53, and a split determination means 54. With.

  The split evaluation value calculation unit 51 (for example, the split evaluation value calculation unit 1032) calculates an evaluation value when the specified node is split at each split point of the dimension using the specified dimension of the explanatory variable. .

  The split point calculation means 52 (for example, the split point search means 1034) is the best split point for splitting the node specified for each dimension of the explanatory variable based on the evaluation value obtained by the split evaluation value calculation means 51. Search for.

  The evaluation value calculation means 53 without split (for example, the evaluation value calculation means 1041 without split) calculates an evaluation value when the designated node is not split.

  The split determination means 54 splits the designated node in the dimension with the best evaluation value based on the evaluation value at the best split point for each dimension of the explanatory variable and the evaluation value when not splitting, or Determine whether to split the specified node.

  In the present invention, the split evaluation value calculation means 51 includes two types of terms that are in a trade-off relationship with respect to the growth of the tree, a term that represents fitting to data with good evaluation values, a branch node, and a leaf. For each node, an evaluation value is calculated using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value according to the number of passing data or the number of data dropped. Further, the evaluation value calculation means 53 without splitting falls in terms of two types of terms that are in a trade-off relationship with respect to the growth of the tree, that is, a term that represents the fitting to the data with good evaluation values and a leaf node. The evaluation value is calculated using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value according to the number of data.

  Therefore, according to the minimum configuration tree model learning apparatus, a tree model having an appropriate size in terms of generalization performance can be obtained at high speed without a test set for pruning. A split evaluation value that determines the quality of a plurality of split candidates is calculated using an evaluation function including two terms that are in a trade-off relationship with respect to tree growth, and also in a trade-off relationship with respect to tree growth. This is because by calculating an evaluation value in the case of not splitting using an evaluation function including a certain two terms, it is possible to determine where to stop the tree growth with validity while maintaining a simple learning method.

  FIG. 10 is a block diagram showing another configuration example of the tree model learning apparatus of the present invention. As shown in FIG. 10, the tree model learning apparatus according to the present invention may include a split evaluation value confidence interval calculation unit 55 and a non-split evaluation value confidence interval calculation unit 56.

  The split evaluation value confidence interval calculation means 55 (for example, the split evaluation value confidence interval calculation means 1033) calculates the confidence interval of the evaluation value calculated by the split evaluation value calculation means 51. Further, the evaluation value confidence interval calculation means 56 without split (for example, the evaluation value confidence interval calculation means 1042 without split) calculates the confidence interval of the evaluation value calculated by the evaluation value calculation means 53 without split.

  In such a case, the split determination means 54 reads the dimension of the explanatory variable each time the evaluation value when the predetermined number of data is read into the node and the evaluation value when the split is performed and the evaluation value when the split is not performed are calculated. Split the node on the dimension with the best evaluation value or do not split the node based on the evaluation value at each best split point and their confidence intervals and the evaluation value and its confidence interval when not splitting Alternatively, it may be determined whether the number of processed data is not sufficient to determine whether to split.

  According to such a configuration, the tree model learning can be performed using only a part of the learning data, so that a tree model of an appropriate size can be obtained at a higher speed in terms of generalization performance.

  Further, the tree model learning device may operate at least evaluation value calculation processing by the split evaluation value calculation means 51 for each dimension in parallel.

  According to such a configuration, an appropriately sized tree model can be obtained at a higher speed in terms of generalization performance.

  Also, if the tree model learning device determines that the number of data processed by the split determination unit 54 is not sufficient to determine whether or not to split, the tree model learning device further reads data into the designated node, The evaluation value and its confidence interval when the specified node is split may be recalculated, and the evaluation value and its confidence interval when not splitting may be recalculated.

  According to such a configuration, it is possible to obtain a tree model of an appropriate size in terms of generalization performance by adjusting the number of data read at a time.

  Further, the tree model learning device may further include a calculation unnecessary dimension deleting unit 57. The calculation-unnecessary dimension deletion unit 57 (for example, the calculation-unnecessary dimension deletion unit 108) determines that the number of data processed by the split determination unit 54 is not sufficient to determine whether or not to split, at the time of the determination Exclude from the recalculation of the evaluation value using the newly read data and the recalculation of that confidence interval the dimension whose upper limit of the confidence interval of the second and subsequent evaluation values is smaller than the lower limit of the confidence interval of the best evaluation value Process.

  According to such a configuration, it is possible to omit recalculation processing of split points of dimensions that do not require calculation, so that a tree model of an appropriate size can be obtained at a higher speed in terms of generalization performance.

  Although the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  For example, another aspect of the present invention may be a computer program that causes a computer to realize each configuration according to each of the above embodiments, or a computer-readable storage medium that records such a program. Also good. This recording medium includes a non-transitory tangible medium.

  Moreover, although a part or all of said embodiment can be described also as the following additional remarks, it is not restricted to the following.

  (Supplementary note 1) For each node in order from the root node, the evaluation value when the node is split at each split point of the dimension is traded off with respect to the growth of the tree. There are two types of terms in the above, which indicate that the fitting to the data is expressed by a good evaluation value, and the branch node and the leaf node tend to worsen the evaluation value according to the number of passing data or the number of falling data, respectively. Calculates using an evaluation function including at least two types of penalty terms, and based on the obtained evaluation value, finds the best split point for splitting the node for each dimension of the explanatory variable, and splits the node The evaluation value in the case of not doing is two types of terms that have a trade-off relationship with the growth of the tree, and the fitting of the data to the evaluation value is good The evaluation at the best split point for each dimension is calculated using an evaluation function that includes at least two types of terms, the term to represent and the penalty term that works in the direction of worsening the evaluation value according to the number of data dropped. A tree model learning method for determining whether to split a node at a dimension having the best evaluation value or not to split a node based on a value and an evaluation value when not splitting.

  (Supplementary note 2) At least the evaluation value at the best split point for each dimension and the evaluation value when not splitting are calculated for each confidence interval, and evaluation when a predetermined number of data is read into the node and split Each time a value and a non-split evaluation value are calculated, the node based on the evaluation value at the best split point for each dimension and its confidence interval and the evaluation value and its confidence interval when not splitting The tree model learning method according to supplementary note 1, wherein it is determined whether to split the node with the dimension having the best evaluation value, not to split the node, or to determine whether the number of processed data is sufficient to determine whether to split .

  (Supplementary Note 3) If it is determined that the number of processed data is not sufficient to determine whether or not to split, the evaluation value and its confidence interval when the data is further read into the node and the node is split The tree model learning method according to attachment 2, wherein the evaluation value and the confidence interval when the split is not performed are recalculated.

  (Supplementary Note 4) When it is determined that the number of processed data is not sufficient to determine whether or not to split, the reliability of evaluation values after the second is lower than the lower limit of the confidence interval of the evaluation value that was most favorable at the time of the determination. The tree model learning method according to supplementary note 3, wherein a dimension having a small upper limit of an interval is excluded from an evaluation value using newly read data and an object of recalculation of the confidence interval.

  (Supplementary note 5) The tree model learning method according to any one of supplementary note 1 to supplementary note 4, wherein the processing for calculating at least the evaluation value in the case of splitting is operated in parallel.

  (Supplementary note 6) Split evaluation value calculation processing and split evaluation value calculation processing for calculating an evaluation value when splitting a specified node at each split point of the dimension using a specified dimension of an explanatory variable in a computer Split point search processing that searches for the best split point for splitting the specified node for each dimension of the explanatory variable based on the evaluation value obtained by, and evaluation without split for calculating the evaluation value when the node is not split Based on the value calculation process and the evaluation value at the best split point for each dimension and the evaluation value when not splitting, determine whether to split the node at the dimension with the best evaluation value or not to split the node Split decision processing is executed, and split evaluation value calculation processing There are two types of terms that are in a trade-off relationship with growth, depending on the number of passing data or the number of dropped data for branching nodes and leaf nodes, respectively, which represent the fitting to the data with good evaluation values. The evaluation value is calculated using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value, and there is a trade-off relationship with the growth of the tree in the evaluation value calculation process without splitting. It includes at least two types of terms: a term that represents fitting to data with good evaluation values, and a penalty term that acts on the leaf node in a direction that degrades the evaluation value according to the number of data that falls. A tree model learning program for calculating evaluation values using evaluation functions.

  (Supplementary note 7) The computer calculates the confidence interval of the evaluation value calculated by the split evaluation value confidence interval calculation processing for calculating the confidence interval of the evaluation value calculated by the split evaluation value calculation processing and the evaluation value calculation processing without the split. Each time an evaluation value when a split is performed and a predetermined number of data is read into the node and split is calculated and an evaluation value when the split is not performed is calculated in the split determination process. Based on the evaluation value and the confidence interval at the best split point for each dimension, and the evaluation value and its confidence interval when not splitting, the node is split at the dimension with the best evaluation value, or the node is Whether to split or the number of processed data is not enough to decide whether to split Tree model learning program according to Note 6 for determination.

  (Supplementary note 8) When it is determined that the number of data processed in the split determination process is not sufficient for the computer to determine whether or not to split, the data is further read into the node and the node is split The tree model learning program according to appendix 7, wherein the evaluation value and the confidence interval thereof are recalculated, and the evaluation value and the confidence interval when not splitting are recalculated.

  (Additional remark 9) When it is determined that the number of data processed in the split determination process is not sufficient for the computer to determine whether or not to split, the lower limit of the confidence interval of the evaluation value that is the best at the time of determination, Supplementary note 8 that executes a calculation-unnecessary dimension deletion process that excludes the evaluation value using newly read data and the recalculation target of the dimension having a small upper limit of the confidence interval of the evaluation value after No. 2 Tree model learning program.

  (Additional remark 10) The tree model learning program in any one of additional remark 6 to additional remark 9 which makes a computer operate | move at least split evaluation value calculation processing about each dimension in parallel.

  The present invention is not limited to regression trees and decision trees, and can be suitably applied to the use of obtaining a model of a tree that represents a division of data space with a tree.

DESCRIPTION OF SYMBOLS 10 Tree model learning apparatus 101 Data storage part 102 Tree model storage part 103 Split point calculation means 1031 Binning means 1032 Split evaluation value calculation means 1033 Split evaluation value confidence interval calculation means 1034 Split point search means 104 Split no calculation means 1041 Evaluation without split Value calculation unit 1042 Evaluation value confidence interval calculation unit without split 105 Binning point calculation unit 106 Split determination unit 107 Tree node generation unit 108 Calculation unnecessary dimension deletion unit 20 Prediction unit

Claims (9)

A split evaluation value calculation means for calculating an evaluation value when the specified node is split at each split point of the dimension using the specified dimension of the explanatory variable;
Split point search means for searching for the best split point for splitting the node designated for each dimension of the explanatory variable based on the evaluation value obtained by the split evaluation value calculation means;
An evaluation value calculation means without split for calculating an evaluation value when the node is not split;
Based on the evaluation value at the best split point for each dimension and the evaluation value when not splitting, it is determined whether to split the node at the dimension with the best evaluation value or not to split the node. Split determination means,
The split evaluation value calculation means includes two types of terms that are in a trade-off relationship with respect to the growth of the tree, and a term that represents the fitting to the data with good evaluation values, a branch node, and a leaf node, respectively. The evaluation value is calculated using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value according to the number of passing data or the number of falling data,
The evaluation value calculation means without splitting is two types of terms that have a trade-off relationship with respect to the growth of the tree, the term representing the fitting to the data by the goodness of the evaluation value, and the number of data dropped with respect to the leaf node A tree model learning device characterized in that an evaluation value is calculated using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value in accordance with. A split evaluation value confidence interval calculation means for calculating a confidence interval of the evaluation value calculated by the split evaluation value calculation means;
A non-split evaluation value confidence interval calculation means for calculating a confidence interval of the evaluation value calculated by the non-split evaluation value calculation means,
The split determination means reads the best split for each dimension of the explanatory variable every time a predetermined number of data is read into the node and the evaluation value when splitting and the evaluation value when not splitting are calculated. Based on the evaluation values at points and their confidence intervals and the evaluation values when not splitting and their confidence intervals, the node is split at the dimension with the best evaluation value, or the node is not split, or The tree model learning device according to claim 1, wherein it is determined whether the number of processed data is not sufficient to determine whether to split or not. If the split determination means determines that the number of processed data is not sufficient to determine whether or not to split, an evaluation value when further reading data into the node and splitting the node The tree model learning apparatus according to claim 2, wherein the tree interval learning section recalculates the confidence interval and the evaluation value when the split is not performed and the confidence interval. When it is determined that the number of data processed by the split determination means is not sufficient to determine whether or not to split, the evaluation values for the second and subsequent evaluation values are lower than the lower limit of the confidence interval of the evaluation value that was the best at the time of the determination. The tree according to claim 3, further comprising: a calculation unnecessary dimension deleting unit that performs processing for excluding an evaluation value using newly read data and a recalculation target of the dimension having a small upper limit of the confidence interval. Model learning device. The tree model learning device according to any one of claims 1 to 4, wherein at least evaluation value calculation processing by the split evaluation value calculation means for each dimension is operated in parallel. In order from the root node,
For each dimension of the explanatory variable, the evaluation value when the node is split at each split point of the dimension is two types of terms that are in a trade-off relationship with the growth of the tree. Using an evaluation function that includes at least two types of terms, that is, a term expressed by a good evaluation value and a penalty term that works in the direction of worsening the evaluation value according to the number of passing data or the number of falling data for branch nodes and leaf nodes, respectively. Calculate
Based on the obtained evaluation value, the best split point for splitting the node for each dimension of the explanatory variable is searched,
The evaluation value in the case where the node is not split is two types of terms that are in a trade-off relationship with the growth of the tree, and the terms representing the fitting to the data with the good evaluation value and the leaf node are dropped. Calculate using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value according to the number of data,
Based on the evaluation value at the best split point for each dimension and the evaluation value when not splitting, it is determined whether to split the node at the dimension with the best evaluation value or not to split the node. A tree model learning method characterized by that. Calculating a confidence interval for each of the evaluation value at the best split point for each dimension and the evaluation value when the split is not performed;
Each time a predetermined number of data is read into the node and the evaluation value when splitting and the evaluation value when not splitting are calculated, the evaluation value at the best split point for each dimension and Based on the confidence interval, the evaluation value when the split is not performed, and the confidence interval, the node is split at the dimension with the best evaluation value, or the node is not split or the number of processed data is split. The tree model learning method according to claim 6, wherein it is determined whether or not it is sufficient to determine whether or not. On the computer,
Split evaluation value calculation processing for calculating an evaluation value when the specified node is split at each split point of the dimension using the specified dimension of the explanatory variable,
Split point search processing for searching for the best split point for splitting the node specified for each dimension of the explanatory variable based on the evaluation value obtained by the split evaluation value calculation processing,
The split node evaluation value calculation process for calculating the evaluation value when the node is not split, and the evaluation value at the best split point for each dimension and the evaluation value when the node is not split, Execute split determination processing for determining whether to split in a dimension with a good evaluation value or not to split the node,
In the split evaluation value calculation process, there are two types of terms that are in a trade-off relationship with respect to the growth of the tree, and the terms representing the fitting to the data with good evaluation values, and the branch node and the leaf node, respectively The evaluation value is calculated by using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value depending on the number of passing data or the number of falling data,
In the evaluation value calculation process without splitting, there are two types of terms that are in a trade-off relationship with respect to the growth of the tree, the term representing the fitting to the data with the goodness of the evaluation value, and the number of data that falls regarding the leaf node A tree model learning program for calculating an evaluation value by using an evaluation function including at least two types of penalty terms that work in the direction of worsening the evaluation value according to. In the computer,
Split evaluation value confidence interval calculation processing for calculating an evaluation value confidence interval calculated by the split evaluation value calculation processing, and no split evaluation value for calculating an evaluation value confidence interval calculated by the non-split evaluation value calculation processing Run the confidence interval calculation process,
In the split determination process, every time a predetermined number of data is read into the node and the evaluation value when the split is performed and the evaluation value when the split is not performed are calculated, the best split point for each dimension is calculated. The node is split at the dimension with the best evaluation value, or the node is not split or processed based on the evaluation value and the confidence interval thereof and the evaluation value when the split is not performed and the confidence interval. The tree model learning program according to claim 8, wherein the tree model learning program is configured to determine whether the number of data obtained is not sufficient to determine whether to split or not.

Images