KR100727555B1 - Creating method for decision tree using time-weighted entropy and recording medium thereof - Google Patents

Abstract

The present invention relates to a method for generating a decision tree using time weighted entropy that can better represent the trend of recent data in consideration of time, and a recording medium recording the same. A decision tree generation method among data mining classification techniques for generating data, comprising: (a) testing to select an attribute to be a node of a current level among attributes of a plurality of case data using time weight entropy; (B) selecting an attribute having the largest amount of information obtained in the test as a node, (c) repeating steps (a) to (b) until all nodes of the current level are selected; (d) If all nodes of the current level are selected, repeat steps (a) through (c) for one level lower level. A step of.

By using the decision tree generation method using the time-weighted entropy as described above and the recording medium recording the same, the influence of the recent data can be seen even if the entire data is taken into consideration because the recent data is more focused.

Data Mining, Decision Trees, ID3, Entropy

Description

CREATING METHOD FOR DECISION TREE USING TIME-WEIGHTED ENTROPY AND RECORDING MEDIUM THEREOF}

1 is a diagram illustrating a DM shipment collection data table and extracted rules according to a conventional decision tree;

2 is a diagram showing weights for respective cases;

3 is a flowchart illustrating a method for generating a decision tree using time weighted entropy according to the present invention;

4 is a diagram showing a data table for a comparative experiment between a conventional ID3 algorithm and a decision tree generation method according to the present invention;

FIG. 5 shows an example of a decision tree generated according to the table shown in FIG. 4; FIG.

The present invention relates to a decision tree among classification techniques of data mining, and more particularly, to a method for generating a decision tree using time weighted entropy that can better express trends of recent data in consideration of time, and a recording medium recording the same. will be.

In the ubiquitous environment, data related to user profile information should be stored in a database, and user profile information data stored in order to provide various services to the user should be analyzed to analyze user trends. . However, in a real ubiquitous environment, the user's tendency often changes over time. For example, user A has done A10 work in A1 situation three months ago, but recently does A10 work in A1 situation or A10 work in B1 situation. As such, when analyzing user profile data, more emphasis should be placed on recent data than on past data.

In general, data mining is a field of knowledge exploration that effectively finds hidden knowledge that can be useful among a large number of data. In other words, it explores and analyzes the types and relationships between the data hidden in the Data Warehouse. Data mining techniques can be used to create associations in which one event correlates with another, to recognize sequences, patterns that cause one event later, and to generate newly classified data. Classification, clustering to discover and visualize previously unknown fact groups, and predictions to predict the future through simple patterns found in the data.

The classification methods commonly used in data mining include Bayesian Theorem, K Nearest Neighbors, Genetic Algorithm, Neural Network, Rule-Based Algorithm, Decision Tree, etc. There is this. Among them, decision trees, whose results are organized in a simple tree format, are widely used for data mining because they can be easily understood and explained by people. In addition, decision trees are the most used method for prediction and classification. In these predictions and classifications, in some cases, only 'how well do we classify or predict?' In other words, a direct mailer is more interested in how well a group can respond to its own mail than how the DM model is organized. . In other cases, however, it may be important to explain 'why did we make this decision?' In this case, the decision tree is usefully used. For example, when the card applicant refuses to issue a card, a decision tree is used to explain the result of the rejection.

The decision tree is a method used to divide the data set into subsets and to identify the characteristics of the partitioned set in order to identify the relationship between the attributes constituting the data and the class. . In other words, it is a tree-type taxonomy that analyzes records of data collected in the past, and represents a pattern existing among these classes as a combination of attributes. The classification model thus created is used to classify new records and to predict properties of the class. This decision tree is similar to a twenty-four game. The first question decides what to do next, and if the question is well selected, the incoming records can be sorted well in a short time. The decision tree is a method of constructing a tree by using recursive partitioning. The root node located at the top of the tree, internal nodes including attribute separation criteria, nodes and It consists of links that connect nodes, and leaf nodes, which means final classification.

One of the forms forming the structure of the crystal tree may be a binary tree structure. This structure is a way for each node to go to the leaf node by creating two child nodes and answering a Yes-No-Query. Not just binary tree shapes, but mixed models. The constructed model measures the effectiveness of the model by applying test data and looking at the prediction rate. At this time, there may be a more effective path among several paths, and in this case, a pruning method that removes ineffective branches should be applied. Decision trees begin by finding the criteria for separation as the separation variables that best separate the data. Then, at the next node, the criteria for separation are found again and the tree is formed until it can no longer be separated.

As an example of the decision tree generation, the DM transmission will be described with reference to FIG. 1.

FIG. 1 is a diagram illustrating a DM shipment collection data table and an extracted rule according to a conventional decision tree. FIG. 1A is an example of collected DM transmission data, and FIG. 1B illustrates a generated classification rule in a tree format.

As shown in Figure 1a, the DM shipment aims to predict the customers that will show a positive response. First, the customer's response to the carrier's question is classified as "Yes" and those who answered "No" according to their characteristics. The data consists of 'responses' corresponding to three attributes and characteristics: 'occupation', 'gender' and 'age', with a total of 10 customers, of which 3 are 'no', 7 People replied yes. The decision tree measures the degree to which each attribute affects the classification of customers, and then selects the most influential attribute among them and assigns it as the root node. In this example, the attribute 'Employment' is designated as the root node, and customers are divided into three branches, 'Office worker', 'Self-employment' and 'Unemployment', depending on the value of the attribute. Here you can find the first rule that says "Yes" regardless of "gender" and "age" if the customer's job is "self-employed." On the other hand, among the total 10 customers, the number of customers belonging to the branch whose occupation is 'office worker' is 5, of which 2 said no and 3 said yes. The decision tree expands the tree to continue to classify these five customers, finding the most influential attribute 'age' and designating it as an internal node. The standard for separation is set at 35 years. In particular, two customers who are under the age of 35 responded with a no answer. Based on this, a second rule is derived that the job is an 'employee' and a customer who is under the age of '35' replies with no. As a result of expanding the tree in this way, a total of four classification rules were created. As shown in FIG. 1B, judging how the DM responds to a 33-year-old male customer who is now an office worker, it can be predicted that the answer is no according to the above rules. Therefore, it would be advantageous to exclude these customers from the list of direct mailers in terms of cost and effort savings.

These decision tree algorithms include ID3 (Interactive Dichotomizer 3), ID5R, C4.5, Classification and Regression Trees (CART), Scalable PaRallelizable INduction of decision Trees (SPRINT), and Chi-squared Automatic Interaction Detection (CHAID). The most representative of these is the ID3 algorithm proposed by Quinlan in 1986.

The ID3 algorithm uses information gain as an attribute selection measure to select an optimal inspection attribute that is a partitioning criterion in the decision tree. This measure uses the concept of entropy in the theory of information, where entropy values are small. Therefore, the attribute having the smallest entropy value among the inspection attributes has the largest amount of information acquisition, so the attribute is selected. The ID3 approach to pattern recognition and classification is an efficient discrimination tree generation process for classifying patterns with nonnumeric attributes or feature values. Because an identity tree can be expressed as a collection of rules, it can be thought of as inductive reasoning for machine learning or rule acquisition. ID3 can be very efficient under some conditions, but should not be used beyond its usefulness. ID3 can be effectively used when the number of patterns is large and each pattern is composed of a long non-numeric attribute value.

As an example of a technique for selecting an optimal decision tree for data mining, Korean Patent Registration No. 10-0497211 (registered on June 15, 2005; an apparatus and method for selecting an optimal decision tree for data mining) ) Is disclosed.

The technique disclosed in the above publication is a probability calculation means for calculating the probability of each of the multi-dimensional data for a given tree using the input multidimensional data and nested decision trees (trees), and a prior probability calculation means for calculating a prior probability for the tree. A post probability calculation means for calculating a post probability according to Bayesian Theorem using the probability of each of the multi-dimensional data of the tree and the prior probability of a given tree; It consists of a decision tree selection means that selects a decision tree to obtain a unified optimal decision tree, and uses posterior probabilities to define a new quantity called Tree Information Criteria (TIC), and then makes an optimal decision based on TIC values. Select trees to build decision trees only once, dramatically speeding up computation An apparatus and method for selecting optimal decision trees for improving data mining are disclosed.

In addition, an example of a decision tree technique used as a statistical classification in data mining is disclosed in Korean Patent Registration Publication No. 10-0498651 (registered on June 22, 2005; classification of decision making tree in data mining, that is, a pure classification of nodes of interest). Method of statistical classification of data.

The technique disclosed in the above publication is based on min (PL 0PL 1, PR 0PR 1) or min (PL 1, PL 0, PR 1, in forming a classification decision tree among decision trees (trees) which is a technique of data mining. As a splitting method based on PR 0) or max (PL 1, PL 0, PR 1, PR 0), an independent variable (x) and its threshold are selected as classification criteria, and the selected classification criteria As a result, the child nodes are classified, the child node is judged whether the classification process is terminated for all the child nodes, and the child node is determined to have not been terminated. As a categorical variable, each observation has an unbalanced tree structure by predicting the class by the decision tree, but because of this it is more explanatory and faster and more concise to find the desired subset. Disclosure of Terminology's Classification Decision trees have been described for statistical classification of data through small variance, that is, pure node of interest classification.

However, when analyzing data that tends to change over time in an ubiquitous environment, there is a problem that the existing ID3 algorithm cannot properly represent time elements.

In addition, the conventional ID3 algorithm has a problem that it is difficult to know the trend of the latest data when processing data that changes over time. In other words, the existing algorithm is difficult to find out which data is the latest in finding the latest data.

An object of the present invention is to solve the problems described above, and to provide a method for generating a decision tree using time-weighted entropy, which can easily find out the latest trends even with all the data, and a recording medium recording the same.

Another object of the present invention is to provide a decision tree generation method using time-weighted entropy capable of predicting user behavior or recognizing propensity by analyzing user preference data in a ubiquitous environment, and a recording medium recording the same.

In order to achieve the above object, a decision tree generation method using temporal weight entropy according to the present invention is performed by a computer to recognize a pattern of a plurality of case data input from the outside and to determine among classification techniques of data mining for generating newly classified data. A method of generating a tree, the method comprising: (a) performing a test for selecting an attribute to be a node of a current level among attributes of a plurality of case data using time weight entropy, (b) obtaining information obtained in the test Selecting the largest attribute as a node, (c) repeating steps (a) to (b) until all nodes of the current level are selected, and (d) if all nodes of the current level are selected Repeating steps (a) to (c) for one step lower level, wherein steps (a) to (b) The tree is generated by determining all lower case data group is unified into one class is generated until the system entropy is zero, characterized by having the time-weighted entropy is greater the more recent trends in weight.

In addition, in the method for generating a decision tree using time weighted entropy according to the present invention, the time weighted entropy is

의 실행에 의해 연산되며, 상기 S는 사례 데이터들의 집합, 상기 T_c는 사례 데이터들이 속하는 클래스의 총 개수, 상기 i는 사례 데이터의 순서를 나타내는 임의의 변수, 상기 W(i)는 i번째 사례 데이터의 최근 경향을 표시하는 가중치, 상기 c는 순서를 나타내는 임의의 변수인 것을 특징으로 하는 시간 가중치 엔트로피를 이용한 결정 트리 생성 방법.

Computed by execution of S, wherein S is a set of case data, T _c is the total number of classes to which the case data belongs, i is any variable representing the order of the case data, and W (i) is the i th case A weight indicating the latest trend of the data, wherein c is an arbitrary variable indicating an order, wherein the decision tree generation method using temporal weight entropy.

In the method for generating a decision tree using time weighted entropy according to the present invention, W (i) is

의 실행에 의해 연산되며, 상기 n은 사례 데이터들의 개수, 상기 i는 사례 데이터의 순서를 나타내는 임의의 변수, 상기 θ는 상기 n을 X축으로 하고 사례 데이터의 신뢰도를 Y축으로 하는 직선비례그래프에서 각 사례 데이터의 가중치를 나타내는 각도이고, 상기

의 범위는 -2/n보다 크거나 같고 2/n보다 작거나 같은 것을 특징으로 한다.

Where n is the number of case data, i is any variable representing the order of case data, and θ is a linear proportional graph with n as the X axis and reliability of the case data as the Y axis. Is an angle representing the weight of each case data,

The range of is greater than or equal to -2 / n and less than or equal to 2 / n.

In addition, in the decision tree generation method using time weight entropy according to the present invention, if the entropy of a specific node is smaller than a predetermined reference value during generation of the tree, the case data belonging to the node is replaced with the class to which the most data belongs. It features.

In addition, in order to achieve the above object, the computer-readable recording medium according to the present invention is determined by a classification method of data mining for the computer to recognize a pattern of a plurality of case data input from the outside and to create newly classified data. A computer-readable recording medium of a tree generation method, comprising: (a) testing to select an attribute to be a node of a current level among attributes of a plurality of case data using time weight entropy, (b (C) repeating the steps (a) to (b) until all nodes of the current level are selected; A program for executing the steps of repeating steps (a) to (c) for one level lower level if all nodes of the current level are selected The decision tree generated by steps (a) through (b) is generated until all subcase data groups are unified into one class and the system entropy is zero, and the time weight entropy is The tendency of to have a greater weight.

Hereinafter, the most preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily practice the present invention. . In addition, in description of this invention, the same code | symbol is attached | subjected to the same part and the repeated description is abbreviate | omitted.

First, we introduce the entropy formula defined in the ID3 algorithm. Entropy is defined as

In Equation 1, S is a set of cases, T _c is the total number of classes belonging to the case, P _c is the ratio of the cases belonging to class c among S. That is, if there are n cases in S and n _{c of} them, then P _c = n _c / n. And 0log ₂ 0 = 0.

If one of S belongs to c, P _c = 1 / n, and if two belong to 2 / n, the effect of each instance of S on the entropy of S is proportional to 1 / n. That is, if the weight indicating the latest trend of the i-th case is expressed as W (i) and W (i) = 1 for all i, P _c may be rewritten as in Equation 2 below. In Equation 2, c (i) is the class to which the i th instance belongs.

2 is a diagram illustrating weights for respective cases.

Figure 2a is represented by one square area as the weight for each case in the ID3 algorithm. For example, in FIG. 2A the portion filled with comb marks represents the weight of the first instance. In FIG. 2A, the weight values of all data are all 1. If n is considered to be time, the larger n means that the data is more recent. Therefore, the corresponding area must also be large according to n. The ID3 algorithm cannot express the effect on time because the area, which is a weight value of all data, is the same.

In general definition of this problem, assuming that the influence of the recent data should be greater, the weighting value should increase according to n, and assuming that the influence of the old data should be greater, the weighting value should decrease according to n. A diagram in which the weight value can be determined using this generalized definition is shown in FIG. 2B.

로 결정된다. 2B is a diagram illustrating weights for respective cases. In FIG. 2B, the Y-axis is the reliability of each case, and the case with high reliability has a large weight value. The sum of the OABC areas shown in FIG. 2B and the n square areas shown in FIG. 2A is equal. That is, in FIG. 2B, the sum of the weights of the n cases and the weight of the n cases in the commonly used entropy are defined to be the same. The new weight straight line AB

Is determined.

In Equation 3, a weight value of the i th case was used. The weight W (i) of the i-th data is obtained as follows.

범위는

=2/n일 때 n번째 사례의 가중치 값은 가장 크며 최근 데이터 n의 영향이 최대가 되고,

=0일 때 가중치 값이 1이 되면서 ID3 알고리즘이 되고,

=-2/n일 때 n번째 사례의 가중치 값은 가장 작으며 최근 데이터 n의 영향이 최소가 된다. Θ shown in [Equation 3] is an angle defined to give each data a weighted value. As shown in FIG. 2B, each case is a linear proportional graph in which n is the X axis and the reliability of the case data is the Y axis. Indicates the weight of the data.

The range is

When = 2 / n, the weight of the nth case is the largest and the influence of the latest data n is the greatest,

When = 0, the weight is 1 and becomes the ID3 algorithm.

When = -2 / n, the weight of the nth case is the smallest and the influence of the latest data n is minimal.

The formula for calculating entropy using this new weight is as follows.

Entropy in Equation 4 is defined as time-weighted entropy. In the present invention, the decision tree is constructed using the ID3 algorithm and time weight entropy.

Next, a decision tree generation method using time weighted entropy according to an embodiment of the present invention will be described with reference to FIG. 3.

3 is a flowchart illustrating a decision tree generation method using time weight entropy according to the present invention.

Decision trees are created starting at the root node and decremented one level down. Since the classification technique of data mining generally classifies a plurality of case data input from the outside using a computer, it will be described assuming that a computer is used even in an embodiment of the present invention. First, a test is performed to select an attribute to be a node of the current level among attributes of a plurality of case data using time weight entropy. That is, a test for selecting a node is started (ST 2010). An initial value of entropy is calculated using the time weight entropy (ST 2020). Next, after deciding one attribute among several attributes, it is assumed as a node to select (ST 2030). Next, the entropy of each branch of the assumed node selected by using the time weight entropy is calculated (ST 2031). The system entropy is calculated using the calculated entropy of each branch (ST 2032). The information acquisition amount, which is the value obtained by subtracting the system entropy from the initial value of entropy, is calculated (ST 2033). In this case, the smaller the system entropy, the greater the amount of information acquisition. Steps ST 2030 to ST 2033 are repeated for all the attributes (ST 2034). When the tests for all attributes that may be assumed to be nodes are completed, the information acquisition amount of each attribute is compared to select an attribute having the largest information acquisition amount as a node (ST 2035). Currently, since it is the first level, the node selected in step ST 2035 becomes the root node.

If the root node is selected, the level must be decreased by one level to select nodes of level-1 (ST 2060). The process of selecting each node of the reduced level by one step is the same as that of the steps ST 2010 to ST 2035 in which the root node is selected, so a detailed description thereof is omitted. When the selection of all nodes of the current level is completed (ST 2040), the level must be reduced by one step again to select the nodes of level-2. This process is repeated until tree generation is completed (ST 2050). That is, all subgroups are unified into one class and repeated until the system entropy becomes zero.

If the entropy of a specific node is smaller than a predetermined reference value during tree generation, the data belonging to the node is replaced with the class that contains the most data.

The present invention demonstrates through the experiment according to Figs. 4 to 5 that the trend of recent data better than the conventional ID3 algorithm.

4 is a diagram showing a data table for a comparative experiment between a conventional ID3 algorithm and a decision tree generation method according to the present invention, and FIG. 5 is a diagram showing an example of a decision tree generated according to the table shown in FIG. .

The tennis data table shown in FIG. 4 is data for knowing whether or not the user played tennis according to 'outlook', 'temperature', 'humidity', and 'windy'. Since these user-related data change over time, we experimented with adding new data that has a different trend from the existing data. Data 1 to 14 of FIG. 4 are existing data, and data 15 to 21 are newly added data. And in order to express the change in the tendency of the user to play tennis, data 15-21 were added to have a different tendency than before. That is, the data 1 to 14 show the same trend as that of FIG. 5A, but the data 8 to 21 are overlapped with the data 15 to 21 in order to be as shown in FIG. 5B. This is because the user's tendency is expected to change slowly.

In order to properly learn these changing user trends, you should be interested in the current data rather than the historical data that already shows other trends. The hard part, however, is how to figure out which data shows the new trend. For example, suppose that data 8-21 show new trends, and if you apply the ID3 algorithm to it, you can freeze the decision tree that reflects the changed trends. However, if all the data are equally important and the decision tree is obtained by applying the ID3 algorithm to the data from data 1 to 21, a complex tree is obtained as shown in FIG. 5D. You get this complex tree because you have a mix of old and new trends. In addition, since the latest data reflects a new trend, it is impossible to freeze an appropriate nodule tree even if only excessive data is considered. For example, when a decision tree for data 15 to 21 is obtained, an overly simple one is obtained as shown in FIG. 5C. It is not easy to create a decision tree that reflects trends in this changing data.

To this end, the present invention proposes a method for generating a decision tree using time weight entropy and ID3 algorithm. In other words, the new entropy is used instead of the existing entropy in the process of generating the decision tree using the ID3 algorithm. And if the entropy of a specific node is smaller than the existing value during the tree creation, the method that replaces the class with the most data belongs to the node. When the method is applied to the data shown in FIG. 4, as shown in FIG. 5E, a decision tree applying a data time weight entropy algorithm to data 1 to 21 is shown. As a result of recent data focus, 'windy' has been selected as the root node. If we look at the case of 'windy = weak', we can see that the past two cases, data 1 and 8, are 'no' and the rest including the latest data is 'yes'. In this case, entropy results in '0.095' because of the low weight of historical data. Since this is less than the predetermined standard value '0.1', it is replaced with 'yes' which is the most frequently appeared class.

As can be seen from this experiment, the tree structure constructed using the time weighted entropy formula proposed in the present invention is simpler and more advantageous than the tree composed of the conventional ID3 algorithm. The reason is that, unlike the ID3 algorithm, it is difficult to find a recent trend. Therefore, the structure proposed by the present invention focuses on the recent data. Comparing the experimental results, the present invention using the time weight than the conventional algorithm generated a simpler and more effective decision tree, and showed a 3-6% higher prediction rate for the test data.

The decision tree generation method using the time-weighted entropy and the recording medium recording the same may predict the user's behavior by inferring and learning the user's behavior with the user's preference data in an ubiquitous environment.

As mentioned above, although the invention made by this inventor was demonstrated concretely according to the said Example, this invention is not limited to the said Example, Of course, it can be variously changed in the range which does not deviate from the summary.

As described above, according to the method for generating a decision tree using time weighted entropy and a recording medium recording the same, according to the present invention, the effect of recent data can be seen even if all data are considered, since the data is more focused on recent data. Obtained.

In addition, according to the method for generating a decision tree using time-weighted entropy and a recording medium recording the same, an effect of generating a decision tree simpler and more effective than a tree composed of the existing ID3 algorithm is obtained.

In addition, according to the method for generating a decision tree using time-weighted entropy and a recording medium recording the same, an effect of predicting a user's behavior or recognizing a user's behavior can be obtained by analyzing user's preference data in a ubiquitous environment.

Claims (5)

In the decision tree generation method of the classification method of data mining for the computer to recognize the pattern of a plurality of case data input from the outside and to create the newly classified data, (a) calculating an initial value of entropy using time weighted entropy, (b) setting one attribute among attributes of the plurality of case data and assuming the node to be selected; (c) calculating system entropy for the hypothesized node using time weighted entropy, (d) calculating an information acquisition amount using the initial value of the entropy and the system entropy, (e) selecting, as nodes of the current level, an attribute having the largest amount of each information acquisition calculated among the attributes; (f) repeating steps (a) to (e) until all nodes of the current level are selected, (g) repeating steps (a) to (f) for one level lower level if all nodes of the current level are selected, The decision tree generated by steps (a) through (e) is generated until all subcase data populations are unified into one class and system entropy becomes zero, The temporal weight entropy has a greater weight as the latest trend has a greater weight. The method of claim 1, The time weight entropy is

Is computed by the execution of S is a set of case data, T _c is the total number of classes to which the case data belongs, i is any variable representing the order of the case data, and W (i) represents the latest trend of the i th case data. The weight, wherein c is an arbitrary variable representing the order, decision tree generation method using a time weight entropy. The method of claim 2, Where W (i) is

Is computed by the execution of N is the number of case data, i is an arbitrary variable representing the order of the case data, and θ is the weight of each case data in a linear proportional graph where n is the X axis and reliability of the case data is the Y axis. Is an angle to represent, remind

The range of is greater than or equal to -2 / n and less than or equal to 2 / n decision tree generation method using time-weighted entropy. The method of claim 1, If the entropy of a particular node is smaller than a predetermined reference value during tree generation, the decision tree generation method using time-weighted entropy is replaced with the class to which the case data belonging to the node belongs most. In a recording medium that can read a decision tree generation method of the data mining classification techniques for recognizing a pattern of a plurality of case data input from the outside and to create a newly classified data, (a) calculating an initial value of entropy using time weight entropy, (b) setting one attribute among attributes of the plurality of case data and assuming the node to be selected; (c) calculating system entropy for the hypothesized node using time weighted entropy, (d) calculating an information acquisition amount using the initial value of the entropy and the system entropy, (e) selecting, as nodes of the current level, an attribute having the largest amount of each information acquisition calculated among the attributes; (f) repeating steps (a) to (e) until all nodes of the current level are selected, (g) if all nodes of the current level are selected, record a program for executing the steps of repeating steps (a) to (f) for one level lower level, The decision tree generated by steps (a) through (e) is generated until all subcase data populations are unified into one class and system entropy becomes zero, The time-weighted entropy has a greater weight as the recent trend has increased.

Images