JPH08161172A - Knowledge correction type learning system - Google Patents

Abstract

PURPOSE: To provide a knowledge correction type learning system capable of acquiring excellent knowledge from a small number of cases by using existing knowledge (a rule, a determination tree, etc.) together. CONSTITUTION: An existing knowledge conversion means 11 generates a virtual case 23 by converting existing knowledge 21 into a unit rule expression constituted only of the product of conditions, the virtual case 23 and a real case 22 are inputted to a weight changing means 12 and a learning case 24 is generated by respectively applying weight to the virtual case 23 and the real case 22. The case 24 is inputted to an inductive learning means 13 to generate corrected knowledge 25 expressed by a rule or a determination tree, the weight of the means 12 is changed by a parameter determining means 14 and a parameter capable of applying the highest discrimination performance to an unknown case is selected by a cross validation method.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a knowledge correction type learning system for acquiring knowledge from observed cases by using existing knowledge together, and more specifically, knowledge for creating rules / decision trees by concept learning from cases. A modified learning system.

[0002]

2. Description of the Related Art A method of learning a rule / decision tree for discriminating a future unknown case class from past observation cases is called "concept learning", and various methods have been proposed. Among them, Quinlan's "ID3" is the most representative one. ID3 has a great feature in that its learning time is almost proportional to the number of cases (learning cases) used for learning, and is known as a highly practical method because of its high speed. Hereinafter, a brief description of ID3 will be given. For details of ID3, refer to Kyoritsu Publishing, "Knowledge Acquisition and Learning Series Vol.

In ID3, for example, the following case set is used as a case. Here, the case [low, blond, blue;
“Low”, “Blonde”, and “Blue” that compose +]
Is the attribute value. In this example, "height""haircolor"
There are three attributes, "eye color". "-" And "+" are classes determined by these attributes. These eight cases are
This is referred to as “case set C”.

[0004] Now, given the case set C above, there is a new case, [Low, Red, Brown ;? ], What should we presume in this class? The purpose of concept learning is to generate a rule (decision tree) for discriminating a class of such an unknown case from a known learning case (above case set C).

In ID3, the rule for determining the class is
It is expressed as a decision tree as shown in FIG. The meaning of this decision tree is to first examine unknown cases by "hair color". As a result, if the hair color is "black", the class is immediately determined to be "-". If the hair color is “red”, the class is determined as “+”. If the hair color is "blond", further examine the case for "eye color", and if the eye color is "blue", class is "+", and if it is brown, Let the class be "-". In this way, in the decision tree, the class determination is performed by sequentially asking the attribute values of unknown cases from the top node.

Next, a method of creating this decision tree from the case set C will be described. With ID3, first of all, regarding the case set C, a tree when it is determined by each attribute is created. FIG. 2A is an example in which determination is made based on the attribute “hair color”. The eight cases are divided into three, one, and four groups. Here, with respect to “black” and “red”, the classes of the cases are unique. But for the "blond" branch, the class is not unique. ID3 is
The quality of discrimination based on attributes is judged by the gain of entropy.

The entropy of the case set C is calculated by the following equation. The number of cases belonging to the class “+” is N
p and the number of cases belonging to the class “−” is “Nn”,

[Equation 1] Given in. However, the log has a bottom of 2.

First, before branching, the entropy of the case set C has 3 classes of 5 kinds and 5 classes of 8 cases.
Since they exist in proportion, the following is obtained.

[0009]

[Equation 2] On the other hand, for the entropy after the determination, it is necessary to calculate the entropy of each of the branch destination case sets and weight the entropy in consideration of the number of branch cases. Specifically, in this example, the hair color is “black” (3 out of 8 cases).
), “Red” (1 out of 8 cases) and “blond” (4 out of 8 cases) have entropy of 0 bit, 0 bit and 1 bit respectively. The expected entropy of

(Equation 3) Becomes Therefore, the entropy gain of "hair color" is
0.954-0.5 = 0.454 bits. on the other hand,
Although not described in detail, the entropy gain determined by "height" is 0.003 bits in the same manner,
The "eye color" is 0.347 bits.

With ID3, when creating a tree, the attribute with the maximum entropy gain is prioritized. That is, in this example, "hair color" is the first determination attribute. As a result of the determination,
When the attribute value whose class is uniquely determined is branched, the processing is stopped. On the other hand, for a branch whose class is not unique, the same entropy gain calculation is used to determine the attribute to be used for the next discrimination. Such determination of the discrimination attribute is repeated in all the branches until the class becomes unique. The case C is generated as shown in FIG.
It is the decision tree of (b). For unknown cases, this decision tree is used to determine the class.

[0011]

If a sufficient number of cases is secured, the above ID3 can be used as a knowledge acquisition support method for an expert system. However, in the actual field, there is a limit to the number of processing cases of experts that can be collected, and even if knowledge (decision tree) is generated from only available cases, the performance of that decision tree is often insufficient. . In order to avoid this problem, conventionally, it has been general to use a concept learning method and human support in combination. Specifically, first, the knowledge is learned from the case, and the learned knowledge is manually corrected. This conventional method can certainly acquire knowledge from a small number of cases, but the finally created knowledge is manually created. Therefore, problems such as contradiction between rules and omission of conditions occur, leaving a problem in quality assurance of knowledge.

In order to solve the above problems, first, a rule should be manually created, and a method for automatically learning the final rule should be used by combining this rule with cases actually collected. . As a result, more accurate rules can be acquired from the knowledge "in the human head" and the available cases. Moreover, in this method, since the final rule is machine-generated, mutual contradiction of rules and omission of conditions can be prevented.

Several learning methods capable of such modified learning have been proposed in conventional machine learning research. However, the AQ algorithm and the like are all related to the learning method having a long processing time. The major reason why the modified learning is not applied in the conventional expert system construction support seems to be the weight of this processing. If this modified learning is possible with ID3, which is a high-speed learning method that can be learned in a learning time that is approximately proportional to the number of cases, it is of high practical value. However, no existing rule modification type version is known for ID3.

The present invention has been made in view of the above,
The purpose is to provide a knowledge correction type learning system that can acquire excellent knowledge from a small number of cases by using existing knowledge (rules, decision trees) together.

[0015]

In order to achieve the above object, the knowledge correction type learning system of the present invention uses a rule or a knowledge correction type learning system in which existing knowledge is used in combination and knowledge is acquired from observed cases. Existing knowledge conversion means for generating a tentative case by converting existing knowledge represented by a decision tree into a unit rule expression consisting only of products of conditions, and the tentative case and observed actual case as inputs, A weight changing means for giving a weight to each of the provisional case and the real case to generate a learning case; and an inductive learning means for receiving the learning case as an input and generating a corrected knowledge expressed by a rule or a decision tree, By changing the weight as a parameter in the weight changing means and selecting the parameter with the best discrimination performance for unknown cases by the cross validation method. And summarized in that and a chromatography data determining means.

[0016]

In the knowledge correction type learning system of the present invention, the existing knowledge expressed by the rule or the decision tree is converted into the unit rule expression composed only of the product of the conditions to generate the tentative case. A learning case is generated by giving a weight to each of the cases, and this learning case is input to generate modified knowledge represented by a rule or a decision tree, the weight in the weight changing means is changed, and the cross validation method is used. The parameter that gives the best discrimination performance for unknown cases is selected.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a knowledge correction type learning system according to an embodiment of the present invention. The knowledge correction type learning system 1 shown in the same figure converts the existing knowledge 21 represented by a rule or a decision tree into a unit rule expression composed only of products of conditions, and thereby the provisional case 23
The existing knowledge converting means 11 for generating the learning case 24 and the weight changing means 12 for generating the learning case 24 by inputting the temporary case 23 and the actual case 22 and weighting each of the temporary case and the actual case.
And the learning case 24 as an input, the inductive learning means 13 for generating the modified knowledge 25 expressed by the rule or the decision tree, and the weights in the weight changing means 12 are changed, and the unknown case is obtained by the cross validation method. And a parameter determining means 14 for selecting a parameter having the best discrimination performance with respect to. Hereinafter, examples of the present invention will be described in detail.

The existing knowledge conversion means 11 has a function of converting existing knowledge given by a rule or a decision tree into a case (provisional case). A case is a vector composed of an attribute value and a class name, and is expressed in the following format, where i is the case number. However, V _{i, j} is the attribute value of the j-th attribute, and Class _i is the class.

[V _{i, 1} , V _{i, 2} , V _{i, 3} , V _{i, 4} ...
•; Class _i ] In the vector illustrated in the prior art, the case is expressed as, for example, [high, blond, blue; +].

Consider the following case.

[? , Blond, blue; +] This example shows that the class is "+", whatever the value of the first attribute. This is clearly the same as the rule if (hair color = blond) & (eye color = blue) then (class = +). That is, "?"(Don't C
Cases with are) are substantially the same as rules. Further, referring to FIG. 2B, this case corresponds to the path from the root of the decision tree to the leaf by selecting blonde and blue as attribute values. If the existing knowledge is converted into the case with “?” (Hereinafter, referred to as “provisional case”), the provisional case set effectively reflects the information of the existing knowledge.

In ID3, although not described in detail in this specification, even if the input includes a case with "?" As described above, there is no problem as a concept learning means. is there. Therefore, the above case is I
Can be thrown into D3. For more information on this feature, see J. Ross
Quilan, Progorams for Machine Learning, Mo
See rgan Kaufman.

From the above, the processing procedure of the existing knowledge conversion means 11 is as follows.

First, the case where the existing knowledge 21 is a decision tree will be described. In this case, (1) the following tentative case is generated for each path from the root node to the leaf node of the decision tree. The provisional case has the attribute value for the attribute that appears as a condition in the path of the decision tree, and for the other attributes,
It is a case vector having "?" As an attribute value.

(2) However, if it is desired that the generated provisional case more accurately reflect an existing decision tree, a value should be given to an attribute that appears anywhere in the decision tree. think of. Specifically, it is an attribute that appears anywhere in the decision tree, and if the attribute value does not appear in the path, the attribute value is randomly assigned.

The above (2) seems to be wasteful at first glance, but when ID3 is actually operated on the generated temporary case set, the same decision tree as the original existing knowledge is generated. In order to do so, it is a necessary process.

Next, the case where the existing knowledge 21 is in the rule format will be described. In this case, (1) If an OR condition such as (hair color = blonde or black) exists in the rule, first convert to a rule without OR. It is obvious to those skilled in the art that this conversion can be easily realized by the conventional technique. As a result,
A product expression rule composed only of product conditions ((hair color = blond) etc.) is obtained.

(2) Next, the above rule is converted into the above-mentioned case of the attribute value expression. However, in order to fully reflect the information that the rules have, values are randomly assigned to the attributes that appear anywhere in all the rules. However, in the case where this assigned random value matches another rule and results in a different class, the assigned random value is not allowed.

By the existing knowledge conversion means 11 described above,
Existing knowledge is converted, and the output is temporary case 23,
It will be stored. At this time, the existing knowledge conversion means 11
Gives each case an attribute indicating whether the case is a tentative case or a real case. For example, when distinguishing by the first attribute, a case where the first attribute is 0 is a tentative case, and a case where the first attribute is 1 is an actual case.

Next, the weight changing means 12 will be described. The weight changing unit 12 gives the above-mentioned temporary case 23 and the actual case 22 a weight instructed by the parameter determining unit 14 to be described later.

When the weight of the case is displayed with the second attribute, the case has the following structure when the case is added with the case. For example, [1, 2.5, high, blonde, blue; +] is a real case, indicating a weight of 2.5. Also, [0, 2.0, low, blonde ,? ;-] Indicates a provisional case, that is, a part of existing knowledge. Further, the weight 2 may be expressed by not actually giving a numerical value but by actually copying a case (limited to an integer) to generate two cases. The specific configuration of the weight changing unit 12 will be apparent to those skilled in the art from the above description, and thus the description thereof is omitted here. By the weight changing means 12, the provisional case and the actual case become a learning case 24.

The inductive learning means 13 is ID3. This is as described in the related art and does not have a special function as compared with the related art. However, as will be described later, the flag indicating whether it is a real case or a tentative case should be referred to when correcting the weight of the learning case, as will be described later, so the inductive learning means 13 This first attribute is irrelevant and is not input to the inductive learning means 13. The weight of the case was originally from J. Ross Quilan mentioned above.
Author, "Progorams for Machine Learning", Morgan Kauf
In man, a processing method is shown for the case of giving a weight to a case, and in fact, the program attached to this book discloses a program of a processing method to which a weight is given internally. For more details, see J. Ross Quilan's book. The decision tree generated by the inductive learning means 13 becomes the corrected knowledge 25.

The parameter determining means 14 realizes one of the most characteristic functions of the present invention. As mentioned above, according to probability statistics theory, existing knowledge corresponds to a priori probability,
A posterior probability is generated from the actual case and this a priori probability. Therefore, when the a priori probability for a class under a certain condition is P and the distribution (probability) of cases is Q, the posterior probability f in statistical analysis is f = P × α + (1-α ) Given by Q. This α (0 ≦ α ≦ 1) means the importance of the a priori probability. The larger this α is, the more reliable the prior probability should be. For example, the case includes almost no noise (case in which the attribute value or class is incorrect),
On the other hand, when one is not confident in prior knowledge, this α should be kept small. Conversely, if the case is noisy and the a priori knowledge seems to be fairly accurate, then the a priori probability P should be valued rather than the case probability Q, so α should be close to 1. is there.

If the value of α can be known in advance, it will never be exceeded. However, in reality, the value of α depends on the nature of the case and the accuracy of existing knowledge,
It is difficult to know in advance. Therefore, in the present invention, this α
A mechanism for experimentally determining the value of is provided. The details will be described below.

In the present invention, as examples corresponding to this α, the cases used for learning are weighted. That is, weight information is added to even one case. For example, if the weight is 2, the ID 3 executes the same processing as when there are actually two identical cases. There are two ways to give cases.

(1) The weight of the temporary case or the actual case is fixed, and the weights of other cases are changed.

(2) The weights of the temporary case and the actual case are changed.

Practically, it can be considered that the “provisional case weight” / (“provisional case weight” + “actual case weight”) at this time corresponds to the above α. Even if the above values are the same, the actual performance of the learning means may change depending on the absolute value of each weight. Therefore, the above (1)
Since the method of (2) alone may not sufficiently improve the performance of corrected knowledge, the method of (2) above should also be usable. However, in the following description, for simplification of description, the weight of the tentative case generated from the existing knowledge is set to 1. However, in the case where the weight of the tentative case is also changed, those skilled in the art can easily analogize the realization method.

The following description will be made assuming that the weight of the temporary case is constant and the weight of the actual case is changed. In this case, in the present invention, the weight that is the parameter of the actual case is
This parameter is defined by the "cross validation method".

The cross validation method will be described. In this cross-validation method, STEP 1: a predetermined number of cases are randomly extracted from the learning case 24 and set as a test case set.

STEP 2: The modified knowledge is created by the inductive learning means 13 from the learning case 24 excluding the test case set. Then, using this corrected knowledge, the test case set is discriminated, and the correct answer rate is calculated and recorded.

STEP3: The above STEP1 to STEP2 are repeated a predetermined number of times (for example, 10 times),
Find the average of the correct answer rates.

The method of obtaining the correct answer rate is well known to those skilled in the art, and the class can be determined by sequentially tracing the created decision tree from the root.

In the parameter determination by the cross-validation method, the processes of STEP1 to STEP3 described above are executed for the weights of different actual cases, and the weight with the highest correct answer rate is used. In the learning cases described above, the attribute that indicates whether the case is a real case or a temporary case is added to the case so that the temporary case and the real case can be distinguished from each other when the weights of the learning cases are changed one after another. This is because

The embodiments of the present invention have been described above. FIG.
Is the result of carrying out the present invention using a part of the Irvine database (data called Soybean Large) which is benchmark data of learning methods provided by California State University in the United States. However, in this example,
From 683 cases, two sets of 200 learning data are taken, one set (200) is a real case, another set is (200)
Generates a decision tree in and uses this as existing knowledge. Learning is a temporary case generated from this generated existing knowledge,
It was conducted with 200 actual cases. The cases for evaluating the correct answer rate of the decision tree generated by the learning system are 283 cases not used for learning. The reason for taking such a method is to prevent existing knowledge from being manually biased.

Further, in FIG. 3, not only the weight of the actual case but also the weight of the temporary case is changed. Although details are omitted, it is noted that the correct answer rate (87.24% (100% -12.76%)) is most improved when the actual case is given a weight slightly less than twice that of the tentative case. I want to. However, when two sets of 200 learning data are all input to ID3 (88.4
6%), the correct answer rate of the present invention (actually, the error rate is displayed. Of course, 100% to the correct answer rate
Is a little inferior). But 2
Compared to the case where only one set of 00 learning data is used, that is, the case where learning is performed only with actual cases observed without existing knowledge (81.81%), the performance of the knowledge is far superior. Have been shown to be

As described above, the knowledge correction type learning system of the present invention aims to improve the discrimination performance for unknown cases by the existing knowledge conversion means 11 by reflecting the existing knowledge as a tentative case and the cross validation method. There is a feature in that.

Further, since the provisional case is a representation of existing knowledge as a case, the existing knowledge can be used for learning. However, in this case, it is a question of which of the temporary case and the actual case should be viewed more heavily. That is, the existing knowledge corresponds to the a priori probability in the probability statistics theory, and the actual observation actual case that is the basis of the posterior probability (the final probability generated from the a priori probability and the case) The degree to which the a priori probability can be relied upon when obtaining is changed depending on the application target of the present invention. Therefore, the cross-validation method is used to determine the weights of provisional cases and actual cases to improve the discrimination performance for the most unknown cases.

[0050]

As described above, according to the present invention, it is possible to perform high-accuracy learning using existing knowledge even in a high-speed concept learning method which has been difficult to use the existing knowledge in the past. In addition, the existence of existing knowledge makes it possible to acquire knowledge that is far superior to that created from only observation cases.

Further, according to the present invention, since the final knowledge is machine-generated, errors and omission of conditions do not occur as compared with the conventional method of manually correcting the learning result from the case.
In addition, high-speed learning can be realized by making the best use of the high speed of ID3.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a knowledge correction type learning system according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining ID3.

FIG. 3 is a diagram showing a result of a cross validation method.

[Explanation of symbols]

 11 Existing Knowledge Converting Means 12 Weight Changing Means 13 Inductive Learning Means 14 Parameter Determining Means 21 Existing Knowledge 22 Actual Cases 23 Temporary Cases 24 Learning Cases 25 Modified Knowledge

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shigeo Kaneda 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation

Claims (1)

[Claims] 1. In a knowledge correction type learning system that acquires existing knowledge by using existing knowledge together, existing knowledge expressed by rules or decision trees is converted into a unit rule expression consisting of products of conditions only. Existing knowledge conversion means for generating a tentative case by converting, and a weight changing means for generating a learning case by giving a weight to each of the tentative case and the actual case with the tentative case and the observed real case as input , An inductive learning means for generating modified knowledge represented by a rule or a decision tree using the learning case as an input, and changing a weight as a parameter in the weight changing means, and discriminating performance for an unknown case by a cross validation method. A knowledge correction type learning system, comprising: a parameter determining unit that selects a parameter that maximizes.