JPH0895791A - Correction type knowledge learning device - Google Patents

Abstract

PURPOSE: To provide the correction type knowledge learning device which generates enough knowledge to explain an instance set partially including instances that do not match background knowledge on the assumption of the background knowledge. CONSTITUTION: A temporary instance and an actual instance where the background knowledge 3 given conditions for determining the classes of instances in an actual instance set 2 from attribute values in the form of a rule, decision tree, etc., and put together to form a training instance set by an instance converting means 4; and a training instance set is inputted and the same values of respective attributes are gathered to decide an attribute whose decrease in index showing the randomness of the instance sets is minimum as a divided attribute when the instance set is divided, the divided attribute is extracted recursively as a node, and the decision tree is generated by a decision tree generating means 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a knowledge learning device that learns general knowledge existing behind a problem solving case of a human expert, and in particular, combines knowledge and case created by a human. And a modified knowledge learning device for acquiring higher quality knowledge.

[0002]

2. Description of the Related Art Collecting a plurality of problem solving cases of human experts and extracting the knowledge behind them from this set of cases is actively studied as a powerful method for creating knowledge for an expert system. Is being promoted. In this specification, the following is considered as an example. That is, A ₁ ,
_{A 2, ·· A j, ··} , A n an attribute value, as the class determined by the attribute value C, _{case, (A 1, A 2,} ···· A n: C) expressed as To do. In this case, it is the responsibility of the knowledge learning device to estimate an unknown function f (A ₁ , A ₂ , ... A _n ) = C that determines the class from the attribute value.

Various methods have been proposed for constructing a knowledge learning device, but the most well-known one is ID3 by J. Ross. Quinlan. ID3 has a function of generating a decision tree as a form of the unknown function f. Hereinafter, a brief description of ID3 will be given. For details, see Kyoritsu Publishing Co., Ltd., Knowledge Acquisition and Learning Series 1,
See Chapter 5, "Introduction to Knowledge Acquisition / Inductive Learning and Applications".

First, consider the following case set S. Here, each case in S has three attributes A ₀ , A ₁ , A ₂ (provided that the attribute value is a binary value of 0 or 1), and a class C (binary value of 0 or 1). Shall consist of That is, the format of the case is (A ₀ , A ₁ , A ₂ : C).

(0,0,1: 1) (1,0,0: 0) (1,1,1: 1) (1,1,0: 1) Now, select the second attribute A ₁ . , Split this set S. In this case, the case is divided into two groups.

Group 0 (A ₁ = 0) Group 1 (A ₁ = 1) (0,0,1: 1) (1,1,1: 1) (1,0,0: 0) (1,1 , 0: 1) Note that splitting is possible in this case with three types of attributes. In ID3, the difference in entropy before and after this division (entropy gain) is obtained. Here, the entropy gain is "entropy of case before division"-"expected entropy of case after division". "Expected entropy of cases after division" is the entropy of the group of each case after division, which is the frequency of occurrence for each group (the number of cases that reached the group divided by the total number of cases before division) It is a weighted sum. Specifically, the entropy gain is calculated as follows. For this case set before division, class 1 is 3
Since there is one and there is one class 0, the entropy is

[Equation 1]-(1/4) log (1/4)-(3/4) log
(3/4) = 0.500 + 0.311 = 0.811. However, the base of the logarithm is 2.

On the other hand, the entropy after division is group 0.
(A ₁ = 0)

(2)-(1/2) log (1/2)-(1/2) log
(1/2) = 1 Group 1 (A ₁ = 1)

[Equation 3]-(2/2) log (2/2)-(0/2) log
(0/2) = 0. Therefore, the expected entropy after division is (2/4) · 1 + (2/4) · 0 = 0.5, and the entropy when the set S is divided by selecting the second attribute (A ₁ ). The gain is 0.811−0.5 = 0.331.

Next, the first attribute (A ₀ ) is selected and this set is divided. The entropy after division in this case is group 0 (A ₀ =
Since the entropy of 0) is 0,

(4) (3/4) × (-(1/3) log (1/3)-
(2/3) log (2/3)) = 0.689. Also, when the third attribute (A ₂ ) is selected and this set is divided, the result is as follows.

Group 0 (A ₂ = 0) Group 1 (A ₂ = 1) (1,0,0: 0) (0,0,1: 1) (1,1,0: 1) (1,1 , 1: 1) The entropy after division in this case is group 1 (A ₂ =
Since the entropy of 1) is 0,

(5) (2/4) × (-(1/2) log (1/2)-
(1/2) log (1/2)) = 0.5. Therefore, in this case, the second attributes A ₁ and 3
Both of the second attributes A ₂ exhibit maximum entropy gain. Here, it is assumed that the division is executed with the second attribute A ₁ . This attribute A ₁ is called a division attribute. Further, the values A ₁ = 0 and A ₁ = 1 of the division attribute are called branch attribute values.

When divided by the second attribute A ₁ , the class of group 1 is only "1", so all the classes of cases flowing to this branch are regarded as "1". Furthermore, two cases (0,0,1: 1) (1,0,0: 0) belonging to the class of group 0 are obviously divided by the third attribute A ₂ , and The class of the group will be unique. Therefore, the division by the third attribute has the largest entropy gain.

The above procedure can be expressed in a decision tree format, as shown in FIG. In ID3, the generated decision tree is used for discrimination with respect to a case of unknown class. As a result, if the class value is f, then f = A
A decision tree equivalent to the logical expression of ₁ + A ₂ is obtained.

The above is the outline of the induction learning algorithm ID3, and if described as a more accurate procedure, the following recursive procedure is performed.

(STEP 1) Let A be the attribute set of a case and S be a given case set. (STEP 2) If the case set is empty or the class to which it belongs is single, the processing is stopped. Otherwise, for the case set S, for each attribute included in the attribute set A,
The entropy gain when the division is performed by the attribute is calculated, and one attribute having the largest entropy gain is selected and designated as a. (STEP 3) The attribute a is deleted from the attribute set A, and this is set as a new A. Then, the case set S is divided into groups according to the attribute a, and the case sets for each group are set as S ₀ , S ₁ ... STEP 2 to STEP 3 are recursively executed for each of S ₀ , S ₁ ...

In the above description, the index of the randomness of the case set before and after the division is expressed by entropy, but the index is not limited to entropy. Any index may be used to represent randomness, and the index may be ordered (total order). For example, the base of the logarithm of entropy may be 10, or it may be simply the number of classes in the group minus 1. However, depending on this indicator,
The performance of the created knowledge (the ability to estimate unknown cases) is affected. The method based on entropy gain of ID3 is recognized as an index showing simple and good performance.

FIG. 3 illustrates the above procedure intuitively (this example is completely different from FIG. 2). In Figure 3, ○
A case where the cases displayed as (class 1) and x (class 0) are distributed in a multidimensional space (two dimensions in this example) having attribute values is shown. The division by the above attribute value is to divide this multidimensional space by one hyperplane. The upper diagram of FIG. 3 shows the result of division by a certain hyperplane. In this case, since there are only three × cases on the right side, the class is unique with “0”. Here, the maximum entropy gain means that when cutting with a hyperplane, the class is cut as little as possible (even if the class is not cut cleanly at that point). .

In the lower diagram of FIG. 3, the division is repeated for the left half region which is not neatly cut. As a result, the multidimensional space is divided into three regions, and the classes are completely unique in each region.

[0017]

The above ID3 is known as an excellent method, and an implementation example in an expert system construction support tool has been reported. However, in the actual application, it is not heard that it was used much. The biggest reason is that in reality, cases are not collected enough to generate knowledge (decision tree) with this ID3. In other words, there are too few cases to create knowledge (decision tree) that is sufficiently practical.

What is or is actually possible is that humans generate knowledge in their heads from a few cases.
This was the job of a Knowledge Engineer (KE) or domain expert. However, humans cannot generate perfect knowledge either. When human-created knowledge is actually used, there are cases (even a small number) that do not match the human-created knowledge. What is needed is to tune existing knowledge into new (exceptional) cases. However, such a modified knowledge learning algorithm is hardly known.

Here, it should be noted that
Generating "corrected knowledge" only from "human-created knowledge + newly generated exceptional case" causes a question of accuracy regardless of which method is used. This is because it is essentially impossible to know which part of human-created knowledge (background knowledge) is reliable knowledge and which part is uncertain knowledge. In other words, in terms of statistical analysis, when the prior probability is not the true prior probability, how much can the prior probability be modified unless there is something like "reliability" of the prior probability? I don't know. In the present invention, an actual case (actual case) from which background knowledge is generated is used as this reliability. That is, the actual case is used in the range in which the division can be executed in the actually observed case, and the background knowledge is used when the actual case cannot be further divided.

The present invention has been made in view of the above,
It is an object of the present invention to provide a modified knowledge learning device that generates knowledge capable of explaining a case set including cases that do not match the background knowledge and that is based on the background knowledge.

[0021]

In order to achieve the above object, the modified knowledge learning apparatus of the present invention comprises a set of attribute values and one class determined corresponding to the set of attribute values. A set of actual cases including a plurality of cases, a background knowledge given in the form of a rule, a decision tree, or the like for a condition for determining the class of the case from an attribute value, a tentative case reflecting the background knowledge, and the actual case And a case conversion unit that forms a training case set together with the training case set as an input, and when the case set is divided by collecting the same values of the attribute for each attribute, the case set is messed up. The attribute with the largest decrease in the index indicating sex is the split attribute,
The gist is to have a decision tree generating unit that recursively extracts the division attribute as a node and generates a decision tree.

[0022]

In the modified knowledge learning apparatus of the present invention, not only the actual case but also the temporary case generated from the background knowledge is used to determine the division attribute. Generated knowledge, few actual cases,
In a region where only temporary cases exist, knowledge that matches background knowledge is generated.

[0023]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a modified knowledge learning device according to an embodiment of the present invention. The modified-type knowledge learning device 1 shown in FIG. 1 includes an actual case set 2 including a plurality of actual cases composed of a set of attribute values and one class determined corresponding to the set of attribute values, and A training case set is formed by combining background knowledge 3 given as conditions for determining a class from attribute values in the form of rules, decision trees, provisional cases reflecting the background knowledge, and the actual cases. When the case conversion unit 4 and the training case set are input, and when the case set is divided by collecting the same values of the attribute for each attribute, the index indicating the disorder of the case set decreases. The maximum attribute is defined as a split attribute, and the split attribute is extracted as a node recursively, and the decision tree generating means 5 is configured to generate a decision tree.

The real case set 2 is a case observed in the real world (hereinafter, referred to as a real case). The case is expressed in the form of (A ₁ , A ₂ , ... A _n : C), but specifically, it may be expressed by a bit string or a list structure. that is,
Although it is within the range of selection by those skilled in the art, it is necessary to have at least the attribute value and class as information. For example, the actual case set is as follows. However, the case has a format of (A ₁ , A ₂ , A ₃ : C).

(1,0,0: 0) (1,0,1: 0) (0,1,0: 1) This is a subset of the following case set and
It is assumed that only individuals are observed.

(0,0,0: 1) (0,0,1: 1) (0,1,0: 1) (0,1,1: 1) (1,0,0: 0) (1 , 0, 1: 0) (1,1,0: 1) (1,1,1: 1) These eight cases are examples of the target concept of f = (not A ₀ ) or A ₁ . However, not represents negation and or represents OR.

The background knowledge 3 is a rule for determining the class of each case belonging to the actual case set 2 from the attribute value,
For example, a decision tree. F = (not A ₀ ) or
A ₁ can be considered as knowledge expressed by a logical expression. The background knowledge can be in any form, but the important thing is
It is that the class can be calculated by the background knowledge when a subset of attribute values is given. here,
Computable also allows the class to be undefined. That is, if the class cannot be specified when other attribute values other than the subset are not determined, it is assumed that the class is undetermined and is explicitly expressed. However, even if all the attribute values are defined, it is not allowed to generate a plurality of classes from the attribute values. There are already various proposals in the field of knowledge processing regarding the method of expressing knowledge and the method of calculating the class thereof, and since they can be expressed by conventional techniques, further description will not be given in this specification.

The case conversion means 4 corresponds to the above background knowledge (hereinafter referred to as "provisional case").
And the above-mentioned actual case are combined to form a training case set. It is one of the constituent elements constituting the feature of the present invention. The details will be described below. In the first place, the tentative case is a case virtually generated in order to bring in background knowledge in the form of a case because the decision tree generating means 5 described later generates a decision tree based on the case. Specifically, it has the following format.

(*, *, *, ...: C ₀ ) (*, *, *, ...: C ₁ ) ... (*, *, *, ... *: Cn _-1 ) Here, "*" shows that the value of the said attribute is not decided. C ₀ , C ₁ , ..., C _n-1 are enumerations of classes. The above means that the class is not determined at the stage when all the attribute values are not determined. The case conversion unit 4 generates a training case set for learning by using the cases included in the real case set 2 composed of the real cases and the temporary cases. For the above specific example, the provisional case is (*, *, *: 0) (*, *, *: 1). Therefore, the training set is real (1,0,0: 0) real (1,0,1: 0) real (0,1,0: 1) tentative (*, *, *: 0) tentative (* , *, *: 1). “Actual” and “Provisional” before the case represent the distinction between the actual case and the temporary case, respectively.

The decision tree generating means 5 receives the above-mentioned training case set as an input, and for each attribute, when the case set is divided by the attribute, the attribute showing the largest decrease in the index indicating the disorder of the case set is set. As a node, it has a function of recursively generating a decision tree. However, some care must be taken regarding the handling of provisional cases and actual cases. That is, the provisional case is originally
It is representative of multiple cases. Therefore, there is a problem in handling with the same weight as the actual case.

Therefore, in the present invention, the decision tree is generated by considering the weights given in advance to the temporary case and the actual case, and treating as many cases as the weights as if they exist there. . The simplest case is when the actual case is given an extremely large weight. In this case, as long as there is a real case in the input case, the split attribute is determined according to the real case, and when the real case disappears, the split attribute is determined based on the temporary case. Become. Hereinafter, as a first embodiment, the case of prioritizing the actual case will be described in detail.

[First Embodiment of Decision Tree Generating Unit 5] The decision tree generating unit 5 using the real case priority strategy operates in the following steps.

(STEP 1) In order to generate a node of the decision tree, the attribute set of the case given when the node is divided is A, and the case set is S _real and S _art . However, S _real is a set of actual cases, and S _art is a set of temporary cases. (STEP 2) If the stop conditions described later are satisfied for the case sets S _real and S _art , the processing is stopped. Otherwise, for the case set S _real, for each attribute included in the attribute set A, calculate the entropy gain when the attribute is divided, and select the one with the largest entropy gain. , And let this be a. However, when the case set S _real is empty, entropy gain is calculated for each attribute included in the attribute set A for the case set S _art , and the entropy gain is calculated to be the largest entropy. One attribute having a gain is selected and designated as a. Here, in order to calculate the entropy gain, it is necessary to execute the affiliation class of the temporary case for each group after division, which will be described later in detail. (STEP 3) The attribute a is deleted from the attribute set A, and this is set as a new A. Then, with the attribute a, the case set S
_Real and S _art are divided into groups, and the set of cases for each group is S ₀ , S ₁ . At this time, for the case set S _art , "*" of the attribute a is rewritten to the attribute value of the branch. (STEP 4) STEP 2 to STEP 3 are recursively executed for each of the above S ₀ , S ₁ ... However, when the class of the actual case is unique, the temporary cases that do not match the class are deleted and a set is generated.

The above processing is stopped under the following conditions. (1) When both the case sets S _real and S _art are empty. (2) When the case set S _real has a single class and S _art is empty. In this case, the class of the branch is the class of the case set S _real . (3) When the classes of S _real and S _art are determined to be single and the classes are the same. In this case, the class of the branch is the class of the case set S _real . (4) When the classes of the case sets S _real and S _art are individually determined and the classes are inconsistent. In this case, the class of the branch is the class of the case set S _real .

The above conditions (1) and (2) are the same as those of ID3 of the prior art. The condition (3) means to stop when the actual case and the temporary case are in the same class. The condition (4) means that both the actual case and the temporary case are of a single class, but if they are inconsistent, the class of the actual case is prioritized. From this condition, even if the actual case becomes empty (or the class of the actual case converges uniquely), the branch processing is continued while the class of the temporary case is not fixed.

In order to execute the class to which the provisional case belongs to each group after division, basically, in the attribute value of "*" of the provisional case, the attribute of the branch corresponding to the branch. It suffices to generate a case in which the value of the attribute is fixed to the value. Then, it is necessary to check whether or not the class can be obtained by checking the background knowledge in the generated provisional case. Of course, not all attribute values are known, so the class is not always available. But,
At a minimum, provisional cases in which a class does not fall into the class even if an unknown unknown attribute value is found should be deleted. If the class cannot be uniquely determined only by the attribute value that has been substituted for “*”, the provisional case is left at least as long as the class may occur.

Various class calculation methods can be considered according to the background knowledge expression method. For example, a rule interpreter, a decision tree interpreter, and a logic function value calculation program can be used. Those skilled in the art can easily estimate these knowledge representation methods and interpreter implementation methods.

In addition, in the entropy gain calculation based on the actual case, when there are a plurality of attributes showing the maximum entropy gain, the entropy gain of the tentative case is examined, and the attribute that maximizes the entropy gain of the tentative case is selected. That seems to be one method.

More specifically, if a temporary case tentative (*, *, *: 0) tentative (*, *, *: 1) is branched with the first attribute, first copy as follows. I take the.

Group 0 (A ₀ = 0) Temporary (0, *, *: 0) Temporary (0, *, *: 1) Group 1 (A ₀ = 1) Temporary (1, *, *: 0) Temporary (1, *, *: 1) However, here, it is assumed that the background knowledge is, for example, f = not A ₀ . Then, both groups 0 and 1 are 1
Each case is absurd from the background knowledge. Therefore,
The group after branching is as follows.

Group 0 (A ₀ = 0) Temporary (0, *, *: 1) Group 1 (A ₀ = 1) Temporary (1, *, *: 0) In both cases, there is only one case class. Therefore, the branching process for at least the temporary case is stopped here. However, if the background knowledge is f = A ₁ , the value of A ₁ is undecided.
Since the class cannot be determined, the possibilities of both “0” and “1” remain, so the group division after branching is as follows.

Group 0 (A ₀ = 0) Temporary (0, *, *: 0) Temporary (0, *, *: 1) Group 1 (A ₀ = 1) Temporary (1, *, *: 0) Temporary (1, *, *: 1) Above, the description of the embodiment of the present invention is completed.

In order to see the effect of the present invention, some processing examples will be shown below. First, it is assumed that a case of real (1,0,0: 0) real (1,0,1: 0) real (0,1,0: 1) is given. The attributes are A ₀ , A ₁ , A ₂
And it is assumed that f = A ₁ is estimated by a person as background knowledge. However, this human background is not perfect. In reality, f = (not A ₀ ) or A ₁ is correct background knowledge. However, it should be noted that the above three actual cases can be explained by this incorrect background knowledge, so that there is no contradiction for the time being.

Next, it is assumed that a new case, real (0, 0, 1: 1), is input. This contradicts the background knowledge f = A ₁ . Therefore, the following is created as a training example of the knowledge learning system of the present invention.

Real (1,0,0: 0) Real (1,0,1: 0) Real (0,1,0: 1) Real (0,0,1: 1) Temporary (*, *, * : 0) Tentative (*, *, *: 1) When this is processed by the knowledge learning system of the present invention, by entropy gain calculation (only the actual case is examined first).
Since the attribute A ₀ has the maximum entropy gain, the following division is performed.

Group 0 (A ₀ = 0) Real (0,1,0: 1) Real (0,0,1: 1) Temporary (0, *, *: 0) Temporary (0, *, *: 1 ) Group 1 (A ₀ = 1) Real (1,0,0: 0) Real (1,0,1: 0) Temporary (1, *, *: 0) Temporary (1, *, *: 1) So, the actual case class became unique. For the tentative case, the class cannot be determined from the background knowledge f = A ₁ . However, since the class of the actual case is determined, it is necessary to trim the temporary case having a class that conflicts with it (in this case, since the class is binary, the class of the temporary case cannot be determined from the background knowledge). Regardless, the class of provisional cases is uniquely determined, but rather this phenomenon is an exceptional phenomenon). Therefore, the case of each group is as follows.

Group 0 (A ₀ = 0) Real (0,1,0: 1) Real (0,0,1: 1) Temporary (0, *, *: 1) Group 1 (A ₀ = 1) Real (1,0,0: 0) Real (1,0,1: 0) Temporary (1, *, *: 0) Since each class has a single real case and a single temporary case, the process is performed at this stage. Stop. The concept actually generated is
f = not A ₀ .

Then, as another example, for the case actual (0,0,1: 1) actual (0,1,1: 1) actual (1,0,0: 0), as background knowledge, f = It is assumed that A ₂ is declared (this background knowledge is completely wrong, but if we learn from the case, even if ID3 is used, it will be like this).

It is assumed that the actual case (1,1,0: 1) is input here. This is inconsistent with background knowledge.

Training Case Set Real (0,0,1: 1) Real (0,1,1: 1) Real (1,0,0: 0) Real (1,1,0: 1) Tentative (*, *, *: 0) Calculate the entropy gain for the temporary (*, *, *: 1). In this case,
The entropy gain is the same in all cases in practice. However, in the case of division by A ₂ , the provisional case is divided into classes neatly, and thus has the division attribute A ₂ . For the group of A ₂ = 1, the class of the actual case and the class of the temporary case are the same, so the processing is stopped. However, for A ₂ = 0 group real (1,0,0: 0) real (1,1,0: 1) tentative (*, *, 0: 0), the actual case class is not constant. , Further division is continued, and from the viewpoint of entropy gain, A ₁ is selected and further division is continued. This allows
Finally, a tree equivalent to the following logical expression is obtained.

F = A ₁ + A ₂ This is not consistent with the target concept. Note, however, that the human-provided knowledge of f = A ₂ is preserved. As described above, according to the present invention, learning can be performed while leaving background knowledge as much as possible.

The calculation amount of ID3 is 1 of the number of cases.
This is the next order and is known to be high-speed, but the method of the present invention does not hinder the characteristics of this ID3 at all, so learning can be executed with a small processing amount.

[Embodiment 2 of Decision Tree Generating Unit 5] In the present invention, the provisional case introduced at the beginning of the decision tree generation uses "*" as the attribute value. This corresponds to all cases that can be input (that is, all cases that can be distributed in a multidimensional space). Therefore, the tentative cases included in each case set of the division determination correspond to points that can be distributed in the multidimensional space divided up to that stage. Therefore, the following questions arise.

(1) For one actual case, one tentative case
Even with an individual, a provisional case actually represents a large number of cases. Therefore, based on background knowledge, is it possible to simply delete a provisional case just because the class of the provisional case is set to one and the class conflicts with the actual case? (2) Actual cases also have noise and are not always true.
Therefore, if only one of the actual cases specifies another class and the remaining actual cases claim the same class as the temporary case, the class of the temporary case may be the class of the actual case set at this stage. Isn't it? In order to deal with these cases, it is advantageous to calculate the entropy gain including both the real case and the provisional case, rather than calculating the entropy gain of the provisional case only when the actual case alone cannot make a decision. It is thought that there are cases.

Therefore, it is conceivable to weight one or both of the real case and the tentative case, and calculate the entropy gain as if there were as many cases as the number of weights. In this case, various weighting methods can be considered, but the following two types are considered as typical ideas.

(A) A fixed numerical value is given in advance to the tentative case. This may be constant for all temporary cases, or the user may be allowed to explicitly specify this weight in advance as the certainty factor for each temporary case. (B) A method is conceivable in which the weight of the tentative case is divided every time the attribute is selected and the case set is divided. For example, it is assumed that an attribute whose attribute value is given by "*" is a split attribute. In this case, there is a method of dividing the weight of the temporary case by the number of branches. For example, if there are 3 branches,
The weight of each branch is one-third.

Hereinafter, the decision tree generating means 5 in the case of weighting each of the provisional case and the actual case of the above (a) operates in the following steps. The configuration other than the decision tree generating means 5 is not different from that of the above-described embodiment.

(STEP 1) In order to generate a node of the decision tree, the attribute set of the case given when the node is divided is A, and the case set is S _real and S _art . However, S _real is a set of real cases, S _art is a set of provisional cases, and the union of these two types of case sets is a case set S. At this time, a weight given in advance is given to each of the provisional case and the actual case. (STEP 2) If the stop condition described later is satisfied for the case set S, the processing is stopped. In this case, the class defined by the case set S is a majority vote of the classes of the case set S. If processing does not stop, case set S
On the other hand, for each attribute included in the attribute set A, the entropy gain in the case where the attribute is divided is calculated,
One attribute having the largest entropy gain is selected and designated as a. (STEP 3) The attribute a is deleted from the attribute set A, and this is set as a new A. Then, the case set S is divided into groups according to the attribute a, and the case sets for each group are set as S ₀ , S ₁ ,. At this time, for the case set S _art , "*" of the attribute a is rewritten to the attribute value of the branch. (STEP 4) STEP 2 to STEP 3 are recursively executed for each of the above S ₀ , S ₁ , ...

Various stopping conditions for STEP 2 have been developed, and those known to those skilled in the art can be applied. For example: (1) When the case set S is empty. (2) When the class of the case set S is single. In this case, the class of the branch is the class of the case set S _real . (3) When the number of cases included in the case set S (however, if a weight is given, it is assumed that there are cases for the weight) below a predetermined limit. (4) No matter which attribute the case set S is divided into, only a gain lower than the lower limit of the entropy gain given in advance can be obtained,
If you are in doubt about the significance of branching.

[Third Embodiment of Decision Tree Generating Unit 5] The configuration of the decision tree generating unit 5 in the case where a given weight is given to a temporary case has been described above. Next, the configuration of the decision tree generating means 5 of (b) described above for reducing the weight of the temporary case for each branch will be described. In this case, it is almost the same as the case of (a) described above, and the only difference is the weight correction in the case of branching. Specifically, it operates in the following steps. The configuration other than the decision tree generating means 5 is not different from that of the above-described embodiment.

(STEP 1) In order to generate a node of the decision tree, the attribute set of the case given when the node is divided is A, and the case set is S _real and S _art . However, S _real is a set of real cases, S _art is a set of provisional cases, and the union of these two types of case sets is a case set S. At this time, a weight given in advance is given to each of the provisional case and the actual case. (STEP 2) If the stop condition for the case (a) is satisfied for the case set S, the process is stopped. In this case, the class defined by the case set S is a majority vote of the classes of the case set S. If the processing does not stop, for the case set S, for each attribute included in the attribute set A, the entropy gain in the case where the attribute is divided is calculated, and the attribute having the largest entropy gain is calculated. Select one and call it a. However, when the case set is divided by the attribute, for the temporary cases in which the attribute value of the attribute is “*”, a value divided by the number of branches is given to each temporary case to branch. (STEP 3) The attribute a is deleted from the attribute set A, and this is set as a new A. Then, the case set S is divided into groups according to the attribute a, and the case sets for each group are set as S ₀ , S ₁ ... However, when the case set is divided according to the attributes, for the provisional cases in which the attribute value of the attribute is “*”, a value divided by the number of branches is given to each provisional case and branched. For the case set S _art , "*" of the attribute a is rewritten to the attribute value of the branch. (STEP 4) STEP 2 to STEP 3 are recursively executed for each of the above S ₀ , S ₁ ...

In the second and third embodiments of the decision tree generating means 5, the "weight" given to the case may not be an integer. It should be noted that even in this case, there is no hindrance to the calculation of entropy gain.
In the entropy calculation, for a set of cases, the number of cases (total weight) in the entire case stack is n, the number of cases belonging to a certain class (total weight) is k, and − (k / n) log It is necessary to calculate (k / n). This calculation need not be an integer for k or n. Therefore, there is no problem in calculating the entropy gain even for the case set having a real number weight.

[0064]

As described above, according to the present invention,
Knowledge that cannot be learned only from cases can be given as background knowledge, and knowledge that matches both cases and background knowledge can be newly learned. Further, even if the learning cannot be done only from the cases, the knowledge learning can be done from the few cases.

Further, according to the present invention, high-speed learning proportional to the number of cases is possible, and even when the actual cases and the background knowledge are inconsistent, the knowledge obtained by integrating both pieces of information according to a predetermined weight is obtained. You can learn.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a modified knowledge learning device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of a conventional technique expressed in a decision tree format.

FIG. 3 is a diagram showing an image of area division.

[Explanation of symbols]

 1 Modified Knowledge Learning Device 2 Actual Case Set 3 Background Knowledge 4 Case Conversion Means 5 Decision Tree Generation Means

Claims (1)

[Claims] 1. A real case set including a plurality of real cases each including a set of attribute values and one class determined in correspondence with the set of attribute values, and a class for determining the class of the case from the attribute values. A background knowledge given in the form of conditions, rules, a decision tree, etc., case conversion means for forming a training case set by combining the temporary case and the actual case reflecting the background knowledge, and inputting the training case set As an example, when the case set is divided by collecting the same values of the attribute for each attribute, the attribute with the largest decrease in the index indicating the randomness of the case set is the division attribute, and the division attribute is the node. And a decision tree generating means for generating a decision tree recursively.