JPH0340170A - Disjunctive normal form learning device - Google Patents

Abstract

PURPOSE:To reduce process load by replacing a learning problem of the class (k-DNF) of a logic formula with a problem which forms a limited tree from a root with use of the positive and negative samples. CONSTITUTION:A sample control part 1 is prepared to perform the sample control and a process required for the sample control together with a tree control part 2 which performs the tree control and a process required for the tree control. Then a k-DNF tree which can express an optional k-DNF is used together with an evaluation standard set based on the thinking way similar to the expected value of the quantity of information, and a cluster of the sample having a label showing the relation between the k-DNF tree and the sample. Then all positive samples are erased by producing successively the following nodes from a root in a k-DNF tree, clustering samples, and erasing some positive samples under a certain condition. As a result, the process load is reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION (1) Field of Industrial Application The present invention relates to a disjunctive standard form learning device that can handle concept formation through learning, design of logic circuits, and the like.

(2) Conventional technology As a conventional technology, the learnability theory (L e a r
11ity theory), its learning algorithm, and a concept learning algorithm called ID3 (Valiant, L, G,
``he theory of the learnable
“Commun, ACM27.111984.11
341], 42. ) (Quinlan, J.R., “Le
arning C1assificationProc
edures and Their Appli
cation to ChessEnd Game
s, in Machine 1. learning:
AnArtificial Intelligence
nce Approach, R,S.

Michalski J, G, Carbonell,
and T, M.

Mitchell (Eds,), Tioga, Pa.
1o Alto, Calif.

1983,).

First, as a preparation, I will explain the following terms. Concept
cept) means n variables (X I+ X 2+ ”
'+ X n ), where:
A disjunctive standard form (Di
This is abbreviated as -DNF.

However, a literal is Xi or its negation, X. For example, XHX3-Xa + Xz Xs + -X5
Expressed as X7 XH. A sample is a binary value (0
, 1) and has a length (number) of n hectares, and samples that are determined to be true by the conceptual logical formula are called positive samples, and other samples are called negative samples.

In learnability theory, any distribution of samples (
However, when the class of the conceptual logical formula (e.g. -DNF) is known for a constant distribution), a certain algorithm can be used to calculate the logical Probably
, Approximately , Computational feasible
Learning is possible when it is possible to obtain (L)
It is called ``earnable''. In other words, if f is a one-element logical formula, g is the logical formula to be sought, 6 is the probability that the algorithm will fail, and ε is the probability that logical formula f and logical formula g do not match, then a logical formula with an arbitrary number of n1 elements is f, the probability that the algorithm fails δ, and the probability that there is a mismatch ε, n, δ−1,
It is said that learning is possible if there exists an algorithm for finding g that satisfies the following equation in polynomial time of ε-1.

P''(P(f≠g)<ε)>1−δ−(1) where P is a probability measure of the appearance of a sample for the distribution, and 2m is the number of samples used for learning.

The conditions equivalent to these are that the following equation is a polynomial of n, and that there exists an algorithm that outputs a logical equation that completely explains this number of samples in polynomial time.

m −ε−′ (logz r +logzδ−′)
-(2) However, T is the number of focuses required to encode and represent any logical expression belonging to the class, and -D
In the case of NF, it is the value of clothes.

1og2 r = (2N)'
- (3) -D N F proposed for this theory
The learning algorithm uses only negative samples, and as samples appear sequentially, it examines all the terms in the currently remaining -DNF (initially there are all (2N) 11 terms) and calculates the samples. This is an algorithm that eliminates all implied terms.

The problem with this algorithm is that examining the terms in a brute force manner as described above is expected to result in a processing load. Furthermore, it is believed that making it possible to use not only negative samples but also positive samples allows for more efficient processing and more concise determination of -DNF.

On the other hand, ID3 is a decision tree (Dec) from a set of samples.
This is an algorithm that creates sample classification rules called sample classification rules. In a decision tree, leaves and other notes have different functions, and the former uses classification results (
true or false); the latter corresponds to one attribute (variable) of that sample when the sample enters this node.
The value of is tested, and depending on the result, it is determined to which child node directly connected to this node the sample should be moved.

That is, by inserting a sample into the root of the decision tree and repeating the test several times and moving the sample, the sample finally reaches the leaves and a classification result can be obtained.

Next, the procedure for creating a decision tree in ID3 will be explained. First, let C be a set of samples, let p' be the probability that a sample will be true in C, and p- be the probability that the sample will be false.

Then, the expected value M of the amount of information in C can be calculated using the following formula.

M (C) --p"logzp"-p-]Ogz
p--(4) Also, the value of C divided by the value of the i-th attribute is expressed as C8,CI, and the probability that the value of the i-th attribute in C becomes 0 is p. ,■ is the probability that T)l
Then, the expected value B of the new information amount due to this division can be calculated using the following formula.

B (C,i) -Po M (Co) +p+ M
(C++) (5) Therefore, the amount of information obtained by this division is obtained by the following formula.

M (C)-B (C,i) From these, for a given set of samples, find the attribute that maximizes the amount of information that can be acquired sequentially from the root, and select it as the attribute to be tested in the notebook. do. Then, a node is created for each of the three divided sample sets, and the process of repeatedly finding the attribute with the maximum amount of information is performed. However, if there are no more positive or negative samples, that node is treated as a leaf and two classification results are given. What should be noted here is that ID3 aims to minimize the expected value of the number of tests when classifying into two categories.

The problem with ID3-DNF learning is that when creating a decision tree, a tree with a depth of more than 100 yen is generally created. In this case, some processing must be performed, which is expected to result in a processing load. Also, ID3 can only perform forward-facing processing.

Even if it turns out that the initially created node is not effective for classification, it cannot be deleted.

Furthermore, considering the structure of a decision tree from the perspective of learnability theory, it is equivalent to a DNF, and it is not clear whether it is learnable.

(3) Problems to be solved by the invention - The problem with the DNF learning algorithm is that the processing load is heavy due to examining terms in a brute force manner, and that only negative samples are used in learning. becomes. Another problem with ID3 is that there is no limit to the depth of the tree created in 1, and there is no mechanism to delete nodes created in 3.

The purpose of the present invention is to solve the above-mentioned problems by replacing the kDNF learning problem with a problem of constructing a constrained tree from the roots using positive and negative samples, thereby solving the problems with conventional devices. The object of the present invention is to provide a disjunctive standard form learning device that can solve the problem.

(4) Means for Solving the Problems The present invention provides a DNF tree that can express a DNF in an arbitrary manner, a simple evaluation scale based on a concept similar to the expected value of information, and a DNF tree and a sample. The main feature is that it uses clusters of samples that have labels that express the correspondence relationship between them.

(5) Effect The outline of the present invention is to create nodes sequentially from the root in a DNF tree, perform clustering of samples, and delete some positive samples under certain conditions. The goal is to eliminate all positive samples. The main items are explained below.

A k-DNF tree is at most deep, each node is labeled with a variable (attribute) name, and.

The tree is such that the label of each node is different from the label of its parent node. Hereinafter, the term created by combining the labels of each node from each node to the root will be referred to as a label attached to a node. Then, -DNF
A tree can be created at any given point using a label attached to each leaf.
DNF can be expressed. Furthermore, in the N-DNF tree, samples are not stored within nodes. This is to avoid complicating sample management since one sample may be stored in multiple notes.

However, the correspondence between nodes and samples is expressed by labels of clusters of samples. The difference between a decision tree and a DNF tree is that the former branches from a parent node to a child node based only on attribute values, whereas the latter branches based on attributes. An example of a decision tree is shown in Figure 2.
An example of an F-tree is shown in Figure 3. In other words, in the case of a decision tree, as shown in FIG.
0'', it branches to a child node, but in the case of a -DNF tree, as shown in Figure 3, if attribute χ3 is true for the sample that entered node X2, it branches to node X3. Branched - node if X5 is true -
X5 is branched.

The evaluation value E for node creation is defined by the following formula. These are called evaluation values of nodes that will be candidates for future creation.

However, for each node, let C be the set of samples corresponding to the node, let C8 be the subset of C where the value of the i-th attribute is true + P++ be the probability that a sample will be true in C8, and Pl be false. The probability that

E(i) =O(p+-=O) 1 Then, a node with the minimum evaluation value is created next. ID3 aims to minimize the expected value of the number of tests when classifying, whereas the present invention aims to quickly find nodes containing only positive samples. That is, samples that reach such a node can be ignored in subsequent processing, making processing more efficient.

A collection of samples is divided into several clusters and managed. Appropriate labels are given to clusters. The samples in the cluster are all true for the attributes that appear as labels of clusters. This is to make it possible to determine which power sample is included in a note just by testing the label.

In addition, for each attribute, the number of samples for which the attribute is true is calculated for each cluster.

This is to facilitate recalculation of the evaluation value for node creation.

(6) Example Figure 1 shows the principle configuration diagram of the present invention, and Figures 6 (A) (B)
) (C) shows a processing flow that constitutes one diagram as a whole. In the figure, ] is a sample management unit that manages samples and performs necessary processing. 2 is a tree management unit that manages the Ni-DNF tree and performs necessary processing therefor. 3 is a cluster of samples having the same label, 4 is an area for storing the label of the cluster, and 5 is an area for storing samples belonging to the cluster. 6 is an area that stores the number of samples that are true for each attribute with respect to the samples included in the cluster. 7 is a node forming the -D N F tree, 8 is an area for storing the label of the node, and 9 is an area for storing a list of labels that cannot be created from the node as a child node.

The operation of one embodiment of the present invention will be explained below using a specific example.

The following is performed as preprocessing. In the sample management section (■), go to (■) in the process shown in FIG.

For a given set of samples C, separate positive samples and negative samples and treat each as one cluster,
Store in cluster sample value area 5 (processing ■, ■
). Further, for each cluster, the number of samples that are true for each attribute is stored in the addition value area 6, and the cluster label area 4 is left empty. On the other hand, the tree management unit 2 creates a root node. and.

The label area 8 and list area 9 of the node are emptied (process ■).

The following process is repeated until there are no more positive samples (process ■, @).

First, an evaluation value of a note to be created as a candidate is calculated at each node (process ①). This process can be easily calculated using the summation value area 6 of the cluster that includes the RA and RU attached to each node as labels. However, evaluation values are not calculated for attributes included in list area 9 for each note.

FIG. 4 is a diagram showing a specific example of evaluation value calculation.

The tree management department starts from the top level NULL and labels Xa.
The cluster is divided into a cluster with a label xb and a cluster with a label xb, and it is managed that a cluster with a label Xc exists for a cluster with a label xb. Although it is not clear from Figure 1, (1) The number of samples for which the attribute W is true for clusters that store positive sample values and whose label is N U L L is pl, and (
ii) It is shown that the number of samples for which the attribute W is true is 01 for clusters in which negative sample values are stored and whose labels are NULL. Similarly (ii
i) The number of samples for which the attribute W is true is P2 for the cluster in which Rahel is Xa in the cluster in which positive sample values are stored, and (iv) the number of samples in which Rahel is in Xa in the cluster in which negative sample values are stored is It is shown that the number of samples for which attribute W is true with respect to It is shown that the number of samples is p4, and (■) the number of samples for which the attribute W is true for Rahel Xa and Xb in the cluster where negative sample values are stored is n4. .

Note that Pl to p6 shown in FIG. 4 correspond to pi'' shown in equation (6), and nl to n6 correspond to p8'' shown in equation (6). In Figure 4
In the calculation of the evaluation value regarding the attribute W of the node Xc, p5. p6. n5n6 is used to perform the calculation corresponding to equation (6), and the attribute W of the root (without Rahel) is
All values in Table 1 are used to calculate the evaluation value for .

Next, the tree management unit 2 determines the candidate node to be created from the entire tree with the minimum evaluation value (process 2). At this time, nodes whose candidate evaluation values are all 2 (p'' does not exist) and nodes whose depth is -1 and which do not have O as a candidate evaluation value are deleted (process ■, [phase]
,[phase]).

Next, the tree management unit 2 actually creates nodes (
Treatment ■, ■, [phase], o, ■). At this time.

Rahel area of the node that creates the attribute used to calculate the evaluation value 8. and its parent note list area 9
Add to. On the other hand, the sample management unit 1 performs cluster division (processing ■). The division rule is 9.For samples of clusters that include Rahel attached to the parent node of the node to be created, 1. Samples for which the attribute corresponding to the Rahel of the node to be created is true, Move the one with Rahel added to a new raster with Rahel. If the evaluation value is O, division is performed only on clusters containing 2 positive samples (processing 0). Also, at this time b, the Rahel associated with the node is extracted as -DNF in the result, no new raster is created, and the positive samples that should be included there are deleted (processing 0).

FIG. 5 is a diagram showing a specific example of note creation and cluster division. Figure 5 shows a new node Xw in the −DNF tree.
The figure below shows an example of cluster division that occurs when a cluster is added, using only the cluster Rahel. In the case shown in the figure, when node Xw is added, since node Xw belongs to the lower level of node Xb, the clusters rxb and 1r with which node
Xa Xb J, rxb Xc”, “Xa X
The cluster “Xb
Xw JrXa Xb Xw J, rxb Xc
Xw J, "χaXb Xc Xw J" is created.

(7) As described in detail of the invention, the learning of the disjunctive standard form can be performed in any -D
NF can be expressed using a DNF tree, based on an idea similar to the expected value of information (evaluation scale, and - using clusters of samples with labels expressing the correspondence between the DNF tree and the samples. By acquiring terms in the disjunctive standard form, it becomes possible to solve the problems of conventional algorithms.

[Brief explanation of drawings]

FIG. 1 shows a basic configuration diagram of the present invention. FIG. 2 shows an example of a decision tree. FIG. 3 shows an example of a -DNF tree. FIG. 4 shows a specific example of evaluation value calculation. FIG. 5 shows a concrete example of node creation and cluster division. Figure 6 (A) (B) (
C) shows a processing flow that constitutes one diagram as a whole. ■... Sample management department, 2... Woodwind department, 3...
Cluster, 4... Rahel area, 5... Sample value area, 6... Addition value area, 7... Note, 8
... Rahel Area, 9... List Area.

Claims (1)

[Claims] In a disjunctive standard form learning device that estimates an original disjunctive standard form using only samples whose truth values are given by a restricted disjunctive standard form, It has at least a sample management section that performs management and the necessary processing, and a tree management section that manages trees and performs the necessary processing, and uses an evaluation scale based on a concept similar to the expected value of information. In addition, a disjunctive standard form learning device is characterized in that the disjunctive standard form is acquired by creating a tree and clustering samples using the evaluation scale.