JPH0298775A - Method and device for generating decision list - Google Patents

Abstract

PURPOSE:To perform the classification and prediction of data outputted frequently from an information source with high accuracy by picking up the data from decision which maximizes an information gain, and classifying the data preferentially from the decision with high identification capacity. CONSTITUTION:The decision with high information gain for measuring data is added sequentially as the decision of a decision list as a logical product which regulates the decision of the decision list under a state where the number of characters comprising one logical product is fixed. And a stopping rule based on an error classification rate is provided, and the addition of the decision is stopped at a point where it is satisfied, then, the decision list obtained at that time is outputted. In such a way, it is possible to classify and predict unknown data generated from the same information source as that for the measuring data with high accuracy.

Description

[Detailed Description of the Invention] (Industrial Application Field) This invention generates a decision list from a plurality of noisy observation data given as a set of a plurality of attributes and classes that take values of {0, 1). Regarding the method.

(Prior art) A decision list is used as a method for generating and expressing a structural relationship between attributes and classes from observed data given as a set of multiple attributes and classes that take values of {0, 1). There is a method for generating classification rules.

The concept of decision lists is introduced in the American magazine Machine Learning, published in 1987.
The article "Learning decisi
on 1st) J, and is known to have the highest expressiveness among the currently known expressions of classification rules as far as binary data is handled. This paper describes a method for constructing a decision list that is consistent with all observational data from observational data without noise.

(Problem to be Solved by the Invention) The method for generating a decision list described in the above paper is a method for constructing a decision list that is consistent with all observation data from observation data without noise. Ta. However, since the observational data that we actually handle generally involves noise, we need a decision list method that can classify data generated from the same information source as accurately as possible, even if it cannot explain all the observational data. , there was no means to generate such a decision list.

SUMMARY OF THE INVENTION An object of the present invention is to provide a method for automatically generating a decision list that minimizes prediction errors for unknown data from noisy observed data.

(Means for Solving the Problems) One of the methods for generating a decision list according to the present invention is to fix the number of characters constituting one logical product, and to generate a logical product that defines the decision of the decision list. A step of sequentially selecting and adding those that maximize the information gain to observation data, and a step of using a decision list obtained when adding decisions according to a predetermined stopping rule as the final decision list. It is characterized by including.

Alternatively, the following modification can be made to the above invention. That is, the step of once generating a decision list using the above method, and then the decision list obtained by sequentially removing decision items from the end of the decision list).
For each element of the series, calculate the amount of description of observed data described using the decision list, and finally find the decision list that minimizes the amount of description in the decision list series. A method for generating a decision list is characterized in that the method includes the step of generating a decision list. (The above two methods are referred to as [Invention 1].) Another method of generating a decision list according to the present invention is to fix the maximum number of characters (referred to as k) that constitute one logical product. Under the above method, the addition of decisions was completed using a step of sequentially selecting and adding the logical product that maximizes the information gain for the observation data as a logical product that defines the decision of the decision list, and a predetermined stopping rule. By the way, the amount of description of observed data to be described is calculated using the step of calculating the obtained decision list for multiple values of k and the decision list generated by repeating the above generation process for each k. The method is characterized in that it includes a step of comparing and determining a decision list that minimizes the amount of description among them as a final decision list.

Alternatively, the following modification can be made to the above invention. That is, by the above method, for each element of the series of decision lists obtained by obtaining a decision list for each fixed k, and sequentially removing decision terms from the end of the decision list, and calculating the amount of description of the observed data to be described using the decision list, and obtaining the best decision list for each k, which minimizes the amount of description in the decision list). and determining the best decision list for a plurality of values of k, and determining the decision list that minimizes the amount of description among them as the final decision list. It is. (Hereinafter, the above two methods will be referred to as "Invention 2J").

One of the decision list generation devices according to the present invention includes means for storing observation data, and observation data as a logical product defining a decision in the decision list, with the number of characters constituting one logical product being fixed. Means for sequentially selecting and adding decisions that maximize the information gain to the data, and outputting the decision list obtained when the addition of decisions is finished according to a predetermined stopping rule as the final decision list. It is also possible to make the following modifications to the above invention. That is, a means for once generating a decision list using the above apparatus, a means for generating a series of decision lists obtained by sequentially removing decision items from the end of the decision list, and a means for generating a series of decision lists obtained by sequentially removing decision items from the end of the decision list, and a means for generating a series of decision lists one by one by using the above apparatus. For,
means for calculating the amount of description of observed data described using the decision list; means for storing the amount of description in a series of decision lists; and a decision list that minimizes the amount of description in the series of decision lists. A device for generating a decision list, comprising means for outputting the decision list as a final decision list. (The above two devices are referred to as "Invention 3.") Furthermore, the decision list generating device according to the present invention includes a means for storing observation data, and a maximum number of characters constituting one logical product (denoted as k). ) is fixed, the logical products that define the decisions of the decision list are sequentially selected and added such that the information gain for the observed data is maximized, and the addition of decisions is finished using a predetermined stopping rule. means for obtaining the decision list obtained at this point for multiple values of k, means for storing the decision list obtained for each k, and the above generation process is repeated for each k. Means for calculating the amount of description of observed data described using a decision list, means for storing the amount of description, and means for outputting a decision list that minimizes the amount of description among them as a final decision list. It is characterized by including.

Alternatively, the following modification can be made to the above invention. That is, means for obtaining a decision list for k fixed by the above apparatus, means for generating a series of decision lists obtained by sequentially removing decision terms from the end of the decision list, and the series elements. means for calculating the descriptive amount of observation data described using the decision list, means for storing the descriptive amount and the decision list series, and A means for determining the best decision list for each k that minimizes the amount of description, a method for storing the best decision list for a plurality of values of k, and a method for storing the best decision list for a plurality of values of k. A decision list generation device characterized by comprising means for comparing description amounts and outputting a decision list that minimizes the amount of description as a final decision list. (
The above two devices are referred to as "Invention 4". ) (Operation) First, the decision list will be explained. Now, let n be the number of variables that take values of {0, 1), and let k be a positive integer less than or equal to 6 Ln''(Xl, X1+'..., xn, Xn
)(i,=:l−x,(i=1.−,n)), and Ln
The element of is called a literal, and the conjunction of literals is called a term.

k-DL(n) is the value V taken for element t, (j=1.-, r) and (o, i) of Tn, θ=1. . . , r) represents a set of items expressed as follows. Here, Tnk represents a set of all terms expressed by at most different literals among Ln (however, it does not include xi at the same time), and the last function tr is a constant function that returns true.

(t,,vl),...,(tr,v,)
(1) One decision list is any x(xn(
xn represents the entire set of n-digit Boolean vectors)
The value of V for the first j such that ti(x)=1 is shown for. (ti, v,) is called the i-th decision (or simply decision). For example, the following is an element of 3-DT(4).

(x, x2, 1), (x2f3x,,O), (x3x4
, 1), (true, O) (2) The process of determining data classification using this determination list is expressed as shown in FIG.

As shown in FIG. 7, the decision list is “if-then-else if-then-
``This is a generalization of the type concept classification rule.It is a decision list that predicts data generated from the same information source as accurately as possible from multiple observed data described by n attributes and one class. The problem is what method or device should be used to generate the decision list, and the present invention is an answer to this problem.However, the observation data contains noise. First, one way to answer this question is to calculate the information gain (entropy, misclassification rate, G1
Decisions are added to the decision list in descending order of the number of decisions (measured by the decrease in the n1 index, etc.), and a stopping rule is set based on the misclassification rate, and when this is satisfied, the addition of decisions is stopped. Either outputting the decision list obtained there, or using that decision list for each element of the series of decision lists obtained by sequentially removing decision items from the end of the decision list obtained there. A decision list that calculates the amount of description (code length for self-contained encoding) of observation data described in One possible method is to do this. The reason why decisions that maximize information gain are picked up is that by classifying data first, starting with decisions that have the highest discrimination ability, it is possible to more accurately classify and predict data that frequently appears from information sources. Also, the reason for choosing a decision list that minimizes the amount of description is that when performing self-contained encoding of observed data using a model, a model that can further compress all data (in this case, a decision list)
is for unknown observation data generated from the same information source.
This is because it is asymptotically established that the model is capable of making more accurate predictions. Regarding this theoretical support, especially when applied to statistical models, the American magazine Annals of Statistics (A.
J, Rissanen (J, R15) published in vol. 14 of pp. 1080-1100 of
[Stochastic Complexity and Modeling]
complexity and modeling)
It is stated as an MDL standard in J. but,
The novelty of the present invention lies in the use of MDL criteria for optimal modeling of logical concept learning such as decision lists as classification rules. The above is the principle of the method of invention 1, and invention 3 realizes the same principle in the form of an apparatus. The information gain mentioned above is defined precisely below. First, the symbols are defined as follows.

[Symbols] Below, the base of all logarithms is 2.

・k, n (N (all natural numbers) is fixed.

・To S (<i, x, c) (learning data); i (target number) (N.

X (attribute value) ({0, 1) n, c (class) ({0, 1
)): Set of learning data (attribute value is attribute variable X1...
- Indicates the data value for tXn. ) −5A(t)=(<i, x, c> (s:t(x)=
1), teT”k8B(t)=(<i, x, c>(s:
t(x)=0), t(T”.

・A(t) = #5A (below tX, #W represents the total number of elements in the set W) B(t) = #5B(t) ・A"(t) = #(<i, x, c> (SA(t):v=
1), t(TnkA-(t)=#(<i, x, c>(S
A(t):v=01tt'T'kB"(t)=#(<i
, x, c> (sB(t): v=1), t(TnkB
−(t)=#(<i, x, c>(sB(t): v=0
), t(T%Here, A(t)=A”(t)+A−(t
), B(t)=B+(t)+B−(t).

・ΔI(t) to IB(t”)−I(t)
(4) is called the information gain due to decision (t, v).

(t is a decision term that appears immediately before t in the decision list.) Here, the following can be considered as a way to select IA(t) and ll3(t).

Second, we accurately define the amount of description of all observed data using a decision list. The amount of data description is given as the sum of the amount of description of the decision list itself and the amount of description of exceptions in the data classified by the decision list. For example, in the first decision list in (2), #T53=131 (the term corresponding to the constant function true is counted as 0), so the term in the first decision in (2) is 1/131. Since it is selected based on probability, a code length of log(131) bits is required to encode it self-contained. In this case, Huffman encoding is used as the encoding method. Also, for one decision, it must be written whether to divide 1 or assign 0 to the data that makes the term true, and this requires 1 bit, so in the end, (2) Log(131)+1 bits is required as the amount of description regarding the initial decision. Regarding the next decision, the terms necessary for the decision are selected from among 130 terms excluding the terms used in the first decision of ll'53, so in the same way, log(130
) + 1 bits of description is required. Similarly,
By finding the amount of description for each decision and calculating their sum, the amount of description of the decision list itself can be calculated. (2
), the total is (log(131) + 1) +
(log(130) + 1) + (log(12)
9) +1) +(log(128)+1) bits are required. In addition, the amount of exception data described is, for example, for one decision, assuming that the decision value is 1, there are 7 pieces of data that make the term true in that decision, and the number of data that makes the term true in that decision is 1. Assuming that there are 2 data classes with 5 and 0, and the data classes are written in descending order of object number, the result is 1101101. At this time, the code length required to encode this sequence self-contained is log(7+1)+log(7C2) bits.
Or, if you know from the beginning that the number of exceptions is always [7 + 1) 723 or less, log([q7 + 1)
/24 + 1) + log (7C2) bits.

In general, by setting the length of the series to N, b = b1 or b2, where b1 = N, b2 = r (N + 1) / 2J, and multiplying the number of exceptions by h9, the amount of description required for the exception description is log (b + 1) + log (NC,) bits
(8) Or, LN(h)+log(NC,)bits
It is given by (9). Here, LN(h) is {
0, 1, ..., N) represents the code length when performing self-contained encoding of natural numbers, and satisfies the following.

LN(0)=1 LN(k)=1+1ogk+Ioglogk+-+CN
(k>1) (10) However, the above sum is taken only for positive terms, and CM is 8.8M2-LN(k)
is a real number that satisfies =1.

- The sum of the description amount of the decision list itself and the description amount of the exception data calculated by the method given above is the description amount of all the learning data by using the decision list. Using the concepts of information gain and description amount described above, the decision list can be optimized for given observational data. For example, among these with the number of attributes 6, 3, 9.4, and 11.1
There are contradictions in the data for 2 and 20.19 and 24. According to invention 1, first, when the value of k is fixed, a decision list is constructed by adding the decision that maximizes the information gain, and stops according to a predetermined stopping rule.
By using the decision list obtained there as the decision list to be sought, the following decision list can be obtained (k, , =
4.5, DL"(4) and DL*(5), respectively).

However, entropy is used for information gain, and as a condition for stopping decision addition at this time, the decision value in the previous decision is V, the term that maximizes the information gain is used, and the following i), ii), 1ii) are used. The following conditions shall be adopted.

i) m1n(A”(t), A-(t))/A(t)>
If a (=0.25), then B + (t's < B - (t
'') and v=1 or B+(t'')>B-(t'') and v=o(true, 1-v) are determined, and an appendix is added to the right and output, and the process is stopped.

ii) m1n(B+(t), B-(t))/B(t)
=13 (=0.29) and B+(t)<B-(t) and v=1 or B+(t)>B-(t,) and v=0
If so, (true, 1-v) is determined, an appendix is added to the right and output, and the process stops.

1ii) If there is no more observation data that has not been decided yet, add (true, 1-v) to the right as a decision and output it,
Stop.

DL*(4) is as follows (X2X3X51'6.1) (XI or bifurcated,, OXX1
．． X2X4X6+1)-(x2x3X, heaven,,1)(t
rue, 0)DL*(5) is as follows (X2X3X5X6,1)(XIX2X4.OXX,
x2x4X5"611) - (R, x2x3x, X, IX
x1x3. F,, IXtrue, 0) Also, according to invention 3, when a predetermined stopping rule condition is satisfied by adding a means for storing observation data and a decision that maximizes information gain in order, When k=4.5, DL*(4) and DL*(5) are generated, respectively, by a device having means for outputting the decision list obtained as the final decision list.

Note that with the above decision list generation method and apparatus, -DL(n) can be generated only for a fixed k. In general, the thicker the value of k, the higher the expressive ability and the finer the precision of dividing the multidimensional data space, but since observed data generally contains noise, if k is larger than necessary, This results in a decision list that is sensitive to statistical fluctuations, and in that case, the classification prediction error for unknown data becomes larger. Therefore, there must be an optimal value of k that minimizes the prediction error for unknown data generated from the same information source, and a method and apparatus for generating a decision list for such k is needed. In order to generate an optimal decision list in the above sense, it is necessary to generate a decision list for various values of k by the method of Invention 1 or the method of Invention 3, and then calculate the observations described using them. A method and apparatus are conceivable that calculate and compare the amount of description of data and ultimately obtain a decision list that minimizes the amount of description. This theoretical basis is also based on the concept of the MDL standard shown in the paper by J. Rissanen mentioned above. The above is the principle of the method of invention 2, and invention 4 realizes the same principle in the form of a device.For example, in the above example, #T64=473. #T65=665, and (number of objects to be decided, number of exceptions) in each decision is DL*(4), (6,0), (6,1), (8,2), ( 7, 2), (
5,1) DL*(5), (6,0), (6,1), (6,1), (5,1), (
2,0), (7J), the description amount can be calculated as follows if b=b1 and (7) is used to calculate the exception description amount.

In the DL room (4), the amount of model description = 48.398 bits The amount of exception description = 24.750 bitsL (4)
= 73.148 bitsDL” (5), Model description amount: 55.230 bits Exception description amount = 20.621 bitsL (5)
= 75.851 bits Therefore, it can be said that DL*(4) is a more suitable model than DL*(5) under the Mpf standard. According to invention 2, a decision list is generated once for k = 4 and = 5 using the inventor's method, and the amount of description for each k is calculated, and they are compared to create a decision list that gives a smaller amount of description. As,DI,”(
4) can be generated. According to invention 4, k=
means for generating a decision list once with the apparatus of invention 3 for 4 and = 5, means for calculating the amount of description for each k, means for storing the decision list for these and each k, and means for calculating the amount of description for each k. If a means for comparing and outputting a decision list to be minimized is provided, DL*(4) can be generated.

(Example) Next, the present invention will be explained in detail with reference to the drawings.

Hereinafter, symbols follow the (action) section.

FIG. 1 is a flowchart illustrating the invention in detail.

In 5tart, there is a set S whose initial value is the entire set of observed data consisting of a target number, n attributes of binary value, and a binary class, and a set T whose initial value is Tnke for fixed n and k. and the initial term t''=0 (term O is not included in Tnko, but is defined as a term that makes all data false. Also, decisions using this term shall be omitted at the time of output) and an initial determined value v=0 is given.In step 11, as a condition of the stopping rule, a positive real number α less than or equal to 1 given from the beginning,
For I3, 1) S = empty set ii) min(A”(t”), A-(t”))/A(t
”)〉αB+(t”)>B-(t”) and v=1 or B+(t”)<B-(t”) and v=O iii) m1n(B”(t”), B-( t”))/B
(t”)≦p and B+(t”)>B-(t”) and v=
1 or B+(t'<B-(t") and v=0, and check whether any of these conditions apply in the order of i) ii) 1ii). Among these conditions, If any of the above apply to you, please proceed to step 1.
Proceed to step 2, add (true, 1-v) to the right as the final decision, output it together with the decisions added so far, and end. If none of these conditions are met, proceed to step 13 and find the term t in T that maximizes the information gain defined in (4) when it is considered as a function of term t. Determine from. (This is referred to as t*.) If there are multiple terms with the maximum information gain, seven of them are randomly selected. Next, in step 14, A+
If (book)>A-(t”), then v=1 and A+(t
”)<A-(t”), then v=0, and (t”, v)
Add it to the right as a decision in the decision list. Next, in step 15, the set obtained by removing the data that makes the term t true from S is redefined as S, the set obtained by removing the term L from T is redefined as T, the seven data is redefined as tlt, and the process returns to step 11.

FIG. 2 is a flowchart illustrating another embodiment of invention 1. Steps 21 to 25 are respectively
It performs the same function as steps 11 to 15 of FIG. 1, and the output of step 22 is the same decision list as the output of FIG. In FIG. 2, for the decision list (denoted as DLmax(k)) as the output of step 22, in step 26, the amount of description of observation data that can be described using DLmax(k) is determined in the (action) term. Calculate this as the sum of the amount of description of the decision list itself and the amount of description of exception data for the decision using the method described in . shall be. Here, the amount of description is not a simple sum of the amount of description of the decision list itself and the amount of description of exception data for the decision, but a weighted sum, that is, λ (description amount of the decision list) + (1-
λ) (description amount of exception data) (O x λ<1) in some cases. The same applies to the description amount calculation described below. Next, in step 27, pruning is performed from the terminal (final) determination of DLmax(k). Specifically, V
(1)=DLm, x(k), for j≧1, V(
The second decision from the right of j) is (t, v), (true,
1-v), these are (true, 1-v”)
Let V(j+1) be the substituted value. However, if t' is the term used to determine t immediately before, then V* is given as follows.

Next, in step 28, the amount of description of the observed data described using V(j+1) is calculated by the method shown in the (effect) section, and this is set to +1. Next, the process proceeds to step 29, and it is determined whether there are any decisions left to reap from V(j+1). If there are no decisions remaining, the process proceeds to step 31, where the values of all description amounts, , are compared and the one that minimizes the Output V(i) for , and terminate. Here, if
If there are multiple minimum values, V(i
) and exit. If the decision to harvest from Vθ+1 remains in step 29, the process proceeds to step 30, sets j to j+1, and returns to step 27.

FIG. 3 is a flowchart illustrating the invention in detail.

In 5tart, for a set S whose initial value is the entire set of observed data consisting of a target number, n binary attributes, and a binary class, a fixed n, and a plurality of k, (k
is the maximum number of characters constituting the logical product used for one decision, and M is the total number of possible numbers of k), Tnk
e Set T(k) as initial value and initial term t''=0
and an initial determined value V=O is given.

First, for each k, a decision list is generated using the method of invention 1. Let 31 be the step of generating a decision list corresponding to k, where k is a target of k, gu<kM. Step 31 is the entirety of FIG. 1 or the entirety of FIG. 2. Next, the amount of data to be described using the generated decision list is calculated. If the method shown in FIG. 2 is carried out at step 31, this step is omitted. Step 32 represents the step of calculating the description amount corresponding to k. Next, in step 33, the calculated description amounts for each value are compared to find the minimum value. Here, the amount of description is not a simple sum of the amount of description of the decision list itself and the amount of description of exception data for the decision, but a weighted sum,
That is, it is assumed that the calculation may be performed as λ (description amount of the decision list) + (1-λ) (description amount of exception data) (0 x λ<1). Next, in step 34, the decision list corresponding to the smallest value of k among these is output, and the process ends.

FIG. 4 is a block diagram showing the apparatus of invention 3. A complete set of observation data consisting of an object number, n binary attributes, and two classes is input, and is temporarily stored in 41 storage devices. A part of the observation data is sent from the storage device to the information gain maximum term determination circuit 42 in response to a command of a control signal generated from the control signal generator. The first data supplied is all observation data. The information gain maximum term determination circuit 42 sets T=T'' as parameters in the initial state.
(k is fixed), t” = o, and 1) S = empty set ii) m1n(A ” (t”), A − (t”))
/A(t”)〉αB + (t”)>B −(t”)
and V=O or B ” (t”) < B − (t”
) and v=1 iii) m1n(B ” (t”), B-(t”)
)/B(t”)≦p and B+(t′su>B −(t
”) and V=O or B + (t'< B −(
The conditions for t' and v=1 are checked in the order of i), ii), and 1ii), and if even one of them is true, the signals P and to are sent to the decision/addition circuit 43 which is vertically attached to 42. Also, if none of the above conditions are satisfied, for a fixed value of k,
For the terms in T, calculate the information gain given by (4) for the data supplied from the storage device, determine the term that maximizes this, and set the state as t''=
t*, T=T-t", and determines tne by the addition circuit 43.
send to The decision addition circuit 43 combines the output of the maximum information gain term decision circuit 42 and the data (42
42 to t
If a city is sent, calculate A+(to) and A-(if A+(yen>A-(is a town), then v=1.A"
If (t”)<A-(to), then let v=0 and (t”,
Add v) to the right as a decision in the decision list, and then send a signal to the control signal generator 44 to generate a control signal instructing the storage device 41 to supply S-8A(to) to 42. . 44 is a storage device that stores S −5A(t”) at 4
A signal is sent to 41 to generate a control signal instructing it to be supplied to 2. 41 receives instructions from 44, and S-
8A(to) is supplied to 41. In response to the command 44, data is newly supplied from the storage device 41. 42 repeats the same operation. When signal P is sent from 42 to 43, if A+(t”)>A-(to), then
v=1. If A, "(t")<A-(car), then v=0
, (true, 1-v) is added to the right of the decision list as the last decision, and the decision list obtained by adding the decisions so far is output as the output of the entire circuit, and the operation of the entire circuit ends.

FIG. 5 is a block diagram showing another embodiment of the decision list generating device of the third invention. Circuits 51, 52, and 54 perform the same input/output functions as circuits 41, 42, and 44 of FIG. 4, respectively. However, as the output of the circuit 53, the first
The same decision list as in the figure output, plus A+(t), A-(t) for each of the terms used in each decision.
The value of is also output. The pruning and description amount calculation circuit 55 takes the output of the circuit 53 as input, prunes it in order from the determination of the end, and calculates the description amount when describing data using these prunings. Here, the amount of description is not a simple sum of the amount of description of the decision list itself and the amount of description of exception data for the decision, but a weighted sum, that is,
It is assumed that the calculation may be performed as λ (description amount of decision list) + (1-λ) (description amount of exception data) (0 x λ<1). Specifically, the two decisions from the right are (t, v) + (t
true, 1-v), these are (true, 1-v).
-v") is repeated until there are no more decisions to reap. However, if t' is the term used in the decision immediately before t, then v"G is given as follows.

55 sends each of the decision lists obtained by pruning together with the description amount calculated for each to the second storage device 56 . The second storage device stores the input sent from 55. 55 also sends only the description amount of each pruned decision list to the description amount comparison circuit 57. A signal 57 instructs to compare the description amounts of the pruned decision lists, find the minimum value, and send a control signal instructing to output the decision list that minimizes the description amount from the second storage device 56. is sent to the second control signal generation circuit 58. 58 received 57's orders,
The second storage device 56 sends a control signal instructing the second storage device 56 to output a decision list that minimizes the amount of description.
send to The second storage device 5'6 receives the command 58 and outputs a decision list that minimizes the amount of description.

FIG. 6 is a block diagram showing an embodiment of a decision list generating device according to the fourth invention. Using the observation data as human power for the entire device, the human power is first sent to the DL''(k) generation circuit 61. 61 generates a decision list for fixed k using the same circuit as the device of invention 3 ( Let DL*(k) be the decision list generated for a fixed k),
This is executed for a plurality of values of k according to instructions of a control signal generated by the first control signal generator 62. 61 is a description amount calculation circuit 63 which sequentially calculates DL''(k) for each k and the values of A+(t) and A~(1) for each of the terms used for each determination.
DL''(k) for each k is stored in the storage device 64.
send to.

63 uses the output of 61 as input, calculates each description amount, and sends the description amount to the description amount comparison circuit 65 and the storage device 6.
Send it to 4. Here, the amount of description is not a simple sum of the amount of description of the decision list itself and the amount of description of exception data for the decision, but a weighted sum, that is, λ (description amount of the decision list) + (1 - λ) ( Exception data description amount) (0×λ<1
). The storage device 64 is 6
The outputs of 1 and 63 are stored manually. 6
5 is the input of the output of 63, and 65 is the DL for each k.
* Compare the description amount of (k), find the minimum value, and store it in the storage device 6.
4, the second signal instructs to send a control signal instructing to output the decision J strike that minimizes the amount of description.
The signal is sent to the control signal generation circuit 66. 66 receives the command from 65 and sends a control signal to the storage device 64 instructing the storage device 64 to output a decision list that minimizes the amount of description.
send to

The storage device 64 receives the command 66 and outputs a decision list that minimizes the amount of description.

(Effect of the invention) According to inventions 1 and 3, for a fixed value of k (maximum number of characters appearing in the logical product that defines the decision), unknown data generated from the same information source as the observed data can be calculated. A decision list can be generated that is theoretically guaranteed to classify as accurately as possible. According to inventions 2 and 4, further k
From a wide set of decision lists obtained by varying the values of , it is possible to choose a decision list that is theoretically generated to classify as accurately as possible unknown data that originates from the same information source as the observed data. I can do it.

[Brief explanation of drawings]

FIG. 1 is a flowchart showing a method for generating a decision list according to invention 1, FIG. 2 is a flowchart showing a method for generating a decision list according to invention 1, and is a flowchart of a modified version of the method shown in FIG. A flowchart showing how to generate a decision list,
FIG. 4 is a block diagram showing a decision list generation device of invention 3, FIG. 5 is a block diagram of a decision list generation device of invention 3, and a block diagram of a modified version of the method shown in FIG. 4. 4 is a block diagram showing the decision list generation device of invention 4. FIG. 5 is a block diagram of the decision list generation device of invention 4, and a block diagram of a modified version of the method shown in FIG. FIG. 2 is a diagram for detailed explanation. In the figure, 41; storage device; 42: maximum information gain term determination circuit;
43: Determination addition circuit, 44: Control signal generator, 51 first storage circuit, 52: Information gain maximum term determination circuit, 5
3: Determination addition circuit, 54: First control signal generator, 55
: pruning and description amount calculation circuit, 56: second storage circuit,
57: description amount comparison circuit, 58: second control signal generator,
61: DL*(k) generation circuit, 62: first control signal generation device, 63: description amount calculation circuit, 64: storage device, 65:
Description amount comparison circuit, 66: second control signal generation device.

Claims (1)

[Claims] In a method for generating a decision list from a plurality of noisy observation data given as a set of a plurality of attributes and classes taking values of 1, {0, 1}, one logical product is constructed. A step of sequentially selecting and adding the logical product that maximizes the information gain for the observed data as a logical product that defines the determination of the decision list, with the number of characters fixed, and a predetermined stopping rule. A method for generating a decision list, comprising the step of using a decision list obtained when adding decisions as a final decision list. 2. The step of once generating a decision list by the method of claim 1, and then sequentially removing decision items from the end of the decision list, for each element of the series of decision lists, It is characterized by including the step of calculating the amount of description of observed data to be described using the decision list, and setting the decision list that minimizes the amount of description in the decision list series as the final decision list. How to generate a list of decisions. 3. In a method for generating a decision list from multiple noisy observation data given as a pair of multiple attributes and classes that take values {0, 1}, the maximum number of characters constituting one logical product ( k) is fixed, and the step of sequentially selecting and adding the logical product that maximizes the information gain for the observed data as a logical product that specifies the decision of the decision list, and the step of adding it by sequentially selecting the logical product that maximizes the information gain for the observation data, and using a predetermined stopping rule. The step of calculating the decision list obtained when adding decisions is completed for multiple values of k, and the step of calculating the decision list obtained by repeating the above generation process for each k. A method for generating a decision list, comprising the steps of calculating and comparing the amounts of description, and determining a decision list that minimizes the amount of description among them as the final decision list. 4. Obtaining a decision list for each fixed k by the method of claim 1, and for each element of the series of decision lists obtained by sequentially removing decision terms from the end of the decision list. On the other hand, the step of calculating the amount of description of the observed data described using the decision list, and finding the best decision list for each k, which minimizes the amount of description among the decision list series. A method for generating a decision list, comprising the steps of: determining the best decision list for a plurality of k values, and selecting the decision list that minimizes the amount of description among them as the final decision list. 5. In an apparatus for generating a decision list from a plurality of noisy observation data given as a set of a plurality of attributes and classes taking values of {0, 1}, a means for storing observation data and one logical product. By fixing the number of characters composing the decision list, we sequentially select and add the logical product that maximizes the information gain for the observed data as a logical product that specifies the decision of the decision list, and according to a predetermined stopping rule. A device for generating a decision list, comprising: means for outputting a decision list obtained when adding decisions is finished as a final decision list. 6. Means for once generating a decision list by the apparatus of claim 5, and then means for generating a series of decision lists obtained by sequentially removing decision items from the end of the decision list, and one of the series elements. means for calculating the amount of description of observed data to be described using the decision list; means for storing the amount of description and the decision list series; means for outputting the decision list to be finally obtained as a decision list to be obtained;
A decision list generation device comprising: 7. In an apparatus for generating a decision list from a plurality of noisy observation data given as a set of a plurality of attributes and classes taking values of {0, 1}, a means for storing observation data and one logical product. By fixing the maximum number of characters (k) constituting the list, we sequentially select and add those that maximize the information gain for the observed data as a logical product that specifies the decision of the decision list, and means for obtaining a decision list obtained when addition of decisions is finished according to a predetermined stopping rule for a plurality of values of k; a means for storing a decision list obtained for a plurality of k; On the other hand, there is a means for calculating the descriptive amount of observed data using the decision list generated by repeating the above generation process, and a decision list that ultimately obtains the decision list that minimizes the technical amount. A device for generating a decision list, comprising: means for outputting it. 8. Means for obtaining a decision list for a fixed k by the apparatus of claim 5, means for generating a series of decision lists obtained by sequentially removing decision terms from the end of the decision list, and the series elements. means for calculating the descriptive amount of observation data described using the decision list, means for storing the descriptive amount, means for storing a series of decision lists, and the decision list. Means for obtaining a decision list that minimizes the amount of description in a series as the best decision list for each k; Means for storing the best decision list for a plurality of values of k; means for comparing the description amounts of the decision lists and outputting the smallest decision list as a final decision list;
A decision list generation device comprising: