JP2005242739A - Data base analysis method by genetic programming and device therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that a plurality of rules can not be extracted from a database, in which instances satisfying different rules are mixed and the whole set of instances can not be explained with a simplex rule, while a conventional rule extraction method in a genetic programming extracts only one rule from a database from which rules are to be extracted. <P>SOLUTION: A database analysis method by genetic programming and a device therefor adequately divide a set of examples with evolutionary optimization, extract a rule from a set of divided part instances, and consequently extract a plurality of IF_THEN rules as the input-and-output relations of the instances in the database. Each rule thus extracted is given a priority determined based on a rate at which the instances satisfying the rules exist in the database and the result of an inference by using the IF_THEN rules. Consequently, an instance classification system, which is more accurate in classification than a conventional one, can be constituted. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

The present invention relates to a knowledge acquisition method for acquiring a plurality of IF_THEN rules that a case in a database satisfies and a priority of the rules using genetic programming, which is a method for evolutionarily optimizing a tree structure program, and the priority thereof. This is related to a method for constructing a system that predicts an output for a case where the output is unknown using a rule to which is assigned.

Genetic programming is one of the methods for extracting rules that represent the input / output relationships of cases in a database. Genetic programming is a kind of evolutionary computation that is an optimization algorithm conceived from the evolution of organisms, and was proposed by Koza (see Non-Patent Document 1).

Genetic programming is an optimization method for tree-structured programs. In genetic programming, first, a function symbol that is an element of a tree node and a terminal symbol that is an element of a tree leaf are set so that the problem solution can be expressed by a tree structure program. For example, when it is desired to express the input / output relationship by a mathematical expression, it is sufficient to use four operators as function symbols and variables and constants as terminal symbols. FIG. 2 shows an example of a tree structure program when {+, −, ×} is used as a function symbol and {x, 1} is used as a terminal symbol. This tree structure program is expressed by the formula (1 + x) (x−1). ). A tree structure program that represents a solution to a problem is called an individual in genetic programming.

Genetic programming generates multiple individuals and evolves the population according to the principle of living the right person in the organism. The flow of this optimization is shown in FIG. First, a plurality of tree structure programs are generated by randomly combining function symbols and terminal symbols (31). This is the initial generation (first generation) individual population. Next, the solution represented by each individual is applied to the problem and its performance is evaluated (32). Each individual is given an evaluation value that indicates how good the problem is. This is called fitness. Based on this fitness, an individual to be a parent of the next generation individual population is selected (33). Individuals with higher fitness are easier to select. In the parent individual set thus selected, two pairs are created, and a part of the tree structure program is exchanged between individuals based on the probability called the crossover rate for each pair (34). This operation is called crossover. In crossover, as shown in FIG. 4, one node is randomly selected in each of the two individuals, and subtrees below the node are exchanged. Further, for each individual, a symbol in the tree structure program is forcibly changed to another symbol based on a probability called a mutation rate (35). This operation is called mutation. Selection based on fitness, crossover, and mutation are called genetic operations.

A next generation individual set is generated by the above operation. Then, the fitness of each individual is evaluated again (36). The operations from (33) to (36) are repeated until the fitness of the individual with the highest fitness in the group (best individual) exceeds the preset end criterion. As a result, a set of tree structure programs can be evolved and a solution more suitable for the problem can be obtained.

In systems where the relationship between input and output signals is unknown, set terminal symbols and function symbols so that the rules representing the input / output relationships can be expressed in a tree structure, and apply genetic programming using known cases as training examples. For example, the rule expressed by the tree structure program of the best individual obtained as a result of optimization appropriately represents the input / output relationship of the case, and by applying the acquired rule to the case whose output is unknown, the case output Can be predicted. Here, a known case is expressed by a set of a feature vector serving as an input to an individual of genetic programming and an output signal corresponding thereto.

In Japanese Patent Laid-Open No. 2003-317083, an input feature vector X = (X1, X2,..., Xn) is classified into one of m predetermined classification results Y1, Y2,. Describes a system for extracting, by genetic programming, rules that realize the classification from a set of training examples having the characteristics described above. Hereinafter, a case set classified into a plurality of predetermined types and having the same classification result is referred to as a class. In the conventional invention, in order to extract a rule for determining whether a case belongs to the class Yi, only the case classified into the class Yi returns a positive value 1, and the case classified other than the class Yi is negative. The teaching signal is set to return a value of −1, and the genetic program is applied so as to reduce this value using the sum of the square of the rule output value and the teaching signal error in all training cases as the fitness evaluation formula. ing. This optimization by genetic programming is performed m times independently for each class up to i = 1,..., M, and a rule that only the case of each class satisfies is extracted. Using the m rules obtained as a result, a classification system for cases whose output is unknown is constructed. When classifying a case whose output is unknown, the feature vector of the case is applied to the rule for each class, and the case is classified into the class indicated by the rule that returned the value closest to 1.

In the embodiment of the present invention, the classification problem of three types of classes α, β and γ is dealt with as a specific example. Here, as shown in FIG. 5, the relationship between the α, β, and γ classes and the teaching signals for obtaining the rules A, B, and C that only the examples of each class satisfy is determined, and the teacher data (input and (A combination of a feature vector and a teaching signal). The rule extraction by genetic programming is performed independently for each target class, and finally one rule (a total of three rules A to C) is determined for each class.

Classification processing can be performed by using the three rules A to C finally determined as described above. As shown in FIG. 6, if the output value of rule A is maximum, it is determined as class α, if the output value of rule B is maximum, it is determined as class β, and if the output value of rule C is maximum, it is determined as class γ.

The problem with the conventional method described above is that only a single rule can be extracted as a rule satisfied by a certain class of cases. Even if they fall into the same class, there may be exceptional cases that do not satisfy the typical rules of that class, so the rules that are valid for all cases of that class are It is not always possible to describe with a single rule. In such a situation, when classifying cases, a plurality of rules are prepared for a certain class, and if any of them is established, it must be determined that the class is the class. Since this process is impossible in the above-described conventional method, the accuracy of the classification ability is lowered.

JP2003-317083 Koza, J., Genetic Programming: On the Programming of Computers by means of Natural Selection, MIT press, 1992

The conventional method for rule extraction from a database using genetic programming is to extract only one rule that minimizes the sum of errors for all training cases. Cases that do not apply to that rule are ignored as noise or exception data. However, it is also important to obtain rules that can be used for a few exceptional cases as well as general rules that apply to the majority of cases. In addition, when a plurality of rules are acquired, an index indicating how general each rule is for the class example is also important. Therefore, an object of the present invention is to extract a plurality of rules satisfied by cases in the database and the priority of each rule. It is another object of the present invention to construct a highly accurate classification system using such a plurality of rules.

Each input signal that constitutes a case and its signal in a database that stores cases that have features classified according to the output signal in one or more of multiple classification candidates prepared in advance and allow two or more A method to express the input / output relationship of cases with the same classification result by IF_THEN rule by a tree structure combining terminal nodes that represent possible values and function nodes that represent the magnitude relationship and logical product of those values, and a database A means to automatically divide a set of cases with any one classification result existing in, into multiple partial case sets that satisfy the same input / output relationship, and a classification accuracy of cases for a plurality of generated tree structures By using a process that repeats the operation of generating a new tree structure group by selecting based on the fitness shown and exchanging subtrees in the tree structure or changing nodes In order to determine the IF_THEN rule that appropriately represents the input / output relationship that each partial case set satisfies, and which IF_THEN rule to adopt when multiple IF_THEN rules extracted from each subset are satisfied A database analysis apparatus having means for assigning priorities.

Means for dividing the case set into a plurality of partial case sets satisfying the same input / output relationship, means for extracting an IF_THEN rule that appropriately represents the input / output relationship satisfied by each partial case set, Arbitrary subsets in the case set with the same classification result in the means to give priority to determine which IF_THEN rule to adopt when multiple IF_THEN rules extracted from the subset are established A case that has an IF_THEN rule that satisfies the above example, and that has a function that returns output according to that rule as an agent, and a set of agents that have the same IF_THEN rule as a group, and that has a single classification result by multiple agents The number of groups configured by multiple agents and each group so that the entire set can be expressed by the rules of any agent It belongs number of agents, and a method of database analysis using an optimization method to search automatically the IF_THEN rule possessed by each agent.

The means for assigning a priority for determining which IF_THEN rule to be adopted when a plurality of IF_THEN rules extracted from each subset according to claim 1 are established has the same IF_THEN rule. The number of listed agents represents the priority of the rule, and the higher the proportion of the IF_THEN rule that corresponds to a case set with any one classification result matches the cases in the case set that belong to the classification result, the higher the priority of the IF_THEN rule. A database that gives priority by setting the fitness so that the higher the rate at which the IF_THEN rule is established for cases belonging to other classification results, the lower the priority of the IF_THEN rule. Has a method of analysis.

A set of cases that have the same classification result in a database that stores cases with features classified according to the output signal in one or more of multiple classification candidates prepared in advance and allows two or more Based on the means to automatically divide an arbitrary class into a plurality of partial case sets satisfying the same input / output relationship and the fitness indicating the classification accuracy of the cases for the generated tree structure The IF_THEN rule that appropriately represents the input / output relationship that each sub-case set satisfies can be obtained by using a process of selecting and sub-trees in the tree structure and changing the nodes to generate a new tree structure group. A means for extracting and a means for assigning a priority for determining which IF_THEN rule to adopt when a plurality of IF_THEN rules extracted from each subset are established. The method to extract IF_THEN rules for all classes existing in the database by repeatedly executing the operation to automatically acquire IF_THEN rules that only the target class satisfies for all classes existing in the database The database analysis apparatus according to claim 1, further comprising:

5. A database that stores cases having characteristics classified according to an output signal in one or more of a plurality of classification candidates prepared in advance or allowed to be duplicated, and obtained by means of claim 4. Use IF_THEN rule set for all classes to classify cases according to which class matches the IF_THEN rule for which case in the database or the same format that does not exist in the database And a means for classifying the rule into the class indicated by the rule having the highest priority among the established rules when a plurality of rules indicating different class characteristics are established for the case and there are a plurality of classification candidates. A database analysis device.

As described above, the rule extraction means based on genetic programming according to the present invention can automatically extract as many IF_THEN rules as necessary when there are cases where different rules are satisfied in the database. it can. Therefore, it is possible to extract rules for a small number of exceptional data included in the database, which is impossible with the conventional rule extraction method based on genetic programming for the purpose of extracting a single rule.

Each of the plurality of extracted rules is automatically assigned a priority determined based on the frequency at which the IF_THEN rule is established for the cases in the database and the inference result using the IF_THEN rule. Knowledge of whether each rule is general or exceptional is obtained from this priority value. Also, since general rules and exceptional rules are explicitly separated, it is easy to understand the rules.

When a plurality of IF_THEN rules for different classes are established for an unknown case, it is possible to determine the classification result according to the priority, and as a result, it is possible to classify with high accuracy.

As shown in FIG. 1, the method and apparatus for database analysis by genetic programming according to the present invention appropriately divides a case set by evolutionary optimization and extracts a rule from each partial case set. Multiple IF_THEN rules that represent the input / output relationship of the middle case can be extracted. Each extracted rule is given a priority determined based on the ratio of cases in the database that match the rule and the inference result using the IF_THEN rule. As a result, it is possible to construct a case classification system with higher classification accuracy than the conventional method. Details and examples are given below.

In the entire training case set, an agent having a tree structure program representing input / output relation rules satisfied by an arbitrary subset of cases is referred to as an agent. In order to extract a plurality of rules from the database, a system having a plurality of agents (referred to as a multi-agent system) is used. The training case set is divided into a plurality of subsets, and a plurality of rules can be extracted by a plurality of agents extracting rules from different subsets.

In order to realize a plurality of rule extraction processes by the multi-agent system described above, it is necessary to determine how to divide the training case set into how many subsets to extract the rules. In the present invention, the number of rules necessary to represent the input / output relationship of all training cases and the rules represented by the tree structure program are obtained by using genetic programming in which the method of expressing an individual or the method of genetic manipulation is changed. To do.

When multiple agents extract rules from the same case subset, they have the same tree structure program. A set of agents having the same tree structure program is called a group. Agents prepared in advance belong to one of the groups. The genetic programming used in the present invention is an optimization method in which the group structure of how many agents are divided and which agents belong to the same group and the tree structure program of each group are searched together during the evolution process. It is. If this optimization technique is applied to rule extraction processing, it is possible to generate a number of different rules appropriate for expressing the input / output relationship of training examples. Also, by analyzing the acquired group structure, the number of rules necessary for expressing the input / output relationship of cases, the frequency of appearance of cases that satisfy each rule, and the priority of each rule based on the inference results using the rules Knowledge can be acquired.

In the present invention, a set of tree structure programs possessed by each agent group is regarded as one individual in genetic programming. That is, each individual in genetic programming represents a multi-agent system. FIG. 7 shows an example of one individual of genetic programming when four agents are used for rule extraction. In this figure, agent 1 and agent 2 form one group (71), and agent 3 and agent 4 form another group (72). As a result, this individual has two tree structure programs (73, 74) for each group. That is, this individual has two rules.

Hereinafter, a specific processing procedure of genetic programming used in the present invention will be described. FIG. 8 is a flow of optimization according to the present invention. First, an initial individual population is generated (81). FIG. 9 shows a procedure for generating one initial individual, and FIG. 10 is a conceptual diagram of an individual group in the present invention. By performing the process shown in FIG. 9, the division number of the group in the initial individual and the distribution of the agent to each group are determined at random. An individual group is created by repeating the process of FIG. 9 for the number of individuals used in the initial generation. As a result, the individual population has various group structures as shown in FIG. The individual (101) in FIG. 10 is a simplified depiction of the individual shown in FIG.

In the genetic manipulation of genetic programming used in the present invention, selection (83) and mutation (86) are the same as the processing in normal genetic programming, but the crossover (85) is greatly different. A group mutation (84) is added as a new genetic operation.

First, the crossover method will be described. A specific processing procedure when two individuals are crossed is shown in Steps 1 to 3 below.

Step 1: One agent is arbitrarily selected for two individuals to be crossed. For each individual, let T and T ′ be the trees that the agent references. These trees are used for crossover.

Step 2: For each parent individual, agent sets A (T) and A (T ′) that refer to the selected trees T and T ′ are obtained. The following three cases can be considered for the relationship of these sets.

Case 1 When A (T) = A (T '): If the set relationship is A (T) = A (T'), the group structure of each individual does not change. Go to step 3.

Case 2 When A (T) ⊃ A (T ') or A (T) ⊂ A (T'): A (T) ⊃ A (T ') or A (T) ⊂ A (T' ), The following processing is performed. If the relationship of the set is A (T) ⊃ A (T '), in the individual with T, a new tree structure program identical to T is generated, and the elements of the set A (T) ∩ A (T') Is transferred to the group that references the tree structure program. The tree structure program newly generated in this way is again designated as T. The group structure of individuals with T 'does not change. Conversely, if the set relationship is A (T) ⊂ A (T '), in the individual having T', a new tree structure program identical to T 'is newly generated, and the set A (T) ∩ A ( The agent that is the element of T ′) is moved to the group that refers to the tree structure program. The tree structure program newly generated in this way is referred to as T ′. The group structure of individuals with T does not change. In this way, the group can be divided by one individual so that only the tree structure program referenced by the agent that is an element of the common subset of the two sets A (T) and A (T ') is used. Occur. FIG. 11 shows a case where two parent individuals (111 and 112) cross over between trees referred to by agent 2. Go to step 3.

Case 3 When one of A (T) and A (T ') is not in a relationship that includes the other (when neither Case 1 nor Case 2 applies): Complement to A (T) and A (T') Assume that the sets are A (T) ˜ and A (T ′) ˜, respectively. If the set relationship is not A (T) ⊃ A (T ') and not A (T) ⊂ A (T'), the set A (T) ~ ∩ A (T ' ) Is transferred to the group having the tree T, and in the individual having the tree T ′, the agent being the element of the set A (T ′) to ∩A (T) is transferred to the group having the tree T ′. At this time, the group having no affiliated agent and the tree structure of the group are deleted. As a result, in both individuals, the agent moves so that the agent that is an element of A (T) ∪A (T ′) refers to the same tree. FIG. 12 shows a case where two parent individuals (121 and 122) cross over between trees referred to by agent 1. Go to step 3

Step 3: In the trees T and T ′, one point is randomly selected from the nodes of each tree, the subtrees below that node are exchanged with two individuals, and the crossover is completed.

As described above, crossover is performed between tree structure programs that are referenced by any same agent. Then, the group structure is changed according to the relationship of the agent set that refers to the tree used for crossover.

The group mutation (86) is used to prevent the individual population from converging into a single group structure. In the group mutation, an operation of moving an agent to an arbitrarily selected group is performed for each agent constituting one individual based on the occurrence probability called a group mutation rate. This operation is performed before crossover because it works to promote the change in group structure due to crossover.

By repeating generation changes using the above operation, the individual population searches for a solution while gradually bringing the group structure closer to the preferred one.

FIG. 13 is a block diagram of an embodiment of the present invention. This figure shows a system (system 1) that extracts IF_THEN rules with priorities, and a classification system (system 2) that is created using all IF_THEN rules that are satisfied by each class extracted by using this system multiple times. ). In the system 1, a rule extraction process (132) using genetic programming is applied to a database (131) storing cases. A plurality of rules (133) possessed by the best individual in the evolved individual group are IF_THEN rules for determining whether the case is class Yi. Further, by using the present invention, a plurality of acquired rules are given priorities based on the frequency with which the rules are established and the inference results using the rules. In order to extract the IF_THEN rule that each class case satisfies, the system 1 is executed by changing the class extraction target class Yi to i = 1,..., M.

The database (131) stores cases where the input feature vector X = (X1, X2,..., Xn) is classified into any of the m classes Y1, Y2,. ing.

In order to determine which class an input vector is given, it is necessary to find a logical expression that only the case of each Yi (i = 1, ..., m) should satisfy. I must. This logical expression is an expression obtained by ANDing a set of a certain item and a value that can be taken.

In this case, the logical expression must return true for cases of class Yk, and the logical expression must return false for cases other than class Yk. Therefore, this expression is an IF_THEN rule indicating that if the logical expression is satisfied, the case is classified into the class Yk. A logical expression corresponding to the antecedent part of the IF_THEN rule is expressed by a tree structure program as shown in FIG.

The IF_THEN rule extraction method that only the class Yk example described above satisfies is described. FIG. 15 shows a method for evaluating the fitness of each individual, and FIG. 16 shows a conceptual diagram thereof. A plurality of trees in an individual of genetic programming used in the present invention each represents a logical expression. Each case is input to the system from a training case set in which m types of classes are mixed (152). It is calculated whether or not the logical expression of each tree held by the individual whose fitness is to be evaluated is established for the input data (156). As shown in data 2 of FIG. 16, if at least one of a plurality of logical expressions of one individual is true (T), the classification result of the input data is regarded as Yk. In addition, if all the logical expressions of one individual return false (F) as in data 1 in FIG. 16, the classification result of the input data is regarded as not Yk. For cases that should be classified as Yk, optimize so that at least one of the trees in the individual outputs true and all trees return false for the other classes of cases I do.

If a logical expression incorrectly returns true for cases of other classes that are not subject to rule extraction, the fitness of the individual will be reduced as a penalty depending on the number of agents with that rule. . This suppresses the distribution of agents to groups that are frequently misidentified.

Further, from the viewpoint of the number of cases in which each agent is responsible for rule extraction, the concept of load on each agent arises. This is calculated from the number of times each group has adopted the rules and the number of agents belonging to each group. The number of times each rule is adopted is counted when the rule correctly returns true for the case of the class subject to rule extraction (160). When counting the number of hires, if a plurality of trees output true as shown in data 3 in FIG. 16, the rule of the group with the largest number of agents is adopted. At this time, if a certain agent a belongs to the group g, the load Wa of the agent is calculated as the following Expression 2.

By equalizing the load of each agent calculated in this way, a large number of agents are distributed to a group with a rule with a large number of hires, and the number of agents with a rule with a small number of hires is It will be less. Looking at the number of agents referencing each rule, it is important to see how often each extracted rule is used, that is, how common each rule represents the nature of the class instance. Knowledge is gained. Thus, the number of agents that refer to each rule is determined based on the ratio of cases in the database that match the rule and the inference result using the rule, and represents the priority of each rule.

In order to satisfy the above requirement, the fitness is calculated by the following equation (3). The IF_THEN rule that only the class Yk example satisfies is extracted by evolving the individual population so that the fitness increases.

Here, miss_target_data is the number that all rules return false for the case of class Yk that is the rule extraction target (158, 161, 162). misrecognition is the number of cases that have been misrecognized by outputting true for cases of other classes (158, 161, 163). fault_agent represents the number of agents belonging to a group having the rule when a certain rule outputs true for a case of another class and misidentifies (164). For this reason, the third term of Equation 3 represents the average number of agents that support the case when another class of cases is erroneously determined to be true. Vw is a load distribution for all agents (166). The sum obtained by weighting these by α, β, and δ is set as the fitness (167). In order to suppress the division of redundant groups, the fitness is multiplied by the (G-1) power of the penalty coefficient γ (where γ> 1) in accordance with the increase in the number of groups G possessed by the individual.

With the optimization using the above fitness, one of the trees returns true for the case that should be classified as Yk, and false for the case that should be classified as the other class. Further, the higher the frequency at which the rule returns true to the case of the class to be extracted, the more agents are allocated. The distribution of agents to groups that are frequently misidentified is suppressed. Therefore, it can be understood that as the number of belonging agents increases, a typical determination rule is often used, and a rule to which only a small number of agents belong is a determination rule for exceptional data that appears rarely. Rules for other classes are extracted by the same process.

Next, using all the rules acquired for each class, a classification system for cases where the class is unknown is constructed. The flow of the classification process is shown in FIG. 17, and its conceptual diagram is shown in FIG. When classifying cases with unknown classes, the cases are applied to the rule set for all classes. As a result, if all the rules extracted for a class return false, it is understood that the class is not. If one of the extracted rules returns true, it becomes the class indicated by that rule. For example, Data 1 in FIG. 18 shows a case where it is classified as class Y2.

For certain cases, multiple rules for different classes may return true. In this case, the number of agents belonging to the group having each rule plays an important role. As a result of optimization, the number of agents belonging to the group with each rule is considered to represent the size of the ratio of cases that match that rule in the database and the high prediction accuracy of the rule. The class with more agents is considered a plausible solution. However, it is possible to recognize that there is a possibility of the other class although the possibility is small. In Data 2 of FIG. 18, the rules for class Y2 and class Ym both output true. In this case, it is classified into a class Y2 with many agents supporting each rule. However, it can be recognized that there may be a case of class Ym. It is one of the advantages of using this system that we can recognize such a small possibility.

As an example, the present invention was applied to knowledge acquisition from a liver gallbladder database in the medical field. This database has 9 test results as input items and 4 classification results (Alcoholic liver damage, Primary hepatoma, Liver cirrhosis, Cholelithiasis) as outputs. FIG. 19 shows an example of each disease case in this database. The database used was composed of 536 cases, of which 322 were used as training cases and the remaining 214 cases were used as test cases for evaluation of learning results. Each inspection item x in the case is discretized in four stages using Equation 4 using the threshold shown in FIG.

The terminal / function symbols shown in FIG. 21 were used as symbols for genetic programming. However, the following restrictions apply to the use of symbols. The terminal symbol is not directly attached to the argument of the “and” symbol, and the check item such as GOT is included in the first argument arg0 of eq, gt, and lt, and the discrete values 0, 1, 2, 3 are included in the second argument arg1. Suppose any one of them enters. Crossovers and mutations that do not meet this requirement were not performed.

Here, the experiment was performed with 50 agents constituting one individual of genetic programming. The rules optimized so that only the case sets of the four diseases Alcoholic liver damage, Primary hepatoma, Liver cirrhosis, and Cholelithiasis are satisfied are shown in FIGS. 22, 23, 24, and 25, respectively. It can be seen that a plurality of rules are extracted to determine a certain disease. For example, in FIG. 23, which is a rule satisfied only by the case of Primary Hepatoma, the 50 agents in the acquired best individual are divided into 11 groups, and the rules with more affiliation agents tend to be employed more frequently. For example, Rule 1 for Primary hepatoma is a typical rule that returns true to 32% of the cases of the target Primary hepatoma, and Rule 11 is a rule that returns true to 2% of the cases of the target Primary hepatoma. It can be seen that rules for exceptional cases with a low appearance frequency can also be extracted.

The classification system created by integrating the acquired rules was applied to 322 cases used for learning, and the classification performance was verified. As a result, 80.4% of classification was successful. Looking at the actual classification, for example, the data for the nine test items, which are 0, 0, 0, 1, 2, 1, 1, 0, 1, respectively, show that Alcoholic liver damage and Cholelithiasis This is a case with two diagnostic results. For this case, both Alcoholic liver damage rule 2 and Cholelithiasis rule 6 returned true. Looking at the number of agents referring to these two rules, they are 8 and 3, respectively. Based on the priority indicated by the number of agents, this case is classified as Alcoholic liver damage. However, it has succeeded in calling attention to the possibility of Cholelithiasis.

When this classification system was applied to 214 unlearned cases, 157 cases were successfully recognized and the recognition rate was 73.4%. However, of the cases that failed to be recognized here, 30 cases mentioned the possibility of correct disease, and the majority of the 25 cases were correct in addition to the wrongly adopted disease. It was highly accurate only for the disease. From the perspective of detection of possibility, it can be seen that 87.4% of cases were successfully recognized. The case data used includes cases that were originally classified incorrectly, which is a high recognition rate.

INDUSTRIAL APPLICABILITY The present invention is effective in a database in which cases where noise and human judgment / preference are associated with data measurement and classification determination based on the measurement result are stored. Such data has characteristics that there are cases having different classification results even with the same input, and conversely, a plurality of cases showing the same classification results cannot be expressed by a single rule.

Application examples of the present invention include a disease diagnosis system in the medical field, knowledge acquisition from a customer purchase history in a sales business, weather prediction based on time-series fluctuations of weather data, stock price fluctuation prediction, and the like.

For example, in the medical field, patient diagnosis results are stored in a database. Finding effective diagnostic rules from a large number of cases is important for medical support based on scientific support. However, actual values such as measured values and test values stored in the database contain noise, and the diagnostic results are largely ambiguous depending on the experience of the doctor, so that the diagnostic rules can be expressed by a single rule. Not exclusively. In the medical field, whether or not a patient can be treated without overlooking a patient different from the general case is the key to realizing high-quality medical care. The present invention, which can acquire knowledge for an exceptionally small number of patients, is useful for extracting diagnostic rules from a medical database. Specifically, as given as an example, prioritized rules are extracted for each disease from the past patient diagnosis result database, and depending on which class of rules is established for a new patient Diagnose. When a plurality of rules for different diseases are established, there is a possibility that both of the diseases are applicable, and the possibility can be recognized by a priority ratio of the rules. Moreover, when all the rules do not apply, it can be regarded as a normal state that is not any disease.

Conceptual diagram of the present invention Example of a tree structure program Process flow of genetic programming Crossover example in genetic programming Examples of conventional inventions and examples of teaching signals Classification of unknown cases in conventional invention Conceptual diagram of one individual of genetic programming in the present invention Process flow of genetic programming in the present invention Method for generating an initial individual of genetic programming in the present invention Conceptual diagram of individual population of genetic programming in the present invention Crossover of genetic programming in the present invention (example in which group division occurs) Crossover of genetic programming in the present invention (example of group integration) Configuration diagram of an embodiment of the present invention Example of logical expression that is an antecedent part of IF_THEN rule Evaluation method of fitness of each individual when extracting IF_THEN rule satisfied by case of class Yk Conceptual diagram of how to determine whether each case belongs to class Yi Classification process flow for cases with unknown classes Conceptual diagram of classification of cases with unknown class Example of liver gallbladder disease data Threshold for discretization of each inspection item Terminal and function symbols for creating IF_THEN rules for hepatobiliary disease Rules extracted so that only the case of Alcoholic liver damage is satisfied Rules extracted to satisfy only the primary hepatoma case Rules extracted to satisfy only Liver cirrhosis cases Rules extracted to satisfy only Cholelithiasis cases

Explanation of symbols

71 Group 72 consisting of agents 1 and 2 73 Group consisting of agents 3 and 4 Tree structure program 74 referred to by group 71 Tree structure program 101 referred to by group 72 One individual of genetic programming in the present invention (the individual shown in FIG. 7) Simplified)
111 One parent individual used for crossing 112 Another parent individual 121 crossed with the individual indicated by reference numeral 111 One parent individual used for crossing 122 Another parent individual 131 crossed with the individual indicated by reference numeral 121 Database 132 storing cases classified into any of the classification results Processing unit 133 that extracts IF_THEN rules that only one class of examples satisfies A set of rules that only a class Yi example satisfies

Claims (5)

Each input signal that constitutes a case and its signal in a database that stores cases that have features classified according to the output signal in one or more of multiple classification candidates prepared in advance and allow two or more A method to express the input / output relationship of cases with the same classification result by IF_THEN rule by a tree structure combining terminal nodes that represent possible values and function nodes that represent the magnitude relationship and logical product of those values, and a database A means to automatically divide a set of cases with any one classification result existing in, into multiple partial case sets that satisfy the same input / output relationship, and a classification accuracy of cases for a plurality of generated tree structures By using a process that repeats the operation of generating a new tree structure group by selecting based on the fitness shown and exchanging subtrees in the tree structure or changing nodes In order to determine the IF_THEN rule that appropriately represents the input / output relationship that each partial case set satisfies, and which IF_THEN rule to adopt when multiple IF_THEN rules extracted from each subset are satisfied Database analysis device comprising means for assigning priorities. Means for dividing the case set into a plurality of partial case sets satisfying the same input / output relationship, means for extracting an IF_THEN rule that appropriately represents the input / output relationship satisfied by each partial case set, Arbitrary subsets in the case set with the same classification result in the means to give priority to determine which IF_THEN rule to adopt when multiple IF_THEN rules extracted from the subset are established A case that has an IF_THEN rule that satisfies the above example, and that has a function that returns output according to that rule as an agent, and a set of agents that have the same IF_THEN rule as a group, and that has a single classification result by multiple agents The number of groups configured by multiple agents and each group so that the entire set can be expressed by the rules of any agent It belongs number of agents, and methods of database analysis using an optimization method to search automatically the IF_THEN rule possessed by each agent. The means for assigning a priority for determining which IF_THEN rule to be adopted when a plurality of IF_THEN rules extracted from each subset according to claim 1 are established has the same IF_THEN rule. The number of listed agents represents the priority of the rule, and the higher the proportion of the IF_THEN rule that corresponds to a case set with any one classification result matches the cases in the case set that belong to the classification result, the higher the priority of the IF_THEN rule. A database that gives priority by setting the fitness so that the higher the rate at which the IF_THEN rule is established for cases belonging to other classification results, the lower the priority of the IF_THEN rule. Analysis method. A set of cases that have the same classification result in a database that stores cases with features classified according to the output signal in one or more of multiple classification candidates prepared in advance and allows two or more Based on the means to automatically divide an arbitrary class into a plurality of partial case sets satisfying the same input / output relationship and the fitness indicating the classification accuracy of the cases for the generated tree structure The IF_THEN rule that appropriately represents the input / output relationship that each sub-case set satisfies can be obtained by using a process of selecting and sub-trees in the tree structure and changing the nodes to generate a new tree structure group. A means for extracting and a means for assigning a priority for determining which IF_THEN rule to adopt when a plurality of IF_THEN rules extracted from each subset are established. The method to extract IF_THEN rules for all classes existing in the database by repeatedly executing the operation to automatically acquire IF_THEN rules that only the target class satisfies for all classes existing in the database The database analysis apparatus according to claim 1, further comprising: 5. In a database storing cases having characteristics classified according to an output signal in one of a plurality of classification candidates prepared in advance or two or more allowing duplication, acquired by means of claim 4. Use IF_THEN rule set for all classes to classify cases according to which class matches the IF_THEN rule for which case in the database or case with the same format that does not exist in the database And a means for classifying the rule into the class indicated by the rule having the highest priority among the established rules when a plurality of rules indicating different class characteristics are established for the case and there are a plurality of classification candidates. Database analysis equipment.