JPH0793160A - Inference device - Google Patents

Abstract

PURPOSE:To obtain a device capable of possessing adaptability for the constitutional change of an object system, expanding the retrieval range of a solution and describing the resolution accurately by keeping the variety of an inference result appropriately by providing an inference part, and a constitution retrieval part which optimizes constitution by performing the change operation of each module by using the inference part. CONSTITUTION:An inference execution mode is the operation of the inference part A after the constitution of a rule module required for inference and the completion of adjustment for a neural network that is a constituent, and infers by applying fetched data to a rule module set 2, and stores it is an inference result storage means 3. In a module constitution retrieval mode, the constitution retrieval part B optimize plural neural networks when the rule module set 2 of the inference part A is constituted of those plural neural networks, and optimizes the constitution by the change operation for the symbol string of the rule module set 2. The learning of the neural network that is the constituent is performed in a module adjusting mode.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an inference device, and more particularly to an inference device for a knowledge base system having learning ability.

[0002]

2. Description of the Related Art In recent years, the development of an inference device that interprets observed or given data based on a knowledge base, and infers and infers conclusions and results has been developed in various fields such as diagnosis, recognition, control, and decision making. It is being actively conducted. These conventional reasoning devices enable specialized problem solving by expressing the empirical knowledge of experts in the target field with production rules and accumulating them in a knowledge base. However, these inference devices have a problem that it takes a lot of time and labor to acquire knowledge from a specialist and update the knowledge.

On the other hand, a neural network that automatically realizes an input / output relationship from learning data is in the limelight, and its application to various fields such as pattern recognition and control is being studied. Neural networks have the advantage that they do not require any prior knowledge of the subject area.

[0004]

However, since the input variables and the output variables are determined in advance in the conventional neural network, when dealing with the change of the target system, for example, the relation between these variables is used. Although it can be updated by learning, it is impossible to switch the input / output variables to be used, and there is a limit in responding to changes in the target system. Furthermore, conventional neural networks use so-called 1: 1.
Although it is only possible to associate variables that satisfy the relationship
In the inference device, there is a situation in which output variables are different for the same input variable when performing backward inference.

In this case, it is desirable to be able to present a plurality of possible outputs in order to increase the number of inference solution candidates, to accurately express the inference result and to search a wide range of solutions, but the conventional neural network is used as an inference device. If applied, this is not possible for the reasons given above. That is, the conventional neural network may output a statistical average value of possible output variables and may not sufficiently match any learning data. The present invention has been made in view of the above circumstances, and not only has adaptability to structural changes of the target system, but also appropriately expands the search range of solutions and maintains solution diversity by maintaining diversity of inference results. The purpose is to provide an inference device capable of accurate description.

[0006]

An inference device according to claim 1 of the present invention interprets given data and applies these data to an inference rule to obtain an inference result. An inference unit including a rule module set having a plurality of inference rules including a fetching unit and a neural network, and an inference storing unit, and each module used in the inference unit is changed to optimize the configuration and each neural network. Learning
In addition, it has a configuration search unit that automatically searches for these optimum rule modules.

An inference apparatus according to [Claim 2] of the present invention comprises:
In [Claim 1], the automatic search in the configuration search unit is performed by a genetic algorithm.

An inference apparatus according to [Claim 3] of the present invention comprises:
In [Claim 1], when learning a neural network which is a realization element of an inference rule, if the learning is not performed with the expected accuracy at the time of learning, the learning data is classified and divided into another neural network. I tried to make them learn.

[0009]

The reasoning device according to the first aspect of the present invention has the following three modes of operation. First,
The first is the inference execution mode, which is the configuration of the rule module necessary for inference in this case, and the operation of the inference unit A after the adjustment of the neural network that is a component has already been completed. Search section B
Irrelevant, the fetched data is applied to each of the rule module sets 2 for inference, and the result is stored in the inference result storage means 3.

The second is a module configuration search mode,
In this case, when the rule module set of the inference unit A is composed of a plurality of neural networks, these neural networks are optimized. Since the configuration of the rule module set is represented by a symbol string, the configuration is optimized by a change operation on this symbol string. Further, learning of the neural network which is a constituent element is performed in the module adjustment mode. In [Claim 2], the accuracy is improved by using the automatic search in the configuration search unit as a genetic algorithm.
In [Claim 3], when the learning is not performed with the expected accuracy, the learning data is sorted and divided by another neural network to be processed.

[0011]

Embodiments will be described below with reference to the drawings. Figure 1
FIG. 1 is a configuration diagram of an embodiment of an inference device according to [Claim 1] of the present invention. Then, this reasoning device is used for medical diagnosis as an example, and the age, sex, height, weight, body temperature,
Diagnosis of the abnormality and estimation of the cause of the disease are performed based on blood pressure, pulse rate, blood components, and other symptoms such as cough associated with the disease. A feature of the diagnostic process here is that inference is performed in multiple stages. That is, the intermediate inference result is once output from the input data, and the inference is repeated based on this to show the final diagnosis result. In addition, when diagnosing an animal or the like, it is necessary to reorganize the configuration of the inference device for diagnosing a new animal.

The configuration of this inference apparatus is roughly divided into an inference section A and a configuration search section B. The reasoning unit A is
It is composed of a data capturing means 1, a rule module set 2 composed of a combination of a plurality of neural networks, and an inference result storage means 3. Configuration search unit B
Is a learning database 4, a module set storage unit 5 for storing a plurality of rule module sets configured by a combination of a plurality of neural networks, an output storage unit 6, a neural network learning unit 7, an error value storage unit 8, and a weight value correction unit. 9, encoder 10, decoder 11, module code table 12, module evaluation value calculation means 1
3, module changing means 14, module calling / writing means 15, module code database 16, optimum module selecting means 17, learning controller 18, changeover switch 19,
It is composed of 20 and 21.

The device thus configured operates in the following three modes. Inference execution mode, module configuration search mode, module adjustment mode. Then, these switching is performed by the learning controller 18. Hereinafter, each function mode of each unit will be described.

1. Inference Execution Mode This inference execution mode is the operation of the inference unit A after the configuration of the rule module set necessary for inference and the adjustment of the neural network which is a component have already been completed.
Therefore, the configuration search unit B is not involved in this operation.
This disconnection is performed by switching the selector switch 19 to the terminal side indicated by (1) in response to a signal from the learning controller 18. In this mode, the rule module set 2 infers the input data given from the data fetching means 1 in accordance with a predetermined rule. The rule module set 2 is composed of a plurality of neural networks. An example of this configuration is shown in FIG.
Is shown in.

In this example, age, sex, height, weight, body temperature, blood pressure, pulse rate, red blood cell count, white blood cell count, hematocrit, albumin, serum protein content, blood glucose, serum total fat content, uric acid, inflammation of the tonsils, From the 17 items of cough,
As the intermediate inference results, output the diagnostic results of 4 items of physical exhaustion, respiratory system abnormalities, circulatory system abnormalities, digestive system abnormalities, and these intermediate inference results and some values of input items. Finally, it outputs the diagnostic results for 8 items including no abnormality, overwork, cold, heart disease, diabetes, liver disorder, renal disorder and arteriosclerosis.

The process of deriving the intermediate inference result from the input item is the neural network 1 (NN1), the neural network 2 (NN2), the neural network 3
(NN3) and neural network 4 (NN4). The process of deriving the final diagnosis result from the intermediate inference result and the partial value of the input item is performed by the neural network 5 (NN5) and the neural network 6 (N
N6), neural network 7 (NN7). Each of these neural networks is composed of an input layer, an intermediate layer consisting of one layer or multiple layers, and an output layer, and correlates input data and output data given in advance as learning data, and is desirable between input and output data. A 1: 1 relationship has been realized.

2. Module Configuration Search Mode In this mode, the configuration search unit B executes optimization of the method of configuring the rule module set of the inference unit A with a plurality of neural networks. The configuration here means input variables and output variables of the neural network, and connections of the neural networks. The configuration of each rule module set is represented by a symbol string described later, and the configuration is optimized by a change operation on this symbol string. Learning of the neural network which is a constituent element is performed in a module adjustment mode described later.
The operation of the configuration search unit B is as follows.

(A) The changeover switch 19 is switched to the terminal side shown by (2) by a signal from the learning controller 18, and the inference unit A and the configuration searching unit B are connected. (B) Similarly, the changeover switches 20 and 21 are switched to the terminal side shown in (2), so that a rule module set group including a plurality of rule module sets is connected to an encoder 10 and a decoder 11 described later. (C) A signal from the learning controller 18 causes the neural network learning device 7 described later to be stopped. (D) The configuration of each of the plurality of rule module sets stored in the module set storage unit 5 is the encoder 10
Is represented by a symbol string and stored in the module code table 12.

The expression method of the symbol string is as follows. -Number the input items, diagnostic results, and intermediate inference results. In the example of FIG. 2, the input items are 1 to 1
7, diagnostic results are numbered 18 to 25, and intermediate inference results are numbered 26 to 29. -One neural network is represented by a combination of its input items and output items. Each item is collectively described by (), and each set of () represents one neural network. The neural network 1 (NN1) of FIG. 2 has {(1, 2, 3, 4, 5,
6, 7, 13), (26)}.

There are parallel and serial connection methods for a plurality of neural networks, which are represented by // and + respectively. In the example of FIG. 2, the neural networks 1, 2,
3, 4 (NN1, NN2, NN3, NN4) are connected in parallel, and neural networks 5, 6, 7 (NN5, NN)
6, NN7) are connected in parallel, and they are connected in series, so {NN1 // NN2 // NN3 // NN4} +
It is expressed as {NN5 // NN6 // NN7}. Note that these neural network connections can be combined in multiple ways. Example: {{A // {B + C}} + D}.

(E) Further, the encoder 10 fetches the error value for the learning data of each rule module set stored in the error value storage 8 described later and stores it in the module code table 12.

(F) The module evaluation value calculation means 13 calculates an evaluation value (hereinafter referred to as a module evaluation value) of each symbol string stored in the module code table 12 based on a signal from the learning controller 18. . The learning accuracy for learning data and the simplicity of the rule module set configuration are evaluated by the following items. An error value E for the learning data of the rule module set. The number N of neural networks that make up the rule module set. The number U of units of each neural network that constitutes the rule module set. Then,

[Equation 1] However, α, β, and γ are weighting coefficients for each item.
Also, the above example shows that the larger the evaluation value, the better.

(G) The module changer 14 uses the calculated module evaluation value as a module code table.
The symbol string stored in 12 is rewritten to search for a better module configuration. In this example, a genetic algorithm is used for this search, which is shown in [Claim 2]. This is done as follows. i. A symbol string is selected from the module code table 12 with a probability proportional to the module evaluation value. This increases the probability that a symbol string having a high evaluation value will be selected.

Ii. The genetic operator is applied to the selected symbol string to create a new symbol string. The following are basic genetic operators of genetic algorithms.・ Crossover: Randomly select an intersection from two symbol strings and replace the character strings after the intersection. -Mutation (sudden transition): The characters that make up the character string are randomly exchanged.・ Inversion (reverse): Reverse the arrangement of subsequences. Here, the following operators are defined by reflecting the characteristics of the module set. The probability that each operator acts is set, and the operator acts accordingly.

Add ... Adds an input or output item of one neural network. Example: {a, b} → {(a,
c), b}. a: original input item, b: original output item, c: added output item. Divide ... Divides an input or output item of one neural network to generate a plurality of neural networks. Example: {(a, c), b} → {a, b} // {c,
b}. Insert ... Inserts a layer indicating an item of an intermediate inference result into one neural network to generate a plurality of neural networks. Example: {a, b} → {a, c} + {c,
b}. Delete ... Deletes one neural network input or output item. This is the reverse operation of adding. Example:
{(A, c), b} → {a, b}.

Synthesizing ... The input or output items of a plurality of neural networks are combined into one neural network. This is the reverse operation of splitting. Example: {a,
b} // {c, b} → {(a, c), b}. Reduction ... Removes the layer indicating the item of the intermediate inference result from a plurality of neural networks to form one neural network. This is the reverse operation of insert. Example: {a,
c} + {c, b} → {a, b}. Crossing ... Replace the modules below the module connection symbol common to the two module sets. At this time, the consistency of the input and output with the preceding and following modules is checked, and the processing for eliminating the mismatched portion or the intersection processing is also controlled. Example: Module 1
{A} + {B}, module 2 {C} + {D} → module 1 ′ {A} + {C}, module 2 ′ {C} +
{B}.

Iii. Replace the newly created symbol string with a symbol string having a low module evaluation value in the module code table 12. When a new symbol string is created, an appropriate symbol string is called from the module code database 16 described later and used. This is a process for preventing the occurrence of an undesired state in which all module evaluation values are caught by the same extreme value and the module set close to the optimum solution is deleted. The symbol string changed by the module changing means 14 is interpreted by the decoder 11 and the corresponding rule module set is updated.

(H) Module calling / writing means 15
Is a random symbol string stored in the module code database 16 or a module code table.
The symbol string is extracted according to the symbol string stored in 12. In addition, when a new symbol string is created by the module changing means 14, it is updated by the module code database 16
It also writes to.

(I) The optimum module selecting means 17 extracts the symbol string having the highest evaluation value according to the signal from the learning controller 18. Then, when this evaluation value has reached a predetermined level, it is sent to the decoder 11. This symbol string is converted into the corresponding rule module set, and the rule module set 2 of the inference unit A is realized. Then, the learning controller 18 switches the operation mode to inference execution by switching the changeover switch 19 to the terminal side indicated by (1). If the evaluation value does not reach the predetermined level, a signal to that effect is sent to the learning controller 18, and the module configuration search processing is continued.

3. Module adjustment mode In this mode, the neural network, which is the realization element of the rule module, learns the input data prepared in advance and the inference result for the rule module set configuration. By the signal from the learning controller 18, the changeover switches 19, 20, 21 are switched to the terminal side shown by (3), and the module set storage unit 5 is disconnected from the module code table 12.

The neural network learning device 7 operates by a signal from the learning controller 18, and the learning database 4
The input data stored in (1) is input to the plurality of rule module sets stored in the module set storage unit 5. In the learning database 4, input items used for diagnosis, intermediate inference results, and inference results corresponding thereto are paired and stored as learning data.

Each neural network infers the data input to the rule module set and stores the result in the output storage unit 6. A correct inference result may not always be obtained before the learning of the neural network is completed. Therefore, the error value storage 8
Compares the correct inference result stored in the learning database 4 with the result stored in the output storage unit 6, and stores this difference in the error value storage 8 as a learning error.

The neural network learning device 7 sends a signal to the load value correcting means 9 when the learning error exceeds the predetermined allowable range and the number of learning times has not reached the predetermined number, and the evaluation amount is calculated. Based on, the weight value of the neural network that constitutes the rule module is corrected. A backpropagation algorithm is used to calculate the weight value correction amount from this evaluation amount. Backpropagation is a well-known learning algorithm for neural networks. This is a modification of how much the weight value should be modified in order to reduce the learning error that is the difference between the actual output of the neural network and the training data. It gives a quantity.

In the above description, the functions of the respective parts have been described for each mode in order to make it easier to understand. However, in the module configuration search mode, each time a new rule module set is created, the mode is switched to the module adjustment mode, and the module components A neural network is trained. After the learning is completed, the mode is switched to the module configuration search mode again, and the module configuration is changed. These processes are repeated until the end condition is satisfied.
When the end condition is satisfied, the mode is switched to the inference execution mode.

According to the above embodiment, the construction of the rule module set and the adjustment of each rule module are automatically performed, so that the apparatus can be constructed easily in a short time. Further, by optimizing the configuration of the rule module set, it is possible to deal with structural changes in the target system that cannot be dealt with only by learning the neural network that is the rule module. This is a medical diagnostic device
This corresponds to the case where the target animal changes in the animal diagnostic device. It should be noted that this embodiment is not limited to the above-mentioned embodiment, but can be applied to the diagnosis processing based on the input data.

FIG. 3 is an embodiment corresponding to [Claim 3] of the present invention, and shows a configuration diagram of a neural network and its learning device (hereinafter referred to as a learning system). This is for learning learning data given in advance, and is basically the same as the module adjustment mode of the configuration searching unit B in the above embodiment. The feature here is that if the relationship between the input data and the output data to be learned by the neural network is not 1: 1, the learning data is divided and this is learned by another neural network. As a result, it becomes possible to learn learning data in which a plurality of outputs correspond to the same input.

FIG. 3 corresponds to the corresponding portions in FIG. 1, and the same portions as those in FIG. 1 are designated by the same reference numerals and the description thereof will be omitted. Newly added in this embodiment are a neural network duplicator 22 and a learning data divider 23. The neural network learning device 7 inputs the input data stored in the learning database 4 into the neural network 1 stored in the neural network storage unit 5. At this point, the only neural network stored is the neural network 1. In the learning database 4, input data and output data are paired and stored as learning data. As shown in FIG. 4, the input / output relationship includes that in which different outputs correspond to the same input.

The output corresponding to the data input to the neural network 1 is stored in the output storage device 6, and the difference between this and the desired output data is calculated and stored in the error value storage device 8. The neural network learning device 7
When this error value exceeds the allowable range and the number of learning times has not reached the predetermined number, a signal is sent to the weight value corrector 9 and the weight value of the neural network is corrected based on this error value. . A backpropagation algorithm is used to calculate the weight value correction amount from this error value.

Back-propagation is a well-known learning algorithm for neural networks, and this is how much the weight value should be modified in order to reduce the learning error which is the difference between the actual output of the neural network and the learning data. It gives a correction amount of kika. However, when the data shown in FIG. 4 is learned, the neural network outputs the average value of the learning data as shown in FIG. 5, and the learning error does not decrease below a certain value. In this case, re-learning is performed by the following procedure.

First, the neural network duplicator 22 duplicates the neural network 2 having the same number of units and the same weight value as the neural network 1 according to the signal from the neural network learning device 7, and stores it in the neural network storage unit 5. Next, using the signal from the neural network learning device 7, the learning data divider
Reference numeral 23 divides the learning data of the learning database 4 into two, and causes the neural network 1 and the neural network 2 to learn respectively. This division is performed by the following method.
The learning data having the largest learning error is separated from the learning data, and the neural network 1 is trained. As a result of re-learning, the one with the largest learning error is separated again, and the learning is repeated. This process is continued until the error value becomes equal to or lower than the predetermined required level.

According to the above embodiment, it is possible to learn the learning data in which a plurality of outputs correspond to the same input.
Therefore, it is possible to present a possible output for a given input, and it is possible to realize backward inference by a neural network, and to appropriately expand the search range of the solution and accurately calculate the solution by maintaining the diversity of inference results. Can be described.

[0042]

As described above, according to the present invention, the adaptability to structural changes of the target system and the appropriate expansion of the search range of solutions by maintaining the diversity of inference results and the accurate description of solutions are provided. Is possible. Further, in [Claims 1 and 2], the device can be constructed in a short time and easily, and by optimizing the configuration of the rule module set, the target system that cannot be dealt with only by learning the neural network which is the rule module. It can respond to structural changes. Further, in [claim 3], possible output for a predetermined input can be presented, and backward inference can be realized by a neural network, and at the same time, by maintaining the diversity of inference results, an appropriate search range of solutions can be obtained. Extending and accurate description of the solution are possible.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an inference apparatus according to an embodiment of [Claim 1] and [Claim 2] of the present invention.

FIG. 2 shows an example of rule module set.

FIG. 3 is a configuration diagram of a learning system according to an embodiment of [Claim 3] of the present invention.

FIG. 4 is an example of learning data having no 1: 1 relationship.

FIG. 5 is a result diagram of learning data of FIG. 4 learned by a conventional learning method.

[Explanation of symbols]

 A inference unit B configuration search unit 1 data importing unit 2 rule module set 3 inference result storing unit 4 learning database 5 module set storing unit 6 output storing unit 7 neural network learning unit 8 error value storing unit 9 weight value correcting unit 10 encoder 11 Decoder 12 Module code table 13 Module evaluation value calculating means 14 Module changing means 15 Module calling / writing means 16 Module code database 17 Optimal module selecting means 18 Learning controller 19, 20, 21 Changeover switch 22 Neural network duplicator 23 Learning data division vessel

Claims (3)

[Claims] 1. An inference device that interprets given data and applies these data to an inference rule to obtain an inference result, and a rule module set having a plurality of inference rules including a data fetching means and a neural network, and inference. A configuration search in which an inference unit including a storage unit and each module used in the inference unit are changed and operated to optimize the configuration, learning of each neural network is performed, and these optimal rule modules are automatically searched. An inference device including a section. 2. The inference apparatus according to claim 1, wherein the automatic search in the configuration search unit is performed by a genetic algorithm. 3. When learning a neural network, which is a realization element of an inference rule, if the learning data is not learned with the expected accuracy at the time of learning, the learning data is separated, and this is divided into different neural networks for learning. The inference device according to claim 1, wherein