JPH08314880A - Method for learning neural network and neural network system - Google Patents

Abstract

PURPOSE: To select an amt. of valid featured by inputting teacher data and an amt. of valid feature and repeatedly performing the learning of a neural network(NN), the selection of an amt. of valid feature and the substitution for an input node. CONSTITUTION: Valid feature amount and data showing the node of the corresponding intermediate layer are given from a feature amount determination processing 14 to an NN update processing 15. In this NN update processing 15, the structure of an NN included in the NN 11 is changed because all the couplings of the node of a selected intermediate layer and the node of the previous layer are deleted and the node is defined as the input node for inputting corresponding valid feature amount. As for each node of an input layer, a contribution degree is calculated and whether the combination of valid feature amount is decided or not is judged. When it is judged that the combination of valid feature amount is not decided, a system (teacher data) is inputted in the input layer, the selected valid feature amount is inputted in the input node, the learning of a neural network is performed and processing above is repeated.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a neural network learning method and a neural network system for performing learning at high speed.

[0002]

2. Description of the Related Art Generally, in a control system, the state of a controlled object is detected by a sensor, and a sensor output, or a characteristic amount created based on the sensor output instead of or in addition to the sensor output is used. Control is performed. Further, also in the information processing system, processing is performed on the input information (data) to be processed, or the characteristic amount created based on the input information instead of or in addition to the input information. Hereinafter, the sensor output, the feature amount based on the sensor output, the input information, and the feature amount based on the input information are simply referred to as the feature amount.

In a system represented by an arithmetic method based on an equation, the feature quantity to be input to the system is uniformly determined. On the other hand, in intelligent reasoning systems such as expert systems, fuzzy systems, and neural network systems, there is no definite guideline for determining the amount of features to be input to the system, and the input features (input variables ) Greatly affects the performance of the intelligent reasoning system. Therefore, when designing an intelligent inference system, the input variables are decided (selected).
Is important. Therefore, the design of the intelligent inference system was carried out jointly by the expert to be handled and the expert of the intelligent inference system.

However, since the selection of the feature amount to be input to such an intelligent inference system is jointly performed by the subject expert and the expert of the intelligent inference system, a great deal of labor and effort is required to construct the system. Need time. Also,
There may be a difference between the subject expert and the system expert, and it may not be possible to construct an intelligent reasoning system with sufficient reasoning performance. Therefore, a method has been proposed in which an effective feature amount (this is called an effective feature amount) is selected from a plurality of feature amounts (this is called a feature amount candidate) using a neural network. There are, for example, the following two methods for selecting the effective feature amount.

[0005] One of them is to select some effective feature quantities from a plurality of feature quantities, input these effective feature quantities to a neural network, and to replace a part of the selected effective feature quantities with another. It seeks an effective feature amount while replacing the feature amount candidate with the feature amount and evaluating the feature amount. However, in reality, the correlation between one effective feature amount and another effective feature amount is important, but the correlation is often not taken into consideration.

The other is to input all the feature quantity candidates to the neural network and let the neural network learn by using the output of the sensor or the input information as teacher data, and for each node of the neural network, By deleting the node that has a small effect on the layer (output layer side), the feature amount input from the finally remaining node of the input layer is selected as the effective feature amount. The node is deleted by deleting the coupling between the input layer node and the intermediate layer node and the coupling degree between the intermediate layer node and the output layer node close to 0 (a coupling that has a small effect on the subsequent layers). Then, it deletes the nodes that are no longer connected to the nodes in the previous layer.

On the other hand, in the neural network, when learning is performed by the error back-propagation method based on the teacher data, the calculation amount of the learning is determined by the total number of nodes. Assuming that the number of nodes in the input layer has doubled and the number of nodes in the intermediate layer has doubled accordingly, the number of connections between the nodes in the input layer and the nodes in the intermediate layer is quadrupled, and the computational complexity of learning is increased. Four
It will be doubled. Also, as the scale of the neural network increases (the number of layers and the number of nodes increase),
The number of times of learning until the learning converges also increases. For example, when the number of nodes in the input layer doubles, the number of learning times may double or more. Therefore, when the number of nodes in the input layer increases, the amount of calculation required until the learning converges is proportional to the cube or more of the increase rate of the number of nodes in the input layer.
In general, the larger the size of the neural network, the higher its redundancy, and the more likely it is to fall into a local solution before it converges to the global optimal solution. There are learning methods that do not fall into local solutions, but they require a great deal of time or computation for learning.

However, in such a conventional feature amount selection device, as the number of feature amount candidates increases, the number of nodes in the input layer also increases accordingly.
Learning will require a great deal of calculation or time. The greater the number of feature quantity candidates, the less efficient. In addition, the possibility of falling into a local solution increases, and in order to avoid this, specialized knowledge about neural networks is required.

[0009]

DISCLOSURE OF THE INVENTION An object of the present invention is to enable learning of a neural network at high speed by learning the neural network while simplifying the structure of the neural network.

An object of the present invention is to enable selection of an effective feature amount from a plurality of feature amounts by using a neural network.

According to the present invention, when the learning of the neural network based on the teacher data is performed, some of the nodes in the intermediate layer are not the same as the behavior of the nodes in the output layer and have a correlation. This is based on the fact that if a feature quantity that behaves similarly to the behavior of a node in the middle layer is found from a plurality of feature quantities, that feature quantity is an effective feature quantity.

A neural network learning method and a neural network system according to the present invention will be described.

The neural network learning method according to the present invention comprises an input layer including a plurality of nodes, a single or a plurality of intermediate layers including a plurality of nodes, and an output layer including the same number of nodes as the input layer. A neural network is provided, a plurality of feature quantities are created in advance based on input teacher data, and the output of the neural network when the teacher data is input to the input layer of the neural network is the teacher. The neural network is trained so as to be equal to the data, and the degree of similarity between the output of each node in the intermediate layer of the neural network and the feature quantity is calculated, and these are calculated. One feature amount is selected as an effective feature amount from the plurality of feature amounts based on the similarity of In both cases, the node in the middle layer corresponding to this effective feature amount is replaced with an input node for inputting the effective feature amount, the teacher data is input to the input layer, and the effective feature amount is input to the input node. -It is a method of repeatedly learning the network, selecting effective features and replacing them with input nodes.

A neural network system according to the present invention is a neural network including an input layer including a plurality of nodes, a single or a plurality of intermediate layers including a plurality of nodes, and an output layer including the same number of nodes as the input layer. Network, learning means for causing the neural network to perform learning so that the output of the neural network when the inputted teacher data is inputted to the neural network becomes equal to the teacher data, The similarity representing the degree of similarity between the output of the node in the middle layer and the feature quantity created based on the teacher data is calculated, and the similarity of the plurality of feature quantities is calculated based on these similarities. One of the features is selected as an effective feature and the effective feature is The structure changing means for replacing the node in the middle layer with the input node for inputting the effective feature quantity, and the learning means for learning the neural network using the teacher data and the effective feature quantity The control means is provided for causing the structure changing means to repeatedly perform the selection and the replacement of the input node.

According to the present invention, a neural network including an input layer including a plurality of nodes, a one-layer or a plurality of intermediate layers including a plurality of nodes, and an output layer including the same number of nodes as the input layer is provided in advance. Has been. A plurality of feature quantities are created in advance based on the input teacher data. The neural network is trained so that the output of the neural network when the teacher data is input to the neural network becomes equal to the input teacher data. Similarities representing the degree of similarity between the output of the node in the middle layer of the neural network and the feature amount are calculated. Based on these similarities, one feature quantity is selected from the plurality of feature quantities as an effective feature quantity, and an intermediate node corresponding to this effective feature quantity is an input node for inputting the effective feature quantity. Is replaced by By replacing the node in the middle layer with the input node of the feature amount, the connection between the input node and the node in the previous layer is eliminated, and the amount of calculation is reduced in learning of the neural network. Learning of a neural network using teacher data and the above-mentioned effective feature amount, selection of the effective feature amount, and replacement with an input node are repeated.

Therefore, since the node in the middle layer is replaced with the input node of the effective feature amount, the connection between the input node and the node in the previous layer is eliminated, so that the amount of calculation of learning corresponding to the decrease in the connection is reduced. Since the number is reduced, the neural network can be learned at high speed. Also, an effective feature amount can be automatically selected from a plurality of feature amounts.

The effective feature amount thus selected is
It is used as an input for an intelligent inference system using a fuzzy system, GA (genetic algorithm), neural network, etc.

According to one embodiment of the present invention, for each node in the input layer, a contribution representing how much the node contributes to the subsequent layer is calculated, and all of these contributions are equal to or less than a predetermined threshold value. When, the learning of the neural network, the selection of the effective feature amount, and the replacement with the input node are completed.

In a preferred embodiment of the present invention,
An input node for inputting one or a plurality of feature quantities to be selected as effective feature quantities is provided in advance in the middle layer of the neural network.

According to this embodiment, since the feature quantity to be selected as the effective feature quantity is provided in the neural network in advance, the necessary effective feature quantity can be selected.

[0021] In a further preferred aspect of the present invention, with respect to the nodes in the intermediate layer of the neural network, the contributions representing how much the node contributes to the subsequent layers are calculated, and the contributions and the contributions are calculated. The effective feature amount is selected based on the similarity.

According to this embodiment, the effective feature quantity is selected based on the contribution degree and the similarity degree. For example, among the nodes having a large contribution, the feature having the highest similarity is selected as the effective feature.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT FIG. 1 is a block diagram showing the overall construction of a jacket tank temperature control system. This temperature control system, in order to produce a product of a certain quality,
The temperature of the raw materials (including the produced product) in the tank is controlled to reach a predetermined target temperature.

At the time of production, the raw material B is put in the tank 1 in advance, the raw material A is then put in the tank 1, and the raw material A and the raw material B are chemically reacted to obtain the product C.

The tank 1 into which the raw material is put is provided with a jacket 2 so as to surround it, and the jacket 2
Cold water (normal temperature), steam, or hot water in which cold water and steam are mixed flows into the inside. Further, the inside of the tank 1 is agitated by the agitator 7.

Temperature in the tank 1 (this is called internal temperature)
And a temperature inside the jacket 2 (this is called an outside temperature) are respectively detected by a temperature sensor (not shown), and an inside temperature signal indicating the inside temperature and an outside temperature signal indicating the outside temperature are input to the controller 6. . By outputting the valve opening signal from the controller 6 to the valves 3 and 4, respectively,
The flow rates of cold water and steam are adjusted by valves 3 and 4, respectively. By flowing cold water or steam into the jacket 2, the raw material in the tank 1 can be cooled or heated.

Further, the amount of the raw material (which is referred to as the raw material A) input to the tank 1 is adjusted by the valve 5. An input amount of the raw material A is detected by a sensor (not shown), and an input amount signal representing the input amount is input to the controller 6. Further, the temperature of the supplied raw material A is detected by a temperature sensor (not shown), and the raw material temperature representing the temperature is input to the controller 6.

The temperature around the jacket 2 (outside temperature) is also detected by a temperature sensor (not shown), and an outside temperature signal representing the outside temperature is input to the controller 6.

The chemical reaction between the raw material A and the raw material B is an exothermic reaction, and this heat generation raises the temperature in the tank 1. If the input amount of the raw material A is too large, the amount of heat generation increases, which exceeds the cooling capacity of the jacket 2 and the temperature inside the tank 1 cannot be controlled to the target temperature. Therefore, it is necessary to control the input amount of the raw material A in order to reduce the heat generation amount.

FIG. 2 shows a tank 1 in one production of a product.
An example of a graph of the internal temperature of the inside, the external temperature of the jacket 2 and the input amount of the raw material A is shown. When the raw material A is charged, the raw materials A and B start a chemical reaction and the internal temperature starts to rise. In order to keep the internal temperature at the target temperature, that is, to cool the inside of the tank 1, cold water flows into the jacket 2 and the external temperature decreases.
Even if the amount of cold water flowing into the jacket 2 is maximized (even if the cooling capacity is maximized), the internal temperature rises, so the input amount of the raw material A is reduced so as to suppress the exothermic reaction in the tank 1, and the internal temperature increases. The temperature is controlled to approach the target temperature. After that, when the internal temperature is lowered, the amount of the raw material A added is increased again to increase the reaction.

As described above, in order to control the internal temperature to the target temperature, the amount of the raw material A charged into the tank 1 and the flow rates of cold water and steam flowing into the jacket 2 must be adjusted by the controller 6. Not, but tank 1
Since the inside is maximally cooled by the jacket 2, the outside temperature (the temperature inside the jacket 2) does not necessarily have to be controlled.
Therefore, as described above, the internal temperature, the external temperature, the external temperature, the input amount, and the raw material temperature are input to the controller 6, but it is not necessary to use all of them for control. Further, in order to control the internal temperature to the target temperature, it is necessary to consider the exothermic reaction in the tank 1. Therefore, the characteristic amount for control is determined based on the internal temperature, the external temperature, the external temperature, the input amount and the raw material temperature. Need to create. The feature values based on the inside temperature, the outside temperature, the outside temperature, the input amount, and the raw material temperature include, for example, the following 50 items.

1) Internal temperature 2) External temperature 3) Raw material input 4) -9) Difference between the present internal temperature and N minutes before (N = 1,2,5,10,20,5)
0) 10) ~ 15) Difference between outside temperature and current N minutes ago (N = 1,2,5,10,20,5
0) 16) ~ 21) Difference between the current input amount and N minutes ago (N = 1,2,5,10,2
0,50) 22) Elapsed time from the start of the reaction 23) Integrated value of the internal temperature from the start of the reaction 24) Integrated value of the input amount from the start of the reaction 25) to 28) Integration of the internal temperature from N minutes after the start of the reaction Value (N = 10,
20,50,100) 29) ~ 32) N minutes after the start of the reaction, the integrated value (N = 1
0,20,50,100) 33) to 36) N minutes before the present internal temperature integrated value (N = 10,20,50,
100) 37) ~ 40) N minutes later, the integrated value of the current input amount (N = 10,20,5
0,100) 41) ~ 44) Time until the internal temperature first exceeds (Target temperature-M degrees) (M = 5,10,15,20) 45) ~ 48) Internal temperature is (Target temperature-M degrees) (M = 5,10,15,20) 49) Raw material temperature before input of raw material A 50) Outside temperature

Some of these feature quantities are unnecessary or redundant for control, and an effective feature quantity (this is called an effective feature quantity) is selected from these feature quantities. The effective feature amount is selected by a feature amount selecting device using a neural network described below. In this feature quantity selection device, we paid attention to the fact that the behavior of the nodes (neurons) in the middle layer of the neural network has a higher density of compressed information for the nodes of the output layer than the behavior of the nodes of the input layer. It is a thing.

When the neural network is trained based on the sensor output detected by the sensor (hereinafter referred to as system output) as teacher data, the behavior of the nodes in the output layer is included in the nodes in the intermediate layer. Something that is not the same as and has a correlation appears. If a feature quantity that behaves similarly to the behavior of the node in the middle layer is found from the feature quantity candidates, the feature quantity is an effective feature quantity.

FIG. 3 is a functional block diagram showing a detailed structure of a feature quantity selection device using a neural network. The feature amount selection device includes a neural network (NN) 11, a similarity degree calculation process 12, a contribution degree calculation process 13, a feature amount determination process 14, and a neural network (N
N) Includes structure modification processing 15.

This feature amount selection device is realized by a computer and software operating on the computer. This feature amount selection device can also be realized entirely by hardware. Further, a part of the feature amount selection device can be realized by hardware and the other part can be realized by software.

The NN11 includes an artificial neural network (hereinafter, simply referred to as NN) and a learning process for learning the NN based on the input teacher data.

FIG. 4 shows an example of the structure of the NN. For example, a Hopfield type NN. The NN includes an input layer, three intermediate layers and an output layer. Input layer and output layer have the same number of nodes (neurons) as the number of system outputs
Is provided. For example, in the system described above, the system output is 5. The middle layer has 6 nodes in each layer. The number of intermediate layers is not limited to three, and one layer, two layers, or four or more layers may be provided, and the number of nodes in each layer may be less than or more than six. The number of intermediate layers and the number of nodes in each layer are determined based on the number of nodes in the input layer, in consideration of the speed of convergence of learning and its stability, and the followability to the teacher data.

The intermediate layers are the first intermediate layer, the second intermediate layer, and the third intermediate layer in order from the input layer side, and the i-th (i = 1 to 5) layer in order from the input layer to the output layer, that is, the input The layer, the first intermediate layer, the second intermediate layer, the third intermediate layer, and the output layer are referred to as a first layer, a second layer, a third layer, a fourth layer, and a fifth layer, respectively. The j-th node in the i-th layer is represented by N _{i, j} , and the number of nodes in each layer is represented by J _i . The output of the k-th node N _{i, k} of the i-th layer is U _{i, k,} and the j-th node N _{i, j of the i-} th layer and the subsequent k-th node N of the (i + 1) _-th layer _The degree of coupling with _{i + 1, k} is W _{i + 1, j, k} .

The learning processing included in NN11 is to perform NN learning by the error back propagation method, for example. In the learning process, the NN may be learned by another learning method.

FIGS. 5 and 6 are flow charts showing the procedure of the effective feature amount selection processing in the feature amount selection device.

In NN11, each system output (teacher data) is input to each node of the input layer, and learning is performed so that the output of each node of the output layer matches the corresponding system output (step 21). .

G learning data (system output) is used for learning the NN. The feature amount V _h (h = 1 to H; H is the total number of feature amount candidates) is calculated based on these system outputs. The h-th feature amount of the g-th group is represented by Y _{h, g} . Also, the j-th node N _{i, j of the i-} th layer obtained as a result of learning based on the system output of the g-th set
_Is represented by U _{i, j, g} .

In the contribution degree calculation process 12, the intermediate layer (i =
2-4) contribution degree D _{i, j} indicating how much the j-th node N _{i, j} contributes to the k-th node N _{i + 1, k} of the subsequent layer (output layer side) Is calculated by the following formula based on the coupling degree w _{i + 1, j, k} finally obtained by learning (step 22).

[0045]

[Equation 1]

Equation (1) uses the sum of squares of the coupling degree as the contribution degree.

The contribution D _{i, j} is obtained by using the equation (2) instead of the equation (1).
Alternatively, either one of (3) may be used.

[0048]

[Equation 2]

[0049]

(Equation 3)

In the equation (2), the contribution degree is the one having the maximum coupling degree, and in the equation (3), the contribution degree is the one having the maximum square of the coupling degree.

The degree of similarity can be calculated by other methods. The contribution degree D _{i, j} is given from the contribution degree calculation processing 12 to the feature amount determination processing 14.

The degree of similarity between the output U _{i, j, g} of the j-th node N _{i, j} in each intermediate layer (i = 2 to 4) and the h-th feature candidate Y _{h, g} The similarity _{h, i, j} indicating whether or not there is calculated is calculated by the following equation in the similarity calculation processing 13 (step 2
3).

[0053]

[Equation 4]

Expression (4) represents the total sum of the differences between the outputs of the G nodes and the feature quantities.

The degree of similarity can be calculated by other methods. The similarity C _{h, i, j} is given to the feature amount selection processing 14 from the similarity calculation processing 13.

In the feature amount selection processing 14, the contribution degree D _{i, j} calculated in the contribution degree calculation processing 12 and the similarity degree calculation processing
1 based on the similarity C _{h, i, j} calculated in 13
One effective feature quantity and the node in the middle layer to which it is input are selected as follows (step 24).

A node having a similarity C _{h, i, j} having a constant value Thc or more and a feature quantity corresponding to the node are found. Contribution D _i among those _{nodes, j} is selected maximum nodes, contribution D _i of the _node, if _j is a constant value Thd or more, the feature is enabled features corresponding to its node Selected as quantity.

If there is no node having a similarity C _{h, i, j} having a constant value Thc or more and no feature quantity, a predetermined number of nodes are selected in descending order of contribution, and the similarity is maximum among those nodes. The node and the feature amount corresponding to the node are selected as the effective feature amount.

Instead of selecting the above-mentioned effective feature quantity, the node having the maximum reliability based on the similarity and the feature quantity and the feature quantity corresponding thereto may be selected as the effective feature quantity.

The reliability C _{h, i, j} is calculated by the following equation.

C _{h, i, j} = L _{h, i, j} × D _{i, j} (5)

The reliability may be calculated by the following equation instead of the equation (5).

[0063]

(Equation 5)

[0064]

(Equation 6)

[0065]

(Equation 7)

The effective feature quantity and the corresponding node may be selected by other methods.

The effective feature amount and the data representing the corresponding intermediate layer node are given from the feature amount determining process 14 to the NN updating process 15.

In the NN update process 15, feature amount selection process
It is included in NN11 by deleting all the connections between the nodes in the middle layer selected by 14 and the nodes in the previous layer, and making that node an input node for inputting the corresponding effective feature quantity. The structure of the NN is changed (steps 25 and 26). For example, FIG. 8 shows the second intermediate layer (third
The first node N _3,1 in the (layer) is changed to an input node for inputting the effective feature amount V ₁ .

For each node in the input layer (i = 1), the contribution is calculated according to the above equation (1) (step 27),
Based on this contribution, it is judged whether or not the combination of effective feature amounts has been decided (step 28). For example, it is determined whether or not the combination of effective feature quantities has been decided depending on whether or not the contributions have all fallen below a predetermined value.

When it is determined that the combination of effective feature amounts is not fixed (NO in step 28), the system output (teaching data) is input to the input layer, and the selected effective feature amount is input to the input node. Neural network learning is performed by inputting (step 31). After that, the process returns to step 22 and the processes of steps 22 to 28 are performed.

When it is determined in step 28 that the combination of effective feature amounts has been determined (YES in step 28), all the nodes in the input layer are deleted (step 29). By deleting the nodes in the input layer, all the connections between those nodes and the nodes in the next middle layer are deleted. Further, the nodes of the middle layer that have no connection from the nodes of the previous layer for the first to third middle layers are deleted (step 30).

In this way, the effective feature amount is selected.
For example, FIG. 8 shows four effective feature quantities V ₁ , V ₂ , V ₂₂ and V.
₂₄ is selected.

If there is a feature quantity to be selected as an effective feature quantity (this is called a deterministic feature quantity), the effective feature quantity is calculated using an NN to which an input node for inputting this deterministic feature quantity is added in advance. Flow charts showing the processing procedure to be selected are shown in FIGS. 9, 10 and 11. The same processes as those shown in FIG. 5 and FIG.

An input node for inputting the definite feature amount is added to the NN (step 41). FIG. 12 shows N shown in FIG.
The structure of NN in which the definite feature amount V ₁ is added to the structure of N is shown.

In the NN11, learning is performed on the NN to which the definite feature quantity is added by using the system output and the definite feature quantity (step 42).

The degree of contribution and the degree of similarity are calculated for each node in each intermediate layer (steps 22 and 23), and one effective feature amount and one input node are selected (step 24). The structure of the NN is changed by changing the node in the medium-simplified layer to the input node (step 25), and the connection between the input node and the node in the previous layer is deleted (step 26).

In the NN structure change processing 15, step 26
The structure of the NN modified in is stored (step
43). This NN structure is called the first structure.

All the nodes of the input layer are deleted (step 44), and the nodes of the intermediate layer which have no connection from the nodes of the previous layer are deleted (step 45). The NN structure in which the nodes of the input layer and the nodes of the intermediate layer are deleted is referred to as a second structure.

For the second structure, NN learning is performed using the system output as well as the deterministic feature amount and the effective feature amount as input and using the system output as teacher data (step 4).
6). This learning is performed so that the effective feature amount is input from the input node and each output of the NN approaches the corresponding teacher data.

An error between the NN output and the system output is calculated (step 47), and it is determined whether the error is sufficiently small, that is, the error is less than a preset sufficiently small threshold value ε (step 48). ).

When the error is equal to or less than the threshold value ε (YES in step 48), the selection of the effective feature amount is ended.

When the error exceeds the threshold value ε (step
48 NO), the structure of the NN is returned from the second structure to the first structure of the NN stored in step 41 (step 49),
Among the NN coupling degrees of the first structure, the coupling degree of the first structure corresponding to the second structure is changed to the coupling degree of the second structure learned in step 46 (step 5
3).

After that, the NN of the first structure is learned so that the system output, the definite feature quantity and the effective feature quantity are input, and the output of the NN becomes the system output (step 5
1), the process returns to step 22, and the processes of steps 22 to 48 are repeated.

In this way, the effective feature amount can be selected.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a temperature control system for a jacket tank.

FIG. 2 is a graph of the temperature in the tank, the temperature in the jacket, and the input amount of raw materials in one production of the product.

FIG. 3 is a functional block diagram showing a detailed configuration of a feature amount selection device.

FIG. 4 shows an example of the structure of an NN.

FIG. 5 is a flow chart showing a procedure of effective feature amount selection processing.

FIG. 6 is a flow chart showing a procedure of effective feature amount selection processing.

FIG. 7 shows a structure of an NN in which one effective feature quantity and a node in the middle layer for inputting the effective feature quantity are selected.

FIG. 8 shows a structure of an NN in which a combination of effective feature amounts is fixed.

FIG. 9 is a flowchart showing a procedure of effective feature amount selection processing when one definite feature amount is added to the NN.

FIG. 10 is a flowchart showing a procedure of effective feature amount selection processing when one definite feature amount is added to the NN.

FIG. 11 is a flowchart showing a procedure of effective feature amount selection processing when one definite feature amount is added to the NN.

FIG. 12 shows an example of an NN structure in which one deterministic feature is added to the NN.

[Explanation of symbols]

 11 NN 12 Contribution calculation processing 13 Similarity calculation processing 14 Feature amount selection processing 15 NN structure change processing

Claims (5)

[Claims] 1. A neural network comprising an input layer including a plurality of nodes, a one-layer or a plurality of intermediate layers including a plurality of nodes, and an output layer including the same number of nodes as the input layer is provided and input. A plurality of feature quantities are created in advance based on the teacher data, and the neural network is configured so that the output of the neural network when the teacher data is input to the input layer of the neural network is equal to the teacher data. The network is trained to calculate the degree of similarity between the output of each node in the intermediate layer of the neural network and the feature quantity, and the plurality of similarity values are calculated based on these similarity degrees. One feature amount is selected from the feature amounts as the effective feature amount, and the node in the middle layer corresponding to this effective feature amount is The input node for inputting the effective feature amount is replaced, the teacher data is input to the input layer, and the effective feature amount is input to the input node to learn the neural network and select the effective feature amount and the input node. A learning method for a neural network in which replacement with is repeated. 2. For each node of the input layer, a contribution degree representing how much the node contributes to the subsequent layer is calculated, and when all of these contribution degrees are equal to or less than a predetermined threshold value, End learning of the neural network, selection of effective features and replacement with input nodes,
The neural network learning method according to claim 1. 3. The neural network learning method according to claim 1, wherein an input node for inputting one or a plurality of feature quantities to be selected as effective feature quantities is provided in advance in an intermediate layer of the neural network. . 4. For each node in the middle layer of the neural network, a contribution representing the degree to which the node contributes to the subsequent layers is calculated, and the validity is calculated based on this contribution and the similarity. The neural network learning method according to claim 1, wherein a feature amount is selected. 5. A neural network comprising an input layer including a plurality of nodes, a one-layer or a plurality of intermediate layers including a plurality of nodes, and an output layer including the same number of nodes as the input layer, and input teacher data. Learning means for causing the neural network to perform learning so that the output of the neural network when inputting to the neural network becomes equal to the teacher data, the output of the node in the intermediate layer of the neural network, The similarity representing the degree of similarity with the feature quantity created based on the teacher data is calculated, and one feature quantity is validated from the plurality of feature quantities based on these similarity degrees. Input for selecting the feature amount and inputting the effective feature amount of the node in the middle layer corresponding to this effective feature amount The structure changing means for replacing the nodes, and the learning of the neural network using the teacher data and the effective feature quantity are repeatedly performed by the learning means, and the selection of the effective feature quantity and the replacement of the input node are repeatedly performed by the structure changing means. Neural network system equipped with control means to make it.