JP2000035955A - Constitution method for hierarchical neural network - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method capable of automatically deciding optimum network constitution at the time of modeling an object system by using a hierarchical neural network. SOLUTION: Constitution larger than the expected one is provided as an initial state and a network where respective neurons are connected to all the neurons belonging to adjacent layers before and behind is set. Individual connection data are set for the respective neurons, the combination of known input and output data is applied to the network and learning calculation is performed. When the learning calculation is converged, the connection weighting coefficient of the respective neuron connections is checked and the neuron connection for which the absolute value is below a prescribed condition, is removed from the connection data. In the case that the neuron whose the input or the output becomes one piece appears as the result of removing the neuron connection, only the connection relation is maintained and the neuron is removed. The process is repeated up to the convergence. As a result, the optimized network constitution is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for optimizing a network configuration when a target system is modeled using a hierarchical neural network.

[0002]

2. Description of the Related Art Hierarchical neural networks can be used to predict the behavior of various target systems because they can model target systems whose characteristics are unknown. For example, it can be used to predict future stock price changes based on past stock price change data. The advantage of a hierarchical neural network is that it can handle nonlinear phenomena, and if an appropriate combination of input data and output data is prepared, the target system can be automatically modeled based on the learning method. The point is that modeling does not take much time.

However, a method for determining the number of layers of the network used for modeling the target system and the optimum value of the number of neurons for each layer has not been established. For this reason, when the number of layers and the number of neurons of the network are small, the effect of learning is not obtained. On the other hand, when the number of layers and the number of neurons are large, the number of calculations increases and it takes a long time to perform the learning. Outside output may be obtained.

As described above, in the hierarchical neural network, the optimum network configuration (that is, the number of layers and the number of neurons for each layer) suitable for the purpose cannot be known in advance. Determine the network configuration. For this reason, there is a problem that it takes time to determine the optimal network configuration.

Further, in the conventional hierarchical neural network, since the neurons in each layer are connected to all the neurons in the immediately preceding and succeeding layers, the total number of connections is large and redundant calculations are avoided. I can't. In particular, when learning to determine neuro characteristics (threshold of each neuro and weight coefficient characteristics of each neuro combination), calculate the output values of the neuros that are less involved and calculate the neuro characteristics of them. Therefore, there is a problem that a long calculation time is required.

[0006]

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems of the conventional hierarchical neural network, and an object of the present invention is to provide an object system using a hierarchical neural network. An object of the present invention is to provide a hierarchical neural network configuration method capable of automatically determining an optimal network configuration when performing modeling.

[0007]

A method for constructing a hierarchical neural network according to the present invention comprises, as an initial state, a hierarchical neural network having a network configuration in which each neuron is connected to all neuros belonging to adjacent and preceding and succeeding layers. The first step of setting up a neural network, and for each neuro,
A second step of individually setting connection data representing a connection relationship with another neuron, and giving a combination of known input data and output data to the hierarchical neural network,
The third step of performing the learning calculation, and when the result of the learning calculation converges, examines the connection weight coefficient of each neuro connection, removes the neuro connection whose absolute value is below a predetermined condition, and removes the relevant connection data. In the fourth step of correcting and, when a neuro having one input or output appears as a result of removing the neuro connection, the neuro is removed while maintaining only the connection relationship, and the related connection data is removed. A fifth step of correcting, and after correcting the joint data, the processing of performing the third, fourth, and fifth steps is repeated again. It is characterized in that the obtained network configuration is obtained.

[0008] In this specification, "neuro identification number" means data for identifying each neuron, and is usually the number of a layer to which the neuron belongs and the number of the neuron in the layer. Consists of The “connection data” means data representing a mutual connection relationship for each neuro, and is usually composed of neuro identification numbers of other neuros directly connected to the neuro. The “connection weight coefficient” means a connection weight between a threshold value of a neuro representing characteristics of each neuro and a connection neuro.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In a conventional hierarchical neural network, each neuron is configured so as to be connected to all neurons belonging to adjacent adjacent layers. Accordingly, each neuron has its own neuro identification number, threshold value, and connection weighting factor for connection with all neuros belonging to the next layer (or previous layer), but does not have individual connection data. .

On the other hand, in the hierarchical neural network to which the configuration method according to the present invention is applied, each neuron has individual connection data in addition to the above-mentioned neuro identification number, threshold value, and connection weight coefficient. To have As the connection data, only the identification number of a neuro belonging to a layer downstream of the neuro among other neuros directly connected to the neuro is sufficient.

A. In the process of optimizing the network configuration, first, the network configuration (the number of layers and the number of neuros belonging to each layer) is set to be larger than the predicted one, and each neuron is set before and after adjacent neurons. A network is configured so as to be connected to all neuros belonging to the layer, and learning calculation is started from this state.

B. Given a combination of known input data and output data (referred to as "teacher data"), a learning calculation is performed, and when the calculation results converge, a connection weight coefficient of each neuro-connection is examined. As a result, a neural connection in which the absolute value of the connection weight coefficient is lower than a predetermined condition is removed.
That is, the neuro connection is removed from the connection data for each neuro.

C. When a neuro having one input appears as a result of removing the neural connection, the neuro is absorbed and removed by the neuro connected to the preceding stage, and the related connection data is corrected. Similarly,
When a neuro with one output appears, the neuro is absorbed and removed by the neuro connected to the subsequent stage, and the related connection data is corrected.

D. The learning calculation is performed again using the network configuration modified by the processes b and c, and then the processes b and c are performed. By repeating this process, unnecessary neuro-connections and redundant neuros are sequentially removed. As a result, the calculation is terminated when the elimination of the neural connection has converged.

As a result, unnecessary neural connections and redundant neuros are finally sorted out to obtain an optimized network configuration. Next, a method of configuring a hierarchical neural network based on the present invention will be specifically described with reference to the drawings.

First, a conventional hierarchical neural network as shown in FIG. 1 is set as an initial state. The configuration of the network includes an input layer, a middle layer, and an output layer, and a neuro is arranged in each layer. In the initial state,
Each neuron is connected to all neurons belonging to the layers before and after it. The number of neurons in the input and output layers is determined by the purpose of use, but the number of intermediate layers and the number of neuros belonging to each layer in the initial state are set to be larger than the predicted optimum network configuration. Keep it. In this example, the network configuration in the initial state is such that the number of neurons in the input layer is 2, the number of neurons in the output layer is 2, the number of intermediate layers is 2, the number of neurons in the first layer of the intermediate layer is 4, The number of neurons in the layer is 3.

The learning of the network is performed as follows. The input data is provided to the network as input from the teacher data prepared in advance, and the output of the network is calculated. Next, the threshold value of each neuro and the connection weight coefficient of each neuro connection are corrected so that the difference between the calculated output and the output data of the teacher data is reduced. By repeating this learning calculation, finally, the threshold value of each neuro and the connection weight coefficient of each neuro connection are obtained.

FIG. 2 shows an example of parameters used in a formula for calculating the output of each neuron. The neuro has multiple inputs and one output. In this example, the number of neuro inputs is three.
Represents the case of In the figure, S1, S2, and S3 are input values from the respective neuro-couplings in the preceding layer, T is the output value of the neuro, W1, W2, and W3 are the "coupling weight coefficients" of the neuro-couplings, and H is the neuron's weight Threshold values are shown.

In the case of the example shown in FIG. 2, the output T from the neuron is expressed by the following equation using a sigmoid function f (X). Note that the sigmoid function f (X) is obtained by setting the input value X to 0.
Is a function that converts the value into a value from 1 to 1.

T = f ((ΣSi · Wi) −H) where, ΣSi · Wi = S1 · W1 + S2 · W2 + S3 · W3 f (X) = 1 / (1 + exp (−X)) That is, the output value of a certain neuro Is a value obtained by multiplying the input value Si from each input-side neural connection (ie, the output value of each preceding-side neuron) by the connection weighting factor Wi of the neuro-connection, and calculating the value of the connection-layer neuron in the preceding layer. From the sum,
This is the value of the sigmoid function obtained by subtracting the neuron threshold value H.

As the learning progresses, the value of the connection weight coefficient converges to a constant value. As a result, if the value of the connection weighting factor is close to 0, the connection is not evaluated, as can be seen from the above equation, and there is no problem in removing it. The determination as to whether or not the removal is possible is made based on a comparison of the connection weight coefficients of the respective neurons, a comparison of the connection weight coefficients of the respective layers, a comparison of the connection weight coefficients of the entire network, and the like.

FIG. 3 shows an example of the network configuration after a part of the neural connections determined to be unnecessary are removed in this way. In the network configuration shown in FIG. 3, there is one input-side neuro connection, one output-side neuro connection,
One neuron is generated in each of the input and output neuron connections. For a neuro with one input neuro-connection, the neuro is absorbed and removed by the neuro connected to the preceding stage, and the related connection is established for the neuro connected before and after the removed neuro. Modify the data. Similarly,
As for the neuro with one neuro connection on the output side, the neuro is absorbed and removed by the neuro connected to the subsequent stage, and the connection related to the neuro connected before and after the removed neuro is related. Modify the data. When the redundant neuro is removed while leaving only the connection relationship, a network configuration as shown in FIG. 4 is obtained.

Next, learning calculation is further performed by using the network configuration thus modified, and unnecessary neural connections and redundant neurons are sequentially removed. As a result, the calculation is terminated when the removal of the neural connection has converged, that is, when there is no possibility that the neural connection is further removed. As described above, unnecessary neuro-connections and redundant neuros are arranged, and an optimized network configuration is obtained.

[0024]

According to the method for constructing a hierarchical neural network according to the present invention, unnecessary neural connections and redundant neurons can be automatically and sequentially removed by learning, so that the optimal network configuration can be achieved. The labor required for the conversion can be greatly reduced.

The hierarchical neural network optimized in this way has no unnecessary neural connections and redundant neurons, so that the time required for the calculation is short, and especially in the learning process in which many calculations are repeated. The time required is greatly reduced as compared with the conventional case.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a network configuration set as an initial state in a method of configuring a hierarchical neural network according to the present invention.

FIG. 2 is a view for explaining parameters used in a neuro output calculation formula.

FIG. 3 is a diagram showing a state after unnecessary neuro-coupling has been removed from the network configuration shown in FIG. 1 based on the method of the present invention.

FIG. 4 is a diagram showing a state after removing redundant neurons from the network configuration shown in FIG. 3 based on the method of the present invention.

Claims (1)

[Claims] 1. a first step of setting a hierarchical neural network having a network configuration in which each neuron is connected to all neurons belonging to adjacent and preceding and succeeding layers as an initial state; A second step of individually setting connection data representing a connection relationship with another neuro; and a third step of performing learning calculation by giving a combination of known input data and output data to a hierarchical neural network. When the result of the learning calculation converges, the fourth step of examining the connection weighting factor of each neuro connection, removing the neuro connection whose absolute value is below a predetermined condition, and correcting the relevant connection data, If a neuro with only one input or output appears as a result of removing the connection, remove the neuro by maintaining only the connection relationship. And a fifth step of correcting related connection data.After correcting the connection data, the processing of performing the third, fourth, and fifth steps is repeated again, and the elimination of the neuro connection has converged. By the way, a method of constructing a hierarchical neural network, wherein the method is terminated to obtain an optimized network configuration.