JPH0769894B2 - Learning method in neural network - Google Patents

Description

The present invention relates to a learning method in a neural network,
In particular, it relates to a learning method effective for recognizing a plurality of data required for pattern recognition.

[Problems to be Solved by Conventional Techniques and Inventions] Conventionally, when learning is performed using a backpropagation learning rule in a multilayer perceptron type neural network, data to be learned such as pattern recognition is complicated. Indeed, learning becomes difficult. For this reason,
Escape from the situation where learning falls into an equilibrium state (learning equilibrium state) without learning specific learning data has been a problem.

Therefore, a main object of the present invention is to allow a neural network in the middle of learning in a back-propagation learning rule to quickly escape from the equilibrium state when the learning net state falls into the learning equilibrium state, and to accelerate the rapid convergence of learning. It is to provide a learning method in a simple neural network.

[Means for Solving the Problem] The present invention is a learning method in a multi-layer perceptron type neural network that performs learning by changing the weight of the connection connecting each of a plurality of input layers and a plurality of output layers. When learning is performed using the gation learning rule, the weight of the connection connected to one of the output layers becomes small, and it becomes difficult to learn the learning data that the corresponding output layer should be in charge of. When the learning equilibrium state is reached, the weights of all the connections connected to the output layer are cleared to zero, and then the learning is continued.

[Operation] In the learning method of the neural network according to the present invention, when the learning equilibrium state is encountered by the conventional learning method, the weight of the connection is forcibly changed to quickly escape the learning equilibrium state and perform learning. Promote fast convergence of.

[Embodiment of the Invention] FIG. 1 is a diagram showing a multilayer perceptron type neural network to which the present invention is applied. Referring to FIG. 1, the neural network includes an input layer 2, an intermediate layer 3 and an output layer 4, which are connected by a connection 1. Data to be learned is input to the input layer 2, and learning is performed by changing the weight of the connection 1 according to the back propagation learning rule.

FIG. 2 shows a state in which a learning equilibrium state is caused as a result of learning by the back-propagation learning rule, and specific learning data cannot be recognized. FIG. 3 shows a learning equilibrium state according to an embodiment of the present invention. FIG. 4 is a diagram showing a state in which the connections connected to the output layer of the neural network are zero-cleared, and FIG. 4 is a diagram showing that the learning equilibrium state is exited.

As a result of performing the learning in which the weight of the collection 1 is changed according to the back propagation learning rule, the weight of the connection connected to a specific unit of the output layer 4 becomes very small as shown in FIG. As a result, it is assumed that the learning equilibrium state in which the learning data to be handled by the output unit becomes difficult to learn. The example shown in FIG. 2 is an example of a learning equilibrium state in which the weights of the connectors connected to the specific unit 5 of the output layer 4 are all too small, so that the unit 5 is in a state where it cannot fire. At this time, the thickness of the line of the connection 6 represents the weight of the connection, and the thicker the connection, the deeper the relationship with the unit to be connected.

In this state, the weight of the portion of the connection 6 connected to the output layer 4 is forcibly set to zero (zero clear). As a result, FIG. 3 shows that the weights of the connections 7 connected to the output layer 4 are all zero. At this time, the connection 1 is the input layer 2 and the intermediate layer 3
Has a non-zero weight, and has a function of extracting the basic feature of the input data and distributing it to each unit of the intermediate layer 3.

After zero-clearing as described above, the backpropagation learning rule continues immediately. At this time, the recognition rate of the neural network temporarily decreases, but the learning equilibrium state is left as it is by recovering to the state before the zero clear by the learning of several tens of times. Then, like the connection 8 shown in FIG. 4, all the units in the output layer 4 are assigned appropriate connection weights.

In the conventional method, when the learning equilibrium state as shown in FIG. 2 is encountered, the learning equilibrium state must be escaped by repeating hundreds or thousands of times of learning. In the present invention, it is possible to escape from the learning equilibrium state by learning several tens of times.

[Effects of the Invention] As described above, according to the present invention, when a multilayer perceptron type neural network is trained on the back propagation learning side and falls into a learning equilibrium state, and specific learning data cannot be learned, The learning equilibrium state can be escaped by a simple operation of clearing the weight of the connection connected to the output layer to zero, and fast convergence in learning of the neural network can be performed.

[Brief description of drawings]

FIG. 1 is a diagram showing a three-layer neural network as an example of the multilayer perceptron type neural network. FIG. 2 is a diagram showing a state in which specific learning data cannot be recognized because the neural network shown in FIG. 1 has fallen into a learning equilibrium state as a result of learning by the back propagation learning rule. FIG. 3 is a diagram showing a state in which the connections connected to the output layer of the neural network in the learning equilibrium state are zero-cleared according to the embodiment of the present invention. FIG. 4 is a diagram showing that the learning equilibrium state is exited. In the figure, 1 is a connection, 2 is an input layer, 3 is an intermediate layer, 4 is an output layer, 5 is an output layer of a neural network in a learning equilibrium state, 6 is a connection weight of the neural network in a learning equilibrium state, 7 Indicates the weight of the connection immediately after the application of the present invention, and 8 indicates the weight of the connection of the neural network that has left the learning equilibrium state.

Claims (1)

[Claims] 1. A learning method in a multi-layer perceptron type neural network for performing learning by changing connection weights respectively connecting a plurality of input layers and a plurality of output layers, wherein learning is performed using the back propagation learning rule. In this case, when the weight of the connection connected to any of the output layers becomes small and the learning data to be handled by the corresponding output layer becomes difficult to learn, the learning equilibrium state occurs. A learning method in a neural network, wherein after the weights of all the connections connected to the output layer are zero-cleared, the learning is continued.