JPH0318964A - Learning system for neural net - Google Patents

Abstract

PURPOSE:To quickly get out a learning balanced state and to urge the high speed convergence of learning by changing forcibly the weight of connection when the learning balanced state is caused. CONSTITUTION:An input layer 2, an intermediate layer 3, and an output layer 4 are connected to each other via the connections 1. The data to be learnt is inputted to the layer 2 and the weight of the connection 1 is changed by a back propagation learning rule for execution of the learning. As a result, the weight of the connection 1 connected to a specific unit of the layer 4 is extremely reduced. Then the weight of a part connected to the layer 4 of the connection 1 is forcibly eliminated to zero in such a learning balanced state where the learning is difficult for the learning data to be taken change of by an output unit. Then the back propagation learning rule is immediately carried on. Thus a state set before a zero-clear mode is recovered with several tens of learning operations although the recognizing factor of a neural net is temporarily deteriorated. Then the learning balanced state can be immediately eliminated.

Description

[Detailed Description of the Invention] [Field of Industrial Application] This invention relates to a learning method in a neural network,
In particular, it relates to a learning method that is effective for recognizing complex data required for pattern recognition.

[Prior art and problems to be solved by the invention] Conventionally, when learning is performed using a backpropagation learning rule in a multilayer bersebutron type neural network, the data to be trained for pattern recognition etc. is complex. I see, learning becomes difficult. For this reason, escaping from a situation where learning falls into an equilibrium state (learning equilibrium state) without being able to learn specific learning data has become a problem.

Therefore, the main purpose of this invention is to quickly escape from the equilibrium state when a neural network in the middle of learning falls into a learning equilibrium state using the backpropagation learning rule, and to promote rapid convergence of learning. The purpose of this study is to provide a learning method for neural networks.

[Means for Solving the Problems] The learning method in the neural network according to the present invention is such that when learning is performed using a backpropagation learning agent in the learning method in the multilayer perceptron type neural network, due to the learning equilibrium state, When specific training data becomes difficult to learn, the weights of all connections connected to the output layer are cleared to zero, and then learning is continued.

[Operation] The neural network learning method according to the present invention quickly escapes from the learning equilibrium state by forcibly changing the connection weights when the learning equilibrium state occurs in the conventional learning method, and the learning promotes rapid convergence.

[Embodiments of the invention]

FIG. 1 is a diagram showing a multilayer perceptron type neural network to which the present invention is applied. Referring to FIG. 1, the neural network includes an input layer 2, an intermediate layer 3, and an output layer 4, each connected by a connection 1. Data to be learned is input to the input layer 2, and learning is performed by changing the weight of the connection 1 using a backpropagation learning agent.

FIG. 2 shows a state in which learning is performed using the backpropagation learning rule, resulting in a state of learning equilibrium in which specific learning data cannot be recognized, and FIG. FIG. 4 is a diagram showing a state in which the connections connected to the output layer of the neural network have been cleared to zero, and FIG. 4 is a diagram showing that the learning equilibrium state has been escaped.

The weight of connection 1 is changed using the backpropagation learning rule (as a result of learning, as shown in Figure 2, the weight of the connection connected to a specific unit of output layer 4 becomes very small, As a result, suppose that the learning data that the output unit is in charge of falls into a learning type & state where it becomes difficult to learn.In the example shown in Fig. 2, the weight of the connection connected to the specific unit 5 of the output layer 4 is This is an example of a learning equilibrium state in which the unit 5 is unable to fire because all of them are too small.

At this time, the thickness of the line of connection 6 represents the weight of the connection, and a thick connection indicates a deep relationship with the unit to which it is connected.

In this state, the weight of the portion of connection 6 connected to output layer 4 is forcibly set to zero (zero clear). FIG. 3 shows that as a result, the weights of the connections 7 connected to the output layer 4 are all zero. At this time, connection 1 connects input layer 2 and intermediate layer 3.
It has a function of extracting basic features of input data and distributing them to each unit of the intermediate layer 3.

After clearing to zero as described above, immediately continue using the backpropagation learning agent. At this time, the recognition rate of the neural network decreases over time, but after several thousand times of learning, it recovers to the state before zero clearing and exits the learning equilibrium state.

Then, as shown in connection 8 shown in FIG.
Appropriate connection weights are assigned to all units in .

In the conventional method, when the learning equilibrium state as shown in Fig. 2 is reached, learning must be repeated hundreds or thousands of times to escape from the learning equilibrium state, but with this invention, It is possible to escape from the learning equilibrium state after several thousand times of learning.

[Effects of the Invention] As described above, according to the present invention, when a multilayer bersebutron type neural network is trained using the pack-broadcasting learning rule and the learning equilibrium state is reached, specific learning data cannot be learned. , it is possible to escape from the learning equilibrium state by simply clearing the weights of connections connected to the output layer to zero, and it is possible to achieve high-speed convergence in neural network learning.

[Brief explanation of drawings]

FIG. 1 is a diagram showing a three-layer neural network as an example of a multilayer perceptron type neural network. FIG. 2 is a diagram showing a state in which the neural network shown in FIG. 1 is trained using the backpropagation learning rule and as a result falls into a learning equilibrium state and is unable to recognize specific learning data. FIG. 3 is a diagram showing a state in which connections connected to the output layer of a neural network in a learning equilibrium state are cleared to zero according to an embodiment of the present invention. FIG. 4 is a diagram showing escape from the learning equilibrium state. In the figure, 1 is a connection, 2 is an input layer, 3 is an intermediate layer, 4 is an output layer, 5 is an output layer of a neural network in a learning equilibrium state, 6 is a connection weight of a neural network in a learning equilibrium state, and 7 is the weight of the connection immediately after applying this invention, and 8 is the weight of the connection of the neural network that has escaped the learning equilibrium state.

Claims (1)

[Claims] In the learning method of a multilayer perceptron neural network, when learning is performed using the backpropagation learning rule, when specific learning data falls into a state where it is difficult to learn due to the learning equilibrium state, it is difficult to connect to the output layer. A learning method for neural networks that is characterized by continuing learning after the weights of all connections are cleared to zero.