JPH0318967A - Learning system for neural net - Google Patents

Abstract

PURPOSE:To evenly recognize each data by learning more number of difficult-to- recognize data than the easy-to-recognize ones in a back propagation learning system. CONSTITUTION:An input layer 2 is prepared together with an intermediate layer 3 and an output layer 4 in a neural net where the learning is carried out by a back propagation learning rule. The recognizing result of each learning data and the value of a unit that ignited most with the output against the relevant learning date are checked before the change of the connection 1. Then the value obtained by multiplying the result and value together is defined as a degree of learning. Thus each learning data is correctly recognized and at the same time the degree of learning and the learning data obtained at that time point are wrong recognized. A function M is set in response to that degree of learning, and the weight of the connection 1 is changed based on the value obtained by a prescribed arithmetic formula. Thus the more number of difficult- to-recognized data are learnt and each data is evenly recognized. Then the convergence of learning is urged.

Description

[Detailed Description of the Invention] [Field of Industrial Application] This invention relates to a learning method in a neural network,
In particular, it relates to a learning method that is effective for recognizing complex data required for pattern recognition.

[Prior art and problems to be solved by the invention] Traditionally, when learning is performed using a backpropagation learning rule in a multilayer perceptron neural network, the more complex the data to be trained, such as pattern recognition, the more difficult the learning becomes. The problem was that it was difficult.

Therefore, the main purpose of this invention is to use the backpropagation learning method to process data that is difficult to recognize.
It is an object of the present invention to provide a learning method in a neural network that can uniformly recognize each piece of data by learning more data than data that is easy to recognize.

[Means for Solving the Problems] The present invention is a learning method for a multilayer perceptron neural network that performs learning using a backpropagation learning rule. 2B Check the value of the most fired unit in the recognition result and the output for the learning data, and consider the value obtained by multiplying them as the learning degree of each data to determine whether each learning data is recognized correctly and the degree of learning degree and A function M is created depending on whether learning data is misrecognized and the degree of learning, and the weight of a connection is changed according to a value calculated by a predetermined arithmetic expression.

[Operation] The learning method in the neural network according to this invention calculates the learning degree of each data before changing the weight of the connection, and changes the weight of the connection according to the value calculated using a predetermined learning formula. By doing this, the system learns more data that is difficult to recognize, allows each piece of data to be recognized uniformly, and promotes convergence of learning.

[Embodiments of the Invention] FIG. 1 is a diagram showing a multilayer perceptron neural network to which the present invention is applied. Referring to FIG. 1, the 221 runnet includes an input layer 2, an intermediate layer 3, and an output layer 4, each connected by a connection 1. Data to be learned is input to the input layer 2, and learning is performed by changing the weight of the connection 1 according to the backpropagation learning rule.

FIG. 2 is a flowchart of the learning method according to the present invention, and FIG. 3 is a diagram showing the convergence state of learning when complex data is learned by the neural network according to the present invention.

Referring to FIG. 2, in the present invention, before learning data is input to input layer 2 and the weight of connection 1 is changed, the learning data is recognized in step (abbreviated as SP in the figure) SPI, In step SP2, the recognition result for each learning data and the value of the unit that fired the most in its output are checked, and the product of these values is calculated as the learning degree for the learning data. Then, in step SP3, it is determined whether the learning data X has been correctly recognized.

Here, for the learning data X, M ( X) = 0, M(
X) = 1, if low, M (X) = k, if X is misrecognized and the learning level of X is low, M (X) = 2k, if medium, M (X) = 4k, if high, M (X) )-6k) Create a function M. Once the function M is completed, learning is performed. In this invention, the conventional backpropagation learning rule is improved and used as follows.

ΔW(n+1)−aΔW(n)−εΣM(j)dE/d
Wj (dE/dWj is the amount of change in connection weight for learning data j) The meaning of this equation is as follows.

In other words, the conventional learning formula ΔW(n+1)−αΔW(n)
-εdE/dW, in order to learn L times more learning data j than other learning data according to the concept of the present invention, the number of learning data j must be increased to L and trained. However, it is difficult to dynamically change the number of learning data during learning, and the calculation time increases. Here, if we multiply the dE/dWj portion of dE/dW by L, additional learning of the learning data j is possible, and the calculation time is almost the same as conventional learning, so dE/dW can be reduced to 2M(j). By changing dE/dWj, it becomes possible to learn the learning data j M(j) times. If the learning data is recognized equally, the values of M(j) will all be equal, and will be the same as the conventional learning formula.

The above operation will be explained with reference to the flow diagram shown in FIG. 2. When it is determined in step SP3 that the learning data X has been correctly recognized, it is determined in step SP4 whether or not X is large, and if it is large, If no data learning is performed in step SP5 and X is large,
In step SP6, it is determined whether or not X is medium, and if it is medium, data learning is performed normally in step SP7, and if
Determine whether or not is small. If X is small, the data will be learned twice as much.

Furthermore, if it is determined in step SP3 that X has been erroneously recognized, it is determined in step 5PIO whether or not X is small, and if X is small, data learning is doubled by 2 in step 5P11, and If not, in step 5P12 it is determined whether or not X is in the middle,
If it is medium, data learning is performed by a factor of 4 in step 5P13. If X is not medium, it is determined in step 5P14 whether or not X is large; if it is, data learning is multiplied by 6 in step 5P15.

By performing the above learning, the conventional method
In contrast to the state 5 to be learned shown in the figure, which gradually converges from the state 6 in the middle of learning, in the learning method according to the present invention, much learning is performed in the part 7 that requires a lot of learning, Part 8, which does not require much learning, can be kept to a minimum level of learning, and convergence to state 5 can be made faster.

[Effects of the Invention] As described above, according to the present invention, complex learning data can be learned quickly by applying a formula that is an improved learning formula on the backpropagation learning side to a multilayer perceptron neural network. I can do it.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing the configuration of a three-layer neural network as an example of a multilayer perceptron neural network. FIG. 2 is a flow diagram of the learning method according to the present invention. FIG. 3 is a diagram showing the convergence state of learning when making a neural network learn complex data. In the figure, l is a connection, 2 is an input layer, 3 is an intermediate layer, 4 is an output layer, 5 is a state to be learned, 6 is a state in the middle of learning, 7 is a part that requires a lot of learning, 8 is a little Indicates parts that do not require learning. Special fraud applicant A.T.R. Co., Ltd. Figure 2 Figure 3

Claims (1)

[Claims] In a learning method for a neural network that performs learning using a backpropagation learning rule in a multilayer perceptron type neural network, before changing connections, the recognition results of each learning data and the output for the learning data are Find out the value of the unit that fires the most, consider the value multiplied by them as the learning degree of each data, and for each learning data X, M(X) = {0, 1, k, 2k, 4k, 6k } (k is a positive constant, if X is correctly recognized and the learning level of X is high, M(X) = 0, if it is medium, M(X) = 1, if it is low, M(X) = k, If X is misrecognized and the learning level of X is low, M(X) = 2k, if it is medium, M(X) = 4k, if it is high, M(X) = 6k). =αΔW(n)−εΣM(j)dE/d
A learning method in a neural network characterized by changing the weight of a connection according to a value calculated by Wj (dE/dWj is the amount of change in the weight of a connection with respect to learning data j).