JP3389684B2 - Learning method of neural network - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hierarchical neural network, and more particularly to an output layer configuration for pattern recognition and a learning process by back propagation. 2. Description of the Related Art Hierarchical neural networks, for example, perform a process of converting speech data in speech recognition into a phoneme sequence, and use the result to perform DTW (Dynamic Programming, D
It is used in a discrete word recognition system that performs word identification by the P matching method. FIG. 4 shows an example of a conventional hierarchical neural network. A unit (neuron) is separated into an input layer S, an intermediate layer A, and an output layer R, each of which is independent of three layers.
The input layer S is composed of a number of neuron models, and converts numerical information of an input pattern to be identified into each unit and converts it into a neural signal. [0004] The intermediate layer A is composed of a large number of neuron models and is connected to all units of the input layer S with a connection weight W. [0005] The output layer R is composed of a number of neuron models corresponding to the categories to be identified on a one-to-one basis, and is connected to all the units of the intermediate layer A with a connection weight. In this configuration, in order to obtain a correct answer corresponding to the input pattern, the output pattern when the input pattern is given is a teacher signal (expected output).
A pattern conversion process for changing each connection weight of the network so as to coincide with is performed. In this pattern conversion processing, an error between the output pattern and the teacher signal is obtained, the error is propagated in the reverse direction from the output layer to the input layer, and the load between the units is adjusted by an amount corresponding to the error. The back-propagation type learning is performed in which an input pattern is given again and an error between the output pattern and the teacher signal is obtained. [0008] In the conventional neural network, when dealing with practical problems, when the boundaries of data between categories are ambiguous,
In many cases, it is difficult to represent one category by one unit, for example, when the characteristics of the data in the category vary greatly. Therefore, the recognition rate decreases as the recognition target such as phoneme identification becomes more complicated. An object of the present invention is to provide a neural network learning method for improving the ability to recognize a complex recognition target. [0010] Means for Solving the Problems The present invention, in order to solve the above problems, comprises an input layer and the intermediate layer and output layer, wherein
Output layer consists of multiple units for one recognition category
Is provided, and the coupling load stored in the storage device is provided between each layer.
Layered Neural Network Connected with Weight
Based on the rules of back propagation
Determine the correction amount of the connection weight from the load and teacher data
The connection weight stored in the storage device can be rewritten.
Learning method of neural network trained by
A procedure for checking whether the teacher data to be set to a certain recognition category X is 1 or not, and if the teacher data is not 1 in the above check, the teacher data to be given to the unit of the recognition category X is set to 0 A procedure for checking whether the teacher data is a unit having the maximum output among the units of one recognition category, a procedure for using the output as it is as the teacher data for a unit having a non-maximum output, 1 unit of teacher data
And having a procedure to. In a multiplexed output layer configuration in which a plurality of units are prepared for each category to be recognized, back propagation is performed.
The teacher data given at the time of the expression learning corresponds to the input pattern.
Teaching to each unit of the category that does not give a correct answer
The master data is set to 0, and the answer is correct for the input pattern.
Of the units in the category, the unit with the largest value
Only the teacher data, and the output to other units.
It is given as teacher data as it is. This enables learning with clear boundaries between categories. FIG. 1 is a diagram of a hierarchical network configuration showing an embodiment of the present invention. The difference between the network of the present embodiment and the conventional hierarchical neural network is that in the conventional configuration, each unit of the output layer corresponds to the category to be recognized on a one-to-one basis, whereas in the present embodiment, each category A, A plurality (three in the figure) of units are prepared for each of B and C. In these units, the category belonging to the unit having the largest output value is used as the recognition result in the same manner as in the prior art. In the configuration of the present embodiment, the learning of the neural network is basically performed by the back propagation method. However, an error is generated in the multiplexed output unit so that the unit identification expression is easily differentiated. Is an algorithm shown in FIG. In FIG. 2, broken line blocks indicate teacher data change processing for each of the categories A, B, and C. It is checked whether or not the teacher data to be set to a certain category X is 1 (step S1), and if it is not 1, the teacher data given to the unit of the category X is set to 0 (step S2). For the category where the teacher data is 1,
It is checked whether or not the unit of the category has the maximum output (step S3), and the output of the unit having the non-maximum output is directly used as the teacher data (step S4). The teacher data 1 is given to the unit having the maximum output in the category (step S5). That is, no learning is performed. The process of changing the teacher data is performed for all categories A, B, and C. FIG. 3 shows a modification example of the teacher data, in which the output values of the categories A, B, and C become the illustrated values, and 1 is given to the category A and 0 is given to the other categories B and C as the teacher data. Show the case. Since the teacher data is 0 for the categories B and C, 0 is given to the teacher data for these categories from the paths of steps S1 and S2. Since the teacher data is 1 in the category A, the unit which does not have the maximum output among the three units has the output 0.1 and 0.5 as the teacher data (steps S3 and S4), One unit of teacher data is given to the unit to be output (steps S3 and S5). Returning to FIG. 2, after completing the process of changing the teacher data given to each unit of each category,
Learning is performed by using a normal back propagation learning algorithm using these teacher data (step S
6). Therefore, according to the present embodiment, a multi-unit configuration in which a category is composed of a plurality of units is set. For learning other than the input category, all the teacher data of the unit are set to 0, and Sets the teacher data of the unit having the maximum value to 1, and sets the output values of the other units as the teacher data. In this way, even in a case where the boundary between one category and another category is ambiguous and identification is difficult in the past, learning can proceed smoothly by multiplexing output units, and good pattern identification can be expected. . As an experimental example based on this embodiment, phoneme identification was performed from speech data using a neural network, and word identification was performed with the DTW using the phoneme identification results. As a result, the phoneme identification ability and word identification ability were also improved. The neural net used in this experiment is
The output layer has 23 units (23 × 2 units in the multiplex output), and the recognition results are shown in Table 1 below in comparison with those of the conventional type. [Table 1] From this table, it can be seen that for the 14th category of phoneme data which the conventional neural network misrecognizes,
In the multiple output type according to the present invention, a correct result is obtained. As described above, according to the present invention, a multiplexed output layer configuration in which a plurality of units are prepared for each category to be recognized is used for back propagation type learning.
If the teacher data given is correct for the input pattern
The teacher data given to each unit of the category without
Of each category that is correct for the input pattern.
Of the knits, only the unit with the maximum value is teacher data
Output to other units as is for teacher data.
Since it is provided as a data , it is possible to perform learning in which boundaries between categories are clarified, and there is an effect that good pattern identification can be performed even for a complicated identification target.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration diagram of a neural network showing one embodiment of the present invention. FIG. 2 is an algorithm for calculating an error in the embodiment. FIG. 3 is an example of how to give teacher data in the embodiment. FIG. 4 is a configuration diagram of a conventional hierarchical neural network. [Description of Signs] S: input layer A: intermediate layer R: output layer

Continued on the front page (58) Fields investigated (Int.Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7/60 G06K 9/62-9/72 G06T 1/40 G06T 7/00-7 / 60 G10L 15/00-17/00 JST file (JOIS) CSDB (Japan Patent Office) INSPEC (DIALOG)

Claims (1)

(57) [Claims 1 further comprising an input layer and the intermediate layer and output layer, wherein
Output layer consists of multiple units for one recognition category
Is provided, and the coupling load stored in the storage device is provided between each layer.
Layered Neural Network Connected with Weight
Based on the rules of back propagation
Determine the correction amount of the connection weight from the load and teacher data
The connection weight stored in the storage device can be rewritten.
Learning method of neural network trained by
A procedure for checking whether or not the teacher data to be set to a certain recognition category X is 1. If the teacher data is not 1 in the above check, the teacher data to be given to the unit of the recognition category X is set to 0 A procedure to check whether the teacher data is a unit having the maximum output among the units of the recognition category of one, a procedure to use the output as it is as the teacher data for a unit which is not the maximum output, the neural network is in the unit, characterized in that it comprises a procedure for the teacher data to 1
Learning method .