JP2699622B2 - Pattern learning device - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a learning device required for pattern recognition.

[Conventional technology]

In the conventional learning in pattern recognition, a large number of actual data are sampled in advance, the data are added with correct category names, stored, and used for learning. When linear analysis such as discriminant analysis, factor analysis, or multiple regression analysis is used, a linear association between an input pattern and an output signal is statistically obtained. In learning in a multilayer neural network, a pattern is input, and its ideal output is used as a teacher signal, as described in the literature (“An Introduction to Computing with Neural Nets”).
by RPLippmann: IEEE.ASSP: April 1987, 00.4-22), and sequentially learns a nonlinear correspondence between an input pattern and an output signal.

[Problems to be solved by the invention]

In the learning method described above, a category name is given to a pattern sampled in advance, and the association between them is learned. As for the learning patterns, a sufficiently large number of patterns must be sampled so that a pattern set serving as a population can be expressed well. To sample the pattern to be actually recognized enough to sufficiently represent the population, it is necessary to predict the deformation of the pattern to appear and to sample such a deformed pattern. It costs. However, it is impossible to use all patterns to be recognized for learning, and it is even more difficult to collect as many learning sample patterns as can sufficiently express a population. In addition, there is no guarantee that an appropriate number of sample patterns used for learning express the population well, and in particular, there are few patterns near category boundaries, and the learned category boundaries become ambiguous. Also,
It is difficult to sample a reject pattern from actual data, and it is not possible to sufficiently learn a part to be rejected.

An object of the present invention is to provide a pattern learning device that solves these problems.

[Means for solving the problem]

A pattern learning device obtained by the first invention comprises: a multilayer neural network calculation unit including an input layer, an intermediate layer, and an output layer for learning by a back propagation method; a learning pattern storage unit for storing a pattern to be learned; Inputting the memory pattern to the input layer of the multilayer neural network,
Means for controlling learning of the weight value of the multilayer signal network by inputting the category name as a teacher signal; inputting the learning pattern to the input layer of the multilayer neural network; inputting the category name for the purpose of transformation; Means for controlling a transformation process for transforming an input pattern so as to belong to a target category by a multi-layer neural network;
Means for inputting a correct category name for each generated pattern; means for adding the input correct category name for each generated pattern, and registering the generated pattern in a learning pattern storage unit Means for controlling repetition of pattern generation and learning; and 4.

A pattern learning device obtained by the second invention includes a multilayer neural network calculation unit including an input layer, an intermediate layer, and an output layer for learning by a back propagation method, and a learning pattern storage unit for storing a pattern to be learned. Input the memory pattern to the input layer of the multilayer neural network,
Means for controlling the learning of the weight value of the multilayer signal network by inputting the category name as a teacher signal; and inputting the learning pattern to the input layer of the multilayer neural network.
If the output signal is correct, a category name similar to the previously stored result category is determined as a category name for the purpose of transformation, and if the output signal is incorrect, the category name obtained from the output signal is determined. Means for defining a transformation target category name, means for controlling a transformation process for transforming an input pattern by a multilayer neural network so as to belong to the decided transformation target category, and displaying a pattern in the middle of the transformation and a result generated. do it,
Means for inputting a correct category name for each generated pattern; means for adding the input correct category name for each generated pattern, and registering the generated pattern in a learning pattern storage unit And means for controlling repetition of pattern generation and learning.

The pattern learning device obtained by the third invention comprises: a multi-layer neural network calculating unit including an input layer, an intermediate layer, and an output layer for learning by the back propagation method; a learning pattern storage unit for storing a pattern to be learned; Inputting the memory pattern to the input layer of the multilayer neural network,
Means for controlling the learning of the weight value of the multilayer signal network by inputting the category name as a teacher signal; and inputting the learning pattern to the input layer of the multilayer neural network.
Only when the output signal is incorrect, means for determining both the name of the category similar to the result category stored in advance and the category name obtained from the output signal as the category name for the purpose of transformation, Means for controlling a transformation process for transforming an input pattern by a multilayer neural network so as to belong to the transformation target category; and displaying a pattern in the middle of the transformation and a result generated,
Means for inputting a correct category name for each generated pattern; means for adding the input correct category name for each generated pattern, and registering the generated pattern in a learning pattern storage unit And means for controlling repetition of pattern generation and learning.

[Action]

In the first invention, an initial learning pattern is learned by a multi-layer neural network, and the multi-layer neural network is configured to belong to an input category by using the learning pattern or a modified learning pattern by noise superimposition or the like as an initial pattern. Use to deform. This process is schematically represented in FIG. 4, where an initial pattern 401 belonging to category 1 (boundary indicated by 409) is given, and the pattern is changed to belong to category 2 (boundary indicated by 410). When gradually deformed, patterns 402, 403, 404, 405, 40
It changes to 6,407 and stops when the pattern 408 is finally reached. A category name or a reject signal is given to the patterns 401 to 408 in the process, and registered as a learning pattern. Thereby, a pattern or a reject pattern near the boundary can be generated. By learning the pattern thus generated and registered and repeating the above procedure, the distribution of the pattern near the boundary of the category can be learned.

The above principle will be described below. Here, as shown in FIG. 5, the multilayer neural network will be described as being composed of a total of three layers: an input layer 501, an intermediate layer 502, and an output layer 503, but this is not an essential problem. In the case of two layers, even in the case of four or more layers, it can be easily expanded. First, the learning will be described with reference to FIG. In the learning, a back propagation learning method is used. Here, a typical backpropagation learning method will be described, but any learning method for a multilayer neural network may be used.

Learning is performed by repeatedly providing an input pattern and a teacher signal. A case where one input pattern and one teacher signal are provided will be described. A weight matrix to be connected to each unit is initialized in advance.

First, a feedforward process for calculating an output signal from an input pattern will be described. The input pattern is input to the input value 501 as a vector value. The input vector value is set to o _i ⁰ (1 ≦ i ≦ N ₀ ), and the first layer weight matrix is set to w
_ij 1 (1 ≦ i ≦ N ₀ , 1 ≦ i ≦ N ₁ ), and the value of the intermediate layer 502 is represented by o ′ _j ¹
If (1 ≦ j ≦ N ₁ ), the intermediate layer 502 Perform the following calculation. Here, θ _j ¹ is a bias value. Results of ^{_{o j 1 (1 ≦ j ≦}} N 1) each, o seek ^{_{j 1 'j 1 = {1}} + tanh (o j 1)} / 2 (2) calculation o by comprising', and the output of the intermediate layer 502 I do.

The weight matrix coupled from the hidden layer 502 to the output layer 503 is
w _jk ² (1 ≦ j ≦ N ₁ , 1 ≦ k ≦ N ₂ ) and the bias value is θ _k ²
Then, with the output value 503, Perform the following calculation. Results of ^{_{o k 2 (1 ≦ k ≦}} N 2) , respectively ^{_{o 'k 2 = {1 +}} tanh (o k 2)} / 2 (4) comprising computing the output signal o of the output layer 503' obtains the _k ^2.
It is assumed that the elements of the output signal are associated with the category to be recognized.

Next, the backpropagation processing for correcting the weight matrix will be described.

A teacher signal 504, which is an ideal value, is input to the output signal obtained by the above-described feedforward processing, and the value t _k (1 ≦ k ≦ N ₂ ) and the output signal o ′ of the output layer 503 are input. _k ² and an error calculation (block 505), and an error signal d ′ _k ² is calculated as d ′ _k ² = t _k −o ′ _k ² (1 ≦ k ≦ N ₂ ) (5). 'evaluates the back propagation error from _k ² Prefecture, d _k ² = d' output signal ^{_{o k 2 × o 'k 2}} × (1-o' k 2) (6) becomes d _k ² by calculation executing the formula (1 ≦ k ≦ N ₂ ) is obtained.

In block 506, w _jk ² = w _jk ² + a × d _k ² × o ′ _j ¹ (7) θ _j ² = θ _j ² + b × d _k from the error evaluation value d _k ² obtained by the equation (6). ² (8) The second layer weight matrix is corrected by the following calculation. Here, a and b are positive value coefficients.

The d _k ² obtained by the equation (6) is calculated in block 507 as follows: After calculating d _j ¹ = d ′ _j ¹ × o ′ _j ¹ × (1−o ′ _j ¹ ) (10), d _j ¹ (1 ≦ j ≦ N ₁ ) is obtained. the ^{_{w ij 1 = w ij 1 +}} a × d j 1 × o i 0 (11) θ i 1 = θ i 1 + b × d j 1 (12) becomes equation, corrects the first Soomomi matrix in block 508.

By repeatedly performing the above-described equations (1) to (12) for each learning pattern, learning of the multilayer neural network can be executed.

Next, pattern recognition processing and pattern deformation processing are performed using the multilayer neural network learned by the above procedure.
A pattern used for learning, a pattern obtained by slightly modifying the learning pattern by affine transformation or the like, a pattern obtained by superimposing random noise on the learning pattern, or a pattern other than the learning pattern is input to the input layer 501 of the multilayer neural network. An output signal of the output layer 503 is obtained by sequentially performing calculations according to Expressions (1), (2), (3), and (4). The category corresponding to the signal having the maximum value among the elements is set as the recognition result. If the user wants to change the pattern by displaying the input pattern and the recognition result on the input layer 501, he or she inputs a category name for the purpose of changing the pattern.

A method of changing the pattern will be described with reference to FIG. A teacher signal 504 that inputs an initial pattern o _i ⁰ (1 ≦ i ≦ N ₀ ) to the input layer 501 and indicates a target category name to be changed
(T _k : 1 ≦ k ≦ N ₂ ) is input. The input pattern is a block 50 from the signal o ′ _k ³ of the output layer 503 and the teacher signal 504 calculated according to the processing shown in the above-described equations (1) to (4).
In the error calculation in 5, the error between the output signal and the teacher signal is obtained by the calculation shown in Expression (5).
By performing the calculations shown in (9) and (10), the error is back-propagated from the output layer, and the error signal d _j ¹ (1
≦ j ≦ N ₁ ) is evaluated. In block 509, an error value d _i ⁰ when this error signal is propagated to the input layer is Is added to the pattern held in the input layer 501, and the pattern p _i (1 ≦ i ≦ N ₀ ) of the input layer is corrected by the calculation of p _i = o _i ⁰ + c × d _i ⁰ (14) I do. Here, c is a positive coefficient and is a value of about 0.2, but is not particularly limited to this value. Pattern p _i
Is set to o _i ⁰ , and the signal of the output layer 503 is obtained again through the intermediate layer 502. After calculating the error from this output signal and the teacher signal 504, The error amount D is obtained according to the following calculation.

Comparing the error amount D and the threshold value, and terminates the correction processing of the input layer pattern if towards the error amount D is small, the deformation results pattern p _i at that time. If the error amount is larger than the threshold value, it is determined that the pattern deformation has not been completed, and the error signal d ′ _k ² is back-propagated in the same manner as the above-described processing, and the input layer 501 is corrected. This is repeated until the error amount D becomes equal to or less than the threshold value, and a pattern deformed to finally belong to the category represented by the given teacher signal is obtained.

Thereby, as shown in FIG. 4, the initial pattern is gradually deformed so as to belong to the given category. In this process, the category name or the reject signal is added to the pattern near the boundary of the category, and the pattern is added to the learning pattern set. After repeating this process a predetermined number of times, the learning pattern generation processing is terminated, the pattern learning is performed using the above-described equations (1) to (12), and the pattern deformation and the learning pattern generation processing are performed. Perform as described above. As a condition for terminating the process of generating the learning pattern, there is a criterion for determining whether or not a similar deformation pattern can be obtained with a certain probability or more, but this criterion is not an essential problem and may be any. This process is repeated a predetermined number of times of the learning and the generation process of the learning pattern, or when the probability of erroneous recognition becomes equal to or less than the threshold value at the stage of the generation process of the learning pattern, the process is terminated.

In the second invention, an initial learning pattern is learned by a multilayer neural network in the same procedure as in the first invention, and a learning pattern or a modified learning pattern is input to the multilayer neural network as an initial pattern. Recognition processing is performed on the input initial pattern according to equations (1) to (4), and whether or not the result is an error is input. If the result is correct, a category name having a similar shape is previously determined as a pair by clustering, and the category name paired with the category name of the recognition result is set as the category name for the purpose of transformation. If the recognition result is incorrect, the category name that is incorrectly recognized is used as the category name for the purpose of transformation. afterwards,
According to the same procedure as that of the first invention, the initial pattern is deformed by using a multilayer neural network so that the initial pattern belongs to a category. The pattern in the process is given a category name or a reject signal and registered as a learning pattern. Thereby, a pattern or a reject pattern near the boundary can be generated. By repeating the procedure for learning the generated and registered pattern, the distribution of the pattern near the boundary of the category can be learned.

As shown in FIG. 4, the initial pattern 401 is a category 1
If it is inside the boundary 409 and is correctly recognized, the initial pattern 401 is deformed by giving a different category name that is close to it, and the pattern is gradually deformed and rejected across the boundary 409 of category 1. Pattern 404,405
Then, across the boundary 410 of category 2, the pattern is transformed into patterns 406, 407, 408 belonging to category 2. A reject pattern near the category boundary can be generated and added to the learning pattern.

Also, as shown in FIG. 6, when the initial pattern 604 belonging to category 1 is erroneously recognized as belonging to category 2, it is considered that the boundary of category 2 has been learned like the dotted boundary 602. Can be Therefore, when the pattern of category 1 is erroneously recognized as belonging to category 2,
By transforming into a pattern belonging to category 2, the initial pattern 604 traverses the boundary 601 of the true category 1 to become reject patterns 605 and 606, and further traverses the true boundary 603 of the category 2 to become the category 2. It is transformed into the belonging patterns 607,608. Thereby, a reject pattern near the category boundary can be generated and added to the learning pattern.

In the third invention, an initial learning pattern is learned by a multilayer neural network in the same procedure as in the first invention, and a learning pattern or a modified learning pattern is input to the multilayer neural network as an initial pattern. Recognition processing is performed on the input initial pattern according to equations (1) to (4), and whether the result is correct or incorrect is input. If the result is correct, it is considered that the vicinity has been correctly learned, and the pattern deformation processing is not performed. If the recognition result is incorrect, both the misrecognized category name and the correct category name are used as the category names for the transformation purpose so as to belong to the category for which the initial pattern is determined by the same procedure as in the first invention. Deform using a multilayer neural network. The pattern in the process is given a category name or a reject signal and registered as a learning pattern. Thereby, a pattern or a reject pattern near the boundary can be generated. By learning the pattern thus generated and registered and repeating the above procedure, the distribution of the pattern near the boundary of the category can be learned.

〔Example〕

FIGS. 1, 7, 8, and 9 show an embodiment of the first invention.
This will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a pattern learning apparatus according to the first invention, FIG. 7 is a diagram showing an example of a multilayer neural network calculation unit 103 in FIG. 1, and FIG. FIG. 9 is a diagram showing an example of the learning control unit 112 in FIG. 9, and FIG. 9 is a diagram showing an example of the entire learning control unit 111 in FIG.

The initial learning pattern is stored in the learning pattern storage unit 106. Initially, the learning control unit 112 sends a learning signal from the learning signal generation unit 806 to the multilayer neural network calculation unit 103, the pattern selector 104, the teacher category name selector 107, and the pattern change unit 105 via the terminal 807, and the learning is performed. Done. The pattern change unit 105 receives the learning signal from the learning control unit 112 and sends the pattern to the pattern selector 104 without changing the pattern sent from the learning pattern storage unit 106. The pattern selector 104 receives the learning signal, cuts off the signal from the multilayer neural network calculation unit 103, and sends the pattern received from the pattern change unit 105 to the pattern storage unit 102. The teacher category name selector 107 receives the learning signal, cuts off the signal from the category name input unit 114, and outputs the category name from the learning pattern storage unit 106 to the teacher signal generation unit 108.
Send to Multi-layer neural network calculator when receiving a learning signal
The learning operation of 103 will be described with reference to FIG.

The learning signal input from the terminal 716 is sent to the first layer weight matrix correction unit 713, the second layer weight matrix correction unit 712, and the input pattern correction unit 714, and starts the weight matrix correction and stops the correction of the input pattern. I do.

The vector value input through the terminal 701 is represented by o _i ⁰ (1 ≦ i
≦ N ₀ ), the matrix value held in the first layer weight matrix storage unit 711 is w _ij 1 (1 ≦ i ≦ N ₀ , 1 ≦ j ≦ N ₁ ), and the vector held in the intermediate layer storage unit 704 The value is o ' _j ¹ (1 ≦
Assuming that j ≦ N ₁ ), the matrix product calculation unit 702 executes the calculation according to the equation (1). Here, θ _j ¹ is a bias value, which is stored in the first-layer weight matrix storage unit 711. The resulting o _j ¹ (1 ≦ j ≦ N ₁ ) is stored in the intermediate layer storage unit 704 as o ′ _j ¹ by the calculation according to the equation (2) in the S function unit 703.

The weight matrix stored in the second-layer weight matrix storage unit 710 is w _jk ² (1 ≦ j ≦ N ₁ , 1 ≦ k ≦ N ₂ ), and the bias values stored in the storage unit 710 are θ _k ² Then, the matrix product calculation unit 705 executes settlement in accordance with Expression (3). Results of ^{_{o k 2 (1 ≦ k ≦}} N 2) each of which obtains a o _'k ² by executing a calculation in accordance with Equation (4) In the S function unit 706, is output from the terminal 707 as an output signal.

An error of the signal output from the terminal 707 is calculated by an error calculator 109 by a calculation according to equation (5) with the teacher signal sent from the teacher signal generator 108, and the error is calculated by the terminal 70.
8 to the second-layer weight matrix correction unit 712 in the multi-layer neural circuit calculation unit 103, and calculates d _k ²
After obtaining (1 ≦ k ≦ N ₂ ), the second-layer weight matrix is corrected by the calculation of Expressions (7) and (8), and the result is stored in the second-layer weight matrix storage unit 710 again. The d _k ² obtained by the calculation for executing the equation (6) in the second-layer weight matrix correcting section 712 is sent to the first-layer weight matrix correcting section 713, and the equations (9) and (10) are calculated. Then, the first layer weight matrix is corrected by calculation for executing equations (11) and (12), and the result is stored in the first layer weight matrix storage unit 710 again.

The error calculated by the error calculation unit 109 is sent to the learning control unit 112 through the terminal 801. The error amount calculation unit 802 calculates the error amount by the calculation according to the equation (15), The comparator 804 compares the learning threshold value in the storage unit 803 with the learning threshold value. If the threshold value is smaller than the threshold value, the counter 811 counts the number and sends the result to the learning signal generation unit 806. The above learning process is repeated for all the learning patterns, and the contents of the counter 811 are stored in the learning pattern storage unit 106.
If the number of patterns matches the number of all stored patterns, the learning is terminated, and a learning process generation signal is sent from the learning signal generation unit 806 through the terminal 807. If the number of the counters 811 is smaller than the total number of learning patterns despite the completion of the learning of all the learning patterns, the counter 811 is reset and learning of the learning patterns is continued again.

The transformation start signal is sent to the pattern
05, pattern selector 104, teacher category selector 107,
When the pattern is sent to the multilayer neural network calculation unit 103, a pattern randomly selected from the learning pattern storage unit 106 is sent to the pattern change unit 105, and random noise is superimposed on the pattern.
Sent to 02. At this time, the pattern selector 104 cuts off the signal from the multilayer neural network 103 in response to the deformation processing start signal. The pattern input to the pattern storage unit 102 is displayed on the pattern display unit 101, and prompts the user to input a category name for the purpose of transformation. The transformation process start signal is sent to the teacher category name selector 107 at the same time, interrupts the signal from the learning pattern storage unit 106, and sends the category name input from the category name input unit 114 to the teacher signal generation unit 108.

The multilayer neural network calculation unit 103 receives the deformation processing start signal input from the terminal 716 and stops the weight matrix correction processing in the first layer weight matrix correction unit 713 and the second layer weight matrix correction unit 712. . Further, the input pattern correction unit 714 is activated. Terminal 70 from pattern storage unit 102
The pattern input through 1 is the matrix product
02, S function section 703, intermediate layer storage section 704, matrix product operation section 70
5. The S function unit 706 obtains the output signal o ′ _k ³ according to the processing shown in the above equations (1) to (4), and transfers the output signal o ′ _k ³ from the terminal 707 to the error calculation unit 109. Further, an error between the output signal and the teacher signal is obtained by the calculation shown in Expression (5) in the error calculation unit 109, sent to the second layer weight matrix correction unit 712 again via the terminal 708, and according to Expression (6). Is calculated, and the result is calculated by the first-layer weight matrix correction unit.
Sent to the 713, the error is back-propagated from the output layer by performing the calculations shown in equations (9) and (10), and the error signal d _j ¹ (1 ≦ j ≦ N ₁ ) in the intermediate layer Is evaluated. Sends the error signal to the pattern modifying unit 714, where an error value d _i ⁰ when the error signal is propagated to the input layer were evaluated by the equation (13), the input according to the equation (14), an error value from the terminal 701 A pattern p _i (1 ≦ i ≦ N ₀ ) is obtained in addition to the pattern to be transmitted, and is sent to the pattern selector 104 through the terminal 715.

The output signal from the terminal 707 of the multilayer neural network calculation unit 103 and the signal from the teacher signal generation unit 108 are sent to the error calculation unit 109, and the error vector obtained according to equation (5) is
8 and to the learning control unit 112 at the same time as sending to the multilayer neural network calculation unit 103. From the error vector input from the terminal 801, an error amount D is obtained by an error amount calculation unit 802 according to equation (15), and is sent to a comparator 804. The error amount D is compared with the deformation processing threshold value in the threshold value storage unit 803. If the error amount D is smaller, a deformation end signal is sent through the learning signal generator 806, and the error amount D is If so, a deformation continuation signal is sent. Deformation continuation signal is pattern selector 10
When the pattern is sent to 4, the signal from the pattern change unit 105 is cut off, and the pattern sent from the terminal 715 of the multilayer neural network calculation unit 103 is input to the pattern storage unit 102.

The deformed pattern is sent from the pattern storage unit 102 to the pattern display unit 101 and displayed. At this time, if the pattern is registered as a learning pattern, by inputting a category name from the category name input unit 114, the pattern in the pattern storage unit 102 is changed to the learning pattern storage unit 106.
And stored with the entered category name. The transformation processing counter 810 counts each time a transformation end signal is generated in the learning signal generation unit 806, and counts the number of patterns subjected to the transformation processing. The number is stored in the threshold storage unit 80
9 Compare the internal value with the comparator 808, and if the value is larger than the threshold value, send a signal to the learning signal generation unit 806, stop the deformation continuation signal, and send the learning signal from the terminal 807. Then, the multilayer neural network calculation unit 103 learns the patterns stored in the learning pattern storage unit 106 according to the above-described procedure. If a signal indicating that the value of the modification processing counter 810 is smaller than the value of the threshold value storage unit 809 is obtained from the comparator 808, the learning signal generation unit 806 outputs a modification processing start signal to the terminal 807. , And the above-described pattern deformation process is performed again on another pattern selected from the learning pattern storage unit 106.

In the all learning control unit 111, the signal transmitted from the learning control unit 112 is monitored, and the number of times when the signal input from the terminal 807 changes from the deformation continuation signal to the learning signal is counted in the learning deformation processing counting unit 909. The number is compared with the value in the threshold value storage unit 910 by the comparator 908, and if the count number is larger, the learning deformation process stop signal generation unit 90
A stop signal is sent from the terminal 5 through the terminal 906, and both processes of learning transformation are stopped. Further, the result of the recognition result detection unit 110 is input from the terminal 901 as a signal of correct or incorrect recognition, and is counted by the recognition rate counting unit 902. That is the threshold storage unit 904
Are compared by the comparator 903. If the recognition rate becomes larger than the threshold value, a stop signal is transmitted from the learning transformation process stop signal generator 905 in the same manner as described above.

After stopping all the learning, the pattern input from the terminal 116 is recognized using the weight matrix after learning, and the result is output from the terminal 115.

Next, an embodiment of the second invention will be described with reference to FIGS.
This will be described with reference to FIGS. FIG. 2 is a block diagram showing an embodiment of the pattern learning apparatus according to the second invention.

The operation of the learning control unit 112 in FIG. 8 and the operation of the entire learning control unit 111 in FIG. 9 are the same as those described in the first invention, and thus description thereof is omitted here. Also, the operation for learning the pattern in the learning pattern storage unit 106 in FIG. 2 is the same as that of the first invention, and the description is omitted. The second invention is different from the first invention in that a part for generating a target category name when deforming a pattern is used.

When the transformation processing start signal is transmitted through the terminal 807, a pattern randomly selected from the learning pattern storage unit 106 is transmitted to the pattern change unit 105, and random noise is superimposed on the pattern, and the pattern noise is superimposed on the pattern storage unit 102 through the pattern selector 104. Sent. At this time, the pattern selector 104
Then, the signal from the multilayer neural network calculation unit 103 is cut off. The pattern input to the pattern storage unit 102 is displayed on the pattern display unit 101. Multi-layer neural network calculator 103
The signal output from the terminal 707 is converted into a category name by the recognition result detection unit 110, and is collated with the correct category name input from the learning pattern storage unit 106 to determine whether it is correct or incorrectly recognized. The correct answer signal or the incorrect recognition signal is sent to the transformation target category name selector 202 together with the category name of the recognition result.

In the modified target category name selector 202, when the correct answer signal is input, the recognition result category name is sent to the similar category storage unit 201, the category name stored in pair therewith is received, and the modified target category name is And Here, it is assumed that the similar category storage unit 201 clusters learning patterns in advance and stores similar categories in pairs. When an erroneous recognition signal is input to the modification target category name selector 202, the category name of the recognition result is set as the modification target category name.

The transformation target category name determined as described above is sent to the teacher signal category name selector 107 to perform a pattern transformation process, give a category name to the pattern in the process from the category name input unit 114, and enter the pattern and the category name. Registered in the learning pattern storage unit 106. The operation of the transformation process, the transformation process, and the control of the learning are performed in the same manner as in the first invention, and the description is omitted here.

Next, an embodiment of the third invention will be described with reference to FIGS.
This will be described with reference to FIGS. FIG. 3 is a block diagram showing one embodiment of the pattern learning apparatus of the third invention.

The operation of the learning control unit in FIG. 8 and the operation of all the learning control units in FIG. 9 are the same as those described in the first invention, and thus description thereof is omitted here. The learning pattern storage unit shown in FIG.
The operation for learning the pattern in 106 is the same as that of the first invention, and therefore the description is omitted. The third invention is different from the first and second inventions in that a part for generating a target category name when deforming a pattern is used.

When the transformation processing start signal is transmitted through the terminal 807,
A pattern randomly selected from the learning pattern storage unit 106 is sent to the pattern change unit 105, a random noise is superimposed on the pattern, and sent to the pattern storage unit 102 through the pattern selector 104. At this time, the pattern selector 1
In 04, the signal from the multilayer neural network calculation unit 103 is cut off. The pattern input to the pattern storage unit 102 is displayed on the pattern display unit 101. Multi-layer neural network calculator 1
The signal output from the terminal 707 of 03 is converted into a category name of the result by the recognition result detection unit 110, and the result is stored in the learning pattern storage unit 106.
Is checked against the correct answer category name input from the above to determine whether the answer is correct or incorrect. The correct answer signal or the incorrect recognition signal is sent to the transformation target category name storage control section 302 together with the category name of the recognition result.

When a correct answer signal is input, the transformation target category name storage control unit 302 outputs a transformation processing skip signal to a terminal 906.
To the learning control unit 112, and sends a transformation process start signal from the learning signal generation unit 806 to read another pattern from the learning pattern storage unit 106 and perform the transformation process without performing the transformation process.

When an erroneous recognition signal is input to the modified target category name storage control unit 302, the recognition result category name is sent to the similar category name storage unit 301, and the category name stored as a pair is received. , And the category name of the recognition result is the second modified target category name. Here, it is assumed that the similar category storage unit 301 clusters learning patterns in advance and stores similar categories in pairs.

First, the first transformation target category name is sent to the teacher signal category name selector 107, the pattern transformation processing is performed, and the category name input from the category name input section 114 is added to the pattern in the course of the transformation processing, and the learning pattern Storage unit 106
Register with. The operation of the transformation process, the transformation process, and the control of the learning are performed in the same manner as in the first invention, and the description is omitted here. When the learning start signal from the learning control unit 111 reaches the deformation target category name storage control unit 302 at the stage when the deformation of the first deformation target category is completed, before the new initial pattern is input to the pattern storage unit 102, The second modification target category name is sent from the modification target category name storage control unit 302 to the teacher signal name selector 107, and the learning control unit 112
The learning control unit 112 transmits a re-deformation signal and the learning control unit 112 transmits a re-deformation continuation signal, so that the pattern of the deformation result for the first deformation target category stored in the pattern storage unit 102 is used as an initial value, The transformation process is performed with the transformation target category as a target. By adding the category name input from the category name input unit 114 to the pattern of the deformation process,
Registered in the learning pattern storage unit 106.

The operation of the learning control unit 111 after the end of this processing is the first operation.
The description is omitted here.

〔The invention's effect〕

As is clear from the above description, the present invention solves the problem that it was difficult to recognize whether or not the result of learning conventionally accurately reflects the actual pattern distribution, and erroneous recognition has occurred. In the vicinity of the pattern, it is assumed that the learned result does not reflect the distribution of the actual pattern, so that the pattern is deformed so that it belongs to the intended category, and a pattern near the category boundary is generated. By doing so, it is possible to learn the distribution of the pattern near the category boundary more accurately.

In the first invention, the user can input a given transformation target category name and give it, and the user can check the learning pattern relatively freely and generate a deformation pattern. Learning can be controlled to perform advanced learning.

In the second invention, the user does not always need to input the transformation target category name, and the apparatus automatically determines the transformation target category name in accordance with the result of recognition or misrecognition, so that the burden on the user is reduced. can do.

Furthermore, in the third invention, since the pattern deformation processing is performed only in the case of erroneous recognition, the burden of assigning a category name to the generated pattern is reduced, and the pattern around the erroneous recognition pattern to be learned more accurately is determined. Can be generated with emphasis.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 to FIG. 3 and FIG. 7 to FIG. 9 are views showing an embodiment of the present invention, and FIG. FIG. 5, FIG. 5 is a diagram showing a processing concept of the multilayer neural network, and FIG. 6 is a diagram schematically showing deformation of a misrecognized pattern. 101: Pattern display unit 102: Pattern storage unit 103: Multilayer neural network calculation unit 104: Pattern selector 105: Pattern change unit 106: Learning pattern storage unit 107: Teacher category name selector 108: Teacher Signal generator 109 Error calculator 110 Recognition result detector 111 All learning controller 112 Learning controller 114 Category name input unit 115 Recognition result output terminal 116 Pattern input terminals 201 and 301 ... Similar category name storage unit 202 ... Modified target category name selector 302 ... Modified target category name storage control unit 401 ... Initial pattern of pattern deformation 402,403 ... Deformed patterns belonging to category 1 404,405 obtained by deformation Reject pattern 406, 407, 408... Deformation pattern belonging to category 2 409... Boundary of category 1 410... Boundary of category 2 501... Input layer 502... Intermediate layer 503. 4 ... Teacher signal 505 ... Output layer error 506 ... Modification block of second layer weight matrix 507 ... Intermediate layer estimation error 508 ... Modification block of first layer weight matrix 509 ... Estimation error of input layer 601 ... ... True boundary of category 1 602... Incorrectly learned boundary of category 2 603... True boundary of category 2 604... Initial pattern of deformation 605, 606... Reject pattern 607, 608. …… Pattern input terminal 702 …… First layer matrix product operation unit 703 …… First layer S function unit 704 …… Intermediate layer storage unit 705 …… Second layer matrix product operation unit 706 …… Second layer S function unit 707 output signal terminal 708 error signal input terminal 710 second layer weight matrix storage section 711 first layer weight matrix storage section 712 second layer weight matrix correction section 713 first layer weight Matrix correction unit 714 Correction section 715: deformation pattern output terminal 716: learning / deformation processing switching signal input terminal 801: error input terminal 802: error amount calculation section 803, 809: threshold value storage section 804, 808: comparator 806 … Learning signal generation unit 807… Control signal output terminal 810… Transformation processing counter 811… Counter 901… Recognition result input terminal 902… Recognition rate counting unit 903,908… Comparator 904,910 …… Learning transformation processing stop signal generator 906 …… Signal output terminal 909 …… Learning transformation processing counting section

Claims (3)

(57) [Claims] 1. A multi-layer neural network calculation section comprising an input layer, an intermediate layer, and an output layer for learning by a back propagation method, a learning pattern storage section for storing a pattern to be learned, and a multi-layer neural network for storing a storage pattern. Means for inputting the category name as a teacher signal to control learning of the weight value of the multilayer signal network; and inputting the learning pattern to the input layer of the multilayer neural network, Means for inputting a target category name and controlling a deformation process for deforming the input pattern so as to belong to the target category by the multi-layer neural network; and displaying the generated pattern during the deformation and as a result. Means for inputting a correct category name for each pattern; and adding the input correct category name for each generated pattern, and storing the generated pattern in a learning pattern. Pattern learning device comprising means for registering, and means for controlling the repetition of the generation and learning pattern, in that it comprises a. 2. A multi-layer neural network calculation section comprising an input layer, an intermediate layer, and an output layer for learning by a back propagation method, a learning pattern storage section for storing a pattern to be learned, and a multi-layer neural network for storing a storage pattern. Means for controlling learning of the weight value of the multilayer signal network by inputting the category name as a teacher signal, and inputting the learning pattern to the input layer of the multilayer neural network; If the output signal is correct, a category name similar to the previously stored result category is determined as a category name for the purpose of transformation, and if the output signal is incorrect, the category name obtained from the output signal is determined. Means for defining a transformation target category name, means for controlling a transformation process for transforming an input pattern by a multilayer neural network so as to belong to the decided transformation target category, and Means for displaying the generated pattern as a result and inputting the correct category name for each generated pattern; and adding the input correct category name for each generated pattern, A pattern learning apparatus, comprising: means for registering the obtained pattern in a learning pattern storage unit; and means for controlling repetition of pattern generation and learning. 3. A multi-layer neural network calculating section comprising an input layer, an intermediate layer, and an output layer for learning by a back propagation method, a learning pattern storing section for storing a pattern to be learned, and a multi-layer neural network for storing a storage pattern. Means for controlling learning of the weight value of the multilayer signal network by inputting the category name as a teacher signal, and inputting the learning pattern to the input layer of the multilayer neural network; Only when the output signal is incorrect, means for determining both the name of the category similar to the result category stored in advance and the category name obtained from the output signal as the category name for the purpose of transformation, Means for controlling a transformation process for transforming an input pattern by a multi-layer neural network to belong to a transformation target category; Means for inputting a correct category name for each pattern, means for adding the input correct category name for each generated pattern, and registering the generated pattern in a learning pattern storage unit; Means for controlling repetition of generation and learning of the pattern learning device.