JP3273568B2 - Hierarchical neural network and learning method thereof - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hierarchical neural network and a learning method thereof.

[0002]

2. Description of the Related Art Attention has been paid to a neural network for realizing information processing by mechanically realizing the mechanism of the human brain nervous system. This neural network is composed of a plurality of neuron elements. By performing predetermined learning, pattern recognition of characters and figures, analysis and synthesis of voice, continuous analog data processing, parallel processing of large amounts of data are performed. Is performed efficiently. The neural network includes a neuron element network composed of a plurality of neuron elements for transmitting data, and a learning control unit for controlling learning of the neuron element network. The neuron element network includes an input layer to which data is input, an output layer to which data is output in response to the input data, and one or a plurality of intermediate layers arranged between the two layers. The neuron elements between the layers of the neuron element network are connected to other neuron elements with a predetermined strength (connection weight), and the output signal changes due to the difference in the value of the connection weight. ing.

In a conventional neural network having such a hierarchical structure, a process called "learning" is performed by changing a connection weight between neuron elements by a learning control unit. The learning is performed in the form of an analog or 2
This is performed by value data (pattern). Now, g ₁ to g ₆ are given as input data, among the, g ₁ to g
₃ is input as a learning pattern from the input layer, and an output signal is output from the output layer in response to the input of the learning pattern. Correct answer to this output signal is g ₄ to g _6, are commonly referred to as a teacher signal. Then, learning is performed by executing processing for correcting the connection weight of each neuron element such that the error between the output signal and the teacher signal becomes small or coincides with each other for a plurality of learning patterns.

As a specific method of correcting the connection weight between the neuron elements in the neuron element network so that the output signal matches the teacher signal, a back propagation method (hereinafter referred to as a BP method) has been conventionally used. ) Is used. The BP method corrects the connection weight between the neuron elements between all the layers constituting the neural network in order to minimize the square error between the output value and the teacher signal in the output layer. That is, the error in the output layer is determined to be the sum of the individual errors generated in the neuron elements of each intermediate layer, and not only the error from the output layer but also the error of each intermediate layer The connection weight is modified so that the error of the neuron element is also minimized. For this purpose, all errors for each neuron element in the output layer and each intermediate layer are calculated. In this calculation process, the error value of each neuron element of the n-th intermediate layer is given as (n
Calculation processing is performed in the opposite direction, such as -1) the error of the intermediate layer,. The correction value of the connection weight is calculated using the error value of each neuron element thus obtained and the connection weight at that time. The above learning process is repeated for all learning patterns until the error from the teacher signal becomes equal to or less than a certain value or a predetermined number of times, thereby completing the learning.

[0005]

However, in the conventional learning of the hierarchical neural network by the BP method, calculation of an error for calculating a correction value is complicated. For this reason, the BP method has a problem that it takes time until the learning is completed. On the other hand, in order to shorten the learning time, it is possible to solve the problem by making the calculation processing unit hardware instead of performing the learning calculation processing by software. However, as mentioned above, BP
Learning a neural network by the method is difficult to implement in hardware because of the complicated calculation processing.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a hierarchical neural network capable of performing learning quickly by a simple calculation process and a learning method therefor.

[0007]

According to the first aspect of the present invention, as shown in principle by a solid line in FIG. 1, a neuron element network comprising a plurality of neuron elements hierarchically connected with a predetermined connection weight. A first calculating means for calculating a square error between an output signal and a teacher signal for the learning pattern input to the neuron element network; and a connection weight of one of the plurality of neuron elements for a predetermined test correction width S First correction means for performing test correction only in at least one of the positive and negative directions, and an output signal and a teacher signal for the same learning pattern as the first calculation means in the neuron element network after the test correction by the first correction means. A second calculating means for calculating the squared error of the test, and a test wherein the squared error calculated by the second calculating means is smaller than the squared error calculated by the first calculating means. Direction determining means for determining the positive or negative direction of the positive width S; third calculating means for calculating a correction value of the connection weight according to the correction direction by the direction determining means and the test correction width S; 3 in calculating means calculates a correction value of the coupling weight of a predetermined number, Ru is provided with a second correction means for correcting the respective connection weight to the correction value to the hierarchical neural network. And in the third calculating means,
When the direction determined by the direction determining means is positive, W ⁿ _ij + S × k
× [E (W ⁿ _ij ) −E (W ⁿ _ij + S)]
When the direction determined by the means is negative, W ⁿ _ij −S × k × [E
(W ⁿ _ij ) −E (W ⁿ _ij −S)]
Is calculated. According to the second aspect of the present invention, the original is indicated by a dotted line in FIG.
2. The hierarchical neural network according to claim 1, wherein the hierarchical neural network comprises:
In the network, the test correction width S and coefficient k are small.
At least one of them is provided with a changing means for changing. Claim
In the invention according to claim 3, the hierarchical neural network according to claim 1 is provided.
In the network, the test modification width S is 0.1 and the coefficient is
Let k be 100.

[0010] In the invention according to claim 4, the original line is indicated by a solid line in FIG.
As shown in the figure, a hierarchical connection is performed with a predetermined connection weight.
Neuron element network consisting of multiple neuron elements
And the learning pattern input to this neuron element network
Calculating the square error between the output signal and the teacher signal
Calculating means, and one of the plurality of neuron elements
The combined weight is tested in the positive and negative directions by a predetermined test correction width S.
First correcting means for correcting the strike, and the first correcting means
The first in the neuron element network after test correction
Output signal and teacher for the same learning pattern as the calculation means
A second calculating means for calculating a square error with respect to the signal;
The square error calculated by the calculating means is calculated by the first calculating means.
Test correction width S that is smaller than the square error
Direction determining means for determining the negative or negative direction, and the direction determining means.
According to the correction direction by means and the test correction width S
A third calculating means for calculating a correction value of the connection weight;
A third calculating means calculates a correction value of a predetermined number of connection weights, and
Correction means for correcting each connection weight to the correction value of
Achieved the above purpose by providing neural network
I do.

According to the fifth aspect of the present invention, a learning pattern is input to a neuron element network composed of a plurality of neuron elements hierarchically connected with a predetermined connection weight, and a square error between an output signal and a teacher signal is obtained. The first step to be calculated and the other connection weights of the plurality of neuron elements are set in the same state as in the first step, and one connection weight is changed by a predetermined test correction width S.
The test error is corrected in the positive and negative directions, and the square error calculated by inputting the same learning pattern as in the first step to the neuron element network after the test correction becomes smaller than the square error calculated in the first step. A second step of determining the positive or negative direction of the test correction width S and calculating a correction value of the connection weight according to the determined correction direction and the test correction width S for a predetermined number of connection weights; In the second step, learning of the hierarchical neural network is performed in a third step in which correction values of a predetermined number of connection weights are calculated in the second step, and then each connection weight is corrected to the correction value calculated.

[0010]

According to the first aspect of the present invention, the first correction means test-corrects one connection weight of the plurality of neuron elements in at least one of the positive and negative directions by a predetermined test correction width S. The calculating means and the second calculating means calculate square errors before and after the test correction for the same learning pattern input. Then, the direction determining means determines the positive or negative direction of the test correction width S in which the square error is smaller than before the test correction, and determines the determined positive or negative direction.
The correction value of the connection weight is calculated by the third calculating means by the equation corresponding to Then, the second correction unit corrects the connection weight to a correction value obtained by calculating a predetermined number of connection weights. Claims 4 and 5
In the invention described in (1), a plurality of neuron elements are
One of the children is positive by a predetermined test correction width S
Test correction in the negative direction and the first calculation means and the second
Calculating means for inputting the same learning pattern
The square errors before and after the test correction are calculated. Soshi
And the square error is smaller by the direction determination means than before the test correction.
Determine the positive or negative direction of the test correction width S
(3) A correction value of the connection weight is calculated by the calculation means. And the second
(2) Combining a predetermined number of connection weights with the correction value calculated by the correction means
Modify the weight. As described above, the correction direction and the correction value of the connection weight are determined by examining the fluctuation of the output error of the neuron element network caused by the single test correction of each connection weight, so that a small amount of calculation is required.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the hierarchical neural network according to the present invention will be described below in detail with reference to FIGS. FIG. 2 shows the configuration of a neuron element network of a hierarchical neural network. In this embodiment, a three-layered neural network will be described for simplicity. The neuron element network has three layers consisting of neuron elements 1, 2, and 3 as an input layer, three layers consisting of neuron elements 4, 5, and 6 as an intermediate layer, and three layers consisting of neuron elements 7, 8, and 9 as an output layer. Each is configured. Then, the connection weights of the neurons 1, 2, and 3 of the input layer to the neurons 4 of the intermediate layer are respectively represented by W
¹ _11, and W ¹ _21, W ¹ _31. Similarly, the connection weights for the neuron elements in each layer are shown in FIG. 2, and the general formula is W ⁿ _ij . Each neuron element is configured by dedicated hardware, but may be configured by software of a Neumann computer.

FIG. 3 shows the configuration of a learning control unit for controlling learning of such a neuron element network. The learning control unit includes an instruction to input a learning pattern to the neuron element network, an output signal corresponding to the input, and a teacher signal.
CPU that performs various controls such as calculation of a squared error and calculation of a correction value
(Central processing unit) 11. This CPU 11
Are connected to the following units via a bus line 12 such as a data bus. That is, the CPU 11 performs various controls in the neural network and calculation of the correction value of the connection weight.
A ROM (Read Only Memory) 13 storing various programs to be executed by the computer is connected. This ROM
13 stores equations (1) and (2) for calculating the correction amount dW ⁿ _ij of the connection weight. dW ⁿ _ij = S × k × [E (W ⁿ _ij ) −E (W ⁿ _ij + S)] (1) dW ⁿ _ij = −S × k × [E (W ⁿ _ij ) −E (W ⁿ _ij− S)] (2) In the equations (1) and (2), S is a test correction width for test-correcting the connection weight W ⁿ _ij during learning, k is a coefficient,
E (W ⁿ _ij ) is the square error between the output signal and the teacher signal with respect to the input of a certain learning pattern, and E (W ⁿ _ij ± S) increases (+ S) or decreases the connection weight W ⁿ _ij by the test correction width S. (−S) shows the square error between the output signal and the teacher signal when the learning pattern is input.

The bus line 12 has an R for storing a learning pattern, a test correction width S, a coefficient k, and other various data.
A connection weight memory 17 having an AM (random access memory) 14, a weight table 15 storing the value of each connection weight of the neuron element network, and a correction value table 16 storing the correction value W ⁿ _ij '. Is connected. Further, the test correction width S stored in the RAM 14
18 as an input device for instructing correction of coefficient k and coefficient k, etc., and connection weight I / I for supplying a correction value W ⁿ _ij ′ of connection weight to each neuron element of the neuron element network.
O (input / output port) 19, an output signal I / O 20 to which each output value from the output layer is supplied, and various I / Os such as a data I / O 21 to which various data such as a learning pattern are input / output are connected. I have. Further, a display device (not shown), a scanner for reading figures and characters, a voice input / output device, and the like are connected as necessary.

Next, the learning operation in the neural network configured as described above will be described. Outline of Learning Operation In an initial state in which all the connection weights W ⁿ _ij are set to predetermined values, a predetermined learning pattern is input to an input layer, and a square error between an output signal of the input learning signal and a teacher signal is calculated. And 1
One of the connection weights W ⁿ _ij, a predetermined test correction range S increases and decreases, and inputs the same learning pattern and learning pattern previously entered test fix. Then, before and after the test correction of the connection weight, the square error between the output of the neural network and the teacher signal is compared to determine the direction of the correction. If the square error decreases when the connection weight is increased, the correction width dW ⁿ _ij is calculated by equation (1). on the other hand,
If the square error decreases when the connection weight is reduced, the correction width dW ⁿ _ij is calculated by equation (2). If the square error decreases in either direction, the correction width dW ⁿ _ij is calculated by equation (1) or equation (2) in the direction in which the square error decreases. If the square weight does not decrease in either case by increasing or decreasing the connection weight, the correction width dW ⁿ _{ij is set} to 0, and no correction is performed. The correction widths of the other connection weights are calculated in the same manner, but the connection weights W ⁿ _ij for which the correction widths have already been calculated are returned to the original values.
That is, the processing is performed in a state where the test correction width S is returned to the initial state except for the connection weight for performing the test correction. After determining the respective correction range dW ⁿ _ij for all connection weights W ⁿ _ij, completed one study to correct the simultaneously coupling weight.

Operation Details FIGS. 4 to 6 are flowcharts showing the operation details. First, the CPU 11 stores the learning pattern supplied from the data I / O 21 and the teacher signal thereof in the RAM 14, and performs the following learning operation. Now, as various conditions, the number of learning patterns is a, and n defining the connection weights W ⁿ _ij of each neuron element, i, the value of j is from respectively 1 b, c, as is d. Then, CPU 11 initializes the connection weight W ⁿ _ij of each neuron elements (step (S)
1). That is, each connection weight W ⁿ _ij is determined by a random value and supplied to each neuron element from the connection weight I / O 19, and the determined value of each connection weight W ⁿ _ij is stored in the weight table 15. In each of the neuron elements 4 to 9, the connection weight W supplied through the connection weight I / O 19
Initially set to ⁿ _ij . Next, the CPU 11
Setting the identifier x of the training patterns stored in the 1 (step 2), in order to determine separately the correction range dW ⁿ _ij for connection weight W ⁿ _ij of each neuron elements 4 to 9, n = 1, i = 1 and j = 1 are set (step 3). Then, the first learning pattern P _{x is} read from the RAM 14.
Is read and input from the data I / O 21 to each of the neuron elements 1, 2, and 3 in the input layer (step 4). The output signal O _x from each neuron elements 7, 8, 9 of the output layer by the input is outputted. This output signal O _x is output signal I
/ Receipt from O20, and which calculates a square error E between the teacher signal T _x (W ⁿ _ij), and stores the value in a predetermined area of the RAM 14 (Step 5).

[0016] Then, (in this case W ¹ ₁₁ corresponds.) The first connection weight W ⁿ _ij and read from the weight table 15, the coupling weight W added thereto to test correction range S ⁿ _ij +
Test correction to S (FIG. 5, step 6). That is, C
The PU 11 converts the connection weight W ⁿ _ij + S into the connection weight I / O 19
And supplied to the corresponding neuron element via a neuron element which receives the supply test modifications carried out by setting the connection weight to W ⁿ _ij + S. The CPU 11 adds a step 4 to the input layer of the neuron element network after the test correction.
Enter the same learning pattern P _x and receives its output signal O _x 'from the output signal I / O20. Then, the square error E (W) between the output signal O _x ′ and the teacher signal difference T _x
ⁿ _ij + S) is calculated (step 7) and stored in a predetermined area of the RAM 14. Next, the modified connection weight W ⁿ _ij +
S is further test-corrected to W ⁿ _ij -S (step 8),
Similarly, a square error E (W ⁿ⁾ between the output signal O _x ″ and the teacher signal difference T _x by inputting the same learning pattern P _x as in step 4 is input.
_ij- S) is calculated (step 9) and stored in a predetermined area of the RAM 14.

The CPU 11 calculates the square errors E (W ⁿ _ij ), E (W ⁿ _ij + S) stored in each predetermined area of the RAM 14,
E (W ⁿ _ij -S) is read out, and these are compared to determine the minimum value (step 10). If the square error E (W ⁿ _ij + S) when the test is corrected in the plus direction is the minimum, RO
Calculating a correction range dW ⁿ _ij by equation (1) stored in M13. On the other hand, when the test was corrected in the minus direction, 2
Multiply the error E (W ⁿ _ij -S) is the minimum case for calculating the correction range dW ⁿ _ij by equation (2) (step 12). If the square error E (W ⁿ _ij ) before the test correction is the minimum, the correction width d
And W ⁿ _ij = 0 (step 13).

[0018] CPU11 is thus the value obtained by adding the correction range dW ⁿ _ij calculated based on the connection weights W ⁿ _ij and correction value W ⁿ _ij 'by, stores it in the corresponding area of the correction value table 16 (Step 14). After the above processing is completed, CPU 11 returns to the original connection weights W ⁿ _ij of the previous test fix the connection weights (step 15). That is, the original connection weight W ⁿ _ij is read from the weight table 15 and supplied to the corresponding neuron element via the connection weight I / O 19. The supplied neuron element changes the connection weight to the original W
Set to ⁿ _ij .

The above processing from step 6 to step 15 is performed for all the connection weights W ⁿ _ij (FIG. 6, steps 16 to 21). After finishing all the connection weights (step 21; Y), the correction value table 1
Fixed stored in 6 values W ⁿ _ij 'connection weights I / O1
All connection weights are updated by supplying them to the corresponding neuron elements via 9 (step 22). Then, the correction value of each connection weight is read from the correction value table 16 and stored in the corresponding area of the weight table 15, and the weight table 15 is updated (step 23).
Thus, one learning pattern ends.

The above operation returns to step 2 and is performed for all the learning patterns P _{1 to} P _a (step 24,
25). Further, returning to step 1, the process is repeated until the square error E (W ⁿ _ij ) becomes equal to or less than a predetermined value for all the learning patterns or the number of repetitions reaches a predetermined limit (step 26). finish.

Next, the result of learning the neural network by the above operation will be described. As a neural network that has performed learning, a neuron is composed of two input layers composed of two neuron elements, two intermediate layers composed of two neuron elements, and one output layer composed of one neuron element. An element network is formed, and learning about exclusive OR is performed. In the learning, ten initial values of the connection weights W ⁿ _ij are prepared using random numbers from 0 to 0.3, and a comparison experiment is performed for each of the initial values. In addition, as a condition for completing the learning, the square errors for all the learning patterns are all 0.0.
It is set when the number becomes 1 or less, or when the number of times of learning exceeds 3000 times. Learning is performed for the following three cases, and the results are shown in FIG. That is, equation (1),
FIG. 7A when the coefficient k of (2) is changed, FIG. 7B when the test correction width S is changed, and FIG. 7 when the coefficient × test correction width S is changed within a constant range. (C). In this figure, the average number of times of learning is the average number of times of learning when learning converges. As shown in this figure, a high convergence rate can be obtained by setting the test correction width S and the coefficient k to appropriate values. The test correction width S and the coefficient are changed by operating the keyboard 18. Further, as shown in this figure, it is understood that the optimum value is obtained when the test correction width S is 0.1 and the coefficient k is 100.

FIG. 8 shows a comparison result between the learning time when the conventional BP method is used and the learning time according to the present embodiment. FIG. 8A shows a comparison in the case of a partial network for speech recognition, and FIG. 8B shows a comparison in the case of performing alphanumeric character recognition. As shown in FIG. 8, according to the present embodiment, the number of times of learning increases as compared with the conventional BP method. However, since the calculation performed in one learning is very simple, the learning is completed in a shorter time in each case than in the BP method.

FIG. 9 shows a flow of another learning operation by the learning control unit. This drawing corresponds to FIG. 5 and shows the processing of steps 31 to 41 corresponding to the processing of steps 6 to 15. In other words, after the processing from step 1 to step 5 in FIG. 4, the process proceeds to step 31, and after the process in step 41, the process proceeds to step 16 in FIG. The processing in FIGS. 4 and 6 is as described above, and a description thereof will be omitted.

Firstly, the first connection weight W ⁿ _ij (W ¹ ₁₁ is applicable) the test modified to W ⁿ _ij + S (step 31).
Then, the same learning pattern P _{x as} in step 4 (FIG. 4)
Enter the calculated square error E (W ⁿ _ij + S) and the output signal O _x 'teacher signal T _x (step 32), the square stored calculated in RAM14 in step 5 (FIG. 4) The magnitude is compared with the error E (W ⁿ _ij ) (step 33). 2
When the squared error E (W ⁿ _ij + S) is smaller (step 3
3; Y), the correction width dW of the connection weight W ⁿ _{ij according} to equation (1).
ⁿ _ij is calculated (step 34).

On the other hand, if the square error E (W ⁿ _ij + S) is not small (step 33; N), the connection weight is set to W ⁿ _ij -S
(Step 35). The same learning pattern P _x inputs to determine the output signal O _x "and the square error E between the teacher signal difference T _x (W ⁿ _ij -S) in (step 36), the square error E (W ⁿ _ij (Step 37). If the square error E (W ⁿ _ij -S) is smaller (Step 37; Y), the correction width dW ⁿ of the connection weight W ⁿ _ij is calculated by the equation (2). _ij is calculated (step 3
8). If towards the square error E (W ⁿ _ij -S) it is not smaller (step 37; N), the W ⁿ _ij to 0 (step 39). CPU11, in addition the correction width dW ⁿ _ij calculated in this Uyo based connection weight W ⁿ _ij, the correction value W ⁿ
_ij 'is stored in the corresponding area of the correction value table 16 (step 40). Further, the connection weight is returned to the original value W ⁿ _ij (step 41), and the processing shifts to step 16 shown in FIG.

As described above, if the square error E (W ⁿ _ij + S) due to the test correction in the plus direction is smaller than the square error E (W ⁿ _ij ) before the test correction, the test correction in the minus direction is omitted. As a result, the number of calculations is reduced and the learning speed is increased as compared with the case where the test correction is performed in both the positive and negative directions by the test correction width S. In this case, the test correction in the minus direction may be performed first. In addition, the test correction may be performed in the same direction as the test correction performed last time. As a result, the number of times of calculating the square error is reduced, and the learning speed is further increased. In this case, the direction of the previous test correction is stored in a predetermined area of the RAM 14. Here, the previous test correction includes both the test correction of another connection weight performed immediately before the connection weight and the test correction of the same connection weight for the previous learning pattern.

[0027] As described above, according to an embodiment of the present invention, the calculation of the square error before and after testing correct the connection weights W ⁿ _ij only test fix width S in a positive or negative direction, the calculation result , It is only necessary to calculate the correction width dW ⁿ _ij based on, and there is no need to perform a complicated calculation such as the BP method.
Learning can be completed with a small calculation formula and a small amount of calculation. Note that for each of the connection weights W ⁿ _ij ,
Type obtain an output signal O _x learning pattern for each testing modified test correction range S. Therefore, the number of times of input of the test pattern is larger than that of the BP method. However, since the neuron element network is constituted by hardware, the time from input to output is very short. It doesn't matter.

Further, according to the present embodiment, when calculating the square error after the test correction in the connection weight, the connection weights other than the connection weight for performing the test correction are calculated in a state where the connection weights are returned to the initial state. Therefore, it is possible to determine only the effect of the change in the connection weight for performing the test correction. Therefore, it is possible to reflect the influence of each connection weight W ⁿ _{ij on} the square error in the initial state.

In the above-described embodiment, the learning is performed on the neuron element network by the control of the CPU according to the program stored in the ROM. In the present invention, however, the first calculation is performed by dedicated hardware. means,
You may make it comprise the 1st correction means, the 2nd calculation means, the direction determination means, the 3rd calculation means, and the 2nd correction means.
In the present invention, since learning is performed by a simple calculation process, dedicated hardware can be easily implemented.
In this way, by adopting a configuration in which learning is performed using dedicated hardware, the learning time is one-tenth or less of the learning time by the conventional BP method.

[0030]

According to the present invention, unlike the BP method, the correction direction and value of the connection weight are determined by examining the fluctuation of the output error of the neuron element network caused by a single test correction of each connection weight. Only a small amount of calculation is required. Further, since the learning method is simple, it is easy to implement hardware. With a neural network implemented with hardware, it takes almost no time from input to output of data, so that it is possible to finish learning quickly.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of the present invention.

FIG. 2 is an explanatory diagram showing a configuration of a neuron element network according to one embodiment of the present invention.

FIG. 3 is a configuration diagram of a learning control unit that controls learning of a neuron element network.

FIG. 4 is a part of a flowchart showing a flow of a learning operation by a learning control unit.

FIG. 5 is a part of a flowchart showing a continuation of the learning operation shown in FIG. 4;

FIG. 6 is a part of a flowchart showing a continuation of the learning operation shown in FIGS. 4 and 5;

7A and 7B are diagrams showing learning results by a hierarchical neural network. FIG. 7A shows a case where a coefficient k is changed.
FIG. 7C is a diagram showing a learning result when the test correction width S is changed, and FIG. 7C shows a learning result when the coefficient × the test correction width S is changed within a constant range.

FIG. 8 is an explanatory diagram showing a comparison result between the learning time according to the conventional PB method and the learning time according to the present embodiment.

FIG. 9 is a part of a flowchart showing a flow of another learning operation by the learning control unit.

[Explanation of symbols]

 1 to 9 neuron element 11 CPU 13 ROM 14 RAM 15 weight table 16 correction value table 19 connection weight I / O 20 output signal I / O 21 data I / O

Claims (5)

(57) [Claims] 1. A neuron element network composed of a plurality of neuron elements hierarchically connected with a predetermined connection weight, and a square error between an output signal and a teacher signal for a learning pattern input to the neuron element network is calculated. First calculating means for performing a test correction of one connection weight of the plurality of neuron elements in at least one of positive and negative directions by a predetermined test correction width S; and a test performed by the first correcting means. A second calculating means for calculating a square error between an output signal and a teacher signal for the same learning pattern as the first calculating means in the corrected neuron element network, and a square error calculated by the second calculating means is Direction determining means for determining the positive or negative direction of the test correction width S which is smaller than the square error calculated by the first calculating means; A third calculating means for calculating a correction value of the connection weight according to the correction direction and the test correction width S; a correction value of a predetermined number of connection weights being calculated by the third calculation means; Second correcting means for correcting the weight ;
And the third calculating means calculates k as a coefficient, E (W ⁿ _ij ) as a square error before test correction, E
(W ⁿ _ij ± S) is the square error after test correction
, When the direction determined by the direction determining means is positive, the W ⁿ _ij + S × k × _{^{[E (W n ij) -E (}} W n ij + S) ], when the direction determined by the direction determining means is negative by W ⁿ _ij -S × k × _{^{[E (W n ij) -E (}} W n ij -S) ], and calculates the respective correction value
Hierarchical neural network. 2. A hierarchical neural network according to claim 1, further comprising a change means for changing at least the other hand, the test fix width S and the coefficient k. 3. The test correction width S is 0.1 and the coefficient k is 100.
The hierarchical neural network according to claim 1, wherein 4. A neuron element network comprising a plurality of neuron elements hierarchically connected with a predetermined connection weight, and a square error between an output signal and a teacher signal for a learning pattern input to the neuron element network is calculated. First calculating means for performing a test correction of one connection weight of the plurality of neuron elements in a positive and negative direction by a predetermined test correction width S ; after the test correction by the first correcting means A second calculating means for calculating a square error between an output signal and a teacher signal for the same learning pattern as the first calculating means in the neuron element network, and the square error calculated by the second calculating means is: Direction determining means for determining the positive or negative direction of the test correction width S smaller than the square error calculated by the first calculating means; And third correction means for calculating a correction value of the connection weight in accordance with the test correction width S; calculating a correction value of a predetermined number of connection weights with the third calculation means; A hierarchical neural network, comprising: a second modifying means for modifying. 5. A first step of inputting a learning pattern to a neuron element network composed of a plurality of neuron elements hierarchically connected with a predetermined connection weight and calculating a square error between an output signal and a teacher signal. The other connection weights of the plurality of neuron elements are set in the same state as in the first step, and one connection weight is test-corrected in a positive and negative direction by a predetermined test correction width S. The square error calculated by inputting the same learning pattern to the net as in the first step is 2 calculated in the first step.
A positive or negative direction of the test correction width S smaller than the squared error is determined, and the determined correction direction and the test correction width S are determined.
Calculating the correction value of the connection weight according to the second step of performing the correction value of the predetermined number of connection weights in the second step; and calculating the correction value of the predetermined number of connection weights in the second step. A method for learning a hierarchical neural network, comprising a third step of correcting connection weights.