JP3367264B2 - Neural network learning method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network applicable to the fields of large-scale logic circuits, pattern recognition, associative memory, data conversion, image processing and the like, and can output desired output signals stably at high speed by simple learning. The present invention relates to a learning method of a neural network that can obtain

[0002]

2. Description of the Related Art As one of neural networks,
Although there is a multilayer neural network, a back propagation algorithm is widely used as a learning method using a teacher signal of this neural network. In the learning process using this algorithm, after the weighting coefficient is initialized, the teacher signals T (teacher signal elements, T ₁ , T ₂ , ..., T _M prepared in advance are prepared.
), The output unit signal from the output layer with respect to the learning input signal input to the input layer is subtracted through a subtracter to obtain an error signal, and the power of the error signal is calculated based on the unit output signal of each layer and the error signal. The weighting coefficient between layers is updated so as to minimize the learning. The learning consisting of the weighting coefficient adaptive control is executed for all learning input signals and repeated until convergence.

In the learning process, when the error power becomes minimum, the error power converges completely, and the output unit signal for the learning input signal matches the teacher signal. However, once the error power drops to the local minimum (local minimum), the learning does not proceed to the minimum after that, and the number of learning increases remarkably because of the minimum state, and the weighting factor There is a problem that there is an initial value dependency of.

On the other hand, in order to improve the learning speed for the binary teacher signals T of 0 and 1, "Parallel Distributed Pr" is used.
In ocessing "DE Rumelhart, MIT Press., a new teacher signal T'of 0.1 and 0.9 is set for each element in the multilayer neural network using the conventional learning shown in FIG. Learning is started after initialization is performed by the control signal from the operation mode controller 9. From the teacher signal T ′ from the terminal 3, the terminal 2 is started.
The output unit signal of the multilayer neural network 1 with respect to the learning input signal is obtained through the subtractor 4 to obtain a subtraction error signal, which is input to the weighting factor controller 5, and the weighting factor is updated by the back propagation algorithm, The process of setting again in the multilayer neural network 1 is repeatedly learned. A binary output unit signal is obtained from the output unit signal through the binary threshold circuit 6 which is binarized by using the binary threshold circuit, and a binary threshold circuit 7 which is binarized by using the binary threshold circuit is provided. A binary teacher signal T is obtained from the teacher signal T'through the detection signal, and the coincidence detector 8 detects a state in which they are completely coincident with each other. Learning ends.

By setting the teacher signals to 0.1 and 0.9 in this way, it is stated that the number of learnings for convergence is reduced as compared with the case of binary teacher signals of 0 and 1. ing. This is because, in the sigmoid function, when the input approaches 0 or 1, the gradient becomes very small and the update rate of the weighting coefficient becomes small. Therefore, the learning speed is improved by setting the teacher signal to 0.1 and 0.9 and increasing the gradient. However, learning will not progress if it falls into a stable local minimum as well.

As described above, these conventional learning methods have a drawback in that they cannot reach the local minimum state and cannot escape, so that a completely converged state cannot be easily reached. In particular, in a three-layer or multi-layer neural network with a large number of input units, a design method for reliably converging general teacher signals has not been clarified, and the initial value of the weighting factor is changed due to the initial value dependence. And increasing the number of hidden units (intermediate units) by trial and error.

[0007]

In the conventional learning processing of a multilayer neural network using a teacher signal, when the weighting coefficient is updated so that the multilayer neural network sends out a desired output unit signal corresponding to the learning input signal. In addition, when the number of times of learning, that is, the number of times of learning iterations until a converged state in which a desired output unit signal is transmitted becomes extremely large, or when learning does not converge, that is, a stable local minimum state is reached, the desired output is output. It has drawbacks such as that the unit signal is not transmitted and the weighting factor depends on the initial value. In particular, when the number of units in the input layer is large and the number of output units is small, convergence by learning becomes very difficult, and a method for freely designing a multilayer neural network that sends a desired output unit signal has not been established. . In addition, there is a method to significantly increase the number of intermediate units to make it easier to converge, but as the generalization capacity deteriorates, the amount of calculation of each increases naturally, and a very large hardware capacity or calculation capacity is required. It Further, from these drawbacks, it is very difficult to realize a multi-layer neural network which learns in a short real time and sets the converged weighting coefficient in a timely manner.

An object of the present invention is to solve the above-mentioned problems and to converge with a very small number of learning times as compared with a conventional learning method of a multilayer neural network, etc.
It is an object of the present invention to provide a new learning method for a multi-layer neural network capable of stably completing learning at 0 times higher speed and easily obtaining a desired output unit signal for a learning input signal.

[0009]

In order to solve the above problems, in a neural network for learning using a teacher signal, the absolute error signal obtained by subtracting the output unit signal of the neural network from the teacher signal If the value is smaller than the first threshold, a weighting factor update error signal having a polarity opposite to that of the error signal and having an amplitude that decreases with distance from the teacher signal is generated. In the region of the threshold of 2, the weighting coefficient update error signal of the same polarity having an amplitude smaller than that of the error signal, and when the absolute value of the error signal is larger than the second threshold, the same weight as the error signal is weighted. A neural network learning method characterized by generating a coefficient update error signal and updating the weight coefficient using the weight coefficient update error signal. It is formed.

[0010]

In the learning method of the multi-layer neural network of the present invention, for the output unit giving the output unit signal greatly deviated from the teacher signal, the error signal obtained by subtracting from the teacher signal is directly used as the weighting coefficient update error. On the other hand, in the output unit which is used as a signal and gives an output unit signal very close to the teacher signal, a weighting coefficient update error signal whose amplitude becomes smaller as the distance from the teacher signal having a polarity opposite to that of the error signal increases. In addition, in the output unit which gives the output unit signal relatively close to the teacher signal, the weight coefficient is corrected by using the weight coefficient update error signal having a slightly smaller amplitude with the same polarity as the error signal. Therefore, it is easy to get out of the stable local minimum and quickly approach the optimum state with a small number of learnings. We have the work. By using such a weight coefficient update error signal, it is possible to significantly reduce the number of learning without the dependence of the weight coefficient on the initial value, and to significantly reduce the number of learning, the number of intermediate units (hidden units), or the number of intermediate layers. Can be reduced.

As described above, the learning method of the present invention is easy to get out of the local minimum of a multilayer neural network, has no dependence on the initial value of the weighting coefficient, and converges very quickly and surely as compared with the conventional method. Since the output unit signal can be easily obtained, a large-scale multivalued neural network can be freely designed. In addition, we have realized a logic system with a learning function that requires re-learning in real time, and a multi-valued logic system using an input signal with an extremely large number of input signal elements and a multilayer neural network with a large number of units. it can. Furthermore, it is possible to easily design and realize pattern recognition, image processing, etc., which were difficult to obtain a desired output signal because the conventional method could not converge stably and at high speed, and also data conversion etc. Becomes

[0012]

EXAMPLES Examples 1-2 of a multilayer neural network using the learning method of the present invention will be given below, and the configuration and operation thereof will be described in detail when a binary teacher signal is used.

(Embodiment 1) FIG. 2 shows an embodiment of a learning process of a multilayer neural network using the learning method of the present invention. The input signal from the terminal 2 is input to the input unit of the input layer, and the output unit signal is output from the output unit of the output layer. Multilayer neural network 1, binary teacher signal, output unit signal, and correct / incorrect determination signal from the learning determiner 11. A weighting factor update error signal generator 10 that outputs a weighting factor update error signal, and a binary threshold circuit 6 that binarizes the output unit signal using a binary discrimination threshold circuit to obtain a binary output unit signal, The weighting factor controller 5 for updating and setting the weighting factor of the multilayer neural network 1 using the input weighting factor updating error signal, the output unit signal, the binary teacher signal, and the binary output unit signal as inputs, and the learning status Judgment is made and the binary teacher signal and the binary output unit signal are compared to detect a binary error in the output unit, and a correct / wrong judgment signal is sent. Learning determiner 11, multi-layer neural network 1
And initial setting of the weighting factor controller 5 and the learning judging device 11, and an operation mode controller 12 for controlling the start and end of learning.
Composed of.

Next, the operation in the learning process will be described. The multilayer neural network 1 performs learning using a learning input signal from the terminal 2 and a binary teacher signal from the terminal 3. In the weighting coefficient update error signal generator 10, for the output unit in which an incorrect binary output unit signal is detected with respect to the binary teacher signal by the correctness / wrongness judgment signal from the learning determiner 11, The error signal obtained by subtracting the output unit signal is output as the weighting coefficient update error signal as it is, while the absolute value of the error signal is given to the output unit giving the correct binary output unit signal. If it is less than or equal to the threshold value, a weighting coefficient update error signal having a polarity opposite to that of the error signal and having a smaller amplitude as the distance from the teacher signal is generated and output, and if the absolute value of the error signal is greater than the threshold value, The weighting factor update error signal having the same polarity or the same or a small amplitude is generated and output.

A method of generating one of the weighting factor update error signals in the case of using the binary or multi-valued teacher signal will be shown below by using an equation.

[0016]   If the multi-valued output unit signal of the output unit is correct (match),   When | Tm−Ym | ≦ dm,   Em = Tm-Ym-Dm1 · sgn (Tm-Ym) (1)                                     However, Dm1> dm

[0017]   When | Tm-Ym |> dm,   Em = Tm-Ym-Dm2 · sgn (Tm-Ym) (2)                                     However, dm ≧ Dm2

[0018]   If the multilevel output unit signal of the output unit is incorrect (incorrect),   Em = Tm-Ym (3)

Here,   sgn (x) = 1: x ≧ 0               = -1: x <0, (4)

M is an output unit number (1≤m≤M), E
m is a weight coefficient update error signal of the output unit m, Tm is a multivalued teacher signal of the output unit m, binary (0 or 1), Ym is a multivalued output unit signal of the output unit m,
dm is a threshold consisting of 0 or a positive constant for the output unit m, Dm1 is 0 or a positive constant for the output unit m, and Dm2 is 0 or a positive constant for the output unit m.

The expressions (1), (2) and (3) are expressed by the multi-value teacher signal Tm and the output unit signal Y in the output unit m.
The general calculation formula for obtaining the weighting factor update error signal by using m and m is shown. In the correct output unit m having no multilevel error in which the multilevel output unit signal and the multilevel teacher signal match, the output unit signal Ym is very close to the multilevel teacher signal Tm, and the output unit signal from the multilevel teacher signal Tm The absolute value of the error signal obtained by subtracting Ym is a constant dm (where ≧
If it is within the range of 0), the amplitude has a constant Dm1 as the error signal has a polarity opposite to that of the teacher signal as the distance from the teacher signal increases, as shown in Expression (1).
A weight coefficient update error signal smaller than (where ≧ 0) is used. Outside this range, a weighting coefficient update error signal is used, which is smaller than the error signal by a constant Dm2 (where ≧ 0) as shown in equation (2). On the other hand, if there is a multi-level error, the error signal is used as it is as the weighting coefficient update error signal as shown in Expression (3). Where dm is the first threshold and the multi-valued discrimination threshold is the second threshold,
If the first threshold is defined as the convergence region, it can be similarly applied to a neural network that handles continuous quantities.

The weight coefficient controller 5 uses the weight coefficient update error signal to correct the weight coefficient by the back propagation algorithm, and repeats learning so that the power of the error signal is minimized. In this way, unlike the conventional case, when the output unit signal is very close to the binary teacher signal, the direction of the error signal is reversed, and when the output unit signal is greatly separated and is erroneous, the weight coefficient is updated using the error signal as it is. Therefore, it is easy to get out of the stable local minimum, and the initial value dependence becomes very small, and it has the feature of quickly approaching the optimum state with a small number of learnings. As a result, the number of learnings can be significantly reduced by a factor of 10 to 100, and unnecessary learnings and intermediate units (hidden units)
The number or the number of intermediate layers can be significantly reduced.

Further, when the number of erroneous binary output unit signals does not decrease even if learning is repeated and falls into a very stable local minimum, the learning decision unit 11 examines a change in the number of binary erroneous signals and changes. Is small, it is possible to easily escape from a very stable local minimum by temporarily increasing the amplitude of the weighting coefficient update error signal in the opposite direction. Alternatively, the minimum value of the distance of the output unit signal from the binary discrimination threshold of the binary threshold circuit 6 is calculated for all the output units whose binary output unit signal does not match the binary teacher signal with respect to the learning input signal. , If the minimum value is larger than the given threshold, it is regarded as a very stable local minimum, and even if the weight coefficient is updated by temporarily increasing the amplitude of the weight coefficient update error signal in the opposite direction, Good. Further, the values of Dm1 and Dm2 may be changed according to the state of Tm ≧ Ym or Tm <Ym.

In the learning decision device 11, after the binary error is eliminated and the convergence state in which the binary teacher signal and all the binary output unit signals are coincident with each other is once achieved, the error signal is used as it is as the weight coefficient update error signal. Then, the weighting factor controller 5 may update the weighting factor. Further, the distance obtained by subtracting the binary discrimination threshold value from the output unit signal is calculated to obtain the minimum distance with respect to the learning input signal, and when this exceeds a preset threshold, the operation mode controller 12 outputs the learning end signal. May be sent to terminate the learning of the multilayer neural network 1. This allows
Avoid unnecessary learning and increase generalization.

The learning method of the present invention is similar to the above method in the neural network that handles a continuous amount of teacher signals in addition to the multi-valued neural network, by providing a region that is regarded as the correct answer for the teacher signals. Can be widely applied to the learning of neural networks using general teacher signals.

(Embodiment 2) A parallel binary neural network 1 is used as a multilayer neural network using the learning method of the present invention as Embodiment 2.
An example when applied to No. 3 will be shown. The main neural network 1 and the correction neural network 16 are connected in parallel to the input, and at the time of execution processing, the correction neural network 16 corrects a binary error in the main binary teacher signal in the binary output unit signal of the main neural network 1. The binary output unit signal is used for correction by the XOR additive logical operation processing. The parallel binary neural network 13 has a learning processing mode for learning a weighting coefficient for a learning input signal and an execution processing mode for sending a binary output signal to the input signal by using the learned weighting coefficient. It operates under two modes: Here, similarly, description will be made on the premise of a parallel binary neural network.

Here, the parallel 2 using the learning method of the present invention is used.
The configuration of the learning process in the value neural network 13 is shown in FIG. A main neural network 1 that performs learning using a previously prepared learning input signal from the terminal 2 and a main binary teacher signal T from the terminal 3, a learning input signal, and a correction 2 from the correction teacher signal generator 17. A weighting coefficient update error signal is generated from the correction neural network 16 that performs learning using the value teacher signal Tc, the output unit signal of the main neural network 1, the main binary teacher signal T, and the correctness determination signal from the learning determiner 14. Weight coefficient update error signal generator 10, weight coefficient update error signal for generating a weight coefficient update error signal from the output unit signal of the correction neural network 16, the corrected binary teacher signal Tc, and the correctness determination signal from the learning determiner 14. Based on the weight coefficient update error signals from the generator 18 and the weight coefficient update error signal generators 10 and 18, the main binary teacher signal T and the correction 2 The weighting coefficient is updated by the backpropagation algorithm so that the power of the error signal obtained by subtracting the output unit signals of the main and correction neural networks 1 and 18 from the teacher signal Tc is minimized, and the main and correction neural networks 1 and 18 are updated. Weighting factor controllers 5 and 19 for setting new weighting factors, and the output unit signal is binarized by a binary discrimination threshold to obtain a binary output unit signal.
The value threshold circuits 6 and 20, the main binary teacher signal T, the output unit signal and the binary output unit signal are input,
The correctness determination signal that detects a binary error that does not match the main binary teacher signal T in the binary output unit signal is output, and the distance from the binary discrimination threshold of the output unit signal in the case of error is obtained, and this minimum Learning determiner 14 for detecting a value,
When the learning of the main neural network 1 is completed, a binary error is obtained from the teacher signal T and the binary output unit signal at that time by an XOR logic operation, and this is used as a corrected binary teacher signal. A coincidence determiner 21 that detects a match between the value output unit signal and the corrected binary teacher signal Tc, sets the initial value of the main neural network 1 and controls learning start and end, and then a corrected teacher signal generator. The operation mode controller 15 performs the generation of the corrected binary teacher signal in 17 and the initial setting of the correction neural network 16 and the control of the start and end of learning, and switches to the execution processing mode when the learning processing ends.

In the weight coefficient update error signal generators 10 and 18, the weight coefficient update error signals given by the equations (1), (2) and (3) are obtained as in the first embodiment, and the weight coefficient update error signals are used. The weighting coefficient is updated and controlled in the controllers 5 and 19. The output unit signal is 2 as in the first embodiment.
If it is very close to the value teacher signal, by reversing the polarity of the weight coefficient update error signal to the error signal, it is possible to easily get out of the stable local minimum state.
An optimal state or a very stable local minimum state can be achieved with a small number of learnings without initial value dependence. Further, if the number of binary errors detected by the learning determiner 14 does not decrease even after repeating the learning of the main neural network 1, the state is in a very stable local minimum, and the learning determination is the same as in the first embodiment. If this condition is detected by the controller 14 and the amplitude of the weighting coefficient update error signal having a polarity opposite to that of the error signal is temporarily increased and input to the weighting coefficient controller 5, a very stable local minimum is obtained. You can even get out. At this time, if the number of binary errors is less than or equal to a given threshold, learning may be terminated.

In the learning decision unit 14, a method of detecting the number of binary errors between the binary output unit signal and the main binary teacher signal T and comparing it with the threshold, or 2 for each incorrect output unit A method to find the distance between the value discrimination threshold and the output unit signal, find the minimum distance between all output units of the incorrect answer to the learning input signal, and compare this with the given threshold (very stable. The learning of the main neural network 1 is terminated via the weighting factor controller 5 by any method such as detection of a local minimum state) or comparison of the number of times of learning with a prescribed number of times.

In particular, when the main neural network 1 performs the learning process and the main binary teacher signal and the binary output unit signal are in a completely converged converged state, the weighting coefficient of the correction neural network 16 is set. All are set to zero, and it is not necessary to learn the correction neural network 16.

On the other hand, if they do not match, a binary error in the binary output unit signal corresponding to the learning input signal with respect to the main binary teacher signal T at the end of learning is corrected in the corrected teacher signal generator 17. At XOR logical operation,
This is stored as the corrected binary teacher signal Tc and used as the corrected binary teacher signal Tc of the correction neural network 16.

When the above operation is completed, the learning of the correction neural network 16 is started using the learning input signal and the corrected binary teacher signal Tc via the operation mode controller 15, and the weighting coefficient update error signal is generated. The weighting factor update error signal is obtained from the corrected binary teacher signal Tc and the output unit signal of the correction neural network 16 by the device 18, and is input to the weighting factor controller 19 by the formulas (1), (2) and (3). , The weight coefficient is updated by the pack propagation algorithm so as to minimize the error power, and a new weight coefficient is set. In this learning process, the match determiner 21
In step 2, learning is repeated until a complete match between the corrected binary teacher signal Tc and the binary output unit signal obtained via the binary threshold circuit 20 is detected. The learning is terminated assuming that the correction neural network 16 is completely converged.
By causing the coincidence determiner 21 to perform the same processing as the learning determiner 14, it is possible to easily get out of a very stable local minimum.

In the parallel binary neural network 13 of this embodiment, the correction neural network 16 corrects the binary error in the binary output unit signal of the main neural network 1 with respect to the learning input signal as described above. The main neural network 1 does not necessarily have to converge as long as it can be learned as the signal Tc and completely converge. For example, if the binary output unit signal of the main neural network 1 does not have a binary error,
The ratio of the value output unit signal reaches about 85% or more with a very small number of times of learning, and the number of types of patterns of the correction binary teacher signal Tc at this time is very small. Signal correction neural network 1
6 completely converges with a very small number of learnings. Therefore,
It is clear that the dependency on the initial value of the weighting coefficient is eliminated as compared with the conventional parallel binary neural network, the number of learning can be further reduced, and the generalization ability can be enhanced.

Next, a configuration example of the execution processing of the parallel binary neural network 13 learned as described above will be described with reference to FIG.
Shown in. Main and correction neural networks 1 and 18 connected in parallel, a binary threshold circuit 6 for binarizing an output unit signal from the main neural network 1 using a binary discrimination threshold, and an output unit from a correction neural network 16. A binary threshold circuit 20 for binarizing a signal using a binary discrimination threshold;
An XOR additive logic operator 22 for performing an XOR logic operation on the corresponding binary output unit signals from the binary threshold circuits 6 and 20, and an operation mode controller for controlling the operation modes of the main and correction neural networks 1 and 16. 15 and.

The parallel binary neural network 13 is
Under the control of the operation mode control circuit 15, the operation processing mode is set to operate. At this time, the weighting factors obtained by learning in the learning processing mode are set in the main and correction neural networks 1 and 16, respectively. With respect to the input signal I, the output unit signals output from the main neural network 1 and the correction neural network 16 are converted into binary output unit signals by the binary threshold circuits 6 and 20, and are input to the XOR additive logical operation unit 22.

The XOR additive logical operation unit 22 processes each by an XOR logical operation, and outputs the binary output signal O of the parallel binary neural network 13 from the terminal 23. The main neural network 1 does not necessarily give a desired binary output unit signal to the learning input signal, but sends a binary output unit signal having a binary error. On the other hand, in the correction neural network 16, since the binary output unit signal equal to this binary error is transmitted, this binary error can be completely corrected by the XOR additive logical operation unit 22, and the parallel binary neural network 13 The desired binary output signal O is sent out from the terminal 23.

Since the number of types of patterns in the corrected binary teacher signal Tc is very small, the corrected neural network 16 is superior to the main neural network 1 in general in generality, but the main neural network 1 is unnecessary. By avoiding excessive over-learning by the learning determiner 14, the generalization ability of the parallel binary neural network 13 is enhanced. For these reasons, it is possible to realize a large-scale parallel binary neural network which does not depend on the initial value, converges quickly with a small number of learnings, and is highly generalizable. Further, it is not necessary to completely converge the main neural network 1 in learning, the number of hidden layers and hidden units can be greatly reduced, and the operation accuracy in the main and correction neural networks for performing a binary threshold circuit and XOR operation processing. Can be low, and the calculation scale can be small.

If the teacher signal is multi-valued with two or more values, 2
As the value threshold circuits 6 and 20, a multi-value threshold circuit that multi-values using a multi-value discrimination threshold is used, and X
The OR additive logical operation unit 22 performs multi-valued addition operation processing in place of the XOR circuit, and multi-valued subtraction operation processing in place of the XOR circuit in the corrected binary teacher signal generator 17 Extensions to neural networks are easily realized.

In the above-described Embodiment 1-2, the explanation has been made on the premise of the multilayer neural network, but a neural network other than the above may be used as long as it is a neural network which performs learning by using a teacher signal. Also,
Although a binary neural network has been described as an example here, the description is omitted, but the present learning method can also be applied to a neural network having a continuous amount of teacher signals instead of discrete integer values.

[0040]

As described above, in the learning method of the neural network of the present invention, the error signal obtained by subtracting from the teacher signal is given to the output unit which gives the output unit signal greatly separated from the teacher signal. Is used as the weighting coefficient update error signal as it is. On the other hand, for the output unit giving an output unit signal relatively close to the teacher signal, the weighting coefficient update error signal having an amplitude less than that error signal is used. In the output unit that gives an output unit signal very close to, a weighting coefficient update error signal with an amplitude that decreases with the polarity opposite to that of the error signal and decreases with the teacher signal is used. You can get out.

Further, when the learning is in a very stable local minimum in which the number of errors between the teacher signal and the multilevel output unit signal does not change, it has a polarity different from that of the error signal. By updating the weight coefficient by temporarily increasing the amplitude of the weight coefficient update error signal,
You can easily get out of a stable local minimum. Therefore, the number of learning times is 10 to 100 times that of the conventional method.
It is significantly shortened by about twice. Further, since it is possible to stably converge to the optimum state by using the minimum required number of intermediate units or the number of intermediate layers, the hardware scale and the amount of calculation can be reduced.

The multi-valued neural network or the parallel multi-valued neural network using the learning method of the present invention uses a smaller number of intermediate layer units or intermediate layers than the conventional method, has no initial dependency, and has low calculation accuracy. With the weighting factor, the weighting factor converges rapidly and stably, and a desired multilevel output signal can be transmitted. From this, it is necessary to freely design and realize a large-scale multivalued logic circuit or the like, which is difficult to be realized by the conventional technology, in a short time by the multivalued neural network using the present invention, and rapid learning so far. In addition, it has a very wide range of effects such as wide application to artificial intelligence systems and search systems, pattern recognition, data conversion, data compression, multi-valued image processing, and communication systems that require complete convergence. There is.

If a plurality of sets of weighting coefficients learned by using different binary teacher signals are prepared and the weighting coefficients of the binary neural network of the present invention are switched and set, the truth table of the binary logic function is obtained. , Etc. can be easily learned in a short time, so that a programmable large-scale variable binary logic circuit with a fixed delay can be easily realized, and learning can be done in a short time according to the situation. A new large-scale binary logic circuit can be realized on the same hardware.

[Brief description of drawings]

FIG. 1 is a configuration example of a learning process in a multilayer neural network according to a conventional learning method.

FIG. 2 is a configuration example of a learning process of a multilayer neural network using the learning method of the present invention in the first embodiment.

FIG. 3 is a configuration example of a learning process of a parallel binary neural network using the learning method of the present invention in the second embodiment.

FIG. 4 is a configuration example of an execution process of a parallel binary neural network according to the second embodiment.

[Explanation of symbols]

1 Multilayer neural network 2 Input signal input terminal 2 ₁ Input signal unit input terminal 2 ₂ Input signal unit input terminal 2 _N Input signal unit input terminal 3 Teacher signal input terminal 3 ₁ Teacher signal unit input terminal 3 ₂ Teacher signal unit input terminal 3 _M teacher signal unit input terminal 4 subtracter _{4 1} subtractor _{4 second} subtracter _{4 M} subtracter 5 weight coefficient controller 6 binary threshold circuit ₆₁ binary threshold circuit _{6 2} binary threshold circuit _{6 M} binary threshold circuit 7 Binary Threshold Circuit 7 ₁ Binary Threshold Circuit 7 ₂ Binary Threshold Circuit 7 _M Binary Threshold Circuit 8 Match Detector 9 Operation Mode Controller 10 Weighting Factor Update Error Signal Generator 10 ₁ Weighting Factor Update Error Signal Generator 10 ₂ weight coefficient updating error signal generator 10 _M weighting coefficient updating error signal generator 11 learning determination 12 operation mode controller parallel binary using the learning method of the present invention 13 the neural network 14 learn determiner 15 operation mode controller 16 corrects the neural network 17 corrects the binary teacher signal generator 18 weight coefficient updating error signal generator 18 ₁ weight coefficient updating error signal generator 18 ₂ weighting factor updating error signal generator 18 _M weighting coefficient updating error signal generator 19 weighting factor controller 20 binary threshold circuit 20 ₁ binary threshold circuit 20 ₂ binary threshold circuit ₂₀ M 2 Value threshold circuit 21 Matching discriminator 22 XOR additive logic operator 23 Output signal output terminal 23 ₁ Output signal unit output terminal 23 ₂ Output signal unit output terminal 23 _M Output signal unit output terminal

Continuation of front page (56) References JP-A-4-15860 (JP, A) JP-A-4-251392 (JP, A) JP-A-3-273444 (JP, A) International Publication 95/5640 (WO, A1) Fumiaki Sugaya, Yotaro Yatsuka, “Convergence Characteristics of Parallel-type Neural Networks”, Proceedings of the 1993 Autumn Meeting of the Institute of Electronics, Information and Communication Engineers, Japan, The Institute of Electronics, Information and Communication Engineers, August 15, 1993, 6th Separate volume, pp. 18 Fumiaki Sugaya, Yotaro Yatsuka, “Fast convergence using code conversion of parallel neural network”, IEICE Spring Conference 1994 Spring Conference, Japan, The Institute of Electronics, Information and Communication Engineers, March 10, 1994, 6th volume, pp. 66 Kajiro Watanabe et al., “Speeding up learning of neural network by adaptive adjustment of learning coefficient”, Transactions of the Society of Instrument and Control Engineers, Japan, 1994, Vol. 30, No. 9, pp. 1093-1099, JST Material No .: S 0104A (58) Fields investigated (Int.Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7/60 JST file (JOIS) CSDB (Japan Patent Office)

Claims (5)

(57) [Claims] 1. In a neural network for learning using a teacher signal, if the absolute value of the error signal obtained by subtracting the output unit signal of the neural network from the teacher signal is smaller than a first threshold, the error signal When the weight coefficient update error signal whose amplitude becomes smaller as the distance from the teacher signal having the opposite polarity is increased, and the absolute value of the error signal is in the region between the first threshold and the second threshold, the error signal If the absolute value of the weighting coefficient update error signal of the same polarity having a smaller amplitude is larger than the second threshold, the same weighting coefficient update error signal as the error signal is generated, and the weighting coefficient update error is generated. A neural network learning method characterized by updating weighting factors using signals. 2. In a multivalued neural network for learning using a multivalued teacher signal, a multivalued output unit obtained from the output unit signal of the multivalued neural network from the multivalued teacher signal through a multivalued threshold circuit. In the output unit in which the signal matches the multi-valued teacher signal, when the absolute value of the error signal obtained by subtracting the output unit signal from the multi-valued teacher signal is less than or equal to the given threshold, the output unit has a polarity opposite to that of the error signal. A weighting factor update error signal whose amplitude decreases with increasing distance from the multi-valued teacher signal is generated. When the absolute value of the error signal is greater than the threshold, the weighting factor update error of the same polarity having an amplitude equal to or less than the error signal is generated. In the output unit that generates an error signal and the multi-value output unit signal does not match the multi-value teacher signal, the same weight coefficient update error signal as the error signal is generated. A multi-valued neural network learning method characterized by generating a signal and updating the weighting coefficient using the weighting coefficient update error signal. 3. The neural network according to claim 2, wherein the polarity of the error signal is inverted when the error number of the output unit signal, which is different from the teacher signal, does not decrease more than a prescribed value even after repeated learning. Multi-valued neural network learning method characterized by updating the weighting coefficient by temporarily increasing the amplitude of the weighting coefficient updating error signal. 4. The multi-valued neural network according to claim 2, wherein the multi-valued output unit signal does not match the multi-valued teacher signal with respect to a learning input signal in all output units, The minimum value is obtained in the distance of the output unit signal from the multi-valued discrimination threshold, and when the minimum value is larger than the prepared threshold, the amplitude of the weighting factor update error signal whose polarity is inverted from that of the error signal. A multi-valued neural network learning method characterized by updating the weighting coefficient by temporarily increasing. 5. A main and a correction neural network using the multivalued neural network according to claim 2, 3 or 4, are connected in parallel, and the main neural network is trained using a main multivalued teacher signal for learning. The multi-valued output unit signal obtained from the output signal of the main neural network through the multi-valued threshold circuit with respect to the input signal is different from the main multi-valued teacher signal. In a parallel multi-valued neural network for sequentially learning as corrected multi-valued teacher signals, all output units in which the multi-valued output unit signal of the main neural network and the main multi-valued teacher signal do not match the learning input signal. , The minimum value of the distance between the multi-level discrimination threshold of the multi-level threshold circuit and the output unit signal is , A parallel multi-valued neural network learning method characterized by terminating learning of the main neural network when a threshold value given in advance is exceeded.