JP3224831B2 - Neural network device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network device, and more particularly, to a neural network device in which different learning methods can be appropriately selected for neurons having the same configuration.

[0002]

2. Description of the Related Art Neural networks have begun to be widely used for identification problems such as speech recognition, motion control problems for robots and the like, various process control problems, and neurocomputers. The basic configuration of a neural network is
As shown in FIG. 5, it is composed of a neuron unit j and a synapse unit connecting the neuron units.

In neural learning, as described in Reference 1: "Mathematics of Neural Networks", Shun-ichi Amari, Sangyo Tosho, 1978, the load (synapse load) Wij set in a synapse unit is the synapse. It is known that the product r · xi of the intensity xi of the input signal to the unit and the learning signal r from the neuron unit j changes as an update amount. At this time, the synapse load wij is updated by Wij (t + 1) = Wij (t) + Cr (t) .xi (t). C is a coefficient that determines the speed of learning.

Here, various learning methods can be obtained depending on what kind of signal is used as the learning signal r. For example, a method using the output of the neuron unit j as the learning signal r is called Hebb learning, and a method using the difference between the teacher signal and the output of the neuron unit j as the learning signal r is called an error correction learning method. . The method of using the teacher signal itself as the learning signal r is called correlation learning. Furthermore, when the difference between the teacher signal and the output of the neuron unit j is used as an evaluation function by a nonlinear optimization method, and a learning signal r that reduces this evaluation function is generated, the error back propagation learning method is realized (Reference 2). : ARTIFICAL NEURAL
SYSTEMS, PERGAMON PRESS, 1990).

[0005] Each of these learning methods has unique capabilities. For example, Hebb's learning has noise suppression capability (Ref. 3: Self-Organization in a Perceptual Networ).
k, R. Linsker et.al, IEEE Computer, March 1988 pp. 105
-117). As for the backpropagation learning method, the feasibility of arbitrary nonlinear mapping has been shown (Reference 4: Neural and Co.)
ncurrent Real-Time Systems, John Wiley & Sons, 1989)
. These abilities are obtained only when learning is correctly performed, and learning often does not proceed or learning is not established due to a phenomenon called over-learning.

Therefore, in order to eliminate the drawbacks of the individual learning methods, a method of connecting neural networks using different learning methods in series or in series / parallel is disclosed in the above-mentioned document 1. However, preparing a plurality of types of neural networks having different learning methods not only increases the work required for integrated circuit design, but also increases the overall circuit scale, thereby causing a significant cost increase. There is.

[0007] Furthermore, in order to avoid the problem of learning failure in the error backpropagation learning method, a method of re-learning with an initial value different from that at the start of learning has also been proposed. Lack of knowledge on how to achieve this.

[0008]

As described above, the method of connecting a plurality of types of neural networks having different learning methods in series or in series / parallel in order to eliminate the disadvantages of the individual learning methods in the neural network requires a large cost. In order to avoid the problem of learning failure in the error back propagation learning method, there is a specific means for re-learning with an initial value different from that at the start of learning. There was no problem.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a neural network device in which a plurality of different learning methods having the same configuration can be arbitrarily selected without preparing a plurality of types of hardware (integrated circuits) for a neuron unit. And

[0010]

In order to solve the above-mentioned problems, the present invention provides a plurality of neuron units and a synapse load updated based on a learning signal to input signals to these neuron units. In a neural network device having a synapse unit as a basic component, each neuron unit generates a plurality of learning signals corresponding to different learning methods, and updates a synapse load from the plurality of learning signals. And a learning signal selecting means for selecting one of the learning signals.

In the present invention, it is preferable that each neuron unit is constituted by the same integrated circuit, and is controlled so as to perform predetermined processing by an external control signal.

The learning signal generating means generates a learning signal corresponding to at least two learning methods of, for example, the Hebb learning method, the differential Hebb learning method and the error back propagation learning method.

[0013]

As described above, according to the present invention, since each neuron unit selects one learning signal from a plurality of learning signals corresponding to different learning methods and supplies it to the synapse unit, the same learning signal is selected. The learning method can be changed by the configured neuron unit. Therefore,
For example, in a multilayer perceptron-type neural network, by changing the learning method for each layer, it becomes possible to perform signal processing having the ability of each learning method. Particularly, in the error back propagation learning method, it is also possible to change the initial value for avoiding the problem of learning failure by switching the learning signal.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of the entire neural network device according to one embodiment of the present invention.
This neural network device realizes a multilayer perceptron type neural network schematically shown in FIG. 6 by an integrated circuit, and connects an intermediate layer neuron chip 1, an output layer neuron chip 2 and both neuron chips 1 and 2 to each other. It consists of synaptic chips 3 to be connected.

Each of the neuron chips 1 and 2 is composed of three neuron units 4 and has a neuron input terminal F for each neuron unit 4 as an input / output terminal.
I, a neuron output terminal FO, a neuron learning signal input terminal EI, and a neuron learning signal output terminal EO. The neuron units 4 used in the neuron chips 1 and 2 all have the same configuration.

On the other hand, the synapse chip 3 is composed of nine synapse units 5 of the same configuration connected in a matrix, and has a synapse input terminal SFI connected as an input / output terminal to the neuron output terminal FO of the intermediate layer neuron chip 1. , A synapse learning signal output terminal SEO connected to the neuron learning signal input terminal EI of the intermediate layer neuron chip 1, a synapse learning signal input terminal SEI connected to the neuron learning signal output terminal EO of the output layer neuron chip 2, and an output layer neuron The chip 2 has a synapse output terminal SFO connected to the neuron input terminal FI.

In the synapse chip 3, each of the three in the row direction
The synapse units arranged one by one are synapse input terminals S
An output signal from the same neuron unit in the intermediate layer neuron chip 1 is input via FI, and the learning signals output from these synapse units are subjected to current addition and then output via a synapse learning signal output terminal SEO. It is output and supplied to the same neuron unit.

Each of three synapse units arranged in the column direction receives a learning signal from the same neuron unit in the output layer neuron chip 2 by a synapse learning signal input terminal S.
Signals input through the EI and output from these synapse units are subjected to current addition, output through the synapse output terminal SFO, and supplied to the same neuron unit.

As shown in FIG. 2, the neuron unit 4 includes a sigmoid function circuit 21, a sigmoid derivative operation circuit 22, a subtractor 23, a multiplier 24, and a function changeover switch 2.
5, 26 and a learning signal selection switch 27.

The inputs of the sigmoid function circuit 21 and the sigmoid derivative operation circuit 22 are input to the neuron input terminal FI.
Are connected in common. The sigmoid function circuit 21 is a threshold processing element, and when an input is a and an output is b,
(A) = b is performed and the output is sent to the neuron output terminal FO. When the input is a and the output is b, the sigmoid derivative operation circuit 22 performs an operation of dF (a) / da = b.

The subtractor 23 subtracts the output of the sigmoid function circuit 21 from the signal input from the neuron learning signal input terminal EI. The function changeover switches 25 and 26 are for switching the function of the neuron unit 4, and when the neuron unit 4 is an intermediate layer, the switches 25 and 26 are used.
Is turned off, the switch 26 is turned on, and when the neuron unit 4 is in the output layer, the switch 25 is turned on.
The switch 26 is turned off. The multiplier 24 includes a signal input from the neuron learning signal input terminal EI or an output signal of the subtracter 23 and the sigmoid derivative operation circuit 2
2 is multiplied by the output signal.

Here, in the neuron unit 4, a plurality (three in this case) of learning signals corresponding to different learning methods are generated. That is, the learning signal corresponding to the back propagation learning method is obtained from the output of the multiplier 24, the learning signal corresponding to the Hebb learning method is obtained from the output of the sigmoid function circuit 21 (the output of the neuron unit 4), and the sigmoid derivative operation circuit The output of the sigmoid function circuit 21
, That is, a learning signal corresponding to the differential head learning method is obtained. Learning signal selection switch 2
7 selects one of these three learning signals by external control and outputs it from a neuron learning signal output terminal EO.

The synapse unit 5 includes multipliers 31 to 33 and a register 34, as shown in FIG. The first multiplier 31 multiplies the signal input to the synapse input terminal SFI by the synapse load held in the register 34 and outputs the result to the synapse output terminal SFO. The second multiplier 32 multiplies the signal input to the synapse learning signal input terminal SEI by the synapse load held in the register 34 and outputs the result to the synapse learning signal output terminal SEO. The third multiplier 33 multiplies the learning signal input to the synapse learning signal by the signal input to the synapse input terminal SFI, and updates the value of the synapse load held in the register 34. That is, the synapse load W is updated by W (T + 1) = W (T) + ΔW (T is a time unit), where ΔW is the multiplication result of the multiplier 33.

FIG. 4 is a block diagram showing in detail a part of the configuration when the multilayer perceptron shown in FIG. 1 is realized using the neuron unit and the synapse unit shown in FIGS. Is coupled to the intermediate layer neuron unit 4a via the synapse unit 5a, the output of the intermediate layer neuron unit 4a is coupled to the output layer neuron unit 4b via the synapse unit 5b, and the output signal y from the output layer neuron unit 4b
Is output. The configurations of the neuron units 4a and 4b and the synapse units 5a and 5b are as shown in FIGS. 2 and 3, respectively.

The function changeover switches 25 and 26 and the learning signal selection switch 27 are controlled by the control circuit 6,
In the intermediate layer neuron unit 4a, the switch 25
Is turned off, the switch 26 is turned on, and the subtractor 2
3 is bypassed, and in the output layer neuron unit 4b, the switch 25 is on and the switch 26 is off. From the intermediate layer neuron unit 4a and the output layer neuron unit 4b, the learning signals r ₂ and r ₁ selected by the learning signal selection switch 27 are output from the synapse units 5a and 5 respectively.
b, and are multiplied by the respective inputs (outputs of the immediately preceding neuron unit) in the synapse units 5a and 5b, thereby calculating the update amount ΔW of the synapse load. However, an external learning signal (or teacher signal) r ₀ is input to the neuron learning signal input terminal of the output layer neuron unit 4b.

In this case, since the learning signal selection switch 27 can be individually switched for each neuron layer or for each neuron group, it is possible to perform signal processing having features of a plurality of learning methods. For example, the first stage of a cascade-connected neural network converts a set of input signal vectors into a set of orthogonal vectors, and the subsequent stage performs associative learning with the orthogonalized input signal, so that complicated processing can be executed by the same hardware. . Further, by performing the orthogonalization learning at the preceding stage, it is possible to avoid the problem that the learning speeds under different environments are different.

Further, when learning fails in the error back propagation learning method, if the learning signal selection switch 27 is used to temporarily switch to another learning method, for example, the Hebb learning method or the differential Hebb learning method, the initial value of the learning can be changed. Since this has been changed, this problem of learning failure can be easily avoided.

Further, since it is possible to easily and freely combine neural networks having different characteristics with the same hardware, when the brain function is elucidated in a constructive manner, the constituent elements of the brain can be extremely easily constructed. It can be realized.

In the above-described embodiment, the learning signals corresponding to the three learning methods of the Hebb learning method, the differential Hebb learning method, and the back propagation learning method can be selected in the neuron unit. It is likewise possible to select learning signals corresponding to various learning methods. In addition, the present invention can be variously modified and implemented without departing from the gist thereof.

[0030]

As described above, according to the present invention, the neuron unit has a function of selecting any one learning signal from a plurality of learning signals corresponding to different learning methods and supplying the selected learning signal to the synapse unit. Thus, with a simple hardware configuration, a desired complicated signal processing can be easily realized by an arbitrary combination of a plurality of different learning methods. Further, the problem of learning failure in the error back propagation learning method can be easily avoided by changing the initial value by temporarily switching to another learning method.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a multilayer perceptron type neural network device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a neuron unit in the embodiment.

FIG. 3 is a block diagram showing a configuration of a synapse unit in the embodiment.

FIG. 4 is a block diagram showing an intermediate layer and an output layer neuron unit and synapse units before and after it in the embodiment.

FIG. 5 is an explanatory diagram showing an outline of learning in a neural network.

FIG. 6 is a diagram schematically showing a multilayer perceptron type neural network device.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Middle layer neuron chip 2 ... Output layer neuron chip 3 ... Synapse chip 4, 4a, 4b
... neuron unit 5, 5a, 5b ... synapse unit 6 ... control circuit 21 ... sigmoid function circuit 22 ... sigmoid derivative operation circuit 23 ... subtractor 24 ... multiplier 25, 26 ... function changeover switch 27 ... learning signal selection switch 31- 33: Multiplier 34: Register

Claims (3)

(57) [Claims] 1. A neural network device comprising a plurality of neuron units and a synapse unit for applying a synapse load modified based on a learning signal to input signals to these neuron units as basic components, The neuron unit includes learning signal generation means for generating a plurality of learning signals corresponding to different learning methods, and learning signal selection means for selecting one learning signal for updating the synaptic load from the plurality of learning signals. A neural network device comprising: 2. The neural network device according to claim 1, wherein the plurality of neuron units are formed of integrated circuits having the same configuration, and are controlled so as to perform predetermined processing by an external control signal. 3. The learning signal generating means according to claim 1, wherein said learning signal generating means generates a learning signal corresponding to at least two of a Hebb learning method, a differential Hebb learning method and an error back propagation learning method. A neural network device as described.