JPH05307624A - Signal processor - Google Patents

Abstract

PURPOSE:To attain rapid processing for the hardware processing of a neural network by executing a forward process and a learning process in parallel. CONSTITUTION:When an input signal is applied, a forward processing executing circuit(FWD) 11 executes a forward process on an intermediate layer and outputs its execution result to a register 12. An FWD circuit 13 reads out the forward process result on the intermediate layer from the register 12, executes a forward process on an output layer and an error forming circuit 48 finds out an error signal on the output layer based upon the execution result of the circuit 13 and a teacher signal read out from a register 17 and stores the error signal in a register 18. A leaning processing execution circuit(LRN) 47 reads out the error signal on the output layer from the circuit 18 and the input signal to the output layer from a register 45 and executes a learning process on the output layer. Similarly an LRN circuit 46 reads out signals from registers 19, 44 and executes a learning process on the intermediate layer.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing device such as a neural computer imitating a nerve cell circuit network applied to character and figure recognition, motion control of robots, associative memory and the like.

[0002]

2. Description of the Related Art The function of a nerve cell (neuron), which is a basic unit of information processing of a living body, is mimicked, and further, this "nerve cell mimicking element" (nerve cell unit) is configured in a network to obtain information. Aiming for parallel processing,
This is a so-called neural network. Many things, such as character recognition, associative memory, and motion control, can be easily achieved by the conventional Neumann computer even if they are performed easily in the living body.

Attempts to solve these problems by imitating the functions of the nervous system of the living body, in particular, the functions peculiar to the living body, that is, parallel processing and self-learning, are being actively made, centering on computer simulation. ..

FIG. 1 is a schematic diagram showing one neuron model, which is composed of a part receiving an input from another neuron, a part converting an input according to a certain rule, and a part outputting a result. A variable weight “Tij” called a coupling coefficient is attached to each of the coupling portions with and to indicate the coupling strength. Learning means changing this value, which changes the structure of the network.

FIG. 2 is a schematic diagram illustrating a hierarchical network of this type. This network is composed of an input layer A1, an intermediate layer A2, and an output layer A3, and one neuron includes many other neurons. Combined with
There is no connection between neurons in each layer, and there is no connection from the output layer A3 to the input layer A2. Note that FIG.
In the above, a three-layer network having one intermediate layer is used, but a multilayer network having a plurality of intermediate layers is also used.

In the forward process, in the case of the network shown in FIG. 2, the syringe input to the input layer A1 propagates to the intermediate layer A2, and the forward process is executed in the intermediate layer. The processing result of the intermediate layer is propagated to the output layer, the forward processing is executed in the output layer, and finally the output of the neural network is obtained. Here, the degree of coupling between neurons is called a coupling coefficient, and the degree of coupling between the i-th neuron and the j-th neuron is generally represented by Tij. There are two types of coupling: excitatory coupling, in which the output of the partner neuron is larger as the signal is larger, and conversely, inhibitory coupling in which the output of the partner is smaller, the output is smaller, and Tij> 0.
Represents excitatory coupling, and Tij <0 represents inhibitory coupling.

Now, assuming that the own neuron is the i-th neuron and the output of the j-th neuron is yj, Tij · yj obtained by multiplying this by the coupling coefficient Tij becomes the input to the own unit. Since each neuron is connected to a large number of neurons, ΣTij · yj, which is the result of adding Tij · yj to those neurons, that is, ΣTij · yj is the input to its own neuron. Become. This is called an internal potential and is expressed by the following mathematical formula 1.

[0008]

[Equation 1] ui = ΣTij · yi

Next, the input is subjected to non-linear processing to obtain the output of the nerve cell unit. The nonlinear function used here is called a nerve cell response function, and for example, a sigmoid function f (ui) as shown in the following formula 2 and FIG. 3 is used.

[0010]

## EQU2 ## f (ui) = 1 / (1 + e- ^x )

From the above, the output yi of the neuron i is expressed by the following equation 3.

[0012]

## EQU00003 ## yi = fi (ui) = fi (.SIGMA.Tij.yi)

As a learning process, a general back propagation algorithm (hereinafter abbreviated as BP algorithm) will be briefly described below. In the learning pattern, when a certain input pattern p is given, the coupling coefficient is changed so as to reduce the error between the actual output value and the desired output value. The algorithm that asks for this change is BP
It is an algorithm.

Now, given a certain input pattern P, the actual output value (ypi) of the unit i and the desired output value (tp
The difference of i) is defined as in Expression 4.

[0015]

[Number 4] ^{Ep = (tpi-ypi) 2} /2

This represents the error of the output layer unit i, t
pi is teacher data given by humans. In learning, the strength of all connections is changed to reduce this error. Actually, the change amount of the coupling coefficient Tij when the pattern p is given is changed by using the following formula 5. Further, from Equation 4, Equation 6 is obtained as a result.

[0017]

[Equation 5] ΔpTij∝−ΞE / ΞTij

[0018]

[Equation 6] ΔpTij = ηδpiypj

Here, ypj is an input value from the unit j to the unit i, and the error δpi differs depending on whether the unit i is the output layer A3 or the intermediate layer A2, and the error signal δpi in the output layer A3 is expressed by the equation 7 Then, the error signal δpi in the intermediate layer A2 is expressed by Equation 8, respectively. However, in these mathematical expressions, fi '(ui) is the first derivative of fi (ui).

[0020]

## EQU7 ## .delta.pi = (tpi-ypi) .fi '(ui)

[0021]

## EQU8 ## δpi = fi '(ui) ΣδpkTki

From the above, Equation 9 is a general formulation of ΔTij, and therefore Equation 10 is derived. Where n
Is a learning order, η is a learning constant, and α is a stabilizing coefficient.

[0023]

## EQU9 ## ΔTij (n + 1) = ηΔpiypj + αΔTij (n)

[0024]

(10) Tij (n + 1) = Tij (n) + ΔTij (n + 1)

The first term on the right side of the equation 9 is the variation ΔT of the coupling coefficient obtained by the equation 6, and the second term reduces the error vibration.
It is added to speed up the convergence.

In this way, the calculation of the change amount ΔT of the coupling coefficient starts from the unit of the output layer A3 and moves to the unit of the intermediate layer A2, and the learning is performed in the direction opposite to the processing of the hair force data, that is, backward. move on. Therefore, learning by back-propagation involves first inputting and outputting training data (forward) and changing all bond strengths (rearward) so as to reduce error in the results. Will be repeated until is converged.

FIG. 4 is a block diagram showing a general circuit in the case where the forward processing of one neuron is performed by a numerical operation. The region 4 including the memory 2 storing the synapse coupling coefficient and the multiplier 3 is a neuron. Where the synapse function is performed, the product of the input signal and the synapse coefficient is calculated. The cell body 8 cumulatively adds the product of the coupling coefficient calculated by the synapse and the input signal by the adder 5, performs processing by the sigmoid function by the conversion table 7, and outputs this value. This corresponds to the function of the cell body of the neuron. The above is the content of the forward process of one neuron.

The conventional hierarchical nerve cell network forms a network as shown in FIG. 2, and the data flow of the forward process of this three-layer hierarchical network is as shown in FIG. The input signals a1 to a4 are given and the output signals b1 to b4 are obtained from the output layer A3.

As shown in FIG. 7, in a multi-layer neural network including an input layer A, a plurality of intermediate layers B1 to Bn-1 and an output layer C, data is similarly transferred from the input layer to each output layer through each intermediate layer. Flowing.

Next, an example of performing the forward process of the three-layer structure will be described with reference to the circuit diagram of FIG. 9 and the timing chart of FIG. Here, FIG. 9 is a circuit diagram of a circuit that configures and executes a three-layer hierarchical network by a circuit that executes the forward process of one neuron shown in FIG. 4, and FIG. 10 is a timing chart of the forward process. is there.

First, in state S1 of FIG. 10, the input signal 10 of the input layer is input to the intermediate layer forward processing execution circuit 11, and the intermediate layer forward processing start signal FWDS1 of the control circuit 15 is activated. Thereby, the forward process by the first input signal is executed, and the result is held in the register 12. And
In the next state S2, the value of the register 12 is input to the output layer forward processing execution circuit 13, and the output layer forward processing start signal FWDS of the control circuit 15 is input.
2 is activated to perform the forward process in the output layer, and the output 14 is obtained. In this way, the forward processing is executed on the input signals 10 successively given in a pipeline manner as shown in FIG.

Also in the multi-layer network having a large number of intermediate layers, as shown in the circuit diagram of FIG. 11 and the timing chart of FIG. 12, the forward process is executed by the pipeline method as in the above-mentioned three-layer network.

Note that the processing by the learning process is currently executed by an external general-purpose computer in many cases.

FIG. 33 is a diagram showing an example in which the above network is realized by an electric circuit (JP-A-62-2951).
No. 88), basically, a plurality of amplifiers 103 having an S-shaped transfer function, and a resistive feedback circuit for connecting the outputs of the amplifiers 103 to the inputs of amplifiers of other layers as shown by a chain line. A net 101 is provided.
A CR time constant circuit 102 including a grounded capacitor C and a grounded resistor R is individually connected to the input side of each amplifier 103. The input currents I1, I2 to IN
Are supplied to the input of each amplifier 103 and the output is obtained from the set of output voltages of these amplifiers 103.

Here, the strength of the input or output signal is represented by a voltage, and the strength of the nerve cell coupling is defined by the resistance 100 (resistive feedback network 10) connecting the input / output lines between the cells.
It is represented by a resistance value of a grid point 1), and the nerve cell response coefficient is represented by a transfer function of each amplifier 103. Further, the connection between nerve cells has excitability and inhibitory property as described above, and is mathematically represented by the sign of the coupling coefficient. But,
Since it is difficult to realize positive and negative with a constant on the circuit, here, by dividing the output of the amplifier 103 into two and inverting one output, two positive and negative signals 103a,
This is realized by generating 103b and selecting it appropriately. Further, an amplifier is used as the one corresponding to the sigmoid function shown in FIG.

34 to 36 are diagrams showing an example in which the neural network is realized by a digital circuit. FIG. 34 is a diagram showing a circuit configuration example of a single nerve cell.
10 is a synapse circuit, 111 is a dendrite circuit, and 112 is a cell body circuit. FIG. 35 is a diagram showing a configuration example of the synapse circuit 110 shown in FIG. 34, and the coefficient circuit 110a.
A rate multiplier 110b to which a value obtained by multiplying the input pulse f by a factor a (a factor of 1 or 2 by which the feedback signal is multiplied) is input is provided, and the weight value w is stored in the rate multiplier 110b. The synapse load register 110c is connected. Also, FIG.
6 is a diagram showing a configuration example of the cell body circuit 112, which is formed by sequentially connecting a control circuit 113, an up / down counter 114, a rate multiplier 115 and a gate 116, and further is provided with an up / down memory 117. There is.

In this case, the input / output of the nerve cell unit is represented by a pulse train, and the pulse density represents the amount of signal. The coupling coefficient is treated as a binary number and stored in the synapse weight register 110c. The signal calculation process is performed as follows.

First, the input signal is applied to the rate multiplier 1
By inputting into 10b and inputting the coupling coefficient into the rate value, the pulse density of the input signal is reduced according to the rate value. This is the Tijy of the above-mentioned BP model Equation 1.
Corresponds to the i part. Also, the Σ part of ΣTijyi is
This is realized by the OR circuit shown by the dendrite circuit 111. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance, and OR is performed for each group.
Take In FIG. 34, F1 indicates excitatory output and F2 indicates inhibitory output.

An output can be obtained by inputting these two outputs to the up side and down side of the counter 114 shown in FIG. 36 and counting respectively. Since this output is a binary number, the rate multiplier 115 is used again to convert it into a pulse density. A neural network can be realized by configuring a network using a plurality of these nerve cell units.

The learning function is realized by inputting the final output of the network to an external computer, performing numerical calculation inside the computer, and writing the result to the synapse weight register 110c which stores the coupling coefficient.

The present applicant has already developed a signal processing device using a nerve cell network composed of nerve cell mimicking elements,
A patent application has been filed (Japanese Patent Application No. 1-343491). In the present invention, the signal processing device is treated as an example of one embodiment. The signal processing device already developed by the applicant will be described below.

In this signal processing device, as an example of a neural network, a signal processing by a nerve cell unit using a digital logic circuit and a network circuit configured by using the nerve cell unit is proposed.

The basic idea here is: Input / output signals, intermediate signals,
The coupling coefficient, the teacher signal, etc. are all represented by a pulse train represented by binary values of "0" and "1". 2. The value of the signal inside the network is represented by the pulse density (the number of "1" within a certain fixed time). 3. Calculation in the nerve cell unit is performed by logical operation between pulse trains. Four. The pulse train of the coupling coefficient is stored in the memory in the nerve cell unit. Five. In learning, an error is posted based on a given teacher signal pulse train, and the coupling coefficient is changed based on this error. At this time, the calculation of the error and the variation of the coupling coefficient are all "0" and "1". It is performed by the logical operation of the pulse train. This will be described in detail below.

FIG. 13 shows the state of forward processing of one neuron element in the pulse density method, and considers the hierarchical neural network shown in FIG. 2 as the network configuration.

First, all the inputs and outputs of the nerve cell unit are binarized into "0" and "1", and the logical product (AND) of the input yi represented by the pulse density and the coupling coefficient Tij is synapse-wise. Ask for. This corresponds to Tij · yj in Expression 1. The output pulse density of this AND circuit stochastically becomes the product of the pulse density of the input signal and the pulse density of the coupling coefficient.

As described above, the neuron-side connection includes excitatory connection and inhibitory connection. In the case of numerical calculation, the sign of the coupling coefficient, for example, plus when excitatory and minus when inhibitory is performed.

In the case of the pulse density method, each coupling is divided into two groups, an excitatory coupling and an inhibitory coupling, depending on whether the coupling coefficient Tij is positive or negative, and the logical sum is obtained by OR operation for each group. This corresponds to the processing of Σ in Equation 3 and the processing of the non-linear saturation function fi.

That is, in the calculation based on the pulse density, when the pulse density is low, the pulse density as a result of the OR processing can be approximated to the pulse density of the OR input. Since the output of the OR circuit gradually becomes saturated as the pulse density becomes higher, the result does not match the sum of the pulse densities, and nonlinearity appears.

In the case of this OR operation, the value P of the pulse density is 0≤P≤1, and since it is a monotonically increasing function with respect to the magnitude of the input, it is the same as the processing by the sigmoid function of Equation 2 or FIG. become.

The output of the nerve cell element according to the pulse density method is the OR output a of the excitatory group obtained as described above.
Is "1" and the OR output b of the inhibitory group is "
"1" is output only when it is "○", that is, it is expressed as follows.

[0051]

[Equation 11] a = ∪ (yi∩Tij) (0 <T = excitability)

[0052]

[Equation 12] b = ∪ (yi∩Tij) (0> T = inhibitory property)

[0053]

Y i = g (a, b) = a∩ * b

The learning process in the pulse density method will be described below. In a non-learned neural network, the output of the network when a certain pattern is input is not always the desired output. Therefore, the coupling coefficient is changed so as to reduce the error between the actual output value and the desired output value by the learning process similarly to the back propagation algorithm described above.

(Signal Error in Output Layer) First, the error signal in the output layer will be described. If the error is expressed numerically, both positive and negative values can be taken. However, since such an expression cannot be made in the pulse density method, two signals, a positive component signal and a negative component signal, are used and the output layer The error in is defined as

[0056]

[0057]

[0058]

[Equation 14]

[0059]

[Equation 15]

That is, the + component (δ +) of the error signal becomes "1" when the output result is "0" and the teacher signal is "1",
Other than that, it becomes "0".

On the other hand, the minus component (δ-) of the error signal becomes "1" when the output result is "1" and the teacher signal is "0".
Other than that, it becomes "0".

These error signals δ + and δ − correspond to the equation 7 for obtaining the error signal of the output layer in the above-mentioned back propagation algorithm.

(Error Signal in Intermediate Layer) The error signal in the intermediate layer according to the pulse density method is also obtained by referring to the equation 8 by the above-mentioned BP algorithm. That is, the error signals in the output layer are collected and used as its own error signal. Here, the connection is divided into two groups depending on whether excitability is inhibitory, the product part is represented by ∩ (AND), and the sum (Σ) part is represented by ∪ (OR). Further, when the error signal in the intermediate layer is obtained, it is divided into four cases of positive and negative of the coupling coefficient Tij and positive and negative of the error signal δ.

First, in the case of excitatory coupling, an error signal plus component δ + of the output layer and the coupling coefficient (δ + i∩Tij) are calculated for all output layer neutrons. Take the OR. This becomes the error signal plus component δ + of the hidden layer neuron.

[0065]

[Equation 16] δ + = ∪ (δ + i ∩ T + ij)

Similarly, an AND (δ-i∩Tij) of the error minus component δ- of the output layer and its coupling coefficient is obtained for all output layer neutrons, and these O
Take R. This becomes the error signal minus component δ − of the hidden layer neuron.

[0067]

[Expression 17] δ- = ∪ (δ-i ∩T + ij)

Next, the case of inhibitory binding will be described.
An AND (δ-i∩T-ij) of the error signal minus component δ- of the output layer and its coupling coefficient is obtained for all neurons of the output layer, and the OR of these is taken. This becomes the error signal plus component of the hidden layer neuron.

[0069]

[Equation 18] δ + = ∪ (δ-i ∩ T-ij)

Similarly, the error signal of the output layer plus the component δ +
AND of its coupling coefficient (δ + i∩T-ij)
Is calculated for all neurons in the output layer, and these are ORed. This is the minus component of the error signal of the hidden layer neurons.

[0071]

[Formula 19] δ- = ∪ (δ + i∩T-ij)

The connection between a neuron in a middle layer and the neuron in the output layer connected to the middle layer includes an excitatory connection and an inhibitory connection. Therefore, as the error signal plus component of the intermediate layer, the logical sum of δ + of the excitatory coupling of Equation 16 and δ + of the inhibitory coupling of Equation 18 is obtained. Similarly, as the error signal minus component of the intermediate layer, the logical sum of δ- of the excitatory coupling of Equation 17 and δ- of the inhibitory coupling of Equation 19 is obtained. That is, Equation 20 and Equation 21 are obtained, and these are B
It corresponds to Formula 8 by the P algorithm.

[0073]

[Equation 20] δ + = {∪ (δ + i∩T + ij)} ∪ {∪ (δ-i
∩ T-ij)}

[0074]

[Formula 21] δ-= {∪ (δ-i∩T + ij)} ∪ {∪ (δ + i
∩ T-ij)}

(Processing by Learning Constant η) In the BP algorithm, the process of the constant η is described as the averaging in the formula 6 for obtaining the correction amount ΔT of the coupling coefficient. In the numerical calculation, the learning constant η may be simply multiplied as in Expression 6, but in the case of the pulse density method, the pulse train is thinned out as shown below according to the value of the learning constant η. To be realized.

[0076]

[0077]

[Equation 22]

[0078]

[Equation 23]

[0079]

[Equation 24]

Next, a method of obtaining the correction amount ΔT of the coupling coefficient by learning will be described. First, the error signal (δ +, δ−) in the output layer or the intermediate layer is processed by the learning constant η, and the logical product with the input signal to the neuron is calculated (δ∩y). However, since there are δ + and δ− in the error signal, they are calculated as T + and T−, respectively, as shown in the following equations 25 and 26. These correspond to Equation 6 for obtaining ΔT in the BP algorithm.

[0081]

[Equation 25] ΔT + = δ + ∩y

[0082]

[Expression 26] ΔT- = δ- ∩ y

Based on these, new coupling coefficient New
Tij is obtained, and it is classified depending on whether the coupling coefficient Tij is excitatory or inhibitory.

First, in the case of excitability, the ΔT + component is reduced with respect to the original T +.

[0085]

[Equation 27] New_Tij ⁺ = Tij ⁺ ∪ΔT ⁺ ∩Δ * T-

Next, in the case of the inhibitory property, the ΔT ⁺ component is decreased and the ΔT ⁻ component is increased with respect to the original T ⁻ .

[0087]

[Number 28] ^{New_Tij - = Tij - ∪ΔT - ∩} * ΔT +

The learning algorithm based on the pulse density method has been described above.

Here, in the hierarchical network of FIG. 2, the processing flow of the forward process and learning process in the pulse density method will be briefly described.

First, in the forward process, when an input signal is first given to the input layer, this input signal propagates to the intermediate layer, and the signal processing of the intermediate layer is performed by the above equations 11 to 13. , Propagate the result to the output layer. In the output layer, these propagated signals are similarly processed by the equations (11) to (13), the output signal is obtained as a result of these, and the forward process is terminated.

In the learning process, after performing the above forward process, a teacher signal is further given to the output layer. In the output layer, an error signal in the number of outputs is obtained by Expressions 14 and 15 and sent to the intermediate layer. At the same time, the error signal is processed by the learning constant η of Equations 22 to 24,
After the logical product of the input signal from the intermediate layer is obtained by the equations 25 and 26, the coupling strength between the output layer and the intermediate layer is changed by the equations 27 and 28.

Next, as the processing in the intermediate layer, the error in the intermediate layer is calculated by the equations 20 and 21 based on the error signal sent from the output layer, and the error signal is calculated by the equations 22 to 24. Processing is performed by a constant η, the coupling strength between the intermediate layer and the input layer is changed by Equations 25 and 26, and the learning process ends. After that, the learning process is repeated until it converges.

Next, an actual circuit configuration based on the above algorithm will be described with reference to FIGS. The structure of the neural network is the same as in FIG. 14
FIG. 15 is a diagram showing a circuit of a portion corresponding to a synapse of a neuron, and FIG. 15 is a diagram showing a circuit of a portion corresponding to a cell body of the neuron. Further, FIG. 16 is a diagram showing a circuit of a portion for obtaining an error signal in the output layer from the output of the output layer and the teacher signal. By forming a network of these three circuits as shown in FIG. 2, a digital neural network circuit capable of self-learning can be realized.

First, FIG. 14 will be described. 20 is an input signal to the nerve cell unit. The synapse coupling coefficient is stored in the shift register 27. The terminal 27A is a data outlet and the terminal 27B is a data inlet. As long as it has a function similar to that of the shift register, another one, for example, one including a RAM and an address controller may be used.

The circuit 28 uses (y in equations 10 and 12).
i ∩ Tij), which is the AND of the input signal and the coupling coefficient. This output must be grouped according to whether the coupling is excitatory or inhibitory, but it is more versatile to prepare outputs 23 and 24 for each group in advance and switch which group is output. high. Therefore, a bit indicating whether the coupling is excitatory or inhibitory is stored in the memory 33, and the signal is switched by the switching gate circuit 32 using the information.

Further, as shown in FIG. 15, a gate circuit 34 having a plurality of OR gates corresponding to the logical sum of Expressions 11 and 12 for processing each input is provided. Further, as shown in the figure, a gate circuit 3 including an AND gate and an inverter that outputs only when the excitatory group is “1” and the inhibitory group is “0”, which is shown in Expression 13.
5 are provided.

Next, the error signal will be described. 14
Is a diagram showing a circuit for generating an error signal in the output layer, where A
It is a logic circuit that combines ND and inverter,
This corresponds to Formula 14 and Formula 15. That is, the error signals 40 and 41 are generated from the output 38 from the output layer and the teacher signal 39. Expressions 16 to 19 for obtaining the error signal in the intermediate layer are performed by the gate circuit 29 having the AND gate configuration shown in FIG. 14, and output 2 depending on + and −.
1, 22 are obtained.

Since the error signal to be used is different depending on whether the coupling is excitatory or inhibitory, it is necessary to make a distinction in that case. In this case, information on excitatory or suppressiveness stored in the memory 33 is used. And the error signals + and −, 25 and 26 are performed by the gate circuit 31 having AND and OR gate configurations. Also, Equation 20 and Equation 2 that collect error signals
1 is performed by the gate circuit 36 having the OR gate structure shown in FIG. Expressions 22 to 24 corresponding to the learning rate are performed by the frequency dividing circuit 37 shown in FIG.

Finally, the part for calculating a new coupling coefficient from the error signal will be described. This is expressed by Equations 25 to 28, and these operations are performed by AND, shown in FIG.
This is performed by the gate circuit 30 having an inverter and OR gate configuration. This gate circuit 30 must also be classified depending on the excitability / inhibition of the coupling, which is performed by the gate circuit 31 shown in FIG.

[0100]

The hierarchical neural network described above forms a network as shown in FIG. Here, as shown in FIG. 5, the input layer (layer A1 on the left side of FIG. 5)
To the output layer (the layer A3 on the right side of FIG. 5) by applying the input signal to the input layer, and as shown in FIG. Consider a learning process in which a signal is given, the coupling coefficient between the output layer and the intermediate layer is changed, and further the coupling coefficient between the intermediate layer and the input layer is changed.

First, in the forward process, when an input signal is first given to the input layer, this input signal propagates to the intermediate layer, and as the signal processing of the intermediate layer, the operations of the above formulas 1 and 2 are performed. And propagate the result to the output layer. In the output layer, for these propagated signals,
Similarly, the operations of the above-mentioned formulas 1 and 2 are executed, and as a result thereof, the output signal is obtained.

In the learning process, after performing the above forward process, a teacher signal is further given to the output layer. In the output layer, the error in the output layer is obtained by the above equation 5, the error is propagated to the intermediate layer, and the strength of the connection between the nerve cell unit of the output layer and the nerve cell unit of the middle layer, that is, the coupling coefficient. Is changed according to Equation 9 and Equation 10.

Next, as processing in the intermediate layer,
The error in the intermediate layer is calculated by the following, and the strength of the connection (coupling coefficient) between the nerve cell unit of the middle layer and the nerve cell unit of the input layer is changed by the above equations 9 and 10 to obtain the learning process. To complete.

The forward process and the learning process are
The complicated processing described above and the enormous amount of numerical operations are required, which takes a considerable amount of time. In particular, the learning process has more processing contents than the forward process, and generally, the learning process needs to be learned not only once but also hundreds or thousands of times, so this calculation time is enormous.
Even if dedicated hardware is used for speeding up, it is still insufficient.

Further, in the hierarchical neural circuit model, not only the three-layer structure as shown in FIGS. 5 and 6, but also the total number of intermediate layers is various as shown in FIGS. 7 and 8. As described above, as the number of layers increases, the time required for the forward process and the learning process described above considerably increases.

FIG. 37 shows the conventional signal flow in the three-layered neural network shown in FIG.

The FWD circuits 4 and 5 are circuits for executing the forward process of the intermediate layer A2 and the output layer A3 in FIG. 2, and correspond to 28 and 32 in FIG. 14 and 34 and 35 in FIG. The learning circuits LRN6 and 7 are the intermediate layer A in FIG.
2 and a circuit that executes the learning process of the output layer A3.
4 corresponds to 29, 30, 31 and 36 and 37 in FIG. The error generation circuit ERR8 is for obtaining an error signal in the output layer and is shown by the circuit in FIG.

Referring to FIG. 37, the processing steps of one learning process will be described. First, when the input signal 1 is given to the input layer, this signal propagates to the intermediate layer, and the FW of the intermediate layer is transmitted.
The D circuit 4 executes the forward process in the intermediate layer.

Next, this result propagates to the FWD circuit 5 of the output layer, where the forward process of the output layer is executed, and the forward process is completed. In the case of the learning process, further, the teacher signal 3 is given to the output layer, the error generating circuit 8 generates an error signal in the output layer, and this is propagated to the learning circuit LRN7 in the output layer, and the error signal in this output layer is generated. Then, the learning process in the output layer is executed from the input signal to the FWD circuit 5. The error signal of the intermediate layer generated here propagates to the learning circuit LRN6 of the intermediate layer, and the learning process in the intermediate layer is executed from the error signal of the intermediate layer and the input signal to the FWD circuit 4. The learning process will be completed.

As described above, particularly in the learning process, there is a problem that the processing speed cannot be sufficiently increased even if the dedicated circuit is used to increase the speed because the processing path is long. Furthermore, when the number of intermediate layers increases as shown in FIGS. 7 and 8, the processing time will increase accordingly.

It is an object of the present invention to provide a signal processing device capable of high speed processing when a neural network is hardened.

[0112]

A first signal processing device according to the present invention is a signal processing device based on a nerve cell network composed of nerve cell mimicking elements, wherein each nerve cell mimicking the nerve cell network is imitated. The element has a means for holding an input signal input during the forward process and a means for holding an error signal input during the learning process, and is characterized in that the forward process and the learning process are performed in parallel.

A second signal processing device according to the present invention is a signal processing device based on a hierarchical nerve cell network composed of nerve cell mimicking elements, wherein a forward process of a neural network consisting of an input layer, an intermediate layer and an output layer. Is divided into several processes, and each divided process is pipelined.

A third signal processing device according to the present invention is a signal processing device based on a hierarchical neural cell network composed of neural cell mimicking elements, wherein the neural network forward process including an input layer, an intermediate layer and an output layer is used. The learning process is divided into several processes, and each divided process is pipelined.

A fourth signal processing device according to the present invention is a signal processing device having a hierarchical neural cell network composed of neural cell mimicking elements, wherein initial values are provided for each of the input layer, the intermediate layer and the output layer. And a means for setting an initial value to the counter of each layer before the transfer of the forward process and giving a common forward process execution signal to the counter of each layer during execution of the forward process. It is characterized in that the execution signals are counted to generate a forward process execution signal having a predetermined value for executing the forward process of the layer.

A fifth signal processing device according to the present invention is a signal processing device based on a hierarchical neural cell network composed of neural cell mimicking elements, wherein counters are provided for each of the input layer, the intermediate layer and the output layer, A register for storing the forward process start state, a comparator for comparing the value of the counter with the value of the register, and a state for starting the forward process of the layer in the register of each layer before the transfer of the forward process, A forward process executing means having a means for giving a common forward process executing signal to each layer at the time of execution, and the counter of each layer operates by the forward process executing signal, and the comparator of each layer is the counter of the layer. Forward to each layer when the value is greater than or equal to the register value Characterized in that it is intended to generate a forward process execution signal for executing the process.

A sixth signal processing apparatus according to the present invention is a signal processing apparatus based on a hierarchical neural cell network composed of neural cell mimicking elements, and an initial value provided for each layer of an input layer, an intermediate layer and an output layer. And a means for setting an initial value to the counter of each layer before shifting to the learning process and giving a common learning process execution signal to the counter of each layer when the learning process is executed. It is characterized in that the execution signal is counted and a learning process execution signal of a predetermined value for executing the learning process of the layer is generated.

A seventh signal processing apparatus according to the present invention is a signal processing apparatus based on a hierarchical neural cell network composed of neural cell mimicking elements, wherein counters are provided for each of the input layer, the intermediate layer and the output layer, A register for storing a learning process start state, a comparator for comparing the value of the counter with the value of the register, and a state for starting the learning process of the layer in the register of each layer before the learning process is transferred to the learning process. A learning process execution means having a means for giving a common learning process execution signal to each layer at the time of execution is provided, and the counter of each layer operates by the learning process execution signal, and the comparator of each layer has a comparator of the layer. Generate a learning process execution signal that causes each layer to execute the learning process when the value is greater than or equal to the value in the register Characterized in that the at it.

An eighth signal processing device according to the present invention is a signal processing device according to a hierarchical nerve cell network composed of nerve cell mimicking elements, wherein the forward process executing means of the fourth or fifth signal processing device according to the present invention is used. When,
The learning process execution means of the sixth or seventh signal processing device according to the present invention, which can execute the learning process independently of the forward process.

A ninth signal processing device according to the present invention is a signal processing device having a hierarchical neural cell network composed of neural cell mimicking elements, wherein a forward process and a learning process are performed for each of the input layer, the intermediate layer and the output layer. A process counter provided for controlling the process, a register for storing the forward process start state, a comparator for the forward process for comparing the value of the register with the value of the process counter, and a learning process start state A register and a learning process comparator that compares the value of this register with the value of the above process counter are provided, and the forward process of the layer is started in the register that stores the forward process start state of each layer before the process transition. Memorize the state to
The state that starts the learning process of the layer is stored in the register that stores the learning process start state of each layer,
Means for giving a common process execution signal to each layer during process execution, the process counter of each layer counts this process signal, and the comparator for the forward process has a value of the process counter of the forward process of the layer. A forward process execution signal for starting the forward process when the register value for storing the start state is equal to or more than the register value for storing the start state. The learning process comparator is a register value for storing the learning process start state of the layer in the value of the process counter. It is characterized in that a learning process execution signal for starting the learning process is generated at the above time.

[0121]

In the first signal processing device according to the present invention, the forward process and the learning process can be executed in parallel by having the means for holding the input signal input during the forward process and the error signal input during the learning process. Therefore, the processing speed can be increased as compared with the case where the forward process and the learning process are serially executed.

In the second signal processing device according to the present invention,
Since the forward process of the neural network including the input layer, the intermediate layer, and the output layer is subdivided and each process is pipelined, the forward process can be processed at a higher speed than in the case of performing a series of processes. Also, the intermediate layer is 1
In the case of using multiple layers instead of layers, if each process is not pipelined, the processing time of the forward process will increase accordingly. However, according to the present invention, the processing speed is the same as in the case of one intermediate layer. It becomes the same, and high-speed processing becomes possible.

In the third signal processing apparatus according to the present invention,
The forward process and the learning process of the neural network consisting of the input layer, the intermediate layer, and the output layer are subdivided, and each process is pipelined. Therefore, the forward process and the learning process are processed faster than a series of processes. be able to. Further, in the case where the intermediate layer is not a single layer but a multi-layered one, if each process is not pipelined, the processing time of the forward process and the learning process will be increased by that much. The speed is the same as in the case of one intermediate layer, and high speed processing is possible.

In the fourth signal processing device and the fifth signal processing device according to the present invention, regardless of the number of layers of the neural network, an initial value is set in the counter of each layer before the transfer of the forward process, and one control signal is generated. , That is,
By providing the common forward process execution signal, the forward process execution signals of the respective layers are generated at different timings, so that the forward processing of each layer can be started at different timings.

In the sixth signal processing apparatus and the seventh signal processing apparatus according to the present invention, regardless of the number of layers of the neural network, the counter of each layer is set to an initial value before the learning process is started, and one control signal is generated. That is, by giving a common learning process execution signal, the learning process execution signal of each layer is generated at different timings, so that the learning process of each layer can be started at different timings.

In the eighth signal processing device according to the present invention,
Regardless of the number of layers in the neural network, set the initial value to the counter of each layer before moving to the forward process,
By giving one control signal, that is, the common forward process execution signal, the forward process execution signals of the respective layers are generated at different timings, so that the forward processing of each layer can be started at different timings and the learning process can be started. Prior to the transition, the counter of each layer is set to an initial value, and one control signal, that is, a common learning process execution signal is given to generate the learning process execution signal of each layer at different timings. Will be able to start at different times.

In the ninth signal processing device according to the present invention,
Regardless of the number of layers in the neural network, initial values are set in the process counter, the forward process register, and the learning process register of each layer before the process transition, and one control signal, that is, a common process execution signal is given. As a result, the forward processing of each layer can be started at different timings, and the learning processing of each layer can be started at different timings.

[0128]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a neural network realized by the above digital circuit will be described below.

As described above, in the case of executing the forward process and the learning process in the neural network as shown in FIG. 2, even if dedicated hardware is used, the processing speed is insufficient.

Therefore, parallel processing, that is, pipeline processing is conceivable as a method for speeding up signal processing in the neural network model shown in FIG. This will be explained in the case of the learning process in the three-layered network shown in FIG. Forward process in the middle layer 2. Forward process in output layer 3. Learning process in output layer 4. The four processing processes of the learning process in the middle layer are performed in parallel.

However, in the learning processes 3 and 4, since the input signals in the forward process 1 and 2 are required, the input signal in the forward process is held and the input signal held in the learning process is held. It is necessary to devise to use.

Considering the above points, FIG.
2 shows a signal processing circuit capable of executing the processing of (1) completely in parallel. 17 is different from FIG. 37 in that a register 1 for holding process intermediate data is used.
2, 17 to 19 and registers 42 to 44 capable of holding the input signal of the forward process for a specific period. The input signal 10 is the input signal 201, the output 14 is the output 202, and the LRN circuit 46 is the LRN circuit 2
06, the LRN circuit 47 to the LRN circuit 207,
The circuit 48 is the ERR circuit 208, and the FWD circuit 11 is the FW.
The D circuit 204 and the FWD circuit 13 correspond to the FWD circuit 205, respectively.

Each processing executed in parallel will be described below.

(1) Input the input signal 1 to the FWD circuit 11
To perform the forward process in the middle tier,
The result is taken into the register 12. Also, input signal 1
0 is taken into the register 42.

(2) The result of the forward process in the intermediate layer is read from the register 12, and the FWD circuit 13 executes the forward process in the output layer.
From this result and the teacher signal read from the register 17, an error signal in the output layer is obtained by the error generation circuit 48 and stored in the register 18. Further, the input signal to the forward process circuit 13 of the output layer is stored in the register 45.
At this time, the input signal to the intermediate layer is transferred from the register 42 to the register 43.

(3) The error signal in the output layer is read from the register 18, the input signal to the output layer is read from the register 45, and the LRN circuit 47 executes the learning process in the output layer. The error signal in the intermediate layer generated at this time is stored in the register 19. At this time, the input signal to the intermediate layer is transferred from the register 43 to the register 44.

(4) The error signal in the intermediate layer is read from the register 19, the input signal to the intermediate layer is read from the register 44, and the learning process in the intermediate layer is executed by the LRN circuit 46.

As described above, a register for storing the input signal and each intermediate data in the intermediate layer and the output layer is added,
Since the necessary data can be read out when needed, the four processes (1) to (4) can be executed in parallel. The timing chart at this time is shown in FIG. As can be seen from this example, in the case of the three-layer structure, the execution speed can be increased to four times that of the conventional method by dividing the series of processes into four and pipelining each process.

Further, as shown in FIGS. 7 and 8, even when the number of layers of the neural network is increased, similarly, the forward process and the learning process are processed in parallel to enable the pipeline processing. When parallel processing is not performed, the time required for the forward process and the learning process increases as the number of layers increases, but when parallel processing as described above is performed, the processing speed becomes the same as in the three-layer configuration, That is, the processing speed does not depend on the number of layers, and speed processing is possible. The number of stages of the registers 42 to 45 when the number of layers is increased may be adjusted according to the number of stages of pipeline processing.

The signal processing apparatus shown in FIG. 19 is also one in which the above-described processing can be executed in parallel completely, and the FWD circuits 51 and 53 perform the forward process of the intermediate layer A2 and the output layer A3 in FIG. In the circuit to be executed, in the pulse density method, 28, 32 and FIG.
34 and 35. The LRN circuits 60 and 58 are circuits for executing the learning process of the intermediate layer A2 and the output layer A3 in FIG. 2, and are 29, 30, 31 in FIG. 14 and 36 in FIG.
It corresponds to 37. The ERR circuit 56 obtains an error signal in the output layer and is shown by the circuit in FIG. Less than,
The processing steps of the learning process executed in parallel will be described with reference to FIGS.

(1) At the same time as applying the input signal 50 to the input layer, the intermediate layer forward processing start signal FWDS1 of the control circuit 62 is activated in the state S1 as shown in FIG. Thereby, the FED circuit 5 of the intermediate layer
1 performs the forward process in the middle tier and stores the result in register 52. The input signal 50 is stored in the register 61 in this state S1.

(2) Control circuit 6 in state S2
When the output layer forward processing start signal FWDS2 of No. 2 is activated, the intermediate layer forward processing result stored in the register 52 is propagated to the output layer FWD circuit 53, where the output layer forward process is executed. , Output 54 completes the forward process. In the case of the learning process, the teacher signal 55 is further applied to the output layer, and the teacher signal 55 generates an error signal in the output layer, which is stored in the register 57.

(3) Control circuit 6 in state S3
No. 2 output layer learning process start signal LRNS2 is activated, and the learning process in the output layer is performed in the LRN 58 according to the error signal in the output layer stored in the register 57 and the execution result of the forward process in the intermediate layer stored in the register 52. Run with. Then, the error signal of the intermediate layer generated here is stored in the register 59.

(4) Control circuit 6 in state S4
2 activates the learning process start signal LRNS1 of the intermediate layer, and the error signal of the intermediate layer stored in the register 59 propagates to the learning circuit LRN60 of the intermediate layer,
The learning process in the middle layer is executed, and the learning process is completed.

The processing steps of the three-layer hierarchical network have been described above, but as shown in FIGS. 7 and 8, the forward process and the learning process are processed in parallel even when the total number of neural networks is increased. This allows pipeline processing. The processing speed of this multilayer neural network is the same as that of the three-layer structure because pipeline processing is performed, and high-speed processing is possible.

In another embodiment of the present invention, the forward processing start timing of the intermediate layer or the output layer can be freely set, and the intermediate layer and the output layer are shown in the block diagram of FIG. Thus, the forward process circuit 76 including the forward process counter (FWD CNT) 70 and the circuit (FWD) 71 that executes the forward process is provided.

The FWD CNT 70 generates a forward processing start signal FWDS and outputs it to the FWD 71. Specifically, before the transfer to the forward process, FWD CNT7
The counter initial value is loaded into 0. Then, when the forward process start signal FWDST to the counter is activated at the start of the forward process, the counter starts to operate, and when the value of the counter becomes “0”, the forward process start signal FWDS is output to the FWD 71 of the layer. To do. FIG. 23 shows a circuit diagram when a three-layer hierarchical network is constructed by the above forward processing circuit 76.

In the forward processing of this three-layer hierarchical network, as shown in the timing chart of FIG. 22, first, the states S0 and S before the transition to the forward processing are performed.
1 to the FWD CNT70 of the intermediate layer as an initial value, for example, FFh (-1 in decimal) in 8-bit binary display,
Similarly, FWD CNT70 of the output layer is set to F as the initial value.
Eh (-2 in decimal) is stored. From this state, when the forward process start signal FWDST of the control circuit 77 of the neural network is activated, the FWD CNTs 70 in the intermediate layer and the output layer start operating and the forward process is started.

After that, in the state S2, the FWD of the intermediate layer
The value of CNT 70 becomes 00h, the forward process start signal FWDS of the intermediate layer becomes active, and the forward process of the intermediate layer is started. Furthermore, state S
At 3, the value of the output layer FWD CNT70 becomes 00h, the output layer forward process start signal FWDS becomes active, and the output layer forward process is started.

As described above, in this three-layer hierarchical network, the forward process start signal of the intermediate layer and the output layer can be shared by one line as shown in FIG. 23, and compared with the conventional example shown in FIG. Control becomes easier. Similarly, in a multi-layer network having a plurality of intermediate layers (see FIG. 7), control lines can be shared as shown in FIG. 24 and controllability is improved as compared with FIG. 11 of the conventional example.

In a further embodiment of the present invention, the forward processing timing of the intermediate layer or the output layer can be set freely as in the previous example shown in FIG. 25, a process counter (PRO CNT) 72, a forward process start state set register (FREG) 73,
A forward processing circuit 76 including a comparator (CMP) 74 and a circuit (FWD) 75 that executes forward processing is provided.

The FREG 73 is loaded with the forward process start state before shifting to the forward process. PRO
CNT72 is reset before the forward processing transition,
The operation is started when the forward process start signal FWDST is activated. CMP74 is F
Enter the value of REG73 and the value of PRO CNT72,
When they are equal to each other, the forward processing start signal FWDS is output to the FWD 75.

The three-layer hierarchical network constructed by the forward processing circuit 76 of FIG. 25 is as shown in FIG. 23. As shown in the timing chart of FIG. 26, first, the states S0 and S before the transfer to the forward processing are performed.
For example, 01h is stored in FREG73 of the intermediate layer in 8-bit binary representation. In addition, the output layer FREG73
In the same way, 02h is stored. Then, when the forward process is started, the PRO CNT 72 is reset and the forward process start signal FWDST is activated, so that the PRO CNT 72 starts operating and the forward process is started.

In the intermediate layer, the value of PRO CNT72 is 0.
When it reaches 1h, that is, in the state S2, the forward process start signal FWDS becomes active, and the forward process starts. In the output layer, PRO CN
When the value of T72 becomes 02h, that is, in the state S3, the forward process start signal FWDS becomes active and the forward process starts.

As described above, the forward process is executed by the pipeline process by the three-layer network.

As shown in FIG. 23, this three-layer hierarchical network can share the forward process start signal of the intermediate layer and the output layer by one line, and is easier to control than the conventional example shown in FIG. Become. Similarly, in a multi-layer network having a plurality of intermediate layers (see FIG. 7), as shown in FIG.
As compared with the conventional example shown in FIG. 1, the control line can be shared as shown in FIG. 26, and the controllability is improved.

In yet another embodiment of the present invention, the start timings of the forward process and learning process of the intermediate layer or the output layer can be set freely, and the idea is to use the forward process of FIG. The starting method is applied not only to the forward process but also to the learning process.

FIG. 27 shows the middle layer and the output layer of this embodiment.
2, a forward processing circuit 76 and a learning processing circuit 80 are provided, and the forward processing circuit 7 and the learning processing circuit 80 are provided.
6 has the same configuration as the forward processing circuit 76 shown in FIG. 21, the description thereof will be omitted to avoid duplication.

The learning processing circuit 80 uses the learn counter (L
RN CNT) 78 and a circuit (L
RN) 79, the LRN CNT 78 outputs the learning process start signal LRNS to the LRN 79 when the learning process start state is loaded before the process transition and the learning process start signal LRNST given from the control circuit 81 becomes active. Composed.

In this three-layer hierarchical network,
As shown in the timing chart of FIG. 28, in the states S0 and S1 before the process transition, for example, in the FWD CNT70 of the intermediate layer, FFh (-1 in decimal) in 8-bit binary display as an initial value, and the FWD C of the output layer.
Similarly, FEh (decimal number -2) is set in NT70, FFh (decimal number -1) is set in the output layer LRN CNT78, and FEh (decimal number -2) is set in the intermediate layer LRN CNT78.

Next, in the state S1, the control circuit 8
When the forward start signal FWDST of 1 is activated, the FWD CNTs 70 of the intermediate layer and the output layer start to be activated, the forward processing of the intermediate layer is started in the state S2, and the forward processing of the output layer is started in the state S3. The processes up to this point are the same as those in each of the above-described forward processes only shown in FIGS.

Here, further, in the state S3, this time, the learning process start signal LRNS of the control circuit 81.
When T is activated, the LRN between the output layer and the middle layer is
CNT78 starts, and LRN CNT78 in the output layer becomes 0.
When it becomes 0h, that is, in the state S4, the output layer L
The learning process start signal LRNS is output from the RN CNT 78 to the LNR 79 of the output layer, and the learning process of the output layer is started. In addition, LRN CNT78 of the intermediate layer is 0
When it becomes 0h, that is, in the state S5, L of the intermediate layer
The learning process start signal LRNS is output from the RN CNT 78 to the LNR 79 of the intermediate layer, and the learning process of the intermediate layer is started.

In this way, the forward process and the learning process of the intermediate layer and the output layer are executed by pipeline processing. Further, this three-layer hierarchical network can share the forward process start signal FWDST between the intermediate layer and the output layer and the learning process start signal LRNST between the intermediate layer and the output layer, respectively, as compared with the conventional example of FIG. Sexuality improves. This also applies to a multi-layer network.

Another embodiment of the present invention is one in which the start process of the forward process and the learning process of the intermediate layer and the output layer can be freely set by a method different from the neurons shown in FIG. The idea is to apply the method of starting the forward process of the embodiment shown in FIGS. 25 to 26 not only to the forward process but also to the learning process.

The intermediate layer and the output layer of this embodiment are shown in FIG.
0, the forward processing circuit 76 shown in FIG.
A forward processing circuit 86 and a learning processing circuit 90 that are configured similarly to the above are provided.

The forward processing circuit 86 has a process counter (PRO CNT) 82, a forward process start state set register (FREG) 83, a comparator (CMP) 84 and a circuit (FWD) 85 for executing the forward processing. The learning processing circuit 90 uses the learning process start state set register (LREG) 8
7, a comparator (CMP) 88, and a circuit (LRN) 89 that executes a learning process.

The FREG 83 is loaded with the forward process start state before shifting to the forward process. PRO
The CNT 82 is reset before the process transition and starts operating when the forward process start signal FWDST is activated. CMP84 is the value of FREG83 and PR
The value of O CNT 82 is input, and when they become equal, the forward processing start signal FWDS is output to the FWD 85.

The LREG 87 is loaded with the learning process start state before shifting to the learning process. CMP84 is LR
The value of EG87 and the value of PRO CNT82 are input, and when they become equal, the learning process start signal LRNS is output to the LRN89.

The circuit configuration when a three-layer hierarchical network is constructed using the forward processing circuit 86 and the learning processing circuit 90 is, for example, as shown in the block diagram of FIG. 32 and as shown in the timing diagram of FIG. To work.

That is, first, in the states S0 and S1 before the process transition, 01h in the 8-bit binary representation in the FREG83 of the intermediate layer, 02h in the FREG83 of the output layer, 03h in the LREG87 of the output layer, and 03h in the intermediate layer. 04h is stored in the LREG 87. In this state, the process start signal PROS of the control circuit 91
When T is activated, PRO CNT82 starts up,
The process starts. After that, the forward processing of the intermediate layer is executed in state S1, the forward processing of the output layer is executed in state 2, the learning processing of the output layer is executed in state S3, and the learning processing of the intermediate layer is executed in state S4. The processing from the forward processing of the intermediate layer to the learning processing of the intermediate layer is executed by pipeline processing.

In this way, the forward process and learning process start signals can be shared, and the controllability of the system is improved as compared with the conventional example shown in FIG. The same applies to the multilayer network as shown in FIG.

[0172]

According to the first signal processing apparatus of the present invention, since the forward process and the learning process can be executed in parallel, the processing speed can be increased as compared with the case where the forward process and the learning process are executed serially. it can.

According to the second signal processing device of the present invention, the forward process of the neural network consisting of the input layer, the intermediate layer and the output layer is subdivided, and each process is pipelined. It can be processed at a higher speed than the case of processing. Further, when the intermediate layer is not a single layer but a multi-layer, if each process is not pipelined, the processing time of the forward process is increased by that much, but according to the present invention, the processing speed is intermediate. It becomes the same as the case of one layer, and high-speed processing becomes possible.

According to the third signal processing apparatus of the present invention, the forward process and the learning process of the neural network composed of the input layer, the intermediate layer and the output layer are subdivided, and each process is pipelined. The learning process can be processed at a higher speed than the case where the learning process is performed by a series of processes. Further, in the case where the intermediate layer is not a single layer but a multi-layered process and each process is not pipelined, the processing time of the forward process and the learning process is increased by that much, but according to the present invention,
The processing speed is the same as in the case of one intermediate layer, and high-speed processing is possible.

According to the fourth signal processing device of the present invention, it is not necessary to generate a forward process start signal for each layer in the control block for controlling the entire network. Therefore, regardless of the number of layers in the control block. The circuit can be simplified and the circuit scale can be reduced.

Further, since only one forward process start signal is required for each layer, the number of wirings in the network can be reduced, which prevents malfunctions due to noise propagation and skew, and improves reliability.

Further, when the circuit is produced on the printed circuit board, the control signal can be shared, so that the number of layers of the neural network can be easily changed.

According to the fifth signal processing apparatus of the present invention, since it is not necessary to generate the forward process start signal for each layer in the control block for controlling the entire network, the control block is controlled regardless of the number of layers in the network. The circuit can be simplified and the circuit scale can be reduced.

Also, since only one forward process start signal is required for each layer, the number of wirings in the network can be reduced, so that malfunction due to noise propagation and skew is less likely to occur, and reliability is improved.

Further, when the circuit is produced on the printed circuit board, the control signal can be shared, so that the number of layers of the neural network can be easily changed.

According to the sixth signal processing device of the present invention, since it is not necessary to generate the learning process start signal for each layer in the control block for controlling the entire network, the control block is controlled regardless of the number of layers in the network. The circuit can be simplified and the circuit scale can be reduced.

Since only one learning process start signal is required for each layer, the number of wires in the network can be reduced,
As a result, malfunction due to noise propagation and skew is less likely to occur, and reliability is improved.

Further, when the circuit is produced on the printed circuit board, the control signal can be shared, so that the number of layers of the neural network can be easily changed.

According to the seventh signal processing device of the present invention, since it is not necessary to generate a learning process start signal for each layer in the control block for controlling the entire network, the control block is controlled regardless of the number of layers in the network. The circuit can be simplified and the circuit scale can be reduced.

Since only one learning process start signal is required for each layer, the number of wires in the network can be reduced,
As a result, malfunction due to noise propagation and skew is less likely to occur, and reliability is improved.

Further, when the circuit is produced on the printed circuit board, the control signal can be shared, so that the number of layers of the neural network can be easily changed.

According to the eighth signal processing device of the present invention, it is not necessary to generate the start signals of the forward process and the learning process for each layer in the control block for controlling the entire network, so that the number of layers of the network is not affected. The circuit of the control block can be simplified and the circuit scale can be reduced.

Since only one forward process start signal and one learning process start signal for each layer are required,
The number of wires in the network can be reduced, which allows
Malfunction due to noise propagation and skew is less likely to occur, and reliability is improved.

Further, when the circuit is produced on the printed circuit board, the control signal can be shared, so that the number of layers of the neural network can be easily changed.

In addition, since the forward processing circuit or the learning processing circuit can be arranged in any layer by performing a simple initialization, there is no restriction on the circuit and the controllability is improved.

According to the ninth signal processing circuit of the present invention, it is not necessary to generate the start signal of the forward process and the learning process for each layer in the control block for controlling the entire network, so that the number of layers of the network is not affected. The circuit of the control block can be simplified and the circuit scale can be reduced.

Further, since only one process signal is required for the forward process start signal and the learning process start signal for each layer, the number of wires in the network can be reduced, which further reduces the possibility of malfunction due to noise propagation and skew. More reliable.

Further, when the circuit is produced on the printed board, the control signal can be shared, so that the number of layers of the neural network can be easily changed.

In addition, since the forward processing circuit or the learning processing circuit can be arranged in any layer by performing a simple initialization, there is no restriction on the circuit and the controllability is improved.

[Brief description of drawings]

FIG. 1 is a schematic diagram of a nerve cell unit of a neutral network.

FIG. 2 is a schematic diagram in which the nerve cell unit of FIG. 1 is configured in a network.

FIG. 3 is a graph showing a sigmoid function.

FIG. 4 is a block diagram showing a general circuit when performing forward processing of one neuron by numerical calculation.

FIG. 5 is an explanatory diagram of a forward process of a three-layer hierarchical network.

FIG. 6 is an explanatory diagram of a learning process of a three-layer hierarchical network.

FIG. 7 is an explanatory diagram of a forward process of a multilayer hierarchical network.

FIG. 8 is an explanatory diagram of a learning process of a multilayer hierarchical network.

FIG. 9 is a circuit diagram of a circuit that executes forward processing of a three-layer hierarchical network.

FIG. 10 is a timing chart of forward processing of a three-layer hierarchical network.

FIG. 11 is a circuit diagram of a circuit that executes forward processing of a multilayer hierarchical network.

FIG. 12 is a timing chart of forward processing of a multilayer hierarchical network.

FIG. 13 is a circuit diagram showing a nerve cell mimicking circuit.

14 is a circuit diagram showing a circuit corresponding to the line (connection) in FIG.

FIG. 15 is a circuit diagram showing a circuit corresponding to the circle (neuronal cell unit) in FIG.

FIG. 16 is a circuit diagram showing a circuit for obtaining an error signal in the output layer from the output of the output layer and the teacher signal.

FIG. 17 is a block diagram of a three-layer hierarchical network according to an embodiment of the present invention.

FIG. 18 is a timing chart of the operation of the embodiment of the present invention.

FIG. 19 is a block diagram of a three-layer hierarchical network according to another embodiment of the present invention.

FIG. 20 is a timing chart of the operation of another embodiment of the present invention.

FIG. 21 is a block diagram of an essential part of still another embodiment of the present invention.

FIG. 22 is a timing chart of the operation of another embodiment of the present invention.

FIG. 23 is a block diagram of a three-layer hierarchical network according to another embodiment of the present invention.

FIG. 24 is a block diagram of a multilayer hierarchical network according to still another embodiment of the present invention.

FIG. 25 is a block diagram of an essential part of still another embodiment of the present invention.

FIG. 26 is a timing chart of the operation of still another embodiment of the present invention.

FIG. 27 is a block diagram of the essential parts of yet another embodiment of the present invention.

FIG. 28 is a timing chart of the operation of yet another embodiment of the present invention.

FIG. 29 is a block diagram of a three-layer hierarchical network according to still another embodiment of the present invention.

FIG. 30 is a block diagram of an essential part of another embodiment of the present invention.

FIG. 31 is a timing chart of the operation of another embodiment of the present invention.

FIG. 32 is a block diagram of a three-layer hierarchical network according to another embodiment of the present invention.

FIG. 33 is a circuit diagram showing an example in which a neural network is realized by an electric circuit.

FIG. 34 is a block diagram showing a circuit configuration of a single nerve cell.

FIG. 35 is a block diagram showing a synapse circuit.

FIG. 36 is a block diagram showing a cell body circuit.

FIG. 37 is a block diagram of a conventional signal processing circuit.

[Explanation of symbols]

A input layer A1 input layer A2 intermediate layer A3 output layer B1 to Bn-1 intermediate layer C output layer 11 forward processing execution circuit (FWD) 12 registers 13 forward processing execution circuit (FWD) 15 control circuit 17 to 19 registers 42 to 45 Register 46 Learning process executing circuit (LRN) 47 Learning process executing circuit (LRN) 48 Error generating circuit 51 Forward process executing circuit (FWD) 52 Register 53 Forward process executing circuit (FWD) 56 Error generating circuit 57 Register 58 Learning process executing circuit 59 register 60 learning process execution circuit 61 register 62 control circuit 70 forward process counter (FWD CNT) 71 forward process execution circuit (FWD) 72 process counter (PRO CNT) 73 forward process start state set Register (FREG) 74 comparator (CMP) 75 circuit for performing forward processing (FWD) 76 forward processing circuit 77 control circuit 78 learn counter (LRN CNT) 79 circuit for performing learning process (LRN) 80 learning processing circuit 81 control circuit 82 Process Counter (PRO CNT) 83 Forward Process Start State Set Register (FREG) 84 Comparator (CMP) 85 Circuit for Performing Forward Processing (FWD) 86 Forward Processing Circuit 87 Learning Process Start State Set Register (LR)
EG) 88 Comparator (CMP) 89 Circuit for executing learning process (LRN) 90 Learning processing circuit 91 Control circuit

Claims (9)

[Claims] 1. A signal processing device using a nerve cell network composed of nerve cell mimicking elements, wherein each nerve cell mimicking element constituting the nerve cell circuit network holds an input signal input during a forward process. And a means for holding an error signal inputted during the learning process, and performing the forward process and the learning process in parallel. 2. In a signal processing device using a hierarchical neural cell network composed of neural cell mimicking elements, a forward process of a neural network consisting of an input layer, an intermediate layer and an output layer is divided into several processes, A signal processing device, characterized in that each divided process is pipelined. 3. A signal processing device using a hierarchical nerve cell network composed of nerve cell mimicking elements, comprising:
A signal processing apparatus, characterized in that a forward process and a learning process of a neural network including an intermediate layer and an output layer are divided into some processes, and each divided process is pipelined. 4. A signal processing device using a hierarchical nerve cell network composed of nerve cell mimicking elements, comprising:
A counter that can set an initial value provided for each layer of the intermediate layer and the output layer and a counter for each layer that sets the initial value before the transfer of the forward process, and the forward process execution signal common to the counters of each layer when the forward process is executed And a means for providing each of the layers, the counter of each layer counts the forward process execution signal to generate a forward process execution signal having a predetermined value for executing the forward process execution signal of the layer. apparatus. 5. A signal processing device using a hierarchical nerve cell network composed of nerve cell mimicking elements, comprising:
A counter provided for each layer of the intermediate layer and the output layer, a register for storing the forward process start state, and a comparator for comparing the value of the counter with the value of the register, and the register for each layer before the transfer to the forward process. A forward process execution means for storing a state for starting the forward process execution of the layer and providing a common forward process execution signal to each layer when the forward process is executed, and the counter of each layer counts by this forward process execution signal The signal processing device, wherein the comparator of each layer generates a forward process execution signal that causes each layer to execute the forward process when the value of the counter of the layer is equal to or greater than the value of the register. 6. A signal processing device using a hierarchical nerve cell network composed of nerve cell mimicking elements, comprising:
A counter that can set an initial value provided for each layer of the intermediate layer and the output layer, and an initial value is set to the counter of each layer before the learning process transitions, and a learning process execution signal common to the counters of each layer when the learning process is executed And a means for giving a learning process execution signal of each layer to count the learning process execution signal to generate a learning process execution signal of a predetermined value for executing the learning process of the layer. apparatus. 7. A signal processing device using a hierarchical nerve cell network composed of nerve cell mimicking elements, comprising:
A counter provided for each layer of the intermediate layer and the output layer, a register for storing a learning process start state, a comparator for comparing the value of the counter with the value of the register, and the register for each layer before the learning process is transferred. A learning process execution means having a state for starting a learning process of a layer and giving a common learning process execution signal to each layer when the learning process is executed is provided, and the counter of each layer counts by this learning process execution signal. The signal processing device, wherein the comparator of each layer generates a learning process execution signal that causes each layer to execute a learning process when the value of the counter of the layer is equal to or greater than the value of the register. 8. A signal processing device using a hierarchical nerve cell network composed of nerve cell mimicking elements, wherein:
Alternatively, a signal processing apparatus comprising: the forward process executing means of claim 5; and the learning process executing means of claim 6 or 7, which can execute a learning process independently of the forward process. 9. A signal processing device using a hierarchical nerve cell network composed of nerve cell mimicking elements, comprising:
For each of the intermediate layer and the output layer, a process counter provided to control the forward process and the learning process, a forward process register that stores the forward process start state, a value of the register, and a value of the process counter And a learning process register for storing the learning process start state, and a learning process comparator for comparing the value of this register with the value of the process counter, and before the process transition. In the register storing the forward process start state of each layer, the state of starting the forward process of that layer is stored, and
The state that starts the learning process of the layer is stored in the register that stores the learning process start state of each layer,
Means for giving a common process execution signal to each layer during process execution, the process counter of each layer counts this process signal, and the comparator for the forward process has a value of the process counter of the forward process of the layer. A forward process execution signal for starting the forward process when the register value for storing the start state is equal to or more than the register value for storing the start state. The learning process comparator is a register value for storing the learning process start state of the layer in the value of the process counter. A signal processing circuit for generating a learning process execution signal for starting a learning process at the above time.