JPH0737016A - Connection coefficient code processing method for signal processing circuit network and device therefor - Google Patents

Abstract

PURPOSE:To provide a signal processor which can change the code of a connection coefficient by the learning processes in the circuit of a neural network. CONSTITUTION:A connection coefficient storing part 51 is provided to store the value of a connection coefficient, together with a code storing part 52 which stores the code of the connection coefficient, an up-down counter 54 which adds the increment of the connection coefficient value of the 1st code to this coefficient value and then subtracts the increment of the connection coefficient value of the 2nd code from the total coefficient value of the 1st code, a code bit storing part 55 which stores the arithmetic result of the counter 54, and an EXOR 53 which secures an exclusive OR between the output sent from the part 55 and the output sent from the part 52. Then, the code of the part 52 is changed by the output of the EXOR 53.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention changes the coupling coefficient code in a signal processing circuit network such as a neural computer that imitates a neural cell network that is applied to character and figure recognition, motion control of robots, associative memory and the like. And a device therefor.

[0002]

2. Description of the Related Art A neural cell mimicking element (neuronal cell unit) that mimics the function of a neural cell (neuron), which is a basic unit of information processing of a living body, is configured in a network to aim at parallel processing of information. , A so-called nerve cell circuit network (neural network). Although character recognition, associative memory, motion control, etc. are performed in the living body in a very simple manner, many cannot be easily achieved by the conventional Neumann computer.

Therefore, attempts to solve these problems by imitating the functions of the nervous system of the living body, in particular, the functions peculiar to the living body, such as parallel processing and self-learning, are being actively made, centering on computer simulation. There is.

FIG. 1 shows a model (nerve cell unit) of one neuron, which is composed of a part for receiving an input from another neuron, a part for converting the input according to a certain rule, and a part for outputting a result. A variable weight "Tij" is attached to each connection portion with another neuron to indicate the connection strength. Changing this value changes the network structure. Learning the network means changing this value.

FIG. 2 shows a case where this is used as a network to form a hierarchical neural network. A in the figure
Reference numerals 1, A2 and A3 represent nerve cell mimicking elements (nerve cell units), respectively. Each nerve cell unit A1, A
2, A3, like the nerve cell unit shown in the schematic view of FIG. 1, is coupled with a large number of nerve cell units, and the signals received from the nerve cell units are processed and output. This hierarchical network consists of an input layer, an intermediate layer, and an output layer.
2 receives a signal from the nerve cell unit A1 in the previous layer (input layer) and outputs the signal to each nerve cell unit A3 in the next layer (output layer). There is no coupling within each layer and no coupling from the output layer to the input layer. One neuron is connected to many other neurons.

Although FIG. 2 shows a three-layer network including an input layer, an intermediate layer, and an output layer, a multi-layer network having a plurality of intermediate layers is also used.

Taking the nerve cell unit shown in FIG. 1 as an example,
The operation of each nerve cell unit A1, A2, A3 in FIG. 2 will be described. First, the forward process will be described.

In the neural network of FIG.
The signal input to the input layer A1 propagates to the intermediate layer A2,
The forward process is executed in the middle layer A2. Middle layer A2
The processing result of is propagated to the output layer A3, and at the output layer A3,
The forward process is executed, and the output of the neural network is finally obtained.

The degree of coupling between neurons is called a coupling coefficient, and the coupling coefficient between the i-th neuron and the j-th neuron is represented by Tij.
There are two types of coupling: excitatory coupling, in which the output of the partner neuron increases as the signal from the partner neuron increases, and inhibitory coupling in which the output of the partner neuron decreases, the output decreases. Excitatory coupling, Tij
When <0, it represents an inhibitory bond.

Now, assuming that the own neuron is the i-th neuron, and the output of the j-th neuron is yj, this is multiplied by the coupling coefficient Tij, and Tij · yj becomes the input to the own neuron.

Since each neuron is connected to many neurons, Tij.yj for these neurons
ΣTij · yj, which is the result of adding up, becomes the input to the own neuron. This is called an internal potential and is represented by the following equation.

[0012]

Mathematical Expression 1 ui = ΣTij · yj (1)

Next, the input is subjected to non-linear processing to output the nerve cell unit Ai. The function at this time is called a nerve cell response function, and for example, a sigmoid function represented by the following equation (2) is used.

[0014]

## EQU00002 ## f (ui) = 1 / (1 + ^e.sup. -ui) (2)

This function is shown in FIG. Value range is "0 to 1"
Then, as the input value increases, it approaches “1”, and as it decreases, it approaches “0”. From the above, the output y of the neuron i
i is represented by the following equation (3).

[0016]

Equation 3 yi = fi (ui) = fi (ΣTij · yj) (3)

Next, the learning function of the neural network will be described. As a learning process used in the numerical calculation, there is the following one called a general back propagation algorithm (hereinafter, abbreviated as BP algorithm).

First, the coupling coefficient between nerve cell units is initially set to a random value. If an input is given to the neural network in this state, the output result is not always desirable. For example, in the case of character recognition, when a handwritten character "1" is given, it is a desirable result that "this character is" 1 "" is output as an output result, but it is not always necessary that the coupling coefficient is random. Not the desired result.

Therefore, as a learning process, a correct answer (teaching signal) is given to this neural network, and each coupling coefficient is changed so that the output result becomes correct when the same input is received again (a desired output is obtained). Let

That is, in the learning process, when a certain input pattern p is given, the coupling coefficient is changed so as to reduce the error between the actual output value and the desired output value. The BP algorithm is an algorithm for obtaining this change amount.

Now, given a certain input pattern p, the difference between the actual output value (ypi) of the unit i and the desired output value (tpi) is defined by the following equation (4).

[0022]

[Equation 4] ^{Ep = (tpi-ypi) 2} /2 ... (4)

This represents the error of unit i in the output layer,
tpi is teacher data given by a human. In learning, the strength of all connections is changed to reduce this error.
Actually, the Tij change amount when the pattern p is given is expressed by the following equation (5).

[0024]

[Formula 5] ΔpTij ∝-ЭE / ЭTij ... (5)

By using this, the coupling coefficient Tip is changed. As a result, the following equation (6) is obtained from the above equation (4).

[0026]

Equation 6 ΔpTij = ηδpiypj (6)

Here, ypi is an input value from the unit j to the unit i. The error δpi differs depending on whether the unit i is the output layer or the intermediate layer. First, the error signal δpi in the output layer
Becomes the following expression (7).

[0028]

## EQU00007 ## .delta.pi = (tpi-ypi) .fi '(ui) (7)

On the other hand, the error signal δpi in the intermediate layer is as shown in the following expression 8.

[0030]

## EQU00008 ## .delta.pi = fi '(ui) .. SIGMA..delta.pkTki (8)

However, fi 'is the first derivative of fi. From the above, ΔTij is generally formulated as the following equation (9).
Is.

[0032]

Equation 9 ΔTij (n + 1) = nδpiypj + αΔTij (n) (9)

Therefore, it is as shown in the equation (10). Here, n is a learning order, η is a learning constant, and α is a stabilizing coefficient. The first term on the right side of the above equation (9) is added to ΔT obtained by the above equation (6), and the second term is added to reduce the error vibration and accelerate the convergence.

[0034]

Equation 10 Tij (n + 1) = Tij (n) + ΔTij (n + 1) (10)

Thus, the calculation of the variation ΔT of the coupling coefficient starts from the unit of the output layer and moves to the unit of the intermediate layer. Learning proceeds in the opposite direction to the processing of the input data, that is, backward. Therefore, in learning by back propagation, first, learning data is input and the result is output (forward). Change the strength of all joins to reduce the resulting error (backward). Input the learning data again. This will be repeated until it converges.

The conventional hierarchical nerve cell network forms a network as shown in FIG. FIG. 4 shows the data flow of the forward process in this network. This is the case in a three-layer hierarchical network. In the forward process, input signals a1 to a4 are given to the input layer (the layer A1 on the left side of FIG. 4) and the output signal from the output layer (the layer A3 on the right side of FIG. 4). b1 to b4 are obtained.

On the other hand, FIG. 5 shows the data flow of the learning process. In the learning process, teacher signals d1 to d4 are given to the output layer (A3 on the right side of FIG. 5) to update the coupling strength between the neurons, and learning is performed so that the output of the neural network matches the teacher signal. Note that the processing by this learning process is currently executed by an external general-purpose computer in most cases.

FIG. 10 shows an example in which the neural network is realized by an electric circuit (see Japanese Patent Application Laid-Open No. 62-295188). This circuit basically comprises a plurality of amplifiers 34 having an S-shaped transfer function, and a resistive feedback network 32 which connects the output of each amplifier 34 to the input of the amplifier of another layer as shown by the chain line. And are provided. A CR time constant circuit 33 including a grounded capacitor C and a grounded resistor R is individually connected to the input side of each amplifier 34. Then, the input current I _1, I ₂ ~I _N is supplied to the input of each amplifier 34, the output is obtained from the set of output voltages of the amplifiers 34.

Here, the strength of the input and output signals is represented by a voltage, and the strength of the coupling of nerve cells is the resistance 31 (the lattice point in the resistive feedback network 32) connecting the input / output lines between the cells. And the nerve cell response function is represented by the transfer function of each amplifier 34. Further, the connection between nerve cells has excitability and inhibitory property as described above, and is mathematically represented by the sign of the connection function. However, since it is difficult to realize positive and negative with a constant on the circuit, here, by dividing the output of the amplifier 34 into two and inverting one output, two positive and negative signals 34a and 34b are generated. Then, it is realized by selecting this. Further, the transfer function of the amplifier 34 is used as the one corresponding to the sigmoid function shown in FIG.

11 to 13 are views showing an example in which the neural network is realized by a digital circuit.
Shows an example of the circuit configuration of a single nerve cell, and the synapse circuit 4
1, a dendrite circuit 42 indicates a cell body circuit 43.

FIG. 12 shows the synapse circuit 4 shown in FIG.
1 is a diagram showing an example of the configuration of No. 1 in FIG.
Alternatively, a rate multiplier 41b to which a value obtained by multiplying 2) is input is provided, and the rate multiplier 41b stores the weighting value w in the synapse load register 41.
c is connected. FIG. 13 is a diagram showing a configuration example of the cell body circuit 43, in which a control circuit 44, an up / down counter 45, a rate multiplier 46 and a gate 47 are sequentially connected, and an up / down memory 48 is further provided.
Is provided.

In this case, the input / output of the nerve cell unit is represented by a pulse train, and the amount of signal is represented by the pulse density. The coupling coefficient is treated as a binary number and stored in the synapse weight register 41c. The signal calculation process is performed as follows. First, the input signal is input to the rate multiplier 41b and the coupling coefficient is input to the rate value, whereby the pulse density of the input signal is also reduced according to the rate value.
This is the T in the backpropagation model equation above.
It corresponds to the part of ij and yi. Also, ΣTij · yi Σ
The part of is realized by the OR circuit shown by the dendrite circuit 42. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance and a logical sum (OR) is taken for each group. In FIG. 11, F ₁
Is excitatory and F ₂ is inhibitory output. An output can be obtained by inputting these two outputs to the up side and down side of the counter 45 shown in FIG. 13 and counting respectively. This output is a binary number, so again the rate multiplier 4
Convert to pulse density using 6. A neural network can be realized by constructing a network using a plurality of these nerve cell units. The learning function is realized by inputting the final output of the network to an external computer, performing numerical calculation inside the computer, and writing the result to the synapse weight register 41c that stores the coupling coefficient.

The present applicant has already filed a patent application for a signal processing device using a nerve cell network composed of nerve cell mimicking elements (see, for example, Japanese Patent Application No. 1-34891). In the present invention, the signal processing device according to this prior application is dealt with as an example of one embodiment, so the signal processing device according to this prior application will be described below.

In the signal processing device according to this prior application,
As an example of the neural network, we have proposed a neural cell unit using a digital logic circuit and signal processing by a network circuit configured by using it.

The basic idea here is as follows. 1. Input / output signals, intermediate signals in the nerve cell unit,
All the teacher signals are represented by a pulse train represented by binary values of "0" and "1". 2. The value of the signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). 3. The calculation in the nerve cell unit is performed by a logical calculation between pulse trains. 4. The pulse train of the coupling coefficient is stored in the memory in the nerve cell unit. 5. In learning, an error is calculated based on a given teacher signal pulse train, and the coupling coefficient is changed based on this error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1".

FIG. 6 shows a state of forward processing of one neuron element in the pulse density method, and considers the hierarchical neural network shown in FIG. 2 as the network configuration.

First, the logical product (AND) of the input yj and the coupling coefficient Tij which are binarized into "0" and "1" and expressed by the pulse density is obtained for each synapse. This corresponds to Tij · yj in the equation (1). The pulse density of the output of the AND circuit stochastically becomes the product of the pulse density of the input signal and the pulse density of the coupling coefficient.

As described above, the connections between neurons include excitatory connections and inhibitory connections. In the case of numerical calculation, the sign of the coupling coefficient, for example, plus when excitatory and minus when inhibitory is performed.

In the case of the pulse density method, the coupling coefficient Tij
Depending on the positive or negative of each, each connection is classified into an excitatory connection and an inhibitory connection.
It is divided into two groups, and the logical sum is calculated by OR operation for each group. This corresponds to the processing of Σ in Equation 3 and the processing of the nonlinear saturation function fi.

That is, in the calculation based on the pulse density,
When the pulse density is low, the pulse density resulting from the OR processing can be approximated to the sum of the pulse densities of the OR input.

Since the output of the OR circuit gradually becomes saturated as the pulse density becomes higher, the result does not match the sum of the pulse densities, and the nonlinearity appears.

In the case of this OR operation, the value P of the pulse density is 0≤P≤1, and since it is a monotonically increasing function with respect to the magnitude of the input, it is the same as the processing by the sigmoid function of equation 2 or FIG. become.

The output of the nerve cell element by the pulse density method is the OR output a of the excitatory group obtained as described above.
Is "1" and the OR output b of the inhibitory group is "0"
"1" is output only when. That is, (1
It is expressed as shown in 1) to (13).

[0054]

(11) a = ∪ (yi∩Tij ⁺ ) − Excitability group (T> 0) (11) T> 0 b = ∪ (yi∩Tij ⁻ ) − Inhibitory group (T <0) (12) ) T <0 yi = g (a, b) = a∩b (13)

The learning process in the pulse density method will be described below. In a non-learned neural network, the output of the network when a certain pattern is input is not always the desired output. Therefore, similar to the BP algorithm described above, the learning process changes the coupling coefficient so as to reduce the error between the actual output value and the desired output value.

(Error Signal in Output Layer) First, the error signal in the output layer will be described. Here, when the error is expressed by a numerical value, both positive and negative values can be taken. However, since such an expression cannot be made in the pulse density method, two signals representing the positive component are used to calculate the error signal in the output layer as follows. It is defined as in (14) and (15) of Expression 12.

[0057]

[Equation 12]

When the output result (y) is "0" and the teacher signal (d) is "1", the error signal + component (δ ⁺ ) is
It becomes "1", and otherwise it becomes "0".

[0059] On the other hand, the error signal - component ([delta] ^-) is a output result (y) is "1", when the teacher signal (d) is "0",
It becomes "1", and otherwise it becomes "0".

These error signals δ ⁺ and δ ⁻ are obtained by the above equation (7) for obtaining the error signal of the output layer in the above-mentioned BP algorithm.
Corresponding to.

(Error Signal in Intermediate Layer) The error signal in the intermediate layer based on the pulse density method is also obtained by referring to the above equation (8) based on the BP algorithm. That is, the error signals in the fishing layer are collected and used as the own error signal. Here, the connection is divided into two groups depending on whether it is excitatory or inhibitory, the product part is represented by ∩ (AND), and the sum (Σ) part is represented by ∪ (OR).

Further, when the error signal in the intermediate layer is obtained, it is divided into four cases of positive and negative of the coupling coefficient Tij and positive and negative of the error signal δ. First, in the case of excitatory coupling, the error signal plus component δ ^{+ of} the output layer and the AND of the coupling coefficients (δ ⁺ i∩T ⁺ ij) are obtained for all the neurons of the output layer, and these ORs are obtained. Take This becomes the error signal plus component δ ⁺ of the hidden layer neuron (Equation 13).
(16)).

[0063]

[Mathematical formula-see original document] δ ⁺ = ∪ (δ ⁺ i ∩ T ⁺ ij) (16)

Similarly, the error signal of the output layer minus the component δ
^- and that it took AND of the coupling coefficient (δ ^- i∩T
⁺ ij) for all output layer neurons,
These are ORed together. This becomes the error signal minus component δ ^{− of the} hidden layer neuron ((17) in Expression 14).

[0065]

Mathematical Expression 14 δ ⁻ = ∪ (δ ⁻ i ∩T ⁺ ij) (17)

Next, the case of inhibitory binding will be described.
An AND (δ ⁻ i∩T ⁻ ij) of the error signal minus component δ ^{− of the} output layer and its coupling coefficient is obtained for all the neurons of the output layer, and the OR of these is taken. This becomes the error signal plus component of the intermediate layer (Formula 1)
5 (18)).

[0067]

[Equation 15] ^{δ + = ∪ (δ - i∩T} - ij) ... (18)

Similarly, the error signal of the output layer plus the component δ +
And those taking the AND between the coupling coefficient (δ ⁺ i∩T ^-
ij) is found for all neurons and these O
Take R. This becomes the error signal minus component of the hidden layer neuron.

[0069]

[Equation 16] ^{δ - = ∪ (δ + i∩T} - ij) ... (19)

The connection between a neuron in a certain middle layer and the neuron in the output layer connected to the middle layer includes excitatory connection and inhibitory connection. Therefore, as the error signal plus component of the intermediate layer, the logical sum of δ ⁺ of the excitatory coupling of the equation (16) and δ ⁺ of the inhibitory coupling of the equation 18 is obtained. Similarly, the error signal minus component of the intermediate layer, excitatory coupling [delta] of the formula (17) ^- ORed with the ^- [delta] inhibitory binding of the formula (19). That is, the equations (20) and (21) are obtained. This corresponds to the above equation (8) according to the BP algorithm.

[0071]

Mathematical Expression 17 δ ⁺ = {∪ (δ ⁺ i∩T ⁺ ij)} ∪ {∪ (δ ⁻ i∩T ⁻ ij)} (20) δ ⁻ = {∪ (δ ⁻ i∩T ⁺ ij) } ∪ {∪ (δ ⁺ i∩T ^- ij)}… (21)

(Processing by Learning Constant η) Processing of the learning constant η in the above equation (6) for obtaining the correction amount ΔT of the coupling coefficient in the BP algorithm will be described. In the numerical calculation, the learning constant η may be simply multiplied as shown in the equation (6), but in the case of the pulse density method, the pulse train is thinned out as shown below according to the value of the learning constant η. It will be realized.

[0073]

[Formula 18]

Next, a method of obtaining the correction amount ΔT of the coupling coefficient by learning will be described. First, the error signal (δ ⁺ , δ ⁻ ) in the output layer or the intermediate layer described above is processed by the learning constant η, and the logical product with the input signal to the neuron is calculated (δ∩y). However, since the error signals have δ ⁺ and δ ⁻ , they are calculated as shown in the following equations, and ΔT ⁺ , Δ
T ^- to.

[0075]

[Equation 19] ^{ΔT + = δ + ∩y ... (} 25) ΔT - = δ - ∩y ... (26)

This corresponds to the above equation (6) for obtaining ΔT in the BP algorithm. Based on these, a new coupling coefficient New_Tij is calculated.
The case is classified according to whether it is excitatory or inhibitory. First, in the case of excitability, the ΔT ⁺ component is increased and the ΔT ⁻ component is decreased with respect to the original T ⁺ . That is, equation (27) is obtained.

[0077]

[Formula 20]

Next, in the case of the inhibitory property, the ΔT ⁺ component is reduced and the ΔT ⁻ component is increased with respect to the original T ⁻ . That is, equation (28) is obtained.

[0079]

[Formula 21]

The learning algorithm based on the pulse density method has been described above. Here, the processing flow of the forward process and the learning process in the pulse density method in the hierarchical network of FIG. 2 will be briefly described. First,
Although it is a forward process, when a signal is first given to the input layer, this input signal propagates to the intermediate layer, and as the signal processing of the intermediate layer, the above equations (11) to (13) are performed, and the result is To the output layer.

In the output layer, the processes of equations (11) to (13) are similarly performed on these propagated signals,
These results in an output signal, ending the forward process.

In the learning process, after performing the above forward process, a teacher signal is further given to the output layer. In the output layer, the error signal in the output layer is obtained by the equations (14) and (15) and sent to the intermediate layer. At the same time, this error signal is processed by η in equations (22) to (24) to obtain equation (2
5) and (26), the logical product with the input signal from the intermediate layer is calculated, and then the coupling strength between the output layer and the intermediate layer is changed by the equations (27) and (28).

Next, as the processing in the intermediate layer, the error in the intermediate layer is obtained by the equations (20) and (21) based on the error signal sent from the output layer, and the error signal is given by the equation (2).
The processing by the learning constant η of 2) to (24) is performed, and the expression (2
5) and (26), the logical product with the input signal from the input layer is obtained, and then the coupling strength between the intermediate layer and the input layer is changed by the equations (27) and (28), and the learning process ends. After that, the learning process is repeated until it converges.

Next, an actual circuit configuration based on the above algorithm will be described with reference to FIGS. The structure of the neural network is the same as in FIG. Figure 7
FIG. 8 is a logic circuit diagram showing a circuit of a part corresponding to a synapse of a neuron, and FIG. 8 is a logic circuit diagram showing a circuit of a part corresponding to a cell body of the neuron. Further, FIG. 9 is a logic circuit diagram showing a circuit of a portion for obtaining an error signal in the output layer from the output of the output layer and the teacher signal. By forming a network of these three circuits as shown in FIG. 2, a digital neural network circuit capable of self-learning can be realized.

First, FIG. 7 will be described. 1 is an input signal to the nerve cell unit. The synapse coupling coefficient is stored in the shift register 8. The terminal 8A is a data outlet and the terminal 8B is a data inlet. Other components such as a RAM and an address controller may be used as long as they have the same function as the shift register. The circuit 9 has the above formulas (11), (1
In the circuit for executing (yi∩Tij) of 2), the input signal is ANDed with the coupling coefficient. This output must be grouped according to whether the coupling is excitatory or inhibitory, but it is more versatile to prepare outputs 4 and 5 for each group in advance and switch which group is output. high. Therefore, a bit indicating whether the coupling is excitatory or inhibitory is stored in the memory 14, and the signal is switched by the switching gate circuit 13 using the information. Further, as shown in FIG. 8, a gate circuit 15 having a plurality of OR gates corresponding to the logical sum of the equations (11) and (12) for processing each input is provided. Further, as shown in the figure, the excitability group shown in equation (13) is “1”.
And output only when the inhibitory group is "0" A
A gate circuit 16 including an ND gate and an inverter is provided.

Next, the error signal will be described. Figure 9
Is a diagram showing a circuit for generating an error signal in the output layer, where A
It is a logic circuit that combines ND and inverter,
This corresponds to the equations (14) and (15). That is, the error signals 21 and 22 are calculated from the output 19 from the output layer and the teacher signal 20.
To generate. Further, the above equations (16) to (19) for obtaining the error signal in the intermediate layer are performed by the gate circuit 10 having the AND gate configuration shown in FIG. 7, and outputs 2 and 3 corresponding to + and − are obtained. . Since the error signal to be used differs depending on whether such coupling is excitatory or inhibitory, it is necessary to make a distinction in that case. In this case, the excitatory or inhibitory information stored in the memory 14 and the error are stored. This is performed by the gate circuit 12 having an AND and OR gate configuration according to the +,-signals 6 and 7 of the signals. Further, the arithmetic expressions (20) and (21) for collecting the error signals are performed by the gate circuit 17 having the OR gate structure shown in FIG. Expressions (22) to (24) corresponding to the learning rate are given by the frequency dividing circuit 18 shown in FIG.
Done by.

Finally, the part for calculating a new coupling coefficient from the error signal will be described. This is the above equation (25).
.About. (28), which is performed by the gate circuit 11 having the AND, inverter, and OR gate configurations shown in FIG.
This gate circuit 11 must also be classified depending on the excitability / inhibition property of the coupling, and this is performed by the gate circuit 12 shown in FIG.

In the neural network circuit (FIGS. 6 to 9) based on the pulse density method of the prior application, the coupling coefficient Tij between neurons is calculated by the above equation (14) by a learning process.
~ (28), that is, the circuits shown in FIGS. However, the sign of the coupling coefficient (14 in FIG. 7) cannot be changed by learning, which causes a decrease in learning speed of the neural network and a situation in which learning does not converge.

The EXOR problem will be described as an example in which learning is not united. FIG. 14 shows a network configuration when solving a general EXOR problem, and Table 1 shows a truth table.

[0090]

[Table 1]

[0091]

When the function of the truth table of Table 1 is realized by the neural network having the configuration shown in FIG. 14, the sign of each coupling coefficient is fixed, and the sign of FIG. There are two ways, b). In addition, + in the figure represents an excitatory bond and − represents an inhibitory bond. Usually, the coupling coefficient and the sign of the coupling coefficient are randomly set at the initialization of the learning process. At this time, excitatory coupling (+) and inhibitory connections (-) to 1 and a half probability to allocate a, 1 probability is the number of bonds 6 because 2 ⁶ minutes allocated as shown in (a) of FIG. 15, Similarly, the probability of allocation is 2 as shown in FIG.
It will be one ^sixth . Therefore, the code assignment of the coupling coefficient when implementing the EXOR function may be as shown in (a) or (b) of FIG. 15, and the probability of assigning the code like this is 2
^{It is 1/5} , that is, 1/32. This means that the EXOR function can be realized only once by performing the code setting 32 times. Therefore, in such a method, there is a large proportion of learning that does not converge, which results in a long learning time.

The present invention has been made to address the above problems, and an object thereof is to solve the above problems by making it possible to change the sign of the coupling coefficient by learning.

[0093]

According to a first aspect of the present invention, in a signal processing device by a nerve cell network realized by a pulse density method, in the excitatory coupling, the excitatory coupling coefficient of If the value obtained by adding the excitatory coupling coefficient increase to the value becomes smaller than the inhibitory coupling coefficient increase, the sign of the coupling coefficient is changed from excitatory coupling to inhibitory coupling, and in the inhibitory coupling, the inhibitory It is characterized in that the sign of the coupling coefficient is changed from the inhibitory coupling to the excitatory coupling when the value obtained by adding the inhibitory coupling coefficient increment to the coupling value becomes smaller than the excitatory coupling coefficient increment.

A second invention of the present invention is, in a signal processing device by a nerve cell network realized by a pulse density method, in the learning process, the value of the coupling coefficient is within a preset range and is preset. When the number of learnings is the same, the sign of the coupling coefficient is changed from excitatory connection to inhibitory connection in excitatory connection and from inhibitory connection to excitatory connection in inhibitory connection. And

[0095]

According to the signal processing method of the present invention, the sign of the coupling coefficient can be changed from excitatory coupling to inhibitory coupling or from inhibitory coupling to excitatory coupling by the learning process, whereby the learning speed and learning can be improved. It is possible to significantly improve the convergence of the.

[0096]

Embodiments of the present invention will be described below with reference to FIGS. This invention relates to a neural network circuit based on the pulse density method of the previous application (see FIG. 6).
9 to 9), the sign of the coupling coefficient can be changed by the learning process.

First, the first method of converting the sign of the coupling coefficient by the learning process according to the present invention will be described. Let the excitatory coupling coefficient be T ⁺ and the inhibitory coupling be T ⁻ .

In the learning process, the values obtained by cumulatively adding the above equations (25) and (26) for obtaining the update amount of the coupling coefficient for one pulse for a certain time (one frame) are ΔT ⁺ , Δ.
T ^- to.

In this embodiment, the sign of the coupling coefficient is learned as follows.

The value obtained by adding the excitatory coupling coefficient increase ΔT ⁺ to the excitatory coupling T ⁺ is the inhibitory coupling coefficient increase Δ.
The sign of the coupling coefficient is excitatory (+) when it is smaller than T ^−.
To suppressive (-). When the value obtained by adding the inhibitory coupling coefficient increment ΔT ⁻ to the inhibitory coupling T ⁻ is smaller than the excitatory coupling coefficient increment ΔT ⁺ , the sign of the coupling coefficient is changed from inhibitory (−) to excitatory (+). Change. The above can be summarized as shown in (29) and (30) of the following formula 22.

[0101]

[Equation 22] Excitatory coupling T ⁺ : When T ⁺ + ΔT ⁺ −ΔT ⁻ <0, the coupling coefficient sign Excitability (+) → Inhibitory (−) (29) Inhibitory coupling T ⁻ : T ⁻ + ΔT ⁻ − When ΔT ⁺ <0: Coupling coefficient sign Inhibitivity (−) → Excitability (+) (30)

FIG. 16 shows an example of a circuit that realizes the above method.
Shown in. The coupling coefficient register 50 and the coupling coefficient storage unit 51 that stores the value of the coupling count, for example, at the time of excitatory coupling,
The code storage unit 52 stores "0" and "1" at the time of inhibitory coupling.

Before the start of the learning process, the load signal (LOA
D) is activated and the value of the coupling coefficient is loaded from the coupling coefficient storage unit 51 into the counter 54. The coupling coefficient code storage unit 52 includes a multiplexer 56 and an EXOR circuit 5.
Connected to 3. In the case of excitatory coupling, the multiplexer 56 outputs the increment signal (INC) of the counter 54.
The output is selected to control the excitatory coupling coefficient increment ΔT ⁺ and the decrement signal (DEC) by the inhibitory coupling coefficient increment ΔT ⁻ .

At the time of inhibitory coupling, the increment signal (INC) of the counter 54 is changed to the increment ΔT ^{− of the} inhibitory coupling coefficient.
, And the output is selected to control the decrement signal (DEC) by the excitatory coupling coefficient increment ΔT ⁺ .

As described above, the operation on the left side of the inequalities of the above equations (29) and (30) can be realized. Formulas (29) and (3
In 0), whether the left side is smaller than 0 is determined by the sign bit of the sign bit storage unit 55 that stores the value of the counter 54. This is because when the value of the counter is negative, that is, when the sign bit of the sign bit storage unit 55 is “1”, the formula (2
9) and (30) are established, and this sign bit becomes a sign inversion signal and is input to the EXOR circuit 53,
As a result, the code of the current coupling coefficient is converted, and the new code is stored in the code storage unit 52 of the register. From the above, the equations (29) and (30) can be realized, and the sign of the coupling coefficient can be changed by learning.

Next, another method of changing the sign of the coupling coefficient of the present invention will be described. Originally, the sign of the coupling coefficient should be the inhibitory coupling (-), but the change of the coupling coefficient due to learning when the excitatory coupling (+) is set is shown in Fig. 17 (a). As can be seen from this, as the learning progresses, the value gradually decreases from the positive initial setting value and reaches near 0. With the conventional method in which the sign of the coupling coefficient cannot be changed, even if the learning is advanced thereafter, it will stay around 0, and the learning will not proceed any further.

Therefore, as shown in FIG. 17B, the sign of the coupling coefficient is changed only when the coupling coefficient enters from 0 to a certain range (ΔW) and this state continues for a set number of times (WNO). To do. Then, as shown in FIG. 17 (b), the inhibitory coupling should be as it should be, and it converges at a certain value, and the learning is completed. Also, if this coupling is meaningless, that is, if the coupling coefficient originally is near 0,
Even if the sign is changed, it will stagnate near 0, and there is no adverse effect on learning.

The circuit for realizing the above is shown in FIG. The combination coefficient register 50 (the combination coefficient storage unit 58, the code storage unit 52) and the EXOR circuit 53 in FIG. 18 are the combination coefficient register 50 (the combination coefficient storage unit 51, the code storage unit 5 in FIG. 16 described above.
2) and the EXOR circuit 53, the same reference numerals are given and the description thereof is omitted here.

FIG. 17 shows how the sign of the coupling coefficient is converted.
This will be described with reference to FIG. When you start learning,
The comparator 62 detects that the value of the coupling coefficient storage unit 51 of the coupling coefficient register 50 has decreased and that this value has become smaller than the value set in the ΔW storage register 61. When this detection signal becomes active, the counter 65 is incremented. When the detection signal is not active, the inverter circuit 64 activates the reset signal of the counter, so that the counter is reset to "0". When the detection signal of the converter 62 becomes active, the counter is incremented from 0, and the counter value becomes the value set in the WNO storage register 63, the comparator 66 causes the coupling coefficient to reach W within ΔW.
It is detected that the vehicle is stagnant more than NO times. As a result, the sign inversion signal becomes active and is realized by the EXOR circuit 53 of this signal and the code signal of the code storage unit 52.
The sign of the coupling coefficient is inverted and the new sign is added to the sign storage unit 52.
Stored in. As a result, as shown in FIG. 17B, the sign of the coupling coefficient is set to a normal value by learning, and learning proceeds.

[0110]

According to the signal processing method of the present invention, it is possible to solve the problem that the sign of the coupling coefficient cannot be changed by learning in the pulse density type neural network apparatus. That is, the sign of the coupling coefficient can be changed by the learning process, and thereby the learning speed and learning convergence can be significantly improved.

Further, as in the prior art, even if the code of each connection of the neural network is randomly assigned, the code can be changed to the original code by learning, so that redundant synapse connection is unnecessary, and therefore, the neural network having the minimum configuration can be obtained. It is configurable. As a result, when the neural network is realized by hardware, the circuit scale can be minimized.

[Brief description of drawings]

FIG. 1 is a schematic diagram of a nerve cell unit.

FIG. 2 is a schematic diagram of a nerve cell network.

FIG. 3 is a graph showing a sigmoid function.

FIG. 4 is a schematic diagram illustrating a forward process.

FIG. 5 is a schematic diagram illustrating a learning process.

FIG. 6 is a circuit diagram of a nerve cell unit.

FIG. 7 is a logic circuit diagram of a portion corresponding to the line (connection) in FIG.

8 is a logic circuit diagram of a portion corresponding to the circle (neuronal cell unit) in FIG.

FIG. 9 is a logic circuit diagram of a circuit that generates an error signal in an output layer.

FIG. 10 is an electric circuit diagram corresponding to a neural network.

FIG. 11 is an electric circuit diagram corresponding to a single nerve cell.

FIG. 12 is a block circuit diagram of a synapse circuit.

FIG. 13 is a block diagram of a cell body circuit.

FIG. 14 is a schematic diagram showing a neural network configuration when solving the EXOR problem.

FIG. 15 is a schematic diagram showing a code example in the configuration of FIG.

FIG. 16 is a block circuit diagram showing a first embodiment of the present invention.

FIG. 17 is a schematic diagram showing changes in the coupling coefficient due to learning.

FIG. 18 is a block circuit diagram showing a second embodiment of the present invention.

[Explanation of symbols]

 50 Coupling coefficient register 51 Coupling coefficient storage section 52 Code storage section 53 EXOR circuit 54 Up / down counter 56 Multiplexer 61 ΔW storage register 62 Comparator 63 WNO storage register 65 Counter 66 Comparator

Claims (4)

[Claims] 1. A signal processing device using a nerve cell network realized by a pulse density method, wherein a value obtained by adding an excitatory coupling coefficient increase amount to a value of an excitatory coupling coefficient in a excitatory coupling is obtained by a learning process. When it becomes smaller than the increase in inhibitory binding coefficient, the sign of the binding coefficient is changed from excitatory binding to inhibitory binding, and in inhibitory binding, the increment of inhibitory binding coefficient is added to the value of inhibitory binding When the value becomes smaller than the increase in excitatory coupling coefficient,
A coupling coefficient code processing method in a signal processing network, characterized in that the sign of a coupling coefficient is changed from inhibitory coupling to excitatory coupling. 2. A signal processing device using a nerve cell network realized by a pulse density method, wherein in a learning process, a value of a coupling coefficient is within a preset range and a preset number of times of learning. In the signal processing circuit, the sign of the coupling coefficient is changed from excitatory coupling to inhibitory coupling in excitatory coupling and from inhibitory coupling to excitatory coupling in inhibitory coupling. Coupling coefficient code processing method. 3. A signal processing device using a nerve cell network realized by a pulse density method, wherein a coupling coefficient storing means for storing a coupling coefficient value, a code storing means for storing a coupling coefficient sign, and a first The increment of the coupling coefficient value of the first code is added to the value of the coupling coefficient of the code, and the added value and the increment of the coupling coefficient of the second code are subtracted. And a means for determining whether to change the code stored in the code storage means according to the output from the means. 4. A signal processing device using a nerve cell network realized by a pulse density method, a coupling coefficient storing means for storing a value of a coupling coefficient, a code storing means for storing a code of the coupling coefficient, and a coupling coefficient. Means for determining whether or not the value of is within a preset range, means for determining whether or not the value of the coupling coefficient is within a preset range, and the number of learning times set in advance, and depending on the output from this means. A signal processing apparatus comprising: a means for changing the code stored in the code storage means.