JPH09185596A - Coupling coefficient updating method in pulse density type signal processing network - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a coupling coefficient updating method faithfully leading out learning algorithm from a neuron output function and realizing learning processing through the use of a simple logical circuit. SOLUTION: The plus component of a differential signal δk<+> (or an inversion signal by an inverter 45) at a neuron (k), the minus component δk<-> of the same signal (or an inversion signal by an inverter 44 and S'kj are inputted in an AND circuit 46 or 47 and processed by η circuits 38 and 38. Then the signal obtained by the processing is set to be the plus component Δkj<+> of the updated quantity of the coupling coefficient or the minus component Δkj<-> of the updated quantity of the coupling coefficient. When the OR output of the Δkj<+> and a coupling coefficient pulse Wkj before update by means of an OR circuit 48 is '1', '1' is added at an up/down counter 41 and when the kj is '1', '1' is subtracted at the up/down counter 41 to set the result value obtained by these adding and subtracting processing to be a new coupling coefficient.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a coupling coefficient updating method and a coupling coefficient updating device in a pulse density type signal processing circuit network applied to character and graphic recognition, motion control of a robot or the like, or associative memory.

[0002]

2. Description of the Related Art A so-called nerve cell circuit is aimed at parallel processing of information by configuring a nerve cell mimicking element that mimics the function of a nerve cell (neuron), which is a basic unit of information processing of a living body, in a network to process information in parallel. It is a network (neural network). Although character recognition, associative memory, motion control, etc. are performed in the living body in a very simple manner, many cannot be easily achieved by the conventional Neumann computer.

Therefore, attempts to solve these problems by imitating the functions of the nervous system of the living body, in particular, the functions peculiar to the living body, that is, parallel processing and self-learning, have been made by computer simulations and trial production of dedicated hardware. It is being actively conducted.

FIG. 1 shows a model (nerve cell unit) of one neuron, which is composed of a part for receiving an input from another neuron, a part for converting the input according to a certain rule, and a part for outputting a result. A variable weight "Wji" is attached to each of the connection parts with other neurons to represent the connection strength. Changing this value changes the network structure. Learning the network means changing this value.

FIG. 2 shows a case where this is used as a network to form a hierarchical neural network. In the figure,
A1, A2 and A3 represent neurons, respectively.
Each of the neurons A1, A2, A3 is connected to a large number of neurons like the neuron shown in the schematic view of FIG. 1, and processes and outputs the signals received from them. The hierarchical network is composed of an input layer, an intermediate layer, and an output layer, and there is no connection within each layer and no connection from the output layer to the input layer. One neuron is connected to many other neurons.

Although FIG. 2 shows a three-layer network including an input layer, an intermediate layer, and an output layer, a multi-layer network having a plurality of intermediate layers is also used.

The operation of each of the neurons A1, A2 and A3 shown in FIG. 2 will be described by taking the neuron shown in FIG. 1 as an example. First, the forward process will be described.

In the neural network of FIG.
The signal input to the input layer A1 propagates to the intermediate layer A2,
Forward processing is executed in the middle layer A2. Middle layer A2
The processing result of is propagated to the output layer A3, and at the output layer A3,
The forward process is executed, and the output of the neural network is finally obtained.

The degree of coupling between neurons is called a coupling coefficient, and the coupling coefficient between the j-th neuron and the i-th neuron is represented by Wji.
There are two types of coupling: excitatory coupling, in which the output of the partner neuron increases as the signal from the partner neuron increases, and inhibitory coupling, in which the output of the partner neuron decreases, the output decreases. Excitatory coupling, Wji
When <0, it represents an inhibitory bond.

Now, assuming that the own neuron is the j-th neuron and the output of the i-th neuron is Oi, this is multiplied by the coupling coefficient Wji, and Wji · Oi becomes the input to the own neuron. Since each neuron is connected to a large number of neurons, Wji ·
ΣWji · Oi, which is the result of adding Oi, becomes the input to the own neuron. This is called the internal potential,
It is shown by the following equation.

[0011]

[Equation 1] net _j = ΣWji · Oi (1)

Next, this input is subjected to non-linear processing and output. The function at this time is called a nerve cell response function, and for example, a sigmoid function represented by the following equation (2) is used.

[0013]

[Number 2] _{f (net j) = 1 /} {1 + exp (-net j)} ... (2)

This function is shown in FIG. Value range is "0 to 1"
Then, as the input value increases, it approaches “1”, and as it decreases, it approaches “0”. From the above, the output O of the neuron j
j is expressed by the following equation (3).

[0015]

## EQU3 ## Oj = f (net _j ) = f (ΣWji · Oi) (3)

Next, the learning function of the neural network will be described. As a learning process used in the numerical calculation, a general back propagation algorithm (hereinafter, abbreviated as BP algorithm) will be briefly described.

In the learning process, when a certain input pattern p is given, the coupling coefficient is changed so as to reduce the error between the actual output value and the desired output value. The BP algorithm is an algorithm for obtaining this change amount.

Now, given a certain input pattern p, the difference between the actual output value (Opk) of the unit k and the desired output value (tpk) is defined by the following equation (4).

[0019]

[Number 4] ^{Ep = (tpk-Opk) 2} /2 ... (4)

This represents the error of the unit k in the output layer,
tpk is teacher data given by a human. In learning, the strength of all connections is changed to reduce this error.
Actually, the change amount of Wkj when the pattern p is given is expressed by the following equation (5).

[0021]

[Equation 5] ΔpWkj∝−ЭE / ЭWkj (5)

Using this, the coupling coefficient Wkj is changed. As a result, the following equation (6) is obtained from the above equation (4).

[0023]

[Equation 6] ΔpWkj = η · δpk · Opj (6)

Here, Opj is an input value from the unit j to the unit k. The error δpk differs depending on whether the unit k is the output layer or the intermediate layer. First, the error signal δpk in the output layer is given by the following equation (7).

[0025]

## EQU00007 ## .delta.pk = (tpk-Opk) .f '(net _k ) (7)

On the other hand, the error signal δpj in the intermediate layer is as shown in the following equation (8).

[0027]

Δpj = f ′ (net _j ) · ΣδpkWkj (8)

However, f '(net _j ) is f (net _j )
The first-order differential, which will be described in detail later. From the above, ΔWji is generally formulated as the following equation (9).

[0029]

ΔWji (t + 1) = η · δpj · Opi + α · ΔWji (t) (9)

Therefore, it is as shown in the equation (10). Here, t is a learning order, η is a learning constant, and α is a stabilizing coefficient. The first term on the right side of the above equation (9) is added to ΔWji obtained by the above equation (6), and the second term is added to reduce the error vibration and accelerate the convergence.

[0031]

(10) Wji (t + 1) = Wji (t) + ΔWji (t + 1) (10)

As described above, the calculation of the variation ΔWji of the coupling coefficient starts from the unit of the output layer and proceeds to the unit of the intermediate layer. Learning proceeds in the opposite direction to the processing of the input data, that is, backward. Therefore, in learning by back propagation, first, learning data is input and the result is output (forward). Change the strength of all joins to reduce the resulting error (backward). Input the learning data again. This will be repeated until it converges.

The conventional hierarchical neural network is
A network as shown in FIG. 2 is formed. FIG. 4 shows the data flow of the forward process in this network. This is the case in a three-layer hierarchical network. In the forward process, input signals Oi1 to Oi4 are given to the input layer (the layer on the left side of FIG. 4), and the output signals Ok1 to Oi from the output layer (the layer on the right side of FIG. 4). Get Ok4.

On the other hand, FIG. 5 shows the data flow of the learning process. In the learning process, the teacher signals tk1 to tk4 are given to the output layer (the layer on the right side of FIG. 5) to update the coupling strength between the neurons and learn so that the output of the neural network matches the teacher signal. It should be noted that the processing by this learning process is
It is executed by an external general-purpose computer.

FIG. 6 shows an example in which the network is realized by an electric circuit (Japanese Patent Laid-Open No. 62-295188). This circuit basically comprises a plurality of amplifiers 4 having an S-shaped transfer function, and a resistive feedback network 2 for connecting the output of each amplifier 4 to the input of an amplifier of another layer as shown by a chain line. And are provided. A CR time constant circuit 3 including a grounded capacitor C and a grounded resistor R is individually connected to the input side of each amplifier 4. Then, the input current I _1, I ₂ ~I _N is supplied to the input of the amplifier 4, the output is obtained from a set of these output voltage of the amplifier 4.

Here, the strength of the input and output signals is represented by a voltage, and the strength of the nerve cell coupling is the resistance 1 (the lattice point in the resistive feedback network 2) connecting the input / output lines between the cells. , And the nerve cell response function is represented by the transfer function of each amplifier 4. Further, the connection between nerve cells has excitability and inhibitory property as described above, and is mathematically represented by the sign of the connection function. However, since it is difficult to realize positive / negative with a constant on the circuit, here, the output of the amplifier 4 is divided into two (4a, 4b), and one output is inverted to obtain two positive / negative signals. Is generated and is selected so that it is realized. Also,
An amplifier is used as the one corresponding to the sigmoid function f (net) shown in FIG.

In general, when the neural network is configured by analog circuits, the area of a single neural circuit element can be reduced, so that the degree of integration of the neural network can be increased or the execution speed is fast. However, on the other hand, the value of each signal is represented by an analog amount such as a potential or current, and each calculation is performed by an analog element such as an amplifier. Therefore, there are variations due to temperature characteristics, and there are variations in the process of forming elements. However, there is also a problem that the response characteristics of each element cannot be made the same and the output value becomes unstable.

It is also difficult to arbitrarily change the value of the coupling coefficient of the neural network by learning, and it is also possible to perform learning by an external computer and download the value of each coupling coefficient after learning to the hardware. Often done. In such a learning method, there are problems that a computer is required for learning and the learning process is significantly slowed down.

Next, an example in which a neural network is composed of digital circuits will be shown. 7 to 9 are diagrams showing an example in which the neural network is realized by a digital circuit, and FIG. 7 shows a circuit configuration example of a single nerve cell. In these figures, 11 is a synaptic circuit, 12 is a dendrite circuit, and 13 is a cell body circuit.

FIG. 8 is a diagram showing a configuration example of the synapse circuit 11 shown in FIG. 7, which is a value obtained by multiplying the input pulse f by a factor a (a factor of 1 or 2 by which the feedback signal is multiplied) via the coefficient circuit 11a. Input rate multiplier 1
1b is provided, and the rate multiplier 11b is connected to a synapse weight register 11c that stores a weighting value w. FIG. 9 is a diagram showing a configuration example of the cell body circuit 13, which includes a control circuit 14 and an up / down counter 1.
5, a rate multiplier 16 and a gate 17 are sequentially connected, and an up / down memory 18 is further provided.

In this case, the input and output of the nerve cell unit is represented by a pulse train, and the signal density is represented by the pulse density. The coupling coefficient is treated as a binary number and stored in the synapse weight register 11c.

The signal calculation process is performed as follows. First, the input signal is input to the rate multiplier 11b and the coupling coefficient is input to the rate value to reduce the pulse density of the input signal according to the rate value. This corresponds to the Wji · Oi part of the expression of the back propagation model described above. The Σ portion of ΣWji · Oi is realized by the OR circuit shown by the dendrite circuit 12. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance and a logical sum (OR) is taken for each group. In FIG. 7, F1 indicates excitatory output and F2 indicates inhibitory output. An output is obtained by inputting these two outputs to the up side and the down side of the counter 15 shown in FIG. 9 and counting respectively. Since this output is a binary number, it is converted into a pulse density again using the rate multiplier 16. A neural network can be realized by constructing a network using a plurality of these nerve cell units. The learning function is realized by inputting the final output of the network to an external computer, performing numerical calculation inside the computer, and writing the result to the synapse weight register 11c that stores the coupling coefficient.

As described above, if the neural network is configured by a digital circuit, the area of a single neural circuit element becomes larger than that of an analog circuit, so that the integration degree of the neural network cannot be increased. However, it is relatively easy to form a circuit without being affected by temperature characteristics and process variations in device formation, which is a drawback of, and the output of nerve cell devices (neurons) is stable and highly reliable. There are advantages.

However, in the examples shown in FIGS. 7 to 9, since learning of the neural network relies on an external computer, there is a problem that a learning computer is required and the learning time is long.

The present applicant has already proposed a signal processing device using a neural network composed of neurons (for example, see Japanese Patent Application No. 5-118087). In the present invention, the signal processing device according to this prior application is treated as an example of one embodiment, and therefore the signal processing device according to this prior application will be described below.

In the signal processing device according to this prior application,
As an example of the neural network, we have proposed a neural cell unit using a digital logic circuit and signal processing by a network circuit configured by using it.

Here, the basic idea is as follows. 1. Input / output signals, intermediate signals in the nerve cell unit,
All the teacher signals are represented by a pulse train represented by binary values of "0" and "1". 2. The value of the signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). 3. The calculation in the nerve cell unit is performed by a logical calculation between pulse trains. 4. The pulse train of the coupling coefficient is stored in the memory in the nerve cell unit. 5. In learning, an error is calculated based on a given teacher signal pulse train, and the coupling coefficient is changed based on this error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1".

FIG. 10 shows a state of forward processing of one neuron element in the pulse density method, and considers the hierarchical neural network shown in FIG. 2 as the network configuration.

First, a logical product (AND) of the input coefficient Oi, which is binarized into "0" and "1" and expressed in pulse density, and the coupling coefficient Wji is obtained for each synapse. This corresponds to Wji · Oi in the equation (1). The pulse density of the output of the AND circuit stochastically becomes the product of the pulse density of the input signal and the pulse density of the coupling coefficient.

As described above, the connections between neurons include excitatory connections and inhibitory connections. In the case of numerical calculation, the sign of the coupling coefficient, for example, plus when excitatory, minus when inhibitory is performed.

In the case of the pulse density method, the coupling coefficient Wji
Depending on the positive or negative of each, each connection is classified into an excitatory connection and an inhibitory connection.
It is divided into two groups, and the logical sum is calculated by OR operation for each group. This corresponds to the processing of Σ in Equation 3 and the processing of the nonlinear saturation function f (net).

That is, in the calculation based on the pulse density,
When the pulse density is low, the pulse density resulting from the OR processing can be approximated to the sum of the pulse densities of the OR input.

Since the output of the OR circuit gradually saturates as the pulse density increases, the result does not match the sum of the pulse densities, and non-linearity appears.

In the case of this OR operation, the value P of the pulse density is 0≤P≤1, and since it is a monotonically increasing function with respect to the magnitude of the input, processing by the equation (2) or the sigmoid function of FIG. 3 is performed. Will be similar to.

The output of the nerve cell element by the pulse density method is the OR of the excitability groups obtained by the above calculation.
And the output net Non _j ⁺ is "1", OR output net Non _j inhibitory group ^- outputs "1" only when "0". That is,
It is expressed as shown in the following equations (11) to (13).

[0056]

[Equation 11]

Next, the learning process in the pulse density method will be described. In a non-learned neural network, the output of the network when a certain pattern is input is not always the desired output. Therefore, similar to the BP algorithm described above, the learning process changes the coupling coefficient so as to reduce the error between the actual output value and the desired output value.

(Error Signal in Output Layer) First, the error signal in the output layer will be described. Here, when the error is expressed by a numerical value, both positive and negative values can be taken, but since such an expression cannot be performed in the pulse density method, two signals, a signal δk ⁺ representing a plus component and a signal δk ⁻ representing a minus component, can be obtained. Using the error signal in the output layer, the following equations (14) and (1
Define as in 5).

[0059]

(Equation 12)

The error signal plus component δk ⁺ is the output result O
When k is "0" and the teacher signal tk is "1", it becomes "1", and otherwise it becomes "0".

[0061] On the other hand, the error signal minus component .delta.k ^- the output result Ok is "1", when the teacher signal tk is "0",
It becomes "1", and otherwise it becomes "0".

The error signals δk ⁺ and δk ⁻ are the above-mentioned B
In the P algorithm, the difference (t _pk − between the teacher signal of the above equation (7) for obtaining the error signal of the output layer and the actual output signal
O _pk ).

Next, as shown in equation (7), these error signals and the output function f (net), which is the first derivative f ′,
The product of (net) and the error signal in the output layer is obtained. Generally, a differential coefficient obtained by differentiating an output signal with an internal potential is required for calculating an error signal during learning. In the BP algorithm described above, the output function f (net) is
When a sigmoid function is used, its first derivative f ′ (ne
t) is shown by Formula (16).

[0064]

F ′ (net) = df (net) / dnet = f (net) · {1-f (net)} (16)

In the pulse density method, referring to the equation (16), the first-order differential f '(net) in the output layer neuron is used.
plus component f '(net) k ⁺ and minus component f of k
′ (Net) k ⁻ is defined as in equations (17) and (18).

[0066]

[Equation 14]

Here, Ok (t-1) is the output signal Ok.
Is a 1-pulse delay value, and Ok (t−2) is a 2-pulse delay value of the output signal Ok. Therefore, the final error signal in the output layer is expressed by the following equations (19) and (20). This corresponds to the equation (7) for obtaining the error signal of the output layer in the above-mentioned BP algorithm.

[0068]

(Equation 15)

(Error Signal in Intermediate Layer) The error signal in the intermediate layer based on the pulse density method is also obtained by referring to the above equation (8) based on the BP algorithm. That is, the error signals in the output layer are collected and used as the error signal in the intermediate layer. Here, the connection is divided into two groups depending on whether it is excitatory or inhibitory, the product part is represented by ∩ (AND), and the sum (Σ) part is represented by ∪ (OR).

Further, the error signal in the intermediate layer is obtained.
In this case, the coupling coefficient Wkj is positive / negative and the error signal δk is positive / negative.
There are two cases. First, in the case of excitatory coupling, the output layer
Error signal plus component δk⁺ And AND of the coupling coefficient
What was taken (δk⁺ ∩ Wkj ⁺ ) For all output layers
Find the uron and take the OR of these. This is medium
Error signal plus component δj of interlayer neuron⁺ Becomes (expression
(21)).

[0071]

[Number 16] ^{δj + = ∪ (δk + ∩Wkj} +) ... (21)

Similarly, the error signal of the output layer minus the component δ
AND of k ⁻ and its coupling coefficient (δk ⁻ ∩
Wkj ⁺ ) is obtained for neurons in all output layers, and these are ORed. This becomes the error signal minus component of the hidden layer neuron (equation (22)).

[0073]

[Number 17] ^{δj - = ∪ (δk - ∩Wkj} +) ... (22)

Next, the case of inhibitory binding will be described.
Output layer error signal minus component δk ^-And its coupling coefficient
ANDed with (δk^- ∩ Wkj^- ) All
For the neurons in the output layer of
You. This is the error signal plus component of the intermediate layer (equation (2
3)).

[0075]

[Number 18] ^{δj + = ∪ (δk - ∩Wkj} -) ... (23)

Similarly, the error signal plus component δk of the output layer
AND of ⁺ and its coupling coefficient (δk ⁺ ∩W
kj ^-) a request for the neurons of all of the output layer,
These are ORed together. This becomes the error signal minus component of the hidden layer neuron.

[0077]

[Number 19] ^{δj - = ∪ (δk + ∩Wkj} -) ... (24)

The connection between a neuron in a certain middle layer and the neuron in the output layer connected to the middle layer includes excitatory connection and inhibitory connection. Therefore, as the error signal plus component of the intermediate layer, δj ⁺ of the excitatory coupling of the equation (21) and the equation (23)
OR the δj ⁺ of the inhibitory couplings of Similarly, the error signal minus component of the intermediate layer, excitatory coupling .delta.j of formula (22) ^- ORed with the ^- .delta.j inhibitory binding of the formula (24). That is, the following equations (25) and (26) are obtained. This is the Σ of the above equation (8) according to the BP algorithm.
Corresponds to δpkWkj.

[0079]

Equation 20] ^{δj + = {∪ (δk +} ∩Wkj +)} ∪ {∪ (δk - ∩Wkj -)} ... (25) δj - = {∪ (δk - ∩Wkj +)} ∪ {∪ (δk ⁺ ∩Wkj ^- )}… (26)

[0080] Then, similarly to the BP algorithm of formula (8), equation (25), .delta.j obtained in (26) ^+, .delta.j ^- in formula (17), the derivative of (18) (f'(net Non )) Is applied. Here, the positive component f ′ (net) j ⁺ and the negative component f ′ (net) j ⁻ of the first-order differential f ′ (net) j in the intermediate layer neuron are (27) and (28).
Is defined as Here, Oj (t-1) is a one-pulse delay value of the output signal Oj of the hidden layer neuron.

Therefore, the error signal (δj in the intermediate layer is
⁺ , Δj ⁻ ) is calculated by the following equations (29) and (30).

[0082]

(Equation 21)

[0083]

(Equation 22)

(Processing by Learning Constant η) Processing of the learning constant η in the above equation (6) for obtaining the correction amount ΔW of the coupling coefficient in the BP algorithm will be described. In the numerical calculation, the learning constant η may be simply multiplied as shown in the equation (6), but in the case of the pulse density method, the pulse train is thinned out as shown below according to the value of the learning constant η. It will be realized.

[0085]

(Equation 23)

Next, a method for obtaining the correction amount ΔW of the coupling coefficient by learning will be described. First, the error signal (δ +, δ−) in the output layer or the intermediate layer is processed by the learning constant η, and the logical product with the input signal to the neuron is taken (δη∩O). However, since there are δ ⁺ and δ ⁻ in the error signal, ΔW ⁺ , ΔW ⁺ ,
ΔW ^- to.

[0087]

[Number 24] ^{ΔW + = δη + ∩O ... (} 34) ΔW - =
δη ^- ∩O… (35)

This corresponds to the above equation (6) for obtaining ΔW in the BP algorithm. A new coupling coefficient New_W is obtained based on these, and cases are classified depending on whether the coupling coefficient W is excitatory or inhibitory. First, in the case of excitability, the component of ΔW ⁺ is increased with respect to the original W ⁺ , and ΔW ^+
- reduce the component of. That is, formula (36) is obtained.

[0089]

(Equation 25)

Next, in the case of the suppressive property, the component of ΔW ⁺ is decreased and the component of ΔW ⁻ is increased with respect to the original W ⁻ . That is, equation (37) is obtained.

[0091]

(Equation 26)

The learning algorithm based on the pulse density method has been described above. Here, the processing flow of the forward process and the learning process in the pulse density method in the hierarchical network of FIG. 2 will be briefly described. First,
Although it is a forward process, when a signal is first given to the input layer, this input signal propagates to the intermediate layer, and as the signal processing of the intermediate layer, the above equations (11) to (13) are performed, and the result is To the output layer.

In the output layer, the processes of equations (11) to (13) are similarly performed on these propagated signals,
These results in an output signal, ending the forward process.

In the learning process, after performing the above forward process, a teacher signal is further given to the output layer. In the output layer, the error signal in the output layer is obtained by the equations (19) and (20) and sent to the intermediate layer. At the same time, the error signal is processed by the learning constant η of the equations (31) to (33), and the logical product of the error signal and the input signal from the intermediate layer is obtained by the equations (34) and (35). , (37), the coupling strength between the output layer and the intermediate layer is changed.

Next, as the processing in the intermediate layer, the error in the intermediate layer is obtained by the equations (29) and (30) based on the error signal sent from the output layer, and the error signal is given by the equation (3).
Processing with the learning constant η of 1) to (33) is performed, and the expression (3
4) and (35), the logical product with the input signal from the input layer is obtained, and then the coupling strength between the intermediate layer and the input layer is changed by the equations (36) and (37), and the learning process ends. After that, the learning process is repeated until it converges.

Next, an actual circuit configuration based on the above algorithm will be described with reference to FIGS. 11 to 13. The structure of the neural network is the same as in FIG. FIG.
FIG. 1 is a diagram showing a circuit of a portion corresponding to a synapse of a neuron, and FIG. 12 is a diagram showing a circuit for obtaining an error signal in the output layer from the output of the neuron cell body of the output layer, the output of the output layer, and the teacher signal. . Further, FIG. 13 is a diagram showing a circuit of a portion for collecting error signals in the cell body of neurons in the intermediate layer and the output layer and obtaining an error signal in the intermediate layer. By forming a network of these three circuits as shown in FIG. 2, a digital neural network circuit capable of self-learning can be realized.

First, FIG. 11 will be described. The synapse coupling coefficient is stored in the shift register 21. The terminal 21A is a data outlet and the terminal 21B is a data inlet. Other components such as a RAM and an address generator may be used as long as they have the same function as the shift register. Shift register 2
The circuit 22 including 1 is a circuit that executes (Oi∩Wji) in the above equations (11) and (12), and ANDs the input signal and the coupling coefficient. This output must be grouped according to whether the connection is excitatory or inhibitory,
It is more versatile to prepare the outputs O ⁺ and O ⁻ for each group in advance and switch which group to output to. Therefore, a bit indicating whether the coupling is excitatory or inhibitory is stored in the memory 23, and the information is used to switch the signal by the switching gate circuit 24. Further, as shown in FIGS. 12 and 13, a gate circuit 31 having a plurality of OR gates corresponding to the logical sum of the equations (11) and (12) for processing each input is provided. Further, as shown in the figure, a gate circuit 32 formed by an AND gate and an inverter that outputs only when the excitatory group represented by the equation (13) is "1" and the inhibitory group is "0".
Is provided.

Next, the error signal will be described. FIG.
The circuits 35 and 36 in FIG. 2 are circuits showing a circuit for generating a differential coefficient of an output function in the output layer, and are logical circuits by a combination of AND (logical product) and an inverter, and are expressed by the equations (17) and (18). Equivalent to. And the circuit 37
In the figure showing the circuit that generates the error signal in the output layer,
(Logical product), a logical circuit by a combination of inverters, which corresponds to the equations (19) and (20). That is, the error signals δk ⁺ , δ in the output layer are obtained by the output Ok from the output layer, the teacher signal tk, and the differential component plus and minus components generated by the differential coefficient generation circuits 35 and 36.
Generate k ⁻ . Further, the above equations (21) to (24) for obtaining the error signal in the intermediate layer are AN shown in FIG.
This is performed by the gate circuit 26 having the D gate configuration, and an output (δ∩W) according to the plus component and the minus component is obtained.
Since the error signal to be used is different depending on whether the coupling is excitatory or inhibitory, it is necessary to make a distinction in that case. In this case, the excitatory or inhibitory information stored in the memory 23 and the error are stored. AND, O depending on the δ ⁺ , δ ⁻ signal of the signal
This is performed by the gate circuit 25 having an R gate configuration. Also,
The arithmetic expressions (25) and (26) for collecting the error signals are performed by the gate circuit 39 having the OR gate structure shown in FIG. A circuit that executes the equations (29) and (30) for obtaining the error signals δj ⁺ and δj ⁻ in the intermediate layer is a signal Oj (t in which the intermediate output Oj is delayed by the circuit 40 in FIG. 13 and the intermediate layer output Oj is delayed by the register 33. −1) and the error signal from the output layer, which is the output of the OR circuit 39, generate error signals δj ⁺ and δj ⁻ . Expressions (31) to (33) corresponding to the learning rate are η shown in FIG.
It is performed by the circuit 38.

The T-FF (trigger type flip-flop) 38a in FIG. 14A inverts the output Q when the rising of the input T is detected. Further, the multiplexer (MUX) 38b, as shown in FIG.
1, S0 selects one of the input signals A, B, C and outputs it. The figure (b) shows the learning coefficient η of the figure (a).
6 is a timing chart showing the operation of the circuit. When the reset signal (RES) is input first, T
-The output of the FF 41 is set to the initialized state, and the value of the learning coefficient η is set by the selection signals S1 and S0 of the multiplexer 38b.

When the error signal δ is input, as the input signal A of the multiplexer 38b, a pulse subjected to the processing of the learning coefficient η = 1/4 (processing to leave every third pulse) is generated (Equation 31). (Corresponding to Equation 32), a pulse on which a learning coefficient η = 1/2 processing (processing to leave every other pulse) is performed is generated as the input signal B (corresponding to Expression 32), and the learning coefficient is input signal C. A pulse subjected to the processing of η = 1 (processing for making the pulse the same as the original signal) is generated (Equation 3).
3).

Then, one of the input signals A, B and C is output as a pulse δη by the selection signals S1 and S0 of the multiplexer 38b. FIG. 14 (a)
Shows a main circuit that executes the processing by the learning coefficient η, and in reality, a signal noise prevention circuit and the like are added.

Finally, the part for calculating a new coupling coefficient from the error signal will be described. This is the above equation (34).
~ (37), which is performed by the gate circuit 27 having the AND, inverter, and OR gate configurations shown in FIG.
This gate circuit 27 must also be classified depending on the excitability / inhibition of the coupling, which is performed by the gate circuit 25 shown in FIG. The conventional example shown above is referred to as a first conventional example.

Next, another conventional example for realizing a signal processing circuit network by the pulse density method (hereinafter, this conventional example will be referred to as a second conventional example) will be described. This is MSTomlinson,
Jr., DJ Walker and MASivilotti, ^ A digital neural
network architecture for VLSI ^, Proc.ij CNN-90II, Sa
n Diego, CA, 1990, 545-550, the state of forward processing of one neuron element is the same as in FIG. 10 described above. Here, in consideration of establishment of the existence of a pulse in the pulse train, the pulse calculation formula for obtaining the neuron output Oj is represented by the following formulas (38), (39) and (40).

[0104]

[Equation 27]

Next, the learning process will be described. The equations for obtaining the error E and the update amount ΔW of the coupling coefficient in the output layer are the same as the equations (4) and (5) obtained by the above-mentioned back propagation (BP) algorithm. First, updating the coupling coefficient when the synaptic coupling is excitatory is described. Assuming that the error signal plus component of the output layer neuron is δk ⁺ , the following equation (41) is obtained from equations (4) and (40).
Is obtained.

[0106]

[Equation 28]

On the other hand, the error signal plus component δj ⁺ of the hidden layer neuron is expressed by the following equation (42).

[0108]

(Equation 29)

Here, Skj ⁺ and Skj ⁻ are expressed by the following equations (43) and (44), respectively.

[0110]

[Equation 30]

Therefore, the change amount of excitatory coupling is expressed by the following formula (45) by the above formula (5).

[0112]

(Equation 31)

Next, updating of the coupling coefficient when synaptic coupling is inhibitory will be described. Here, assuming that the error signal minus component of the output layer neuron is δk ⁻ , equation (4),
From (40), the following equation (46) is obtained.

[0114]

(Equation 32)

On the other hand, the error signal minus component δj ⁻ of the hidden layer neuron is expressed by the following equation (47).

[0116]

[Equation 33]

Here, Skj ⁺ and Skj ⁻ are the same as those shown in the above equations (43) and (44). Therefore, the update amount of the inhibitory binding is expressed by the following equation (48) by the above equation (5).

[0118]

(Equation 34)

[0119]

However, in the above-mentioned first conventional example, the equation (2) in the case of the back propagation (BP) algorithm for the output function of the neuron is given by
A learning algorithm is derived by approximating the sigmoid function shown in FIG. 3 and assuming that the differential coefficient equation (16) is represented by equations (17) and (18) obtained by referring to the equation (16). Therefore, there are the following problems.

That is, in practice, the neuron output in the case of the pulse density method depends on the establishment of the pulse and is not a sigmoid function in the strict sense. Therefore, the learning algorithm obtained by approximating the sigmoid function does not It doesn't work.

Further, as shown in FIG. 11, since the groups are classified into inhibitory and excitatory groups in advance and the update amount of the coupling coefficient is obtained for each of the inhibitory coupling and the excitatory coupling, the excitement of the coupling coefficient is calculated. There is also a drawback in that the sign indicating whether the character is active or inhibitory cannot be changed by learning.

On the other hand, in the second conventional example, the neuron output is represented by the stochastic equations of the above equations (38) to (40), and these functions are shown in the equations (41) to (48) by the steepest descent method. As such, it faithfully seeks a learning algorithm. The value of the coupling coefficient updated by this learning is applied to the signal processing circuit network shown by the logic circuit in FIG. 10, and the neural network processing is executed. Therefore, there is no drawback that the above learning is not successful, and the learning ability is high. I can say.

However, in the second conventional example, since the equations (45) and (48) representing the update amount of the coupling coefficient are obtained by numerical calculation by an external computer, a learning computer is required and the learning time is increased. Has the problem of becoming longer.

In view of the above circumstances, the present invention provides a pulse density type signal processing circuit network in which the learning algorithm is faithfully derived from the neuron output function and the learning process can be realized by using a simple logic circuit. An object of the present invention is to provide a method for updating a coupling coefficient in a pattern signal processing network.

[0125]

A method of updating a coupling coefficient in a pulse density type signal processing circuit according to the present invention is a method of updating a coupling coefficient in a pulse density type signal processing circuit in which neuron mimicking elements are connected in a net form. In the method of updating the synapse coupling coefficient indicating the increase / decrease degree of the signal when propagating the signal, when the coupling coefficient Wkj of the neuron k and the neuron j is updated, the error signal plus component in the neuron k is added to the first signal. , The error signal minus component in the neuron k is the second signal, the output Oj of the neuron j is the third signal, the coupling coefficient Wkm with the neuron m connected to the neuron k excluding the neuron j, and the output of the neuron m All the logical products with Om are obtained, and the inverted output of the logical product of these is taken as the fourth signal. Update amount of the coupling coefficient is obtained by subjecting the logical product of the first signal, the inverted value of the second signal, the third signal, and the fourth signal to the processing by the learning coefficient. A positive component, an inverted output of the first signal,
The logical product of the second signal, the third signal, and the fourth signal, which has been processed by the learning coefficient, is used as the update amount minus component of the coupling coefficient, and the update amount plus component Processing for adding "1" when the OR output with the coupling coefficient pulse before update is "1", and processing for subtracting "1" when the minus component pulse of the update amount is "1" And the result values obtained by these addition and subtraction processes are used as new coupling coefficients.

Further, the coupling coefficient updating method in the pulse density type signal processing circuit of the present invention propagates a signal from another neuron to the neuron of the pulse density type signal processing circuit network in which neuron mimicking elements are connected in a mesh. In the method of updating the synaptic coupling coefficient indicating the increase / decrease degree of the signal at the time, the coupling coefficient Wk between the neuron k and the neuron j
When updating j, the error signal plus component in the neuron k is used as the first signal, the error signal minus component in the neuron k is used as the second signal, and the output Oj of the neuron j is used as the third signal. , And the output Om of the neuron m is calculated, and the inverted output of the logical sum of these is taken as the fourth signal. An inverted value of the first signal and the second signal,
The logical product of the third signal and the fourth signal, which has been subjected to the processing by the learning coefficient, is used as the update amount of the coupling coefficient plus the component, and the inverted output of the first signal and the second signal The logical product of the signal, the third signal, and the fourth signal, which has been subjected to processing by the learning coefficient, is used as the updating amount minus component of the coupling coefficient, and the updating amount plus component pulse is “1”. When it is "," 1 is added to the coupling coefficient value before updating, and when the updating amount minus component pulse is "1", "1" is subtracted from the coupling coefficient value before updating. It is characterized in that the processing is performed, and the result value obtained by the addition / subtraction processing with respect to the coupling coefficient value before updating is set as a new coupling coefficient.

In any of the above configurations, since the learning algorithm is faithfully derived from the neuron output function, the learning ability is enhanced. In addition, since the learning process can be performed by logical sum or logical product for each signal, the circuit for implementing the coupling coefficient updating method can be simplified, and the logical calculation is performed in this way. Since it does not depend on calculation, an external microcomputer can be eliminated and the processing speed can be improved. Further, the addition / subtraction processing described above is performed by, for example, up / down.
Since a down counter can be used and the sign bit of the up / down counter can represent the sign of the coupling coefficient, it is possible to perform the code conversion by learning as well as updating the coupling coefficient.

[0128]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention faithfully derives a learning algorithm based on the output function of a neuron in a pulse density type signal processing circuit network. The method of faithfully performing numerical calculation by the steepest descent method based on the neuron output function is as shown in Expressions (41) to (48). In the present invention, the learning algorithm based on the neuron output function is derived. It is realized by a logical operation using a logical product (AND), a logical sum (OR), etc. instead of numerical calculation.

(Embodiment 1) An embodiment of the present invention will be described below. In this embodiment, updating of the coupling coefficient between the intermediate layer neuron j and the output layer neuron k will be described.

First, when the synaptic connection is excitatory, based on the error signal plus component δk ⁺ of the output layer neuron k,
Considering the value of the teacher signal Tk and the output Ok of the output layer neuron k, the plus component δk ⁺⁺ and the minus component δk ⁺ -are defined as in the following equations (49) and (50), respectively.

[0131]

(Equation 35)

Here, NETk ⁻ is the above-mentioned equation (12).
It is the sum of the inhibitory values of the output layer neuron k obtained in. These expressions (49) and (50) are the expressions (4
This corresponds to 1).

[0133] On the other hand, if the coupling coefficient is inhibitory, an error signal minus component .delta.k output layer neurons k ^- based on, taking into account the value of the output Ok of the output layer neurons k a teacher signal Tk, the positive component .delta.k ^{- −} And the minus component δk ^{− +} are defined by the following equations (51) and (52).

[0134]

[Equation 36]

Here, NETk ⁺ is the above-mentioned equation (11).
It is the total excitability of the output layer neurons obtained in. These equations (51) and (52) are equivalent to the equation (4
This corresponds to 6).

When obtaining the update amount of excitatory coupling,
An equation corresponding to the above equation (43) in the case of the numerical operation is represented by the following equation (53).

[0137]

(37)

Therefore, the renewal amount of excitatory coupling plus the component Δ
Wkj ⁺ and the update amount minus component ΔWkj ⁻ are represented by the following equations (54) and (55) from the above equations (45), (49), (50) and (53), respectively.

[0139]

(38)

On the other hand, when obtaining the update amount of the inhibitory binding, the equation corresponding to the above equation (44) in the case of numerical calculation is shown by the following equation (56).

[0141]

[Equation 39]

Therefore, the update amount of the inhibitory bond plus the component Δ
The Wkj ⁺ and the update amount minus component ΔWkj ⁻ are represented by the following equations (57) and (58) from the above equations (48), (51), (52) and (56), respectively.

[0143]

(Equation 40)

Now, the updating of the coupling coefficient is performed based on the coupling coefficient pulse Wkj before the updating and the values of the plus component ΔWkj ⁺ of the update amount and the minus component ΔWkj ⁻ of the update amount by the following equations (59) and (60). It is done like.

[0145]

[Equation 41]

The above equation resets a counter (specifically, an up / down counter 41, which will be described later) in which the updated coupling coefficient Wkj is stored before shifting to the learning process. In executing the learning process for one frame, when Wkj∪ΔWkj ⁺ = 1, the counter is incremented, and ΔWkj ⁻ = 1.
In this case, it can be realized by decrementing the counter.

FIG. 15 shows the combination coefficient pulse Wkj before updating.
And the value of the plus component ΔWkj ⁺ of the update amount and the minus component ΔWkj ⁻ of the update amount, the circuit showing the circuit for updating the coupling coefficient. That is, this circuit generates the update amount component ΔWkj and the equation (59),
This is a circuit that realizes the operation of (60) using an AND circuit, an OR circuit, or the like. In FIG. 15,
The same reference numerals are given to the same functional portions as the conventional functional portions shown in FIG. 12 and the like.

The plus component ΔWkj ⁺ of the update amount is calculated by the AND circuit 46 having four input parts shown in the figure and this AN
It is generated by the η circuit 38 which is connected to the output side of the D circuit 46 and executes the processing by the learning coefficient η. That is, the plus component ΔWkj ⁺ of the update amount is calculated by the equations (54) and (5
7) are those obtained, and S'kj is an element of this equation, the output Oj of the neuron j, the error signal plus component .delta.k ⁺ (i.e., .delta.k in .delta.k ⁺⁺ or inhibitory in excitatory ^-) And the error signal minus component δk
^- (i.e., .delta.k + in excitatory ^- .delta.k in or inhibitory - ⁺⁾ and the inverting output of the (inverted by inverter 44), these logical product to the four input is inputted from the AND circuit 46 Is output, and the logical product output is the η circuit 38.
Is input to and processed, the plus component ΔWkj ⁺ of the update amount is output.

[0149] Similarly, the update amount of the negative component DerutaWkj ^-
Is an AND circuit 47 having four input parts shown in the figure.
And an η circuit 38 that is connected to the output side of the AND circuit 47 and executes processing with the learning coefficient η. That is, the minus component ΔWkj ⁻ of the update amount is obtained by the above equations (55) and (58), and S′kj which is an element of this equation and the output Oj of the neuron j.
When the error signal plus component .delta.k ⁺ (i.e., the aforementioned .delta.k ⁺⁺ or .delta.k ^-) and the inverting output of the (inverted by inverter 45), the error signal minus component .delta.k ^- (i.e., the aforementioned .delta.k + ^- or .delta.k ^-+ ) Is input to each of the four input sections of the AND circuit 47 to output a logical product of these, and the logical product output is input to the η circuit 38 and processed,
The minus component ΔWkj ^{− of the} update amount is output.

The combination coefficient pulse Wkj before updating is held in the combination coefficient storage register (W-REG) 42 as a binary value. The coupling coefficient pulse Wkj before updating is supplied from the coupling coefficient storage register 42 to the pulse generator 43.

The pulse generator 43 generates a pulse having the value of the coupling coefficient as the existence of the pulse, and supplies the generated pulse to the input part of the OR circuit 48.

The OR circuit 48 has two inputs, one of which receives the pulse from the pulse generator 43, and the other of which the positive component ΔWk of the update amount.
j ⁺ is input, and the logical sum of these two signals is obtained and output. This process corresponds to the process of Wkj∪ΔWkj ⁺ in equation (59).

The logical sum output from the OR circuit 48 is the increment input portion (IN
C). The decrement input unit (DEC) of the up / down counter 41 is supplied with the minus component ΔWkj ⁻ of the update amount.

When the signal "1" is input to the increment side input section (INC) of the up / down counter 41, the up / down counter 41 executes the increment count operation, and the decrement side input section (DEC) receives the signal "1". When input, the decrement count operation is executed. This operation corresponds to “1 then new-Wk” in Expression (59).
This corresponds to the processing of “j = Wkj + 1” and the processing of “1 then new−Wkj = Wkj−1” in Expression (60).

That is, the count value of the up / down counter 41 by the counting operation is output as the updated coupling coefficient pulse Wkj (New-Wkj). Further, the updated coupling coefficient pulse Wkj is output as the pre-updated coupling coefficient pulse Wkj in the next update, so that the above-described coupling coefficient storage register (W-RE) is output.
G) It is designed to be held in 42.

The up / down counter 41 performs a reset operation when a reset signal is input to its reset terminal (RES), and starts the count operation after this reset.

The operation of the above circuit will be briefly described. First, when the reset signal is input to the reset terminal of the up / down counter 41, the up / down counter 41 is reset and the counting operation is started. To do. In this counting operation, the update amount of the coupling coefficient plus the component ΔW obtained by the AND circuit 46 and the η circuit 38.
When the logical sum of kj ⁺ and Wkj pulse is “1”,
The up / down counter 41 performs increment counting operation, and when the update amount minus component ΔWkj ^{− of the} coupling coefficient obtained by the AND circuit 47 and the η circuit 38 is “1”, the up / down counter 41 performs decrement counting operation. I do. This counting operation is 1
This is executed during the frame, and the value of the up / down counter 41 after this execution becomes the updated coupling coefficient New-Wkj. Then, the updated coupling coefficient New-Wkj
Are stored in the coupling coefficient register 42 for the next learning process. The learning is performed once by the series of processes.

As described above, according to the coupling coefficient updating method in the pulse density type signal processing circuit of this embodiment, the learning algorithm is faithfully derived from the neuron output function in the pulse density type signal processing circuit. As a result, learning ability is improved.

Further, the processing for updating the coupling coefficient is performed by the logic circuits such as the AND circuits 46 and 47 and the OR circuit 48 described above.
Also, since it can be performed by using a simple circuit such as the up / down counter 41, the circuit for implementing the coupling coefficient updating method can be simplified, and the numerical value can be obtained by the logical calculation as described above. Since it does not depend on calculation, an external microcomputer can be eliminated and the processing speed can be improved.

Further, the above-mentioned up / down counter 4
According to 1, since the sign of the coupling coefficient can be represented by the sign bit, the sign of the coupling coefficient can be converted by learning, and the algorithm of the algorithm can be compared with the conventional pulse density type signal processing circuit network. Can be improved.

(Second Embodiment) Next, another embodiment of the present invention will be described. This embodiment is common to the first embodiment in that the coupling coefficient between the intermediate layer neuron j and the output layer neuron k is described, but the first embodiment is updated by the equations (59) and (60). In contrast to the latter coupling coefficient being obtained, in this embodiment, the coupling coefficient Wkj before updating is increased by 1 when the plus component ΔWkj ⁺ of the update amount is 1, and when the minus component ΔWkj ⁻ of the updating amount is I am trying to update it by reducing it by 1. That is, the difference is that the updated coupling coefficient is obtained by the following equations (61) and (62).

[0162]

(Equation 42)

The above equation is used, for example, when the coupling coefficient Wkj before update is loaded into the counter (up / down counter 50 described later) before the learning process is executed and the learning process is executed for one frame. , ΔWkj ⁺ =
When it is 1, the counter is incremented by "1" and ΔW
kj - ⁼ 1 for when can be realized by reducing the "1" in the counter.

FIG. 16 shows the update amount component Wk of the coupling coefficient.
AND the generation of j and the operations of equations (61) and (62).
FIG. 6 is a circuit diagram showing a circuit to be executed by using a circuit and an OR circuit. The same functional parts as those shown in FIG. 15 of the first embodiment are designated by the same reference numerals.

The plus component ΔWkj ⁺ of the update amount is obtained by the AND circuit 46 having four input parts shown in the figure and this AN
It is generated by the η circuit 38 which is connected to the output side of the D circuit 46 and executes the processing by the learning coefficient η. That is, the plus component ΔWkj ⁺ of the update amount is calculated by the equations (54) and (5
7) are those obtained, and S'kj is an element of this equation, the output Oj of the neuron j, the error signal plus component .delta.k ⁺ (i.e., the aforementioned .delta.k ⁺⁺ or .delta.k ^- a),
The error signal minus component δk ⁻ (that is, δk ^{+ −} or δk ^{− +} described above) and its inverted output (inverted by the inverter 44) are input to the four input sections of the AND circuit 46 and their logics are input. The product is output, and this logical product output is the η circuit 3
By being input to 8 and processed, the plus component ΔWkj ⁺ of the update amount is output.

[0166] Similarly, the update amount of the negative component DerutaWkj ^-
Is an AND circuit 47 having four input parts shown in the figure.
And an η circuit 38 that is connected to the output side of the AND circuit 47 and executes processing with the learning coefficient η. That is, the minus component ΔWkj ⁻ of the update amount is obtained by the above equations (55) and (58), and S′kj which is an element of this equation and the output Oj of the neuron j.
When the error signal plus component .delta.k ⁺ (i.e., the aforementioned .delta.k ⁺⁺ or .delta.k ^-) and the inverting output of the (inverted by inverter 45), the error signal minus component .delta.k ^- (i.e., the aforementioned .delta.k + ^- or .delta.k ^-+ ) Is input to each of the four input sections of the AND circuit 47 to output a logical product of these, and the logical product output is input to the η circuit 38 and processed,
The minus component ΔWkj ^{− of the} update amount is output.

The coupling coefficient Wkj before being updated is held as a binary value in the coupling coefficient storage register (W-REG) 42 and is loaded into the data-in terminal (DI) of the up / down counter 50. ing.

When the load signal is input to the load signal terminal (LD) of the up / down counter 50, the up / down counter 50 takes in the coupling coefficient Wkj before updating from the data-in terminal (DI). Further, as in the first embodiment, when the signal "1" is input to the increment side input unit (INC), the increment count operation is executed, and when the signal "1" is input to the decrement side input unit (DEC), Performs a decrement count operation. This operation is performed by the equation (6
1) "1 then new-Wkj = Wkj"
+1 ”processing and“ 1 the in equation (62)
"n new-Wkj = Wkj-1".

As described above, according to the coupling coefficient updating method in the pulse density type signal processing circuit of this embodiment, as in the first embodiment, in the pulse density type signal processing circuit, from the neuron output function, A learning algorithm is faithfully derived. Then, the processing of updating the coupling coefficient is performed by the logic circuits of the AND circuits 46 and 47, and
It is realized by using a simple circuit such as the counter 50. Further, according to the up / down counter 41 described above, since the sign of the coupling coefficient can be represented by the sign bit, the sign of the coupling coefficient can be converted by learning, and the conventional pulse density type signal can be converted. The algorithm can be improved compared to the processing circuit network.

(Embodiment 3) Next, another embodiment of the present invention will be described. In this embodiment, the updating method of the coupling coefficient described in the first embodiment is applied to the updating of the coupling coefficient between the input layer neuron i and the intermediate layer neuron j. It should be noted that in the first aspect, k and j are attached to the neurons, but this is for convenience only and does not mean that the relationship between the neurons i and j in this embodiment is not included.

(−∂E / ∂Oj) in the equations (42) and (47) for obtaining the error signal plus component δj ⁺ and the error signal minus component δj ⁻ in the intermediate layer neuron j is given by the following equation (63). Can be shown as: Here, the plus component (δk ⁺⁺ , δk ^{− +} ) and the minus component (δk ^+- , δ) based on the error signals δk ⁺ and δk ⁻ of the output layer neuron.
k ^-) is determined by the above formula (49) to (52).

[0172]

[Equation 43]

The positive component ej ⁺ and the negative component ej ⁻ of (−∂E / ∂Oj) in the above equation (63) are expressed by the following equations (64) and (65).

[0174]

[Equation 44]

The error signal plus component δj ⁺⁺ and the minus component δj of the hidden layer neuron in the case of excitatory coupling
⁺ -Is represented by the following equations (66) and (67) by the above equations (42), (63) to (65).

[0176]

[Equation 45]

Here, NETj ⁻ is the above-mentioned equation (12).
It is the sum of inhibitory properties in the middle-layer neurons obtained in.

When obtaining the update amount of excitatory coupling,
An equation corresponding to the above equation (43) is represented by the following equation (68).

[0179]

[Equation 46]

Therefore, the renewal amount of excitatory coupling plus the component Δ
Wji ⁺ and the update amount minus component ΔWji ⁻ are calculated by the following equation (6) from the equations (45), (66) and (67).
9) and (70).

[0181]

[Equation 47]

Next, the error signal plus component δj ^{− +} and the minus component δj ^{− of the} hidden layer neuron in the inhibitory coupling.
Is represented by the following equations (71) and (72) from equations (47), (63) to (65).

[0183]

[Equation 48]

Here, NETj ⁺ is the above-mentioned equation (11).
It is the total excitability in the middle-layer neurons obtained in.

Then, when obtaining the update amount of the inhibitory bond,
An equation corresponding to the above equation (44) is shown by the following equation (73).

[0186]

[Equation 49]

Therefore, the update amount of the inhibitory bond plus the component Δ
Wji ⁺ and the update amount minus component ΔWji ⁻ are calculated by the following equation (7) from the equations (48), (71), and (72).
4) and (75).

[0188]

[Equation 50]

By the way, the coupling coefficient is updated by the following equations (76) and (77) using the coupling coefficient pulse Wji before updating and the values of the plus component ΔWji ⁺ of the update amount and the minus component ΔWji ⁻ of the update amount. It is carried out as follows.

[0190]

(Equation 51)

Also in this case, similar to the first embodiment, for example, before the learning process, the updated coupling coefficient W is updated.
Reset the counter that stores ji, and set Wji∪ΔWj
When i ⁺ = 1 the increment count operation is performed,
This can be realized by performing a decrement counting operation when ΔWji ⁻ = 1 and executing a learning process for one frame.

Also, a concrete example for carrying out this method is
The circuit configuration is the circuit shown in FIG. 15 used in the first embodiment.
It is similar to the figure. Note that equations (69), (70), (7
In 4) and (75), the number of elements other than the learning coefficient η is 3
However, the above equations (69), (70), (7
Δj in 4) and (75)⁺⁺, Δj^+-, Δj^-+, Δj
^-Are equations (66), (67) and equations (71), (72)
Error plus component ej ⁺And error minus component
ej^-Since it includes the AND term of
The ND circuits 46 and 47 have four inputs in this embodiment as well.
It can be said that there is no problem in being shown.

(Fourth Embodiment) Next, another embodiment of the present invention will be described. In this embodiment, the method for updating the coupling coefficient described in the second embodiment is applied to updating the coupling coefficient between the input layer neuron i and the intermediate layer neuron j. In the second aspect, k and j are attached to the neurons, but this is for convenience only and does not mean that the relationship between the neurons i and j in this embodiment is not included.

[0194] Updating of the coupling coefficient, the coupling coefficient pulses Wkj before update, when updating of the positive component ΔWkj ⁺ = 1 "1" increases, the update amount of the negative component ΔWkj ^- = "1" reduce it when 1 I am going to do it. That is, the updated coupling coefficient is obtained by the following equations (78) and (79).

[0195]

[Equation 52]

Also in this case, as in the second embodiment, for example, the coupling coefficient W before update is set before the learning process.
When kj is loaded into the counter and the learning process is executed for one frame, “1” is increased in the counter when ΔWkj ⁺ = 1 and “1” is decreased in the counter when ΔWkj ⁻ = 1. It can be realized.

The specific circuit configuration for carrying out this method is the same as the circuit diagram shown in FIG. 16 used in the second embodiment.

[0198]

As described above, according to the present invention, since the learning algorithm is faithfully derived from the neuron output function, the learning ability is enhanced. In addition, since the learning process can be performed by logical sum or logical product for each signal, the circuit for implementing the coupling coefficient updating method can be simplified, and the logical calculation is performed in this way. Since it does not depend on calculation, an external microcomputer is unnecessary and the processing speed can be improved. Further, the addition / subtraction processing can use, for example, an up / down counter, and the sign bit of this up / down counter can be made to represent the sign of the coupling coefficient. The effect that it becomes possible to perform.

[Brief description of the drawings]

FIG. 1 is a schematic diagram of a nerve cell unit.

FIG. 2 is a schematic diagram of a neural network.

FIG. 3 is a graph showing a sigmoid function.

FIG. 4 is a schematic diagram illustrating a forward process.

FIG. 5 is a schematic diagram illustrating a learning process.

FIG. 6 is an electric circuit diagram corresponding to a neural network.

FIG. 7 is an electric circuit diagram corresponding to a single nerve cell.

FIG. 8 is a block diagram of a synapse circuit.

FIG. 9 is a block diagram of a cell body circuit.

FIG. 10 is a schematic diagram showing a state of forward processing of one neuron element in the pulse density method.

FIG. 11 is a block diagram showing a circuit of a portion corresponding to a synapse of a neuron.

FIG. 12 is a logic circuit diagram of a circuit that generates an error signal in an output layer.

FIG. 13 is a logic circuit diagram of a circuit that generates an error signal in the intermediate layer.

14A is a block diagram showing an η circuit, and FIG. 14B is an operation timing chart of the circuit.

FIG. 15 is a block diagram illustrating a circuit that implements a coupling coefficient updating method according to the first embodiment of this invention.

FIG. 16 is a block diagram showing a circuit that implements a coupling coefficient updating method according to a second embodiment of the present invention.

[Explanation of symbols]

 38 η circuit 41 up / down counter 42 coupling coefficient storage register 43 pulse generator 46 AND circuit 47 AND circuit 48 OR circuit 50 up / down counter

Claims (2)

[Claims] 1. A method for updating a synapse coupling coefficient representing a degree of increase or decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network by a pulse density method in which neuron mimicking elements are connected in a mesh. When updating the coupling coefficient Wkj between the neuron k and the neuron j, the error signal plus component in the neuron k is set as the first signal, the error signal minus component in the neuron k is set as the second signal, and the output Oj of the neuron j is set. Is taken as the third signal, all the logical products of the coupling coefficient Wkm with the neuron m connected to the neuron k excluding the neuron j and the output Om of the neuron m are obtained, and the logical sum of these is calculated. The inverted output is the fourth signal, and the logic values of the first signal, the inverted value of the second signal, the third signal, and the fourth signal The value obtained by applying the processing by the learning coefficient to the obtained coefficient is taken as the update amount of the coupling coefficient plus the component, and the inverted output of the first signal, the second signal, the third signal, and the fourth signal If the logical sum output of the above-mentioned update amount plus component and the combination coefficient pulse before update is “1”, the product of the logical product of The process of adding “1” is performed, the process of subtracting “1” is performed when the minus component pulse of the update amount is “1”, and the result value obtained by these addition / subtraction processes is used as a new coupling coefficient. A method for updating a coupling coefficient in a pulse density type signal processing network, wherein: 2. A method of updating a synapse coupling coefficient representing a degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network of a pulse density method in which neuron mimicking elements are connected in a mesh. When updating the coupling coefficient Wkj between the neuron k and the neuron j, the error signal plus component in the neuron k is set as the first signal, the error signal minus component in the neuron k is set as the second signal, and the output Oj of the neuron j is set. Is taken as the third signal, all the logical products of the coupling coefficient Wkm with the neuron m connected to the neuron k excluding the neuron j and the output Om of the neuron m are obtained, and the logical sum of these is calculated. The inverted output is the fourth signal, and the logic values of the first signal, the inverted value of the second signal, the third signal, and the fourth signal The value obtained by applying the processing by the learning coefficient to the obtained coefficient is taken as the update amount of the coupling coefficient plus the component, and the inverted output of the first signal, the second signal, the third signal, and the fourth signal The value obtained by subjecting the logical product of the above to the processing by the learning coefficient is used as the update amount minus component of the coupling coefficient, and when the above-mentioned update amount plus component pulse is "1", the combination coefficient value before update is "1". "" Is added, and when the update amount minus component pulse is "1", "1" is subtracted from the pre-update coupling coefficient value, and these pre-update coupling coefficient values are added. A method of updating a coupling coefficient in a pulse density type signal processing circuit, characterized in that a result value obtained by the addition / subtraction processing is used as a new coupling coefficient.