JPH096743A - Method and device for updating coupling coefficient for pulse density type signal processing circuit network - Google Patents

Abstract

PURPOSE: To provide a device for updating coupling coefficient for pulse density type signal processing circuit network with which learning processing with an arbitrary learning coefficient is enabled in a pulse density type neural network. CONSTITUTION: This device is provided with a storage means (η-Mem) 51 for holding a learning coefficient value η, random number generator (Ran-Gen) 50 for generating a random number value, comparator (CMP) 52 for generating a learning coefficient pulse having the learning coefficient value as pulse density by comparing the learning coefficient value with the random number value, and AND circuit 53 for ANDing the learning coefficient pulse and the error signal pulse sequence of a neuron.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a coupling coefficient updating method and a coupling coefficient updating device in a pulse density type signal processing circuit network applied to character and graphic recognition, motion control of a robot or the like, or associative memory.

[0002]

2. Description of the Related Art A so-called nerve cell circuit is aimed at parallel processing of information by configuring a nerve cell mimicking element that mimics the function of a nerve cell (neuron), which is a basic unit of information processing of a living body, in a network to process information in parallel. It is a network (neural network). Although character recognition, associative memory, motion control, etc. are performed in the living body in a very simple manner, many cannot be easily achieved by the conventional Neumann computer.

Therefore, attempts to solve these problems by imitating the functions of the nervous system of the living body, in particular, the functions peculiar to the living body, that is, parallel processing and self-learning, have been made by computer simulations and trial production of dedicated hardware. It is being actively conducted.

FIG. 1 shows a model (nerve cell unit) of one neuron, which is composed of a part for receiving an input from another neuron, a part for converting the input according to a certain rule, and a part for outputting a result. A variable weight "Wji" is attached to each of the connection parts with other neurons to represent the connection strength. Changing this value changes the network structure. Learning the network means changing this value.

FIG. 2 shows a case where this is used as a network to form a hierarchical neural network. In the figure,
A1, A2 and A3 represent neurons, respectively.
Each of the neurons A1, A2, A3 is connected to a large number of neurons like the neuron shown in the schematic view of FIG. 1, and processes and outputs the signals received from them. The hierarchical network is composed of an input layer, an intermediate layer, and an output layer, and there is no connection within each layer and no connection from the output layer to the input layer. One neuron is connected to many other neurons.

Although FIG. 2 shows a three-layer network including an input layer, an intermediate layer, and an output layer, a multi-layer network having a plurality of intermediate layers is also used.

The operation of each of the neurons A1, A2 and A3 shown in FIG. 2 will be described by taking the neuron shown in FIG. 1 as an example. First, the forward process will be described.

In the neural network of FIG.
The signal input to the input layer A1 propagates to the intermediate layer A2,
Forward processing is executed in the middle layer A2. Middle layer A2
The processing result of is propagated to the output layer A3, and at the output layer A3,
The forward process is executed, and the output of the neural network is finally obtained.

The degree of coupling between neurons is called a coupling coefficient, and the coupling coefficient between the j-th neuron and the i-th neuron is represented by Wji.
There are two types of coupling: excitatory coupling, in which the output of the partner neuron increases as the signal from the partner neuron increases, and inhibitory coupling, in which the output of the partner neuron decreases, the output decreases. Excitatory coupling, Wji
When <0, it represents an inhibitory bond.

Now, assuming that the own neuron is the j-th neuron and the output of the i-th neuron is Oi, this is multiplied by the coupling coefficient Wji, and Wji · Oi becomes the input to the own neuron. Since each neuron is connected to a large number of neurons, Wji ·
ΣWji · Oi, which is the result of adding Oi, becomes the input to the own neuron. This is called the internal potential,
It is shown by the following formula.

[0011]

[Equation 1] net _j = ΣWji · Oi (1)

Next, this input is subjected to non-linear processing and output. The function at this time is called a nerve cell response function, and for example, a sigmoid function represented by the following equation (2) is used.

[0013]

[Number 2] _{f (net j) = 1 /} {1 + exp (-net j)} ... (2)

This function is shown in FIG. Value range is "0 to 1"
Then, as the input value increases, it approaches “1”, and as it decreases, it approaches “0”. From the above, the output O of the neuron j
j is expressed by the following equation (3).

[0015]

## EQU3 ## Oj = f (net _j ) = f (ΣWji · Oi) (3)

Next, the learning function of the neural network will be described. As a learning process used in the numerical calculation, a general back propagation algorithm (hereinafter, abbreviated as BP algorithm) will be briefly described.

In the learning process, when a certain input pattern p is given, the coupling coefficient is changed so as to reduce the error between the actual output value and the desired output value. The BP algorithm is an algorithm for obtaining this change amount.

Now, given a certain input pattern p, the difference between the actual output value (Opk) of the unit k and the desired output value (tpk) is defined by the following equation (4).

[0019]

[Number 4] ^{Ep = (tpk-Opk) 2} /2 ... (4)

This represents the error of the unit k in the output layer,
tpk is teacher data given by a human. In learning, the strength of all connections is changed to reduce this error.
Actually, the change amount of Wkj when the pattern p is given is expressed by the following equation (5).

[0021]

[Equation 5] ΔpWkj∝−ЭE / ЭWkj (5)

Using this, the coupling coefficient Wkj is changed. As a result, the following equation (6) is obtained from the above equation (4).

[0023]

[Equation 6] ΔpWkj = η · δpk · Opj (6)

Here, Opj is an input value from the unit j to the unit k. The error δpk differs depending on whether the unit k is the output layer or the intermediate layer. First, the error signal δpk in the output layer is given by the following equation (7).

[0025]

## EQU00007 ## .delta.pk = (tpk-Opk) .f '(net _k ) (7)

On the other hand, the error signal δpj in the intermediate layer is as shown in the following equation (8).

[0027]

Δpj = f ′ (net _j ) · ΣδpkWkj (8)

However, f '(net _j ) is f (net _j )
The first-order differential, which will be described in detail later. From the above, ΔWji is generally formulated as the following equation (9).

[0029]

ΔWji (t + 1) = η · δpj · Opi + α · ΔWji (t) (9)

Therefore, it is as shown in the equation (10). Here, t is a learning order, η is a learning constant, and α is a stabilizing coefficient. The first term on the right side of the above equation (9) is added to ΔWji obtained by the above equation (6), and the second term is added to reduce the error vibration and accelerate the convergence.

[0031]

(10) Wji (t + 1) = Wji (t) + ΔWji (t + 1) (10)

As described above, the calculation of the variation ΔWji of the coupling coefficient starts from the unit of the output layer and proceeds to the unit of the intermediate layer. Learning proceeds in the opposite direction to the processing of the input data, that is, backward. Therefore, in learning by back propagation, first, learning data is input and the result is output (forward). Change the strength of all joins to reduce the resulting error (backward). Input the learning data again. This will be repeated until it converges.

The conventional hierarchical neural network is
A network as shown in FIG. 2 is formed. FIG. 4 shows the data flow of the forward process in this network. This is the case in a three-layer hierarchical network. In the forward process, input signals Oi1 to Oi4 are given to the input layer (the layer on the left side of FIG. 4), and the output signals Ok1 to Oi from the output layer (the layer on the right side of FIG. 4). Get Ok4.

On the other hand, FIG. 5 shows the data flow of the learning process. In the learning process, the teacher signals tk1 to tk4 are given to the output layer (the layer on the right side of FIG. 5) to update the coupling strength between the neurons and learn so that the output of the neural network matches the teacher signal. It should be noted that the processing by this learning process is
It is executed by an external general-purpose computer.

FIG. 6 shows an example in which the network is realized by an electric circuit (Japanese Patent Laid-Open No. 62-295188). This circuit basically comprises a plurality of amplifiers 4 having an S-shaped transfer function, and a resistive feedback network 2 for connecting the output of each amplifier 4 to the input of an amplifier of another layer as shown by a chain line. And are provided. A CR time constant circuit 3 including a grounded capacitor C and a grounded resistor R is individually connected to the input side of each amplifier 4. Then, the input current I _1, I ₂ ~I _N is supplied to the input of the amplifier 4, the output is obtained from a set of these output voltage of the amplifier 4.

Here, the strength of the input and output signals is represented by a voltage, and the strength of the nerve cell coupling is the resistance 1 (the lattice point in the resistive feedback network 2) connecting the input / output lines between the cells. , And the nerve cell response function is represented by the transfer function of each amplifier 4. Further, the connection between nerve cells has excitability and inhibitory property as described above, and is mathematically represented by the sign of the connection function. However, since it is difficult to realize positive / negative with a constant on the circuit, here, the output of the amplifier 4 is divided into two (4a, 4b), and one output is inverted to obtain two positive / negative signals. Is generated and is selected so that it is realized. Also,
An amplifier is used as the one corresponding to the sigmoid function f (net) shown in FIG.

In general, when the neural network is configured by analog circuits, the area of a single neural circuit element can be reduced, so that the degree of integration of the neural network can be increased or the execution speed is fast. However, on the other hand, the value of each signal is represented by an analog amount such as a potential or current, and each calculation is performed by an analog element such as an amplifier. Therefore, there are variations due to temperature characteristics, and there are variations in the process of forming elements. However, there is also a problem that the response characteristics of each element cannot be made the same and the output value becomes unstable.

It is also difficult to arbitrarily change the value of the coupling coefficient of the neural network by learning, and it is also possible to perform learning by an external computer and download the value of each coupling coefficient after learning to the hardware. Often done. In such a learning method, there are problems that a computer is required for learning and the learning process is significantly slowed down.

Next, an example in which a neural network is composed of digital circuits will be shown. 7 to 9 are diagrams showing an example in which the neural network is realized by a digital circuit, and FIG. 7 shows a circuit configuration example of a single nerve cell. In these figures, 11 is a synaptic circuit, 12 is a dendrite circuit, and 13 is a cell body circuit.

FIG. 8 is a diagram showing a configuration example of the synapse circuit 11 shown in FIG. 7, which is a value obtained by multiplying the input pulse f by a factor a (a factor of 1 or 2 by which the feedback signal is multiplied) via the coefficient circuit 11a. Input rate multiplier 1
1b is provided, and the rate multiplier 11b is connected to a synapse weight register 11c that stores a weighting value w. FIG. 9 is a diagram showing a configuration example of the cell body circuit 13, which includes a control circuit 14 and an up / down counter 1.
5, a rate multiplier 16 and a gate 17 are sequentially connected, and an up / down memory 18 is further provided.

In this case, the input and output of the nerve cell unit is represented by a pulse train, and the signal density is represented by the pulse density. The coupling coefficient is treated as a binary number and stored in the synapse weight register 11c.

The signal calculation process is performed as follows. First, the input signal is input to the rate multiplier 11b and the coupling coefficient is input to the rate value to reduce the pulse density of the input signal according to the rate value. This corresponds to the Wji · Oi part of the expression of the back propagation model described above. The Σ portion of ΣWji · Oi is realized by the OR circuit shown by the dendrite circuit 12. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance and a logical sum (OR) is taken for each group. In FIG. 7, F1 indicates excitatory output and F2 indicates inhibitory output. An output is obtained by inputting these two outputs to the up side and the down side of the counter 15 shown in FIG. 9 and counting respectively. Since this output is a binary number, it is converted into a pulse density again using the rate multiplier 16. A neural network can be realized by constructing a network using a plurality of these nerve cell units. The learning function is realized by inputting the final output of the network to an external computer, performing numerical calculation inside the computer, and writing the result to the synapse weight register 11c that stores the coupling coefficient.

As described above, if the neural network is configured by a digital circuit, the area of a single neural circuit element becomes larger than that of an analog circuit, so that the integration degree of the neural network cannot be increased. However, it is relatively easy to form a circuit without being affected by temperature characteristics and process variations in device formation, which is a drawback of, and the output of nerve cell devices (neurons) is stable and highly reliable. There are advantages.

However, in the examples shown in FIGS. 7 to 9, since learning of the neural network relies on an external computer, there is a problem that a learning computer is required and the learning time is long.

The present applicant has already proposed a signal processing device using a neural network composed of neurons (for example, see Japanese Patent Application No. 5-118087). In the present invention, the signal processing device according to this prior application is treated as an example of one embodiment, and therefore the signal processing device according to this prior application will be described below.

In the signal processing device according to this prior application,
As an example of the neural network, we have proposed a neural cell unit using a digital logic circuit and signal processing by a network circuit configured by using it.

Here, the basic idea is as follows. 1. Input / output signals, intermediate signals in the nerve cell unit,
All the teacher signals are represented by a pulse train represented by binary values of "0" and "1". 2. The value of the signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). 3. The calculation in the nerve cell unit is performed by a logical calculation between pulse trains. 4. The pulse train of the coupling coefficient is stored in the memory in the nerve cell unit. 5. In learning, an error is calculated based on a given teacher signal pulse train, and the coupling coefficient is changed based on this error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1".

FIG. 10 shows a state of forward processing of one neuron element in the pulse density method, and considers the hierarchical neural network shown in FIG. 2 as the network configuration.

First, a logical product (AND) of the input coefficient Oi, which is binarized into "0" and "1" and expressed in pulse density, and the coupling coefficient Wji is obtained for each synapse. This corresponds to Wji · Oi in the equation (1). The pulse density of the output of the AND circuit stochastically becomes the product of the pulse density of the input signal and the pulse density of the coupling coefficient.

As described above, the connections between neurons include excitatory connections and inhibitory connections. In the case of numerical calculation, the sign of the coupling coefficient, for example, plus when excitatory, minus when inhibitory is performed.

In the case of the pulse density method, the coupling coefficient Wji
Depending on the positive or negative of each, each connection is classified into an excitatory connection and an inhibitory connection.
It is divided into two groups, and the logical sum is calculated by OR operation for each group. This corresponds to the processing of Σ in Equation 3 and the processing of the nonlinear saturation function f (net).

That is, in the calculation based on the pulse density,
When the pulse density is low, the pulse density resulting from the OR processing can be approximated to the sum of the pulse densities of the OR input.

Since the output of the OR circuit gradually saturates as the pulse density increases, the result does not match the sum of the pulse densities, and non-linearity appears.

In the case of this OR operation, the value P of the pulse density is 0≤P≤1, and since it is a monotonically increasing function with respect to the magnitude of the input, processing by the equation (2) or the sigmoid function of FIG. 3 is performed. Will be similar to.

The output of the nerve cell element by the pulse density method is the OR of the excitability groups obtained by the above calculation.
And the output net Non _j ⁺ is "1", OR output net Non _j inhibitory group ^- outputs "1" only when "0". That is,
It is expressed as shown in the following equations (11) to (13).

[0056]

[Equation 11]

Next, the learning process in the pulse density method will be described. In a non-learned neural network, the output of the network when a certain pattern is input is not always the desired output. Therefore, similar to the BP algorithm described above, the learning process changes the coupling coefficient so as to reduce the error between the actual output value and the desired output value.

(Error Signal in Output Layer) First, the error signal in the output layer will be described. Here, when the error is expressed by a numerical value, both positive and negative values can be taken, but since such an expression cannot be performed in the pulse density method, two signals, a signal δk ⁺ representing a plus component and a signal δk ⁻ representing a minus component, can be obtained. Using the error signal in the output layer, the following equations (14) and (1
Define as in 5).

[0059]

(Equation 12)

The error signal plus component δk ⁺ is the output result O
When k is "0" and the teacher signal tk is "1", it becomes "1", and otherwise it becomes "0".

[0061] On the other hand, the error signal minus component .delta.k ^- the output result Ok is "1", when the teacher signal tk is "0",
It becomes "1", and otherwise it becomes "0".

The error signals δk ⁺ and δk ⁻ are the above-mentioned B
In the P algorithm, the difference (t _pk − between the teacher signal of the above equation (7) for obtaining the error signal of the output layer and the actual output signal
O _pk ).

Next, as shown in equation (7), these error signals and the output function f (net), which is the first derivative f ′,
The product of (net) and the error signal in the output layer is obtained. Generally, a differential coefficient obtained by differentiating an output signal with an internal potential is required for calculating an error signal during learning. In the BP algorithm described above, the output function f (net) is
When a sigmoid function is used, its first derivative f ′ (ne
t) is shown by Formula (16).

[0064]

F ′ (net) = df (net) / dnet = f (net) · {1-f (net)} (16)

In the pulse density method, referring to the equation (16), the first-order differential f '(net) in the output layer neuron is used.
plus component f '(net) k ⁺ and minus component f of k
′ (Net) k ⁻ is defined as in equations (17) and (18).

[0066]

[Equation 14]

Here, Ok (t-1) is the output signal Ok.
Is a 1-pulse delay value, and Ok (t−2) is a 2-pulse delay value of the output signal Ok. Therefore, the final error signal in the output layer is expressed by the following equations (19) and (20). This corresponds to the equation (7) for obtaining the error signal of the output layer in the above-mentioned BP algorithm.

[0068]

(Equation 15)

(Error Signal in Intermediate Layer) The error signal in the intermediate layer based on the pulse density method is also obtained by referring to the above equation (8) based on the BP algorithm. That is, the error signals in the output layer are collected and used as the error signal in the intermediate layer. Here, the connection is divided into two groups depending on whether it is excitatory or inhibitory, the product part is represented by ∩ (AND), and the sum (Σ) part is represented by ∪ (OR).

Further, when obtaining the error signal in the intermediate layer, the coupling coefficient Wkj is positive or negative and the error signal δk is positive or negative.
There are two cases. First, in the case of excitatory coupling, an error signal plus component .delta.k output layer ^+, calculated for that which took AND coupling coefficient (δk ⁺ ∩Wkj ⁺⁾ neurons of all output layer, taking these OR . This becomes the error signal plus component δj ⁺ of the hidden layer neuron (equation (21)).

[0071]

[Number 16] ^{δj + = ∪ (δk + ∩Wkj} +) ... (21)

Similarly, the error signal of the output layer minus the component δ
AND of k ⁻ and its coupling coefficient (δk ⁻ ∩
Wkj ⁺ ) is obtained for neurons in all output layers, and these are ORed. This becomes the error signal minus component of the hidden layer neuron (equation (22)).

[0073]

[Number 17] ^{δj - = ∪ (δk - ∩Wkj} +) ... (22)

Next, the case of inhibitory binding will be described.
Error signal minus component .delta.k output layer ^- and that taking the AND between the coupling coefficient (δk ^- ∩Wkj ^-) asking for neurons of all output layer, taking these OR. This is the error signal plus component of the intermediate layer (equation (2
3)).

[0075]

[Number 18] ^{δj + = ∪ (δk - ∩Wkj} -) ... (23)

Similarly, the error signal plus component δk of the output layer
AND of ⁺ and its coupling coefficient (δk ⁺ ∩W
kj ^-) a request for the neurons of all of the output layer,
These are ORed together. This becomes the error signal minus component of the hidden layer neuron.

[0077]

[Number 19] ^{δj - = ∪ (δk + ∩Wkj} -) ... (24)

The connection between a neuron in a certain middle layer and the neuron in the output layer connected to the middle layer includes excitatory connection and inhibitory connection. Therefore, as the error signal plus component of the intermediate layer, δj ⁺ of the excitatory coupling of the equation (21) and the equation (23)
OR the δj ⁺ of the inhibitory couplings of Similarly, the error signal minus component of the intermediate layer, excitatory coupling .delta.j of formula (22) ^- ORed with the ^- .delta.j inhibitory binding of the formula (24). That is, the following equations (25) and (26) are obtained. This is the Σ of the above equation (8) according to the BP algorithm.
Corresponds to δpkWkj.

[0079]

Equation 20] ^{δj + = {∪ (δk +} ∩Wkj +)} ∪ {∪ (δk - ∩Wkj -)} ... (25) δj - = {∪ (δk - ∩Wkj +)} ∪ {∪ (δk ⁺ ∩Wkj ^- )}… (26)

[0080] Then, similarly to the BP algorithm of formula (8), equation (25), .delta.j obtained in (26) ^+, .delta.j ^- in formula (17), the derivative of (18) (f'(net Non )) Is applied. Here, the positive component f ′ (net) j ⁺ and the negative component f ′ (net) j ⁻ of the first-order differential f ′ (net) j in the intermediate layer neuron are (27) and (28).
Is defined as Here, Oj (t-1) is a one-pulse delay value of the output signal Oj of the hidden layer neuron.

Therefore, the error signal (δj in the intermediate layer is
⁺ , Δj ⁻ ) is calculated by the following equations (29) and (30).

[0082]

(Equation 21)

[0083]

(Equation 22)

(Processing by Learning Constant η) Processing of the learning constant η in the above equation (6) for obtaining the correction amount ΔW of the coupling coefficient in the BP algorithm will be described. In the numerical calculation, the learning constant η may be simply multiplied as shown in the equation (6), but in the case of the pulse density method, the pulse train is thinned out as shown below according to the value of the learning constant η. It will be realized.

[0085]

(Equation 23)

Next, a method for obtaining the correction amount ΔW of the coupling coefficient by learning will be described. First, the error signal (δ +, δ−) in the output layer or the intermediate layer is processed by the learning constant η, and the logical product with the input signal to the neuron is taken (δη∩O). However, since there are δ ⁺ and δ ⁻ in the error signal, ΔW ⁺ , ΔW ⁺ ,
ΔW ^- to.

[0087]

[Number 24] ^{ΔW + = δη + ∩O ... (} 34) ΔW - = δη - ∩O ... (35)

This corresponds to the above equation (6) for obtaining ΔW in the BP algorithm. A new coupling coefficient New_W is obtained based on these, and cases are classified depending on whether the coupling coefficient W is excitatory or inhibitory. First, in the case of excitability, the component of ΔW ⁺ is increased with respect to the original W ⁺ , and ΔW ^+
- reduce the component of. That is, formula (36) is obtained.

[0089]

(Equation 25)

Next, in the case of the suppressive property, the component of ΔW ⁺ is decreased and the component of ΔW ⁻ is increased with respect to the original W ⁻ . That is, equation (37) is obtained.

[0091]

(Equation 26)

The learning algorithm based on the pulse density method has been described above. Here, the processing flow of the forward process and the learning process in the pulse density method in the hierarchical network of FIG. 2 will be briefly described. First,
Although it is a forward process, when a signal is first given to the input layer, this input signal propagates to the intermediate layer, and as the signal processing of the intermediate layer, the above equations (11) to (13) are performed, and the result is To the output layer.

In the output layer, the processes of equations (11) to (13) are similarly performed on these propagated signals,
These results in an output signal, ending the forward process.

In the learning process, after performing the above forward process, a teacher signal is further given to the output layer. In the output layer, the error signal in the output layer is obtained by the equations (19) and (20) and sent to the intermediate layer. At the same time, the error signal is processed by the learning constant η of the equations (31) to (33), and the logical product of the error signal and the input signal from the intermediate layer is obtained by the equations (34) and (35). , (37), the coupling strength between the output layer and the intermediate layer is changed.

Next, as the processing in the intermediate layer, the error in the intermediate layer is obtained by the equations (29) and (30) based on the error signal sent from the output layer, and the error signal is given by the equation (3).
Processing with the learning constant η of 1) to (33) is performed, and the expression (3
4) and (35), the logical product with the input signal from the input layer is obtained, and then the coupling strength between the intermediate layer and the input layer is changed by the equations (36) and (37), and the learning process ends. After that, the learning process is repeated until it converges.

Next, an actual circuit configuration based on the above algorithm will be described with reference to FIGS. 11 to 13. The structure of the neural network is the same as in FIG. FIG.
FIG. 1 is a diagram showing a circuit of a portion corresponding to a synapse of a neuron, and FIG. 12 is a diagram showing a circuit for obtaining an error signal in the output layer from the output of the neuron cell body of the output layer, the output of the output layer, and the teacher signal. . Further, FIG. 13 is a diagram showing a circuit of a portion for collecting error signals in the cell body of neurons in the intermediate layer and the output layer and obtaining an error signal in the intermediate layer. By forming a network of these three circuits as shown in FIG. 2, a digital neural network circuit capable of self-learning can be realized.

First, FIG. 11 will be described. The synapse coupling coefficient is stored in the shift register 21. The terminal 21A is a data outlet and the terminal 21B is a data inlet. Other components such as a RAM and an address generator may be used as long as they have the same function as the shift register. Shift register 2
The circuit 22 including 1 is a circuit that executes (Oi∩Wji) in the above equations (11) and (12), and ANDs the input signal and the coupling coefficient. This output must be grouped according to whether the connection is excitatory or inhibitory,
It is more versatile to prepare the outputs O ⁺ and O ⁻ for each group in advance and switch which group to output to. Therefore, a bit indicating whether the coupling is excitatory or inhibitory is stored in the memory 23, and the information is used to switch the signal by the switching gate circuit 24. Further, as shown in FIGS. 12 and 13, a gate circuit 31 having a plurality of OR gates corresponding to the logical sum of the equations (11) and (12) for processing each input is provided. Further, as shown in the figure, a gate circuit 32 formed by an AND gate and an inverter that outputs only when the excitatory group represented by the equation (13) is "1" and the inhibitory group is "0".
Is provided.

Next, the error signal will be described. FIG.
The circuits 35 and 36 in FIG. 2 are circuits showing a circuit for generating a differential coefficient of an output function in the output layer, and are logical circuits by a combination of AND (logical product) and an inverter, and are expressed by the equations (17) and (18). Equivalent to. And the circuit 37
In the figure showing the circuit that generates the error signal in the output layer,
(Logical product), a logical circuit by a combination of inverters, which corresponds to the equations (19) and (20). That is, the error signals δk ⁺ , δ in the output layer are obtained by the output Ok from the output layer, the teacher signal tk, and the differential component plus and minus components generated by the differential coefficient generation circuits 35 and 36.
Generate k ⁻ . Further, the above equations (21) to (24) for obtaining the error signal in the intermediate layer are AN shown in FIG.
This is performed by the gate circuit 26 having the D gate configuration, and an output (f∩W) according to the plus component and the minus component is obtained.
Since the error signal to be used is different depending on whether the coupling is excitatory or inhibitory, it is necessary to make a distinction in that case. In this case, the excitatory or inhibitory information stored in the memory 23 and the error are stored. AND, O depending on the δ ⁺ , δ ⁻ signal of the signal
This is performed by the gate circuit 25 having an R gate configuration. Also,
The arithmetic expressions (25) and (26) for collecting the error signals are performed by the gate circuit 39 having the OR gate structure shown in FIG. A circuit that executes the equations (29) and (30) for obtaining the error signals δj ⁺ and δj ⁻ in the intermediate layer is a signal Oj (t in which the intermediate output Oj is delayed by the circuit 40 in FIG. 13 and the intermediate layer output Oj is delayed by the register 33. −1) and the error signal from the output layer, which is the output of the OR circuit 39, generate error signals δj ⁺ and δj ⁻ . Expressions (31) to (33) corresponding to the learning rate are η shown in FIG.
It is performed by the circuit 38.

A T-FF (trigger type flip-flop) 41 in FIG. 14A inverts the output Q when it detects the rising edge of the input T. Further, the multiplexer (MUX) 42, as shown in FIG.
0 selects one of the input signals A, B, and C and outputs it. FIG. 11B is a timing chart showing the operation of the learning coefficient η circuit shown in FIG. When the reset signal (RES) is input first, the T-FF
The output of 41 is set to the initialization state, and the multiplexer 42
The value of the learning coefficient η is set by the selection signals S1 and S0.

When the error signal δ is input, as the input signal A of the multiplexer 42, a pulse subjected to the processing of the learning coefficient η = 1/4 (processing to leave every third pulse) is generated (Equation 31). (Corresponding to Eq. 32), a pulse with a learning coefficient η = 1/2 processing (leaving every other pulse) is generated as the input signal B (corresponding to Expression 32).
As a result, a pulse subjected to processing with the learning coefficient η = 1 (processing for making the pulse the same as the original signal) is generated (Equation 33).
Equivalent to).

Then, the selection signal S of the multiplexer 42
1, S0 outputs one of the input signals A, B, C as a pulse δη. In addition, FIG.
This shows a main circuit that executes processing by the learning coefficient η, and in actuality, a signal noise prevention circuit and the like are added.

Finally, the part for calculating a new coupling coefficient from the error signal will be described. This is the above equation (34).
~ (37), which is performed by the gate circuit 27 having the AND, inverter, and OR gate configurations shown in FIG.
This gate circuit 27 must also be classified depending on the excitability / inhibition of the coupling, which is performed by the gate circuit 25 shown in FIG.

When the neural network processing is executed by numerical calculation, as described above, the processing by the learning coefficient η is simply the value of the learning coefficient η as shown in the equation (6). It is realized by multiplying by.
However, in the coupling coefficient updating method in the learning processing of the pulse density neural network of the prior application, the processing by the learning coefficient η uses the circuit of FIG.
As shown in (33), it is performed by thinning out the pulses, and since this thinning-out operation is realized by a logical operation between the pulses, it is difficult to realize the processing with an arbitrary learning coefficient η value.

That is, in the configuration of FIG. 14 (a) of the prior application, the learning coefficient η can be selected from only three values of 1, 1/2 and 1/4, and the processing by the arbitrary learning coefficient η cannot be realized. . In the learning processing of the neural network, it is necessary to finely adjust the value of the learning coefficient η depending on the learning problem, but if this fine adjustment cannot be made, it is trapped in a local minimum (local solution) on the error curved surface, or The wall around the global minimum causes vibration, and learning does not converge.

In view of the above situation, the present invention provides an arbitrary learning coefficient η in the pulse density neural network.
It is an object of the present invention to provide a coupling coefficient updating method and apparatus for a pulse density type signal processing circuit network which enables learning processing by the method.

[0105]

The first method of the present invention is as follows.
In a method of updating a synapse coupling coefficient indicating a degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network using a pulse density method in which neuron mimetic elements are connected in a mesh, a learning coefficient value and a disturbance A learning coefficient pulse having a learning coefficient value as a pulse density is generated by comparing with a numerical value, and a signal obtained by ANDing the learning coefficient pulse and the error signal pulse train of the neuron is learned as an error signal in the neuron. It is characterized in that the signal processed by the coefficient is used and the coupling coefficient is updated.

The second method of the present invention is the same as the first method, except that two random number values are generated and the error signal plus component in the neuron is processed by the learning coefficient based on one of the random number values. The generated signal is generated, the error signal minus component in the neuron is processed by the learning coefficient based on the other random value, and the coupling coefficient is updated.

The third method of the present invention represents the degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of the signal processing circuit network of the pulse density method in which neuron mimicking elements are connected in a mesh. In the method of updating the synaptic coupling coefficient, an error signal pulse for one frame in the neuron is converted into a binary value, a multiplication value of the binary value error signal and a learning coefficient value is obtained, and the multiplication value and a random value are obtained. By comparing, a pulse train signal having a value of the product of the learning coefficient and the error signal in the pulse density is generated, and this pulse train signal is used as a signal obtained by processing the error signal in the neuron with the learning coefficient. It is characterized by updating.

The fourth method of the present invention represents the degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of the signal processing circuit network of the pulse density method in which neuron mimicking elements are connected in a mesh. In the method of updating the synapse coupling coefficient, the error signal plus component in the neuron is an increment (or decrement) signal of a counter that counts the error signal for one frame,
An error signal minus component in the neuron is used as a decrement (or increment) signal of the counter that counts an erroneous calculation signal for one frame, and after counting for one frame, a multiplication value of this counter value and a learning coefficient value is calculated. Then, a pulse train signal having a product value of the learning coefficient and the error signal as a pulse density is generated by comparing this multiplication value with a random number value, and when the value of the counter is positive (or negative), the pulse train The signal is a signal obtained by processing the error signal plus component in the neuron with the learning coefficient, and when the value of the counter is negative (or positive), the pulse train signal is processed into the error signal minus component in the neuron with the learning coefficient. The given signal,
The feature is that the coupling coefficient is updated.

Further, the first device of the present invention represents the degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of the signal processing circuit network of the pulse density method in which neuron mimicking elements are connected in a mesh. In the device that updates the synapse coupling coefficient, a storage unit that holds the learning coefficient value,
A random number generator for generating a random number value, a comparing means for generating a learning coefficient pulse having a learning coefficient value as a pulse density by comparing the learning coefficient value and the random number value, an error between the learning coefficient pulse and the neuron And a logical product circuit for obtaining a logical product with the signal pulse train.

The second device of the present invention represents the degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of the signal processing circuit network of the pulse density method in which neuron mimicking elements are connected in a mesh. In the device that updates the synapse coupling coefficient, a storage unit that holds the learning coefficient value,
A first random number generator for generating a first random number value, a second random number generator for generating a second random number value, the learning coefficient value, and a first
First comparing means for generating a learning coefficient pulse having a learning coefficient value as a pulse density by comparing with a random number value of
Second learning means for generating a learning coefficient pulse having a learning coefficient value as a pulse density by comparing the learning coefficient value with a second random value; and the learning coefficient pulse from the first comparing means. A first AND circuit for obtaining a logical product of the error signal plus component pulse train of the neuron, and a second AND circuit for obtaining the logical product of the learning coefficient pulse from the second comparing means and the error signal minus component pulse train of the neuron. And a logical product circuit.

The third device of the present invention represents the degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of the signal processing circuit network of the pulse density method in which neuron mimicking elements are connected in a mesh. In an apparatus for updating synapse coupling coefficient, a random number generator for generating a random value, a binary value conversion means for converting an error signal pulse for one frame in the neuron into a binary value, and a storage means for holding a learning coefficient value. , A multiplication means for obtaining a multiplication value of the binary value error signal from the binary value conversion means and the learning coefficient value from the storage means, and a multiplication value from the multiplication means and a random number value from the random number generator. Thus, a comparison means for generating a pulse train signal having a product value of a learning coefficient and an error signal as a pulse density is provided.

The fourth device of the present invention represents the degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of the signal processing circuit network of the pulse density method in which neuron mimicking elements are connected in a mesh. In the device that updates the synapse coupling coefficient, a storage unit that holds the learning coefficient value,
A random number generator that generates a random number value, an error signal plus component in the neuron is an increment (or decrement) signal, and an error signal minus component in the neuron is a decrement (or increment) signal,
A counter for counting an erroneous calculation signal for one frame; and a multiplication means for obtaining a multiplication value of the count value for one frame by the counter and the learning coefficient value from the storage means,
Comparing means for generating a pulse train signal having a pulse density of a product value of a learning coefficient and an error signal by comparing a multiplication value from the multiplying means and a random number value from the random number generator, and a value of the counter. Is positive (or negative), the pulse train signal is output as a signal obtained by processing the error signal plus component in the neuron with a learning coefficient,
When the counter value is negative (or positive), the pulse train signal is output as a signal obtained by processing the minus component of the error signal in the neuron with a learning coefficient.

[0113]

According to the first method and apparatus described above, the learning coefficient pulse having the learning coefficient value in the pulse density is generated by comparing the learning coefficient value with the random number value. Therefore, any value can be set as the learning coefficient value. By setting the error signal in the neuron, a signal obtained by processing the error signal with an arbitrary learning coefficient value can be generated to update the coupling coefficient.

According to the second method and apparatus, the error coefficient plus component and the error signal minus component in the neuron are processed with arbitrary learning coefficient values to generate a signal, and the coupling coefficient is updated. be able to.

According to the third method and apparatus, in the method of converting the error signal pulse for one frame in the neuron into the binary value, an arbitrary value is set as the learning coefficient value and the error signal in the neuron is set. The coupling coefficient can be updated by generating a signal that has been processed with an arbitrary learning coefficient value.

According to the fourth method and apparatus described above, in the method of converting the error signal pulse for one frame in the neuron into the binary value, the error signal plus component and the error signal minus component in the neuron are arbitrarily set. The coupling coefficient can be updated by generating a signal processed by the learning coefficient value.

[0117]

Embodiments of the present invention will be described below. In order to solve the above-mentioned problems in the prior art, the present invention improves the η circuit 38 shown in FIG. 14A so that an arbitrary value can be selected as the learning coefficient η. .

(Embodiment 1) FIG. 15 shows an η circuit in the first embodiment.

The storage means (η-Mem) 51 stores the value of any learning coefficient η in binary data (binary number display data).
Memorize in the form of. Of course, the storage unit 51 is a writable memory and can rewrite the value of the learning coefficient η.

The random number generator (Ran-Gen) 50 is adapted to generate a random number value within a predetermined range.

The comparator (CMP) 52 compares the random number value generated by the random number generator 50 with an arbitrary learning coefficient η read from the storage means 51, and the random number value is less than or equal to the learning coefficient η. Sometimes it produces a pulse. This allows
A pulse train (η pulse) having a learning coefficient η in pulse density is generated.

The AND circuit 53 inputs the η pulse and the error signal (error signal input pulse) δ and takes the logical product (AND) of both. The signal obtained by taking the logical product is the error signal δη obtained by processing the error signal δ with the learning coefficient η, that is, the value (η × δ) of the product of the miscalculation signal δ and the learning coefficient η and the pulse density Pulse train δη. In addition, in FIG. 15, the single line indicates the propagation of the pulse train,
Double lines indicate the propagation of binary data.

The pulse train δη is input to the circuit shown in FIG. 11 to update the coupling coefficient, and the updated value is stored in the register 22.

According to the above configuration, since the pulse train (η pulse) having the learning coefficient η in the pulse density is generated by comparing the learning coefficient η with the random number value, any value can be set as the learning coefficient η. By setting the error signal δ in the neuron, a signal obtained by processing the error signal δ with an arbitrary learning coefficient η can be generated to update the coupling coefficient.

(Embodiment 2) FIG. 16 shows an η circuit in the second embodiment. For convenience of explanation, members having the same functions as those in the first embodiment are designated by the same reference numerals.

The η circuit of this embodiment stores the learning coefficient η, a storage means 51, a first random number generator (Ran-Gen1) 50a for generating a first random number value, and a second random number value. Second random number generator (Ran-Gen2) 50b
And a first comparator 52a for generating a learning coefficient pulse having a learning coefficient value in pulse density by comparing the learning coefficient η with a first random value, the learning coefficient η and a second random value. A second comparator 52b for generating a learning coefficient pulse having a learning coefficient value as a pulse density by comparing with the learning coefficient pulse (η pulse 1) from the first comparator 52a and the error signal plus component of the neuron. The first AND circuit 53a for obtaining the logical product of (pulse train) δ ⁺ , the learning coefficient pulse (η pulse 2) from the second comparator 52b, and the error signal minus component (pulse train) δ ⁻ of the neuron. A second AND circuit 53b for obtaining a logical product.

In order to change the random number sequence of the first and second random number generators 50a and 50b, for example, in the case of well-known M-series random numbers, different values are added to the error signal plus the initial value and the generator polynomial. It may be given as a component (MODE1) and a minus component (MODE2).

According to the above configuration, for each of the error signal plus component δ ⁺ and the error signal minus component δ ⁻ ,
By the same method as in the first embodiment, the error signal δη ⁺ and the error signal δη ⁻ processed by the learning coefficient η can be obtained.

Here, as shown in FIGS. 12 and 13, the error signal generated in the cell body circuit of the neuron is the error signal plus component δ ⁺ and the error signal minus component δ.
^- because there is a, in performing these processes by learning coefficient eta, the error signal plus component [delta] ⁺ and the error signal minus component [delta] ^- so there is no possible correlation, it is necessary to improve the randomness of the pulse train, With the configuration of this embodiment, such a request can be realized.

Further, according to the configuration of this embodiment, the learning coefficient η is shared by the error signal plus component generation and the error signal minus component generation while realizing the above requirement, and one storage means 51 is provided. Since the configuration is adopted, the circuit can be simplified. Furthermore, random number generators 50a and 50b, a comparator,
The AND circuit and the AND circuit can be used in common, and the cost can be reduced by sharing the circuit components.

(Third Embodiment) The η circuit of the present embodiment is configured so that the η circuit of the first and second embodiments processes the error signal (pulse) δ for each pulse. They are different in that they are configured to perform each process (pipeline process).

The η circuit in this embodiment is shown in FIG.
For convenience of explanation, members having the same functions as those in the first embodiment are designated by the same reference numerals.

The storage means (η-Mem) 51 stores the value of any learning coefficient η in binary data (binary number display data).
In the form of a random number generator (Ran-Gen) 50,
A random number value is generated within a predetermined range.

Counter (binary value conversion means) 57
Inputs an error signal (pulse train) δ and counts for one frame.

The multiplier (multiplier) 56 finds the product of the count value (binary value error signal) from the counter 57 and the learning coefficient η from the memory 51, and outputs this to the comparator 52.

The comparator (CMP) 52 compares the multiplication value from the multiplier 56 with the random number value from the random number generator 50 to determine the product value of the learning coefficient η and the error signal as the pulse density. Generate a pulse train signal having.

In the above arrangement, the counter 57
As described above, the error signal (pulse train) δ is input and counting is performed for one frame. By this, the error signal (pulse train) δ for one frame in the neuron is converted into a binary value. Become. Therefore, the multiplier 56 receives the error signal pulse number and the random number value, both of which are binary values, and obtains the product of them. The comparator 52 compares the product from the multiplier 56 with the random number value from the random number generator 50, and outputs a pulse when the random number value is less than or equal to the product value. This allows
A pulse train having a pulse density having the value of the above product is generated. That is, a pulse train δη obtained by processing the error signal δ with the learning coefficient η is obtained.

As a result, in the method of converting the error signal pulse for one frame in the neuron into the binary value, an arbitrary value is set as the learning coefficient η, and the error signal δ in the neuron is processed by the arbitrary learning coefficient η. It is possible to update the coupling coefficient by generating a signal subjected to.

(Embodiment 4) In the η circuit of this embodiment, the η circuit of the first and second embodiments is configured to process the error signal (pulse) δ for each pulse, whereas It is common to the third embodiment in that it is configured to perform each process (pipeline process), and the learning coefficient η for each of the error signal plus component δ ⁺ and the error signal minus component δ ^−. Error signal δη processed by
^This is common to the second embodiment in that ⁺ and the error signal δη ⁻ are obtained.

The η circuit in this embodiment is shown in FIG.
For convenience of explanation, members having the same functions as those in the third embodiment are designated by the same reference numerals.

The storage means (η-Mem) 51 stores the value of any learning coefficient η in binary data (binary number display data).
In the form of a random number generator (Ran-Gen) 50,
A random number value is generated within a predetermined range.

The up / down counter 58 uses the error signal plus component δ ⁺ in the neuron as an increment signal, and the error signal minus component δ in the neuron.
^- a decrement signal, counts the miscalculation signals for one frame. Note that, due to the configurations of the circuit 37 of FIG. 12 and the circuit 40 of FIG. 13, which are error generation circuits, both the error signal plus component δ ⁺ and the minus component δ ⁻ have a pulse (both have a “1” level). ), The positive component δ ⁺ serves as an increment signal and the negative component δ ⁻ serves as a decrement signal.
There is no problem in operation even if input to.

The multiplier (multiplier) 56 finds the product of the count value for one frame by the up / down counter 58 and the learning coefficient η from the memory 51.

The comparator (comparing means) 52 compares the multiplication value from the multiplier 56 with the random number value from the random number generator 50 to obtain the product value of the learning coefficient η and the error signal δ as the pulse density. To generate a pulse train signal.

The selection circuit (OUTSEL) 59 inputs the positive / negative signal (SGN) from the up / down counter 58, and when the value of the up / down counter 58 is positive, the pulse train signal output from the comparator 52. the output as signal .DELTA..eta ⁺ subjected to processing by the learning coefficient η in the error signal plus component [delta] ⁺ in the neurons, and when the value of the counter 58 is negative, the error signal minus component [delta] said pulse train signal in said neurons ^- the The signal is output as a signal δη ⁻ processed by the learning coefficient η.

According to the above arrangement, the up / down counter 58 increments by the error signal plus component δ ^{+ and} decrements by the error signal minus component δ ^{− for} one frame, and the total value of the error inputs is multiplied by the multiplier 56. To the counter value plus / minus signal (S
GN) to the selection circuit 59. Then, the multiplier 56 multiplies the binary error signal from the up / down counter 58 by the learning coefficient η from the storage means 51, and outputs the product value to the comparator 52. The comparator 52 outputs a pulse when the random number value from the random number generator 50 is smaller than the multiplication value output from the multiplier 56, and outputs the pulse to the selection circuit 59. The selection circuit 59 outputs the input from the comparator 52 as an A output when the positive / negative signal of the up / down counter 58 is positive, that is, an error signal plus component δη ⁺ , and the positive / negative signal of the up / down counter 58 is negative. In this case, B output, that is, the error signal minus component δη ⁻ is output.

As a result, in the method of converting the error signal pulse for one frame in the neuron into the binary value, the error signal plus component and the error signal minus component in the neuron are processed with arbitrary learning coefficient values. A signal can be generated to update the coupling coefficient. Further, since the minus component δ ⁻ is added to the plus component δ ⁺ of the error signal and the error signal output can be obtained by the total value thereof, stable learning ability with less variation can be obtained.

[0148]

As described above, according to the present invention, an arbitrary learning coefficient can be set, so that the learning coefficient can be finely adjusted at the time of learning the neural network.
The ability to reduce traps to the local minimum and vibration between the walls around the global minimum on the error curved surface is improved, the convergence rate of learning is increased, and the learning ability can be improved. In addition, it is possible to obtain a stable learning ability with little variation.

[Brief description of drawings]

FIG. 1 is a schematic diagram of a nerve cell unit.

FIG. 2 is a schematic diagram of a neural network.

FIG. 3 is a graph showing a sigmoid function.

FIG. 4 is a schematic diagram illustrating a forward process.

FIG. 5 is a schematic diagram illustrating a learning process.

FIG. 6 is an electric circuit diagram corresponding to a neural network.

FIG. 7 is an electric circuit diagram corresponding to a single nerve cell.

FIG. 8 is a block diagram of a synapse circuit.

FIG. 9 is a block diagram of a cell body circuit.

FIG. 10 is a schematic diagram showing a state of forward processing of one neuron element in the pulse density method.

FIG. 11 is a block diagram showing a circuit of a portion corresponding to a synapse of a neuron.

FIG. 12 is a logic circuit diagram of a circuit that generates an error signal in an output layer.

FIG. 13 is a logic circuit diagram of a circuit that generates an error signal in the intermediate layer.

14A is a block diagram showing a conventional η circuit, and FIG. 14B is an operation timing chart of the circuit.

FIG. 15 is a block diagram showing an η circuit according to the first embodiment of the present invention.

FIG. 16 is a block diagram showing an η circuit according to a second embodiment of the present invention.

FIG. 17 is a block diagram showing an η circuit according to a third embodiment of the present invention.

FIG. 18 is a block diagram showing an η circuit according to a fourth embodiment of the present invention.

[Explanation of symbols]

 38 η circuit 50 random number generator 50a first random number generator 50b second random number generator 51 storage means 52 comparator 52a first comparator 52b second comparator 53 AND circuit 53a first AND circuit 53b second AND circuit Circuit 56 Multiplier 57 Counter 58 Up-down counter 59 Selection circuit

Claims (8)

[Claims] 1. A method for updating a synapse coupling coefficient representing a degree of increase or decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network by a pulse density method in which neuron mimicking elements are connected in a mesh. A learning coefficient pulse having a learning coefficient value as a pulse density is generated by comparing the learning coefficient value and a random value, and a signal obtained by performing a logical product of the learning coefficient pulse and the error signal pulse train of the neuron is output to the neuron. A method for updating a coupling coefficient in a pulse density type signal processing network, characterized in that the error signal is processed by a learning coefficient and the coupling coefficient is updated. 2. Two random number values are generated, an error signal plus component in the neuron is processed by a learning coefficient based on one of the random number values, and a signal is generated based on the other random number value in the neuron. 2. The method for updating a coupling coefficient in a pulse density type signal processing circuit network according to claim 1, wherein a signal obtained by processing the error signal minus component with a learning coefficient is generated and the coupling coefficient is updated. 3. A method of updating a synapse coupling coefficient, which represents a degree of increase or decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network based on a pulse density method in which neuron mimetic elements are connected in a mesh. The error signal pulse for one frame in the neuron is converted into a binary value, the multiplication value of the binary value error signal and the learning coefficient value is obtained, and the multiplication value and the random number value are compared to obtain the learning coefficient and the error. A pulse characterized by generating a pulse train signal having a product of the signal and the pulse density as a pulse density, and using this pulse train signal as a signal obtained by processing an error signal in the neuron with a learning coefficient, and updating the coupling coefficient. A method for updating coupling coefficient in a density type signal processing network. 4. A method of updating a synapse coupling coefficient representing a degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit by a pulse density method in which neuron mimicking elements are connected in a mesh. The error signal plus component in the neuron is used as the increment (or decrement) signal of the counter that counts the error signal for one frame, and the error signal minus component in the neuron is decremented by the counter that counts the miscalculation signal for one frame. (Or increment) signal, after counting for one frame, the multiplication value of the value of this counter and the learning coefficient value is obtained, and the multiplication value and the random number value are compared to determine the product of the learning coefficient and the error signal. A pulse train signal having a pulse density equal to the value of When the value of the counter is negative (or positive), the pulse train signal is the error signal in the neuron. A coupling coefficient updating method in a pulse density type signal processing circuit network, wherein a signal obtained by processing a negative component with a learning coefficient is used to update the coupling coefficient. 5. A device for updating a synapse coupling coefficient representing a degree of increase or decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network of a pulse density method in which neuron mimicking elements are connected in a mesh. Storage means for holding a learning coefficient value, a random number generator for generating a random number value, and comparison means for generating a learning coefficient pulse having a learning coefficient value as a pulse density by comparing the learning coefficient value and the random number value. A coupling coefficient updating device in a pulse density type signal processing circuit network, comprising: a logical product circuit for obtaining a logical product of the learning coefficient pulse and an error signal pulse train of the neuron. 6. A device for updating a synapse coupling coefficient, which represents a degree of increase or decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network of a pulse density method in which neuron mimicking elements are connected in a mesh, Storage means for holding the learning coefficient value, and a first for generating a first random value
A random number generator, a second random number generator that generates a second random number value, and a learning coefficient pulse having a learning coefficient value as a pulse density are generated by comparing the learning coefficient value and the first random number value. A first comparing means, a second comparing means for generating a learning coefficient pulse having a learning coefficient value as a pulse density by comparing the learning coefficient value with a second random value, and the first comparing means. Of the learning coefficient pulse from the second comparison means and the learning signal pulse from the second comparing means and the error signal minus component pulse train of the neuron. A coupling coefficient updating device in a pulse density type signal processing circuit network, comprising: a second AND circuit for obtaining a logical product. 7. A device for updating a synapse coupling coefficient representing a degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network by a pulse density method in which neuron mimicking elements are connected in a mesh. A random number generator for generating a random number value, a binary value conversion means for converting an error signal pulse for one frame in the neuron into a binary value, and a storage means for holding a learning coefficient value,
Multiplication means for obtaining a multiplication value of the binary value error signal from the binary value conversion means and the learning coefficient value from the storage means, and comparing the multiplication value from the multiplication means with the random number value from the random number generator. According to another aspect of the present invention, there is provided a coupling coefficient updating device in a pulse density type signal processing circuit network, comprising: a comparison means for generating a pulse train signal having a pulse density of a product value of a learning coefficient and an error signal. 8. A device for updating a synapse coupling coefficient representing a degree of increase / decrease of a signal when a signal from another neuron is propagated to the neuron of a signal processing circuit network of a pulse density method in which neuron mimicking elements are connected in a mesh. A storage unit that holds the learning coefficient value, a random number generator that generates a random number value, an error signal plus component in the neuron is an increment (or decrement) signal, and an error signal minus component in the neuron is a decrement (or increment) signal. A counter for counting miscalculation signals for one frame, multiplication means for obtaining a multiplication value of the count value for one frame by the counter and the learning coefficient value from the storage means, and a multiplication value from the multiplication means. By comparing the random number value from the random number generator with the learning coefficient and the error signal When the value of the comparing means for generating a pulse train signal having a product value of the pulse density as a pulse density and the counter is positive (or negative), the pulse train signal is processed by the learning coefficient as an error signal plus component in the neuron. Selecting means for outputting the pulse train signal as a signal obtained by processing the minus component of the error signal in the neuron with a learning coefficient when the counter value is negative (or positive). Coupling coefficient updating apparatus for pulse density type signal processing network.