JPH05108597A - Signal processor - Google Patents

Abstract

PURPOSE:To improve an arithmetic effect by weakening the correlation of output signals between the connection of neurons by realizing digital configuration so as to secure operations and to facilitate turning to a hardware. CONSTITUTION:This device is equipped with a memory to hold respective coupling coefficients to plural input signals expressing pulse density, ANDing means to AND the input signals and the coupling coefficients of pulse density expressions, grouping means to divide the calculated result of this ANDing means into an excited coupling group and a suppressed coupling group, ORing means 33a, 33b to OR the resoective groups, and logical calculating means 34 to obtain the output signal of pulse density expression by logically calculating the ORed results of these ORing means 33a, 33b and further, a signal output means 38 is provided with a counting means 35 to count the pulse of the output signal and a transforming means 36 to transform the counted result into the random pulse density expression again.

Description

Detailed Description of the Invention

[0001]

INDUSTRIAL APPLICABILITY The present invention is applicable to control of various movements such as recognition of images and voices, position control of robots, temperature control of air conditioners, orbit control of rockets, etc. The present invention relates to a signal processing device such as a neural computer.

[0002]

2. Description of the Related Art The function of a nerve cell (neuron), which is a basic unit of information processing of a living body, is mimicked, and further, this "nerve cell mimicking element" (nerve cell unit) is connected to a network to process information in parallel. The aim is a so-called neural network. Although it is easy to perform character recognition, associative memory, motion control, etc. in a living body, there are many things that conventional Neumann computers cannot easily achieve. There have been many attempts to solve these problems by imitating a neural system of a living body, in particular, a function peculiar to the living body, that is, parallel processing, self-learning, etc. by a neural network.

First, a conventional neural network model will be described. FIG. 23 is a diagram showing a certain nerve cell unit A, and FIG. 24 is a network thereof. A ₁ , A ₂ , and A ₃ each represent a nerve cell unit. One neuronal cell unit is coupled to a number of other neuronal cell units and processes the signals received from them to produce an output. In the case of FIG. 24, the network is hierarchical, and the nerve cell unit A ₂ receives a signal from the nerve cell unit A _{1 in the} previous layer (left side) and the nerve cell unit A in the next layer (right side). Output to ₃ .

A more detailed description will be given. First, in the nerve cell unit A of FIG. 23, the degree of coupling between other nerve cell units and its own unit is called a coupling coefficient, and the coupling coefficient of the i-th nerve cell unit and the j-th nerve cell unit. The coupling coefficient is generally represented by T _ij . For the coupling, excitatory coupling in which the larger the signal from the other unit (the unit that sends a signal to the own unit) is, the larger the own unit output is, and the larger the signal from the other unit is, the own unit output Is an inhibitory bond, where T _ij > 0 represents an excitatory bond, and T _ij <0 represents an inhibitory bond. Now, let's say that our nerve cell unit is the jth unit, and the output of the ith nerve cell unit is y.
_{If it is i} , T _ij y _{i obtained} by multiplying this by the coupling coefficient T _ij becomes the input to the own unit. As described above, since one nerve cell unit is connected to many nerve cell units, ΣT _ij y _{i, which} is the result of adding T _ij y _i for these units, is It becomes an input to the unit. This is called the internal potential and is represented by u _j .

[0005]

[Equation 1]

Next, a threshold value is added to this input (internal potential) to perform non-linear processing, and the result is output from the nerve cell unit. The function used at this time is called a nerve cell response function, and the sigmoid function as shown in equation (2) and FIG. 25 is used as the nonlinear function.

[0007]

[Equation 2]

When such a nerve cell unit is constructed in a network as shown in FIG. 23, each coupling coefficient T
By providing _ij and sequentially calculating equations (1) and (2), parallel processing of information becomes possible and a final output is obtained.

In such a hierarchical neural network, a desired neural network is constructed by performing learning such that the coupling coefficient T _ij is updated so that a desired result is output for a certain input. .. The most widely used such learning method is the error back-propagation method, so-called back-propagation method.

An example of such a network realized by an electric circuit is shown in FIG. This is disclosed in Japanese Patent Application Laid-Open No. 62-295188, and basically, a plurality of amplifiers 1 having an S-shaped transfer function, and the output of each amplifier 1 is input to the input of the amplifier of another layer. A resistive feedback network 2 is provided which is connected as shown by the dashed line. The input side of each amplifier 1 has a C connected by a grounded capacitor and a grounded resistor.
The R time constant circuits 3 are individually connected. Then, the input current _{I 1, I 2, ~,} I n is supplied to the input of the amplifier 1, the output is obtained from a set of these amplifiers 1 output voltage.

Here, the signal strength of the input and output to the network is represented by a voltage, and the strength of the coupling between the nerve cell units is the resistance 4 (in the resistive feedback network 2) connecting the input / output lines between the cells. Is represented by the resistance value of each of the amplifiers 1, and the nerve cell response function is represented by the transfer function of each amplifier 1. That is, in FIG. 25, a plurality of amplifiers 1 have an inverting output and a non-inverting output, and the input of each amplifier 1 has a CR time constant circuit 3 as an input current supply means. The feedback network 2 connects each output of the amplifier 3 to the input by a resistor 4 (T _ij ) having a value of 1 or a second value selected in advance. Resistor 4 represents the transconductance between the i-th amplifier output and the j-th amplifier input and is selected to create a plurality of minima that the network balances, minimizing the energy function with a plurality of minima. I am trying to. Also, the coupling between nerve cells has excitability and inhibitory property, and is mathematically represented by the positive / negative sign of the coupling coefficient, but it is difficult to realize positive / negative with a constant on the circuit. Then, by dividing the output of the amplifier 1 into two and inverting one output, positive and negative 2
It is realized by generating two signals and selecting them appropriately. An amplifier is used as the one corresponding to the sigmoid function shown in FIG.

On the other hand, an example in which a neural network is realized by a digital circuit will be described with reference to FIGS. 27 to 29. FIG. 27 shows a circuit configuration of a single nerve cell, in which each synapse circuit 6 is connected to a cell body circuit 8 via a dendrite circuit 7. FIG. 28 shows a constitutional example of the synapse circuit 6 in which the input pulse f is multiplied by a factor a through the coefficient circuit 9.
A rate multiplier 10 to which a value obtained by multiplying (a multiplication factor of 1 or 2 by the feedback signal) is input is provided, and the rate multiplier 10 is connected to a synapse weight register 11 that stores a weighting value w. .. Further, FIG. 29 shows a configuration example of the cell body circuit 8, and the control circuit 1
2, an up / down counter 13, a rate multiplier 14 and a gate 15 are sequentially connected, and an up / down memory 16 is further provided.

In this case, the input and output of the nerve cell unit is represented by a pulse train, and the pulse density represents the amount of signal.
The coupling coefficient is represented by a binary number and stored in the memory 16. By inputting the input signal into the clock of the rate multiplier 14 and inputting the coupling coefficient into the rate value,
The pulse density of the input signal is reduced according to the rate value. This is the T _{ij of the} backpropagation model equation.
It corresponds to the part of y _i . Next, the Σ portion of ΣT _ij y _i is realized by the OR circuit shown by the dendrite circuit 7. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance and OR is taken for each group. An output is obtained by inputting these two outputs to the up side and down side of the counter 13 and counting. This output is 2
Since it is a decimal number, the rate multiplier 14 is used again to convert it into a pulse density. A neural network can be realized by making this unit a network.

[0014]

However, the former analog circuit system has the following problems. The signal strength is represented by an analog value such as potential or current,
When the internal calculation is also performed in an analog manner, its value changes due to temperature characteristics, drift immediately after power-on, and the like. Since it is a network, a large number of elements are required, but it is difficult to make the respective characteristics uniform. When the accuracy or stability of one element becomes a problem, it may cause a new problem when it is used as a network, and the behavior when viewed in the whole network cannot be predicted. Since the coupling coefficient T _ij is fixed and the value learned in advance by another method such as simulation is used, self-learning cannot be performed.

Also, in the latter case of the network configuration of digital circuits, the up / down counter 1 is actually used.
3. Use of the rate multiplier 14 results in a very complicated and large-scale circuit.

As described above, according to the conventional technique, the analog circuit system has no certainty in operation, and the learning method by numerical calculation is also complicated in calculation. The circuit configuration is large and complicated.

In order to solve such a drawback, a digital neuron model with a self-learning function is disclosed in Japanese Patent Application No.
No. 4,12,448 and Japanese Patent Application No. 3-29342 have been proposed by the present applicant. However, in order to perform the calculation in each neuron more effectively, it is better that the signals between the connections of each neuron have no correlation. For that reason, the randomness of the transfer pulse train signal in the forward process is improved. There is a problem in terms of.

[0018]

According to a first aspect of the present invention, a memory that holds a coupling coefficient for each of a plurality of input signals represented by a pulse density, and the memory represented by the input signal and the pulse density. A logical product calculating means for calculating a logical product with the coupling coefficient, a grouping means for dividing the calculation result by the logical product calculating means into an excitatory coupling group and an inhibitory coupling group, and a logical sum in each group. The logical OR operation means for performing the operation, the logical operation means for performing the logical operation of the operation results of these OR operation means to obtain the output signal expressed by the pulse density, and the input of the plurality of error signals expressed by the pulse density A new coupling coefficient is calculated based on the coupling coefficient changing means for changing the coupling coefficient on the memory, a counting means for counting the pulses of the output signal, and a counting result. It provided a signal processing means and a signal output means by the conversion means for converting the pulse density representation.

In place of the signal output means described in claim 1, in the invention described in claim 2, the counting means for counting the pulses of the error signal and the converting means for converting the counting result into the pulse density expression again. An error signal output means is provided, and in the invention of claim 3, the counting means for counting the pulses of the positive and negative error signals and the difference in the counting result between the positive and negative error signals are converted into a pulse density expression. The error signal output means is provided by the conversion means for
Counting means for counting the pulses of the positive and negative error signals respectively, conversion means for converting the comparison result of the counting results of the pulses of the positive and negative error signals and the counting result, and the converted signals The error signal output means is provided by the operation means for performing the logical operation of.

According to the invention of claim 5, a plurality of signal processing means are connected in a net form, and counting means for counting the pulses of the output signal of the output layer and the teacher signal respectively, and the difference between these counting results. An error signal output means including a conversion means for converting to a pulse density expression and outputting a positive or negative error signal to the front layer is provided, and also in the invention according to claim 6, a plurality of signal processing means are connected in a net form and output. Counting means for respectively counting the pulses of the layer output signal and the teacher signal, converting means for converting the comparison result of these counting results and each counting result into a pulse density expression, and logical operation of the converted signals The error signal output means is provided by a calculation means for outputting a positive or negative error signal to the front layer.

On the other hand, in the invention described in claim 7, the error signal selecting means for calculating the logical sum of each of the positive and negative error signals and discarding at least one of the error signals is provided.

[0022]

When outputting the calculation result or the error signal in the signal processing means to the other signal processing means, it is more effective to perform the calculation and learning in the signal processing means if the pulse trains having the pulse density expressions do not have correlation. According to the invention described in claims 1 to 6, the output signal, the error signal, and the positive and negative error signals are once counted, converted into a random pulse train, and output. Therefore, this effect can be secured.

According to the seventh aspect of the invention, at least one of the positive and negative error signals collected by taking the logical sum of the error signals is discarded, and the positive and negative error signals are discarded. Since they are not output at the same time, the calculation and learning effects in the signal processing means are improved.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to FIGS. First, a nerve cell unit (neuron element) and a neural network using a digital logic circuit having a self-learning function will be described.

First, the neurons and neural networks of this embodiment are digitalized and implemented as hardware. The basic idea is to input / output signals, intermediate signals, and
The coupling coefficient, the teacher signal, etc. are all represented by a binary pulse train of "0" and "1". The amount of signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed in the memory. Learning is realized by rewriting this pulse train. For learning, the error is calculated based on the given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1". It was done like this.

The idea will be described below. First,
Regarding signal processing by a digital logic circuit, signal processing in the forward process will be described. FIG. 2 shows a portion corresponding to one nerve cell unit (nerve cell mimicking element) 20, and the neural network as a whole is of a hierarchical type as shown in FIG. Input and output are all
The one that is binarized into “1” and “0” and synchronized is used. The value (intensity) of the input signal y _i is represented by a pulse density, and is represented by the number of states of “1” within a certain time as in the pulse train shown in FIG. 4, for example. That is, the example of FIG.
4/6, and the signal "1" is 4 in 6 sync pulses.
There are two "0" s. At this time, it is desirable that the arrangement of "1" and "0" is random.

On the other hand, the coupling coefficient T _ij indicating the degree of coupling between the nerve cell units 20 is similarly expressed by pulse density,
The pulse train of "0" and "1" is prepared in advance in the memory. The example of FIG. 5 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of "1" and "0" is random.

Then, the coupling coefficient pulse train is sequentially read from the memory in response to the synchronous clock, and the AND gate 21 takes the logical product with the input signal pulse train as shown in FIG. 2 (y _i ∩ T _ij ). This is used as an input to the nerve cell j. In the case of the above example, the input signal is “10
When input as "1101", a pulse train is called from the memory in synchronism with this, and "101000" as shown in FIG. 6 is obtained by sequentially performing AND, which means that the input y _i is the coupling coefficient T _ij. It is shown that the pulse density becomes 2/6 by the conversion.

The pulse density of the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This is because the longer the signal train is, the more "1"
The more random the arrangement of "0" and "0", the closer to the product of numerical values it has. When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no data to be read, the head of the data may be returned to and the reading may be repeated.

Since one nerve cell unit 20 has multiple inputs, there are many "ANDs of the input signal and the coupling coefficient" described above, and the OR circuit 22 then takes the logical sum of these. Since the inputs are synchronized, for example, the first data is "101000" and the second data is "01000".
In the case of “0”, the OR of both is “111000”. If you calculate and output multiple inputs (m) at the same time,
For example, as shown in FIG. This corresponds to the sum calculation and the non-linear function (sigmoid function) in the analog calculation.

When the pulse density is low, the pulse density of the OR of the pulse density is approximately equal to the sum of the pulse densities. As the pulse density increases, the OR circuit 22
Since the output of is gradually saturated, it does not match the sum of pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, although the connection has excitability and inhibitory property, it is assumed that the present embodiment can have both of them in connection between neurons. First, depending on whether the coupling coefficient is the excitatory coupling coefficient T _{ij (+)} or the inhibitory coupling coefficient T _{ij (-)} , each coupling is divided into an excitatory coupling group and an inhibitory coupling group.
Share Then, A of the pulse train of the input signal and each coupling coefficient
The OR of ND outputs is calculated for each group. And
"1" is output only when the OR result of the excitatory coupling group is "1" and the OR result of the inhibitory coupling group is "0", and "0" is output otherwise.

When expressed by a logical expression, it is expressed by the following expressions (3) to (5).

[0034]

[Equation 3]

The network of nerve cell units 20 is
Hierarchical type similar to backpropagation (ie, Figure 3)
And If the entire network is synchronized, each layer can be calculated by the functions described above.

As will be described in detail later, when a plurality of such nerve cell units 20 are connected, it is better that the pulse signals between the respective connections have no correlation with each other in the nerve cell units 20. The calculation becomes effective.

Next, the signal calculation processing in learning (back propagation) will be described. Basically, the error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method of c.

First, the error signal in the final layer will be described as a. In the final layer, the error signal in each nerve cell unit is calculated by the output signal and the teacher signal. Here, a desired output with respect to the input at that time is given by a pulse train or a pulse number as a teacher signal. Generally, when the error is expressed by a numerical value, both positive and negative can be taken, but since it cannot be expressed simultaneously by the pulse density, an error signal is obtained by using two kinds of a signal representing a + component and a signal representing a − component. Express. That is, the error signal of the j-th nerve cell unit counts the output signal y _j and the teacher signal d _j for one data (it is not necessary to count when the teacher signal d _j is given by the pulse number). The output is based on the size comparison. That is, when the count value of the teacher signal d _j is larger, only the error signal + (δ _{j (+)} ) produces a pulse by the difference. Further, when the count value of the output signal is larger, only the error signal − (δ _{j (−)} ) outputs a pulse by the difference. It is desirable that these error signals are also random. The coupling coefficient is changed based on such an error signal pulse as described later.

Next, a method for obtaining an error signal in the intermediate layer as b will be described. First, the above-mentioned error signal is back-propagated to change not only the coupling coefficient between the final layer and the layer immediately before it, but also the coupling coefficient of the layer before that. Therefore, it is necessary to calculate the error signal in each nerve cell unit in the middle layer. Signals were propagated from the neuron unit with an intermediate layer to each neuron unit in the next layer, which is exactly the reverse of the procedure of collecting error signals in each neuron unit in the next layer. And use it as its own error signal. This can be performed in the same manner as the above-described arithmetic expressions (3) to (5) and the cases shown in FIGS. 4 to 8 in the nerve cell unit. However, the difference from the above-mentioned processing in the nerve cell unit is that y is one signal, while δ has two signals as positive and negative signals, and both signals are considered. It is necessary. Therefore, it is necessary to divide into two cases depending on whether the coupling coefficient T is positive or negative.

First, the case of excitatory coupling will be described. In this case, for a nerve cell unit having an intermediate layer, the error signal + at the jth nerve cell unit of the layer immediately before (the last layer in FIG. 3), the nerve cell unit and self (intermediate layer in FIG. 3) For each nerve cell unit, an AND (δ _{j (+)} ∩ T _ij ) of the coupling coefficient with a certain nerve cell unit) is obtained, and the OR between them is taken {∪ (δ _{j (+)} ∩ T _ij )}. This is the error signal + of this nerve cell unit. That is, it becomes as shown in FIG.

Similarly, the error signal of the nerve cell unit of the previous layer is ANDed with the coupling coefficient, and the OR of these is taken to obtain the error signal of this nerve cell unit. That is, it becomes as shown in FIG.

Next, the case of inhibitory binding will be described. In this case, the AND of the error signal in the nerve cell unit of the layer one layer ahead and the coupling coefficient of the nerve cell unit and the self is taken, and further the OR between them is taken. This is the error signal + of this nerve cell unit. That is, it becomes as shown in FIG.

Further, by ANDing the error signal + which is one ahead and the coupling coefficient and further ORing them, the error signal − of this nerve cell unit is similarly obtained. That is, it becomes as shown in FIG.

Further, the error signal + of the excitatory coupling and the error signal + of the inhibitory coupling of the nerve cell unit are ORed, and the error signal − of the excitatory coupling − and the error signal − of the inhibitory coupling − are obtained. Is ORed. Then, the error signal δ _{i (+)} of this unit is set to “1” only when the former OR result is “1” and the latter OR result is “0”. On the contrary, the error signal δ _{i (-)} of this unit is set to "1" only when the former OR result is "0" and the latter OR result is "1". This processing corresponds to the invention described in claim 7, and at least one of the OR results of the positive and negative error signals is discarded. In cases other than the above, both may be discarded.
Such error signal selection processing weakens the correlation between the positive and negative error signals.

The above is summarized as shown in equation (6).

[0046]

[Equation 4]

Further, the error signal processing a in the final layer a
As in the case of, the number of pulses of each error signal δ _{i (+)} , δ _{i (-)} is counted, and when the count value of the error signal δ _{i (+)} is larger, the difference is δ _{i (} Alternatively, it may be output as a ₊₎ pulse, and conversely, when the count value of the error signal Δ _{i (−)} is larger, the difference may be output as a Δ _{i (−)} pulse.

Further, the same or different learning rates (learning constants) may be provided for the input error signals. This can be realized by thinning out the pulse train. For example, by using a counter-like concept, FIG. 13 and FIG.
It may be as shown in. In this example, when the learning rate η = 0.5, every other pulse train of the original signal is thinned out, but even if the pulses of the original signal are not evenly spaced, it is possible to thin out the original pulse train. 13 and 14,
When η = 0.5, every other pulse is thinned out, and η =
In the case of 0.33, every other pulse is left and η = 0.6.
In the case of 7, every other pulse is thinned once.

In this manner, the learning rate function is provided by thinning out the error signal. Such thinning out of the error signal can be easily realized by logically operating the output of a counter commercially available or using a flip-flop. In particular, when a counter is used, the value of the learning constant η can be set arbitrarily and easily, so that the characteristics of the network can be controlled.

Further, as c, a method of changing each coupling coefficient by such an error signal will be described. The output y _i from the immediately preceding neuron unit and the error signal δ _{j (+)} or δ _{j (} -of the own neuron unit corresponding to the line to which the coupling coefficient to be changed belongs (see FIG. 3) ₎ And (δ _j ∩ y _i ) (see FIGS. 15 and 16).
The two signals thus obtained are respectively expressed by ΔT _{ij (+)} ,
Let ΔT _{ij (-)} .

Then, based on this ΔT _ij , a new T
_ij is obtained. Since this T _ij is an absolute value component, it is classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the component of ΔT _{ij (+)} is increased with respect to the original T _ij , and ΔT
Reduce the components of _{ij (-)} . That is, it becomes as shown in FIG. On the contrary, in the case of the suppressive property, the component of ΔT _{ij (+)} is reduced and the component of ΔT _{ij (−)} is increased with respect to the original T _ij . That is, FIG.
As shown in 8.

The above is summarized as shown in equation (7).

[0053]

[Equation 5]

The network is calculated based on the above learning rule.

Next, an actual circuit configuration based on the above algorithm will be described. FIG. 1 and FIGS. 19 to 22 show examples of the circuit configuration. The configuration of the entire network is shown in FIG.
Is the same as. 19 shows a circuit of a portion corresponding to a line (connection) in FIG. 3, and FIG. 1 shows a circuit of a portion corresponding to a circle (each nerve cell unit 20) in FIG. A digital neural network capable of self-learning can be realized by making the circuits of the configurations of FIGS. 19 and 1 into a network as shown in FIG.

First, FIG. 19 will be described. In the figure, 25 is an input signal to the nerve cell unit and corresponds to FIG. The value of the coupling coefficient as shown in FIG. 5 is stored in the shift register 26 as a memory. The shift register 26 has an outlet 26a and an inlet 26b, but it may have the same function as a normal shift register, and may be, for example, a combination of a RAM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are A as an AND operation means.
The AND is performed by the logic circuit 28 including the ND gate 27 and performing the processing shown in FIG. The output of this logic circuit 28 must be grouped according to whether the coupling is excitatory or inhibitory.
It is more versatile to prepare 9 and 30 and switch which one of them is output. For this reason, in this embodiment, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31, and the information is used for switching by the switching gate circuit 32 serving as a grouping unit. The switching gate circuit 32 includes two AND gates 32.
a and 32b and an inverter 32c interposed between one input
And consists of.

Further, as shown in FIG. 1, each input process (see FIG.
The gate circuits 33a and 33b serving as a logical sum operation means are provided in a plurality of OR gate configurations that correspond to (1). Further, as shown in the figure, the AND gate 34a and the inverter 3 which output "1" only when the excitatory coupling group shown in FIG. 8 is "1" and the inhibitory coupling group is "0".
There is provided a gate circuit 34 which serves as a logical operation means by 4b.

When outputting the calculation result in the nerve cell unit to the next nerve cell unit, it is necessary to convert the output signal from the gate circuit 34 into a random pulse train again in order to reduce the correlation of each pulse train. Therefore, in this embodiment, the output signal from the gate circuit 34 is set to 1
The data 35 is counted by a counter 35 as counting means, and the counting result is sent to a random pulse train generator 36 as converting means, from which a new pulse train corresponding to the number of pulses is output as an output signal 37. The random pulse train generator 36 may be a general random pulse generator composed of a random number generator and a comparator. The counter 35 and the random pulse train generator 36 constitute the signal output means 38 of the present invention.

Next, the learning process will be described. First,
The error signals 41 and 42 must be generated from the output signal 39 from the output layer and the teacher signal 40. FIG. 20 shows the structure of the generation circuit corresponding to the invention described in claim 5. The pulse of the output signal 39 and the pulse of the teacher signal 40 are respectively counted by the counters 43 and 44 serving as counting means, the difference between these counting results is calculated by the subtractor 45, and the random number as the converting means is calculated according to the difference. The error signal 41 or 42 converted into the pulse density representation is output by the pulse train generation device 46 or 47. That is, as a result of the subtraction processing by the subtractor 45, when the teacher signal 40 is larger, the error signal 41 becomes a pulse train signal and the error signal 42 becomes “0” for one data. On the contrary, when the output signal 39 is larger, the error signal 42 becomes a pulse train signal and the error signal 41 becomes “0” for one data. In this way, the error signal output means 48 is constructed.

The subtractor 45 may be omitted, and the configuration shown in FIG. 21 may be used. FIG. 21 shows an error signal output means 49 corresponding to the sixth aspect of the invention. That is, the pulses of the output signal 39 and the pulses of the teacher signal 40 are counted by the counters 43 and 44, respectively, and the results are converted into pulse trains again by the random pulse train generators 46 and 47, respectively. At the same time, the comparator 50 compares the output signal 39 and the teacher signal 40 with each other. The output of the comparator 50 and the random pulse train generator 46,
The output 47 is processed by a gate circuit 51 composed of an AND gate and an inverter to output an error signal 41 or 42.
Is output. In the comparison of the comparator 50, the error signal pulse becomes "1" only when the pulse having the larger value is "1" and the pulse having the smaller value is "0". Specifically, when the teacher signal 40 is larger, the error signal 41 becomes a pulse train signal for one data and becomes “0” on the error signal 42 side. On the contrary, when the output signal 39 is larger, the error signal 42 becomes a pulse train signal for one data, and becomes “0” on the error signal 41 side.

Then, in the circuit shown in FIG.
1, 42 are input. These error signals 41 and 42 are collected by a plurality of OR gate-configured gate circuits 52 (processing of equation (8)) and output as error signals 53 and 54. These error signals 53 and 54 are A shown in FIG.
A gate circuit 55 composed of an ND gate and an inverter, which serves as an error signal selecting means, and a gate circuit 56 composed of an AND gate, perform the processing shown in FIGS. The error signals 57 and 58 to be output to the nerve cell unit of the previous layer are obtained according to + and −.

As described above, it is necessary to classify the connection depending on whether the connection is an excitatory connection or an inhibitory connection. In this case, information indicating whether the connection is excitatory or inhibitory is stored in the memory 31, and
Depending on the + and-signals 53 and 54 of the error signal, AND,
This is performed by the gate circuit 59 having an OR gate structure.

Further, processing corresponding to the learning rate (see FIG. 1)
3 and the processing of FIG. 14) are performed by the frequency dividing circuit 60 shown in FIG. Further, a coupling coefficient variable circuit 61 having an AND gate, an inverter, and an OR gate for performing the processing shown in FIGS. 15 to 18 for calculating a new coupling coefficient based on such error signals 53 and 54 is provided. And is connected to the data inlet 26b side of each shift register 26. That is, the contents of the shift register 26 used in the forward process are read again, and at the same time, the output from the previous nerve cell unit is also stored in the shift register 26 by transmitting it as a pulse signal using the counter value. The coupling coefficient is updated and rewritten. In the case of this gate circuit 61 as well, it is necessary to separate the cases depending on the excitability and the restraint of coupling, but this is performed by the gate circuit 59.

As with the forward process, it is necessary to convert the error signal into a random pulse train again. Therefore, as shown in FIG. 19, the error signal output from the gate circuit 56 for one data is counted by the counter 6 as a counting means.
2 and 63 are counted, and the counting results are converted back into pulse trains by the random pulse generators 64 and 65 which serve as conversion means to generate error signals 57 and 58. The random pulse generators 64 and 65 in this case may be general random pulse generators including a random number generator and a comparator. The counters 62 and 63 and the random pulse generators 64 and 65 constitute error signal output means 66 corresponding to the second aspect of the invention.

The error signal output means 66 shown in FIG. 19 may be replaced with the error signal output means 48 shown in FIG. 20 or the error signal output means 49 shown in FIG. In either case, the output signal 39 and the teacher signal 40 shown in FIGS.
4 (actually, the error signal that has passed through the gate circuit 56) may be input. The former case corresponds to the invention described in claim 3, and the latter case corresponds to the invention described in claim 4.

The circuit shown in FIG. 19 may be replaced as shown in FIG. Instead of the gate circuit 55, this is the error signal output means 48 shown in FIG.
Alternatively, 49 is used. According to this, during one data, the error signal always has a value of either + or −.

In these processes described above, the coupling coefficient is held in the shift register 26 in a binary number state, converted into a pulse train at the time of calculation, and after the calculation, it is counted again by the counter and the value is set to 2 You may make it hold | maintain as a base number.

It should be noted that the present invention is not limited to the above-mentioned configuration examples as long as it has equivalent functions.
Further, some or all of them may be realized by software without configuring all by hardware.

[0069]

As described above, according to the present invention, a pulse train to be transmitted such as an output signal to another signal processing means or an error signal for self-learning is expressed in a pulse density. Since the counting means once counts and then the converting means converts the pulse density expression again, a simple digital logic circuit structure for processing the signal of the pulse density expression is formed, and the pulse train to be transmitted in the forward process or the learning process. The randomness of the pulse trains can be increased, and therefore the mutual correlation between the pulse trains on each connection can be weakened to improve the calculation and self-learning ability in each signal processing means, and improve the ability of the network as a whole. You can

Also, in the invention described in claim 7, at least one of the positive and negative error signals collected by taking the logical sum of the error signals is discarded, and the positive and negative error signals are simultaneously sent. Since it is not output, the correlation between the two can be weakened, and the calculation and learning effects in the signal processing means can be enhanced.

[Brief description of drawings]

FIG. 1 is a logic circuit diagram showing a main part of an embodiment of the present invention.

FIG. 2 is a logic circuit diagram for performing basic signal processing.

FIG. 3 is a schematic diagram showing a network configuration example.

FIG. 4 is a timing chart showing an example of logical operation.

FIG. 5 is a timing chart showing an example of logical operation.

FIG. 6 is a timing chart showing an example of logical operation.

FIG. 7 is a timing chart showing an example of logical operation.

FIG. 8 is a timing chart showing an example of logical operation.

FIG. 9 is a timing chart showing an example of logical operation.

FIG. 10 is a timing chart showing an example of logical operation.

FIG. 11 is a timing chart showing an example of logical operation.

FIG. 12 is a timing chart showing an example of logical operation.

FIG. 13 is a timing chart showing an example of logical operation.

FIG. 14 is a timing chart showing an example of logical operation.

FIG. 15 is a timing chart showing an example of logical operation.

FIG. 16 is a timing chart showing an example of logical operation.

FIG. 17 is a timing chart showing an example of logical operation.

FIG. 18 is a timing chart showing an example of logical operation.

FIG. 19 is a logic circuit diagram showing a configuration example of each unit.

FIG. 20 is a logic circuit diagram showing a configuration example of each unit.

FIG. 21 is a logic circuit diagram showing a configuration example of each unit.

FIG. 22 is a logic circuit diagram showing a configuration example of each unit.

FIG. 23 is a conceptual diagram showing one unit configuration showing a conventional example.

FIG. 24 is a conceptual diagram of the neural network configuration.

FIG. 25 is a graph showing a sigmoid function.

FIG. 26 is a circuit diagram showing a specific configuration of one unit.

FIG. 27 is a block diagram illustrating a digital configuration example.

FIG. 28 is a circuit diagram of a part thereof.

FIG. 29 is a different partial circuit diagram.

[Explanation of symbols]

 20 signal processing means 26 memory 27 logical product calculating means 32 grouping means 33a, 33b logical sum calculating means 34 logical calculating means 35 counting means 36 converting means 38 signal outputting means 39 output signal 40 teacher signal 41, 42 error signal 43, 44 Counting means 46, 47 Converting means 48, 49 Error signal outputting means 51 Computing means 55 Error signal selecting means 61 Coupling coefficient changing means 62, 63 Counting means 64, 65 Converting means 66 Error signal outputting means

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Toshiyuki Furuta 1-3-6 Nakamagome, Ota-ku, Tokyo Inside Ricoh Co., Ltd.

Claims (7)

[Claims] 1. A memory for holding a coupling coefficient for each of a plurality of input signals represented by pulse density, and a logical product operation for computing a logical product of the input signal and the coupling coefficient represented by pulse density. Means, grouping means for dividing the result of the operation by the AND operation means into an excitatory coupling group and an inhibitory coupling group, an OR operation means for operating the OR in each group, and an OR of these. A logical operation means for performing a logical operation of an operation result of the operation means to obtain an output signal expressed by a pulse density, and a new coupling coefficient is calculated on the basis of inputs of a plurality of error signals expressed by the pulse density on the memory. Of the output signal, counting means for counting the pulses of the output signal, and conversion means for converting the counting result into a pulse density expression again. A signal processing device having a signal processing means having a signal output means for 2. A memory for holding a coupling coefficient for each of a plurality of input signals represented by pulse density, and a logical product operation for computing a logical product of the input signal and the coupling coefficient represented by pulse density. Means, grouping means for dividing the result of the operation by the AND operation means into an excitatory coupling group and an inhibitory coupling group, an OR operation means for operating the OR in each group, and an OR of these. A logical operation means for performing a logical operation of an operation result of the operation means to obtain an output signal expressed by a pulse density, and a new coupling coefficient is calculated on the basis of inputs of a plurality of error signals expressed by the pulse density on the memory. Of the error signal, counting means for counting the pulses of the error signal, and conversion means for converting the counting result into a pulse density expression again. And a signal processing unit having an error signal output unit. 3. A memory for holding a coupling coefficient for each of a plurality of input signals represented by pulse density, and a logical product operation for computing a logical product of the input signal and the coupling coefficient represented by pulse density. Means, grouping means for dividing the result of the operation by the AND operation means into an excitatory coupling group and an inhibitory coupling group, an OR operation means for operating the OR in each group, and an OR of these. A logical operation means for performing a logical operation of an operation result of the operation means to obtain an output signal expressed by a pulse density, and a new coupling coefficient is calculated on the basis of inputs of a plurality of error signals expressed by the pulse density on the memory. Coupling coefficient changing means for changing the coupling coefficient of the positive and negative error signals, and counting means for counting the pulses of the positive and negative error signals, respectively. A signal processing device comprising: a signal processing means having an error signal output means by a conversion means for converting to a density expression. 4. A memory for holding a coupling coefficient for each of a plurality of input signals expressed by pulse density, and a logical product operation for calculating a logical product of the input signal and the coupling coefficient expressed by pulse density. Means, grouping means for dividing the result of the operation by the AND operation means into an excitatory coupling group and an inhibitory coupling group, an OR operation means for operating the OR in each group, and an OR of these. A logical operation means for performing a logical operation of an operation result of the operation means to obtain an output signal expressed by a pulse density, and a new coupling coefficient is calculated on the basis of inputs of a plurality of error signals expressed by the pulse density on the memory. Coupling coefficient varying means for changing the coupling coefficient of the positive and negative error signals, and counting means for counting the positive and negative error signal pulses respectively. It is characterized in that a signal processing means is provided having a converting means for converting the result of comparison and the counting result of the result into a pulse density representation and an error signal output means by an operating means for performing a logical operation of the converted signals. Signal processing device. 5. A memory for holding a coupling coefficient for each of a plurality of input signals expressed by pulse density, and a logical product operation for calculating a logical product of the input signal and the coupling coefficient expressed by pulse density. Means, grouping means for dividing the result of the operation by the AND operation means into an excitatory coupling group and an inhibitory coupling group, an OR operation means for operating the OR in each group, and an OR of these. A logical operation means for performing a logical operation of an operation result of the operation means to obtain an output signal expressed by a pulse density, and a new coupling coefficient is calculated on the basis of inputs of a plurality of error signals expressed by the pulse density, on the memory. A plurality of signal processing means each having a coupling coefficient varying means for changing the coupling coefficient of the above are connected in a mesh form to count the pulses of the output signal of the output layer and the teacher signal respectively. A signal processing device comprising: an error signal output means including a number means and a conversion means for converting the difference between these counting results into a pulse density expression and outputting a positive or negative error signal to the previous layer. 6. A memory for holding a coupling coefficient for each of a plurality of input signals represented by pulse density, and a logical product operation for computing a logical product of the input signal and the coupling coefficient represented by pulse density. Means, grouping means for dividing the result of the operation by the AND operation means into an excitatory coupling group and an inhibitory coupling group, a logical sum computing means for computing a logical sum in each group, and a logical sum of these. A logical operation means for performing a logical operation of an operation result of the operation means to obtain an output signal expressed by a pulse density, and a new coupling coefficient is calculated on the basis of inputs of a plurality of error signals expressed by the pulse density, on the memory. A plurality of signal processing means each having a coupling coefficient varying means for changing the coupling coefficient of the above are connected in a mesh form to count the pulses of the output signal of the output layer and the teacher signal. Numerical means, conversion means for converting the comparison result of these count results and each count result into a pulse density representation, and logical operation of the converted signals to output a positive or negative error signal to the previous layer. A signal processing device comprising an error signal output means by a calculation means. 7. A memory for holding a coupling coefficient for each of a plurality of input signals represented by a pulse density, and an AND operation for computing a logical product of the input signal and the coupling coefficient represented by the pulse density. Means, grouping means for dividing the result of the operation by the AND operation means into an excitatory coupling group and an inhibitory coupling group, a logical sum computing means for computing a logical sum in each group, and a logical sum of these. A logical operation means for performing a logical operation of an operation result of the operation means to obtain an output signal expressed by a pulse density, and a new coupling coefficient is calculated on the basis of inputs of a plurality of error signals expressed by the pulse density, on the memory. Coupling coefficient varying means for changing the coupling coefficient of the error signal, and an error signal for discarding at least one of the error signals by calculating a logical sum of the error signals depending on whether the error signal is positive or negative. A signal processing device comprising a signal processing means having a selecting means.