JPH056351A - Signal processor - Google Patents

Abstract

PURPOSE:To speedily execute processing for obtaining binary data on the final output stage in a neural network provided with a digital self-learning function. CONSTITUTION:The final output stage of the neural network provided with the digital self-learning function is provided with a memory 56 for storing a final output result, a signal extracting means for extracting the result stored in the memory 56 after the lapse of a prescribed time and an up/down counter 58 for determining up/down and enable operation based upon the current final output result and the preceding output result extracted from the memory 56 by the signal extracting means to output binary data and constituted so that binary data can be successively obtained from the up/down counter 58 without waiting the input interval of plural reference clocks.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing device such as a neural computer which imitates nerve cells.

[0002]

2. Description of the Related Art The aim of parallel processing of information is to imitate the function of a nerve cell (neuron), which is a basic unit of information processing of a living body, and further to make this "nerve cell mimicking element" a network, and to process information in parallel. This is a so-called neural network. Although it is easy to perform character recognition, associative memory, motion control, etc. in a living body, there are many things that conventional Neumann computers cannot easily achieve. Therefore, attempts are being actively made to solve these problems by imitating the nervous system of the living body, particularly the functions peculiar to the living body, that is, parallel processing, self-learning and the like. However, many of these attempts are carried out by computer simulation, and parallel processing is necessary to bring out the original functions, and for that purpose, it is necessary to implement neural network hardware. Some have already tried to use hardware, but the self-learning function, which is one of the features of neural networks, cannot be realized, which is a major obstacle. Further, most of them are realized by analog circuits, and there is a problem in operation.

These points will be examined in more detail. First, a conventional neural network model will be described. FIG. 31 shows one nerve cell unit (nerve cell mimicking element) 1, and FIG. 32 shows this as a network. That is, one nerve cell unit 1 is combined with many other nerve cell units 1 to receive a signal, process the signal, and output the signal. In the case of FIG. 32,
The network is hierarchical, and receives a signal from a unit in the previous (left) layer and outputs it to a unit in the next (right) layer.

Here, in the nerve cell unit 1 of FIG. 31, the degree of coupling between another nerve cell unit and its own nerve cell unit is called a coupling coefficient, and the i-th unit and the j-th unit. The combined count with a unit is generally denoted by T _ij . There are two types of coupling: excitatory coupling, in which the output of the partner unit increases as the signal from the partner unit increases, and conversely, inhibitory coupling in which the output of the partner unit decreases, the output decreases. _ij > 0 represents excitatory coupling, and T _ij <0 represents inhibitory coupling. If the input from the i-th unit is y _i when it is the j-th unit, T _ij y _{i obtained} by multiplying this by the coupling coefficient T _ij becomes the input to the own unit. As described above, since each unit is connected to a large number of units, ΣT _ij y which is the result of adding T _ij y _i for these units
_i becomes the input to your unit. This is called the internal potential and is represented by u _j as in the equation (1).

[0005]

[Equation 1]

Next, the input is subjected to non-linear processing and output. The function at this time is called a nerve cell response function, and the sigmoid function as shown in equation (2) and FIG. 33 is used as the nonlinear function.

[0007]

[Equation 2]

When a network is formed as shown in FIG. 32, a final output is obtained by giving each coupling coefficient T _ij and successively calculating equations (1) and (2).

On the other hand, as an example of realizing such a network by an electric circuit, there is one as shown in FIG. This is disclosed in Japanese Patent Laid-Open No. 62-295188, and basically, a plurality of amplifiers 2 having an S-shaped transfer function, and the output of each amplifier 2 is connected to the input of the amplifier of another layer. And a resistive feedback network 3 connected as shown by the chain line. A CR time constant circuit 4 composed of a grounded capacitor and a grounded resistance is individually connected to the input side of each amplifier 2. And
Input currents I ₁ , I ₂ , ..., _IN are supplied to the inputs of each amplifier 2 and the output is obtained from the set of output voltages of these amplifiers 2.

Here, the strength of the input or output signal is represented by a voltage, and the strength of the coupling between nerve cells is determined by the resistance 5 (the grid point in the resistive feedback network 3) connecting the input / output lines between the cells. ), And the nerve cell response function is represented by the transfer function of each amplifier 2. Further, the coupling between nerve cells includes excitatory coupling and inhibitory coupling as described above, and is mathematically represented by the sign of the coupling coefficient. However, since it is difficult to realize positive and negative with a constant on the circuit, here, the output of the amplifier 2 is divided into two, and one of the outputs is inverted to generate two signals of positive and negative. Is properly selected. An amplifier is used as the one corresponding to the sigmoid function shown in FIG.

However, in these circuits, the strength of a signal inside the network is represented by an analog value such as a potential or current, and the internal calculation is also performed in an analog manner. , Its value will change. Since it is a network, a large number of elements are required, but it is difficult to make the respective characteristics uniform. When the accuracy or stability of one element becomes a problem, if it is used as a network, a new problem may occur, and the behavior of the entire network cannot be predicted. Since the value of the coupling coefficient T _ij is fixed and the value learned in advance by another method such as simulation is used, self-learning cannot be performed. There is a problem such as.

On the other hand, as a learning law used in the numerical calculation, there is the following one called back propagation.

First, each coupling coefficient is first randomly given. If an input is given in this state, the output result is not always desirable. For example, in the case of character recognition, if a handwritten “1” character is given, the output result “This character is“ 1 ”” is a desirable result, but the coupling coefficient is not always random. Not the desired result. Therefore, a correct answer (teaching signal) is given to this network, and each coupling coefficient is changed so that the correct answer is given when the same input is given again. At this time, the algorithm to find the amount to change the coupling coefficient is
This is called back propagation.

For example, in the hierarchical network shown in FIG. 32, if the output of the j-th nerve cell unit in the final layer is y _j and the teacher signal for that nerve cell unit is d _j , then equation (3)

[Equation 3] (4) so that E represented by

[Equation 4] _Is used to change the coupling coefficient T _ij .

More specifically, first, to obtain the coupling coefficient between the output layer and the layer immediately before it, the equation (5) is used.

[Equation 5] When the error signal δ is calculated using, and the coupling coefficient between the layers before that is calculated, equation (6) is used.

[Equation 6] The error signal δ is calculated using

[Equation 7] And change T _ij . Where η is a learning constant,
α is called a stabilization constant. Since it is not possible to obtain each logically, we ask empirically. Further, f ′ is a first-order differential function of the sigmoid function f, and ΔT _ij ′ and T _ij ′ are values at the time of previous learning.

Learning is carried out in this manner, and thereafter, the input is given again and the output is calculated and the learning is carried out. By repeating this operation many times, the coupling coefficient T _ij is finally determined so as to obtain a desired result for a given input.

However, if such a learning method is to be implemented as hardware by some method, a large amount of four arithmetic operations are required for learning, which is difficult to realize. The learning method itself is not suitable for hardware implementation.

On the other hand, an example in which a neural network is realized by a digital circuit will be described with reference to FIGS. FIG. 35 shows a circuit configuration of a single nerve cell, in which each synapse circuit 6 is connected to a cell body circuit 8 via a dendrite circuit 7. FIG. 36 shows an example of the structure of the synapse circuit 6 in which a rate multiplier to which a value obtained by multiplying the input pulse f by a factor a (a factor for multiplying a feedback signal by 1 or 2) is input via a coefficient circuit 9. 10
And a synapse weight register 11 storing a weighting value w is connected to the rate multiplier 10. Further, FIG. 37 shows a configuration example of the cell body circuit 8,
A control circuit 12, an up / down counter 13, a rate multiplier 14 and a gate 15 are sequentially connected,
Further, an up / down memory 16 is provided.

The input / output of the nerve cell unit is represented by a pulse train, and the amount of signal is represented by the pulse density.
The coupling coefficient is represented by a binary number and stored in the memory 16. By inputting the input signal into the clock of the rate multiplier 14 and the combined count into the rate value,
The pulse density of the input signal is reduced according to the rate value. This is the T _{ij of the} backpropagation model equation.
It corresponds to the part of y _i . Next, the Σ portion of ΣT _ij y _i is realized by the OR circuit shown by the dendrite circuit 7. Since coupling has excitatory and inhibitory properties, it is divided into groups in advance, and OR is taken for each group. An output is obtained by inputting these two outputs to the up side and down side of the counter 13 and counting. This output is 2
Since it is a decimal number, the rate multiplier 14 is used again to convert it into a pulse density. A neural network can be realized by making this unit a network. The learning is realized by inputting the final output to an external computer, performing numerical calculation inside the computer, and writing the result in the coupling coefficient memory 16. Therefore, there is no self-learning function. Also, the circuit configuration is complicated because the pulse density signal is once converted into a numerical value (binary number) using a counter and then converted into the pulse density again.

[0020]

As described above, in the case of the prior art, the analog circuit system is not reliable in operation,
The learning method by numerical calculation is also complicated in calculation, is not suitable for hardware implementation, and has a complicated circuit configuration in a digital method that ensures reliable operation. In addition, there is a drawback that self-learning cannot be performed on hardware.

In order to solve such a drawback, a pulse density type neuron model with a learning function is disclosed in Japanese Patent Application No. 2-41.
No. 2448, and Japanese Patent Application No. 2-67
No. 942 proposes a method for obtaining a digital or analog signal by using such a neuron. However, in the system shown in the proposed example, it takes time to convert it into a digital or analog signal, which is a problem in applications requiring high speed such as motion control.

In order to clarify this point, the configuration and operation of the proposed example will be described with reference to FIGS.
Will be described with reference to. First, the neural network in the proposed example is hardwareized by a digital configuration, but the basic idea of the proposed example is:
Input / output signals, intermediate signals, coupling coefficients, teacher signals, etc. relating to the nerve cell unit are all represented by a pulse train represented by binary values of "0" and "1". The amount of signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed in the memory. Learning is realized by rewriting this pulse train. For learning, an error is calculated based on the given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1". It was done like this.

This idea will be described below. First, regarding the signal processing by the digital logic circuit, the signal processing in the forward process will be described. FIG. 2 shows a portion corresponding to one nerve cell unit (nerve cell mimicking element) 20, and the entire neural network is of a hierarchical type similar to the case shown in FIG. 32, for example. All inputs and outputs are binarized to “1” and “0” and synchronized. The intensity of the input signal y _i is represented by a pulse density, and is represented by the number of states of “1” within a certain fixed time as in the pulse train shown in FIG. 3, for example. That is, the example of FIG. 3 represents 4/6, and the signal is 4 "1" and 2 "0" in 6 sync pulses. At this time, "1" and "0"
It is desirable that the arrangement of is random.

On the other hand, the coupling coefficient T _ij indicating the degree of coupling between the nerve cell units 20 is similarly expressed by pulse density,
A bit string of "0" and "1" is prepared in advance in the memory. The example of FIG. 4 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of "1" and "0" is random.

Then, this bit string is sequentially read from the memory in response to the synchronous clock, and as shown in FIG.
The ND gate 21 takes the logical product of the input signal pulse train (y _i ∩ T _ij ). This is used as an input to the nerve cell j. In the case of the above example, the input signal is “10110
When "1" is input, a pulse train is called from the memory in synchronism with this and by sequentially performing AND, "101000" as shown in FIG. 5 is obtained, which means that the input y _i is the coupling coefficient T _ij. And the pulse density is 2/6
It shows that it becomes.

The pulse density of the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This is because the longer the signal sequence, the more "1"
The more random the arrangement of "0" and "0", the closer to the product of numerical values it has. If the pulse train of the coupling coefficient is shorter than the input pulse train and there is no data to be read, it is sufficient to return to the beginning of the data again and repeat the reading.

Since one nerve cell unit 20 has multiple inputs, there are also many "ANDs of the input signal and the coupling coefficient" described above, and the OR circuit 22 then takes the logical sum of these. Since the inputs are synchronized, for example, the first data is "101000" and the second data is "01000".
In the case of “0”, the OR of both is “111000”. When this is calculated for m inputs for multiple inputs simultaneously and used as outputs, for example, as shown in FIG. This corresponds to the sum calculation and the non-linear function (sigmoid function) in the analog calculation.

When the pulse density is low, the pulse density of the OR of the pulse density approximately matches the sum of the respective pulse densities. As the pulse density increases, the OR circuit 22
Since the output of is gradually saturated, it does not match the sum of pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, the coupling has excitability and inhibitory property. In the case of numerical calculation, it is represented by the sign of the coupling coefficient. In the case of an analog circuit, when T _ij is negative (inhibitory coupling), an amplifier is used. It is used to invert the output and _connect it to another nerve cell unit with a resistance value corresponding to T _ij . In this respect, in the proposed digital method, first, each coupling is divided into two groups of excitatory coupling and inhibitory coupling according to the positive or negative of T _ij , and then “A of the pulse train of the input signal and the coupling coefficient is divided into two groups.
The OR of “ND” is calculated for each group. If the result of the excitatory group and the result of the inhibitory group thus obtained do not match, the result of the excitatory group is output. That is, if the result of the excitatory group is “0” and the result of the inhibitory group is “1”, “0” is output, and the result of the excitatory group is “1” and the result of the inhibitory group is “0”. , "1" is output. When the results of the excitatory group and the results of the inhibitory group match,
Either "0" or "1" may be output, or
You may output the separately prepared second input signal,
Alternatively, a logical product of such a second input signal and the contents of the memory provided for the second input signal may be calculated and output.

In order to realize this function, in the case of outputting "0", the output of the excitatory group and the negation of the output of the inhibitory group may be ANDed.
FIG. 7 shows an example of this, and when expressed by mathematical formulas, (8) to (10)
It becomes like a formula.

[0031]

[Equation 8]

In the case of the example of outputting "1", the negation of the output of the excitatory group and the output of the inhibitory group may be ANDed. FIG. 8 shows this example, which is expressed by equations (11) to (13).

[0033]

[Equation 9]

In the case of an example of outputting the second input signal, the second input signal is represented by E as shown in FIG. 9, which is expressed by equations (14) to (16). become.

[0035]

[Equation 10]

Further, in the case of the example of the fourth method, assuming that the content (coefficient) of the memory provided for the second input signal E is T ', it becomes as shown in FIG. And, it becomes like (17)-(19).

[0037]

[Equation 11]

The network of nerve cell units 20 is
Hierarchical type similar to backpropagation (ie, FIG.
2). If the entire network is synchronized, each layer can be calculated by the functions described above.

Next, the signal calculation processing in learning (back propagation) will be described. Basically, the error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method of c.

First, the error signal in the final layer will be described as a. The error signal in each nerve cell unit is calculated in the final layer, and the coupling coefficient relating to that nerve cell unit is changed based on the error signal. The calculation method of the error signal for that purpose is described. Here, the "error signal" is defined as follows. When the error is expressed numerically, it is generally +,-
However, in the case of pulse density, both positive and negative cannot be expressed at the same time.
Express the error signal using two types: a signal representing the component. That is, the error signal of the jth nerve cell unit is shown in FIG. That is, the + component of the error signal is the difference between the teacher signal pulse and the output pulse (1,
0) or (0, 1) is a pulse existing on the teacher signal side, and the − component is a pulse also existing on the output side. In other words, if the error signal + pulse is added to the output signal y _j and the error signal−pulse is removed, the teacher signal d _j
Will be. That is, these positive and negative error signals δ
_{When j (+)} and δ _{j (-)} are expressed by logical expressions, they become the expressions (20) and (21), respectively. In the formula, XOR represents exclusive OR. The coupling coefficient is changed based on such an error signal pulse as described later.

[0041]

[Equation 12]

Next, a method of obtaining an error signal in the intermediate layer as b will be described. First, the above-mentioned error signal is back-propagated to change not only the coupling coefficient between the final layer and the layer immediately before it but also the coupling coefficient of the layer before that. Therefore, it is necessary to calculate the error signal in each nerve cell unit in the middle layer. Signals are propagated from a neuron unit with an intermediate layer to each neuron unit in the next layer, which is exactly the reverse of the procedure of collecting error signals in each neuron unit in the next layer. And use it as its own error signal. This can be performed in the same manner as the above-described arithmetic expressions (8) to (10) and the cases shown in FIGS. 3 to 8 in the nerve cell unit. However, what is different from the above-mentioned processing in the nerve cell unit is that y _j is one signal that is always positive, whereas δ _j has two signals that represent positive and negative. It is necessary to consider the signal of. Therefore, _whether the coupling coefficient T _ij is positive or negative,
It is necessary to divide into four cases depending on whether the error signal δ _j is positive or negative.

First, the case of excitatory coupling will be described. In this case, for a nerve cell unit with an intermediate layer, the error signal δ at the kth nerve cell unit in the layer immediately ahead
_{k (+)} and the coupling coefficient T _jk between the nerve cell unit and self
Those taking AND of _{(δ k (+) ∩T jk} ) look for each neuron unit, further, take the OR of these with each other _{{∪ (δ k (+)} ∩ T jk)}. This is the error signal δ _{j (+)} of this intermediate layer. That is, assuming that the number of nerve cell units in the previous layer is n, the result is as shown in FIG. If these are shown in order by mathematical expressions, they will be as shown in Expressions (22) to (24).

[0044]

[Equation 13]

Similarly, the error of this intermediate layer is obtained by _ANDing the error signal δ _{k (-)} and the coupling coefficient T _{jk in} the nerve cell unit of the layer one ahead and further ORing them. Let the signal δ _{j (-)} . That is, it becomes as shown in FIG. 13, and when these are expressed in order by mathematical expressions, they become as in Expressions (25) to (27).

[0046]

[Equation 14]

Next, the case of inhibitory binding will be described. In this case, the error signal δ in the nerve cell unit of the next layer
The AND of the coupling coefficient T _jk between _{k (-)} and its nerve cell unit and self is taken, and the OR of these is taken. This is the error signal δ _{j (+) of} this intermediate layer. That is, in FIG.
As shown in, the mathematical expression of these in order is (28) ~
It becomes like the formula (30).

[0048]

[Equation 15]

Similarly, the AND of the preceding error signal δ _{k (+)} and the coupling coefficient T _jk is performed, and the OR of these is performed, and similarly, the error signal δ _{j (-) of} this layer is also obtained. And
That is, it becomes as shown in FIG. 15, and when these are expressed in order by mathematical expressions, they become as in Expressions (31) to (33).

[0050]

[Equation 16]

Since some nerve cell units are excitatoryly coupled to other nerve cell units and some are inhibitoryly coupled, the error signal δ obtained as shown in FIG. 12 is used. _{j (+)} and the error signal δ obtained as shown in FIG.
The OR with _{j (+)} is taken to be the error signal δ _{j (+) of} its own nerve cell unit. Similarly, the error signal [delta] _j determined as in FIG. 13 _(-) and the error signal [delta] _j determined as in FIG. 15 _(-)
Is taken as the error signal δ _{j (-)} of the own nerve cell unit.

Summarizing the above, equation (34)

[Equation 17] As shown in.

Alternatively, it becomes as shown in equation (35).

[0054]

[Equation 18]

Further, a function corresponding to the learning rate (learning constant) may be provided. When the rate is 1 or less in the numerical calculation, the learning ability is further enhanced. This can be realized by thinning out the pulse train in the pulse train calculation. Here, a counter-like concept is adopted, and the ones shown in FIGS. 16 and 17 are adopted. For example, at the learning rate η = 0.5, every other pulse train of the original signal is thinned out, but even if the pulses of the original signal are not evenly spaced, the original pulse train can be thinned out. 16 and 17, in the case of η = 0.5, every other pulse is thinned out, in the case of η = 0.33, every two pulses are left, and in the case of η = 0.67, two pulses are left. It indicates that one thinning is performed every other time.

In this way, the function of the learning rate is provided by thinning out the error signal. Such decimation of the error signal can be easily realized by logically operating the output of a counter commercially available or using a flip-flop. In particular, when a counter is used, the value of the learning constant η can be set arbitrarily and easily, so that the characteristics of the network can be controlled.

Further, as c, a method of changing each coupling coefficient by such an error signal will be described. Line to which the coupling coefficient you want to change belongs (see Figure 32)
The signal flowing through and the error signal are ANDed (δ _j ∩ _{y j} ).
However, here, since there are two signals of + and-in the error signal, they are respectively calculated and obtained as shown in FIGS.

The two signals thus obtained are designated as ΔT _{ij (+)} and ΔT _{ij (-)} , respectively. Next, this ΔT
_A new T _ij is obtained based on _ij . Since this T _ij is an absolute value component, it is classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the component of ΔT _{ij (+)} is increased and the component of ΔT _{ij (−)} is decreased with respect to the original T _ij . That is, it becomes as shown in FIG. On the contrary, in the case of the suppressive property, the component of ΔT _{ij (+)} is reduced and the component of ΔT _{ij (−)} is increased with respect to the original T _ij . That is, it becomes as shown in FIG.

The contents of these FIG. 20 and FIG. 21 are expressed by mathematical expressions as shown in equations (36) and (37).

[0060]

[Formula 19]

The network is calculated based on the above learning rule.

Next, an actual circuit configuration based on the above algorithm will be described. 22 to 27 show examples of the circuit configuration thereof, the configuration of the entire network 2 is the same as that of FIG. 22 to 25 show a circuit of a portion corresponding to a line (connection) in the hierarchical network as shown in FIG. 32, and FIG. 26 corresponds to a circle in FIG. 32 (each nerve cell unit 20 in the proposed example). The circuit of the part to be shown is shown. Also, FIG.
Reference numeral 7 denotes a circuit of a portion for obtaining an error signal in the final layer from the output of the final layer and the teacher signal. A digital neural network capable of self-learning can be realized by forming a network of these three circuits having the configurations of FIGS. 22 to 24 as in the case of FIG.

First, FIG. 22 will be described. In the figure, 25 is an input signal to the nerve cell unit as shown in FIG. The value of the coupling coefficient as shown in FIG. 5 is stored in the shift register 26. The shift register 26 has an outlet 26a and an inlet 26b, but may be any one having the same function as a normal shift register, for example, R
It may be a combination of an AM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are ANDed by a logic circuit 28 having an AND gate 27 and performing the processing shown in FIG. The output of the logic circuit 28 must be divided into groups depending on whether the coupling is excitatory or inhibitory. However, it is better to prepare the outputs 29 and 30 for each group in advance and switch which one of them is output. It is highly versatile. Therefore, in the proposed example, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31, and the information is used to switch by the switching gate circuit 32. The switching gate circuit 32 includes two AND gates 32a and 32b and an inverter 32c interposed at one input.

When it is not necessary to switch, they may be fixed. For example, FIG. 23 shows the case of excitability and FIG. 24 shows the case of inhibition. Further, for one input, both a memory for a bit indicating excitability and a memory for a bit indicating inhibitory property may be prepared. FIG. 25 shows this example. In the figure, 26A is a bit memory for a coupling coefficient indicating excitability, and 26B is a bit memory for a coupling coefficient indicating inhibition.

Further, as shown in FIG. 26, a gate circuit 3 having a plurality of OR gates for performing each input process (corresponding to FIG. 6).
3a and 33b are provided. Further, as shown in the figure, a gate formed by an AND gate 34a and an inverter 34b which outputs an output "1" only when the excitatory coupling group shown in FIG. 7 is "1" and the inhibitory coupling group is "0". A circuit 34 is provided. Even when the processing results illustrated in FIGS. 8 to 10 are obtained, it can be easily realized by the logic circuit.

Next, the error signal will be described. The error signal in the final layer is generated by the logic circuit 35 based on the combination of AND and exclusive OR shown in FIG. 27, which corresponds to the equations (6) and (7). That is, the error signals 38 and 39 are generated by the output 36 from the final layer and the teacher signal 37. Among the equations (35) for calculating the error signal in the intermediate layer, E _{j (+)} , E
_{The process of obtaining j (-)} is performed by the gate circuit 42 having the AND gate configuration shown in FIG.
3,44 is obtained. As described above, it is necessary to classify the connection depending on whether it is excitatory or inhibitory. In this case, the information on excitatory or inhibitory stored in the memory 31 and the +,-signals 45 and 46 of the error signal are used. AND, OR depending on
This is performed by the gate circuit 47 having a gate configuration. The remaining part of the equation (8) for collecting error signals, that is, the equation (35), is performed by the gate circuit 48 having the OR gate configuration shown in FIG. 16 and 17 corresponding to the learning rate.
This process is performed by the frequency dividing circuit 49 shown in FIG.
This can be easily realized by using a flip-flop or the like.

Finally, the part for calculating a new coupling coefficient from the error signal, that is, the part corresponding to the processing of FIGS. 18 to 21, is performed by the gate circuit 50 having the AND, inverter and OR gate configurations shown in FIG. Done, shift register 2
The contents of 6, that is, the value of the coupling coefficient is rewritten. This gate circuit 50 also needs to be divided into cases depending on the excitability and inhibitory property of the coupling, but it is performed by the gate circuit 47. Figure 2
In the case of FIG. 3 and FIG. 24, since excitability and inhibitory property are fixed, a circuit corresponding to the gate circuit 47 is unnecessary. Figure 2
In the case of the method of 5, the input circuit has both excitability and inhibitory property with respect to one input. Therefore, when the gate circuit 50A is excitatory,
This corresponds to the case where the gate circuit 50B has the suppressing property.

As described above, the method of expressing a signal with a pulse density is useful not only for actual circuits but also for simulation on a computer. On the computer, the calculation is performed serially, but compared with the calculation using the analog value, only the binary logical calculation of "0" and "1" is performed, so that the calculation speed is significantly improved. Generally, the real number arithmetic operations require many machine cycles for one calculation, but the number of arithmetic operations is small. In addition, it is also easy to use low-level languages for high-speed processing when only logical operations are performed.

By the way, it is not necessary to implement all the circuits in implementing the above-described processing method, and some or all of them may be implemented by software. Further, the circuit itself may be replaced with another circuit having an equivalent logic, or the logic may be replaced with a negative logic.

An example of actual measurement will be described taking the case of a self-learning character recognition device using such a neural network as an example. First, scan the handwritten characters with a scanner and
As shown in FIG. 8, the mesh was divided into 16 × 16 meshes, the mesh with the character portion was “1”, and the mesh without the character portion was “0”. 256 pieces of such data were input to the network, and the position of the output having the largest output among the units having five outputs was set as the recognition result. for that reason,
When a number from "1" to "5" was input, learning was performed so that the output of the number corresponding to the number was the largest. Next, regarding the network configuration, the first layer is 256
It was assumed that the nerve cell unit 20 consisted of 20 units, the second layer was 20 units, and the third layer was 5 units. First, if each coupling coefficient is given randomly, the output result is not always the desired one. Therefore, the self-learning function of this circuit is used to newly obtain each coupling coefficient and repeat this several times so that a desired output can be obtained. In the proposed example, since the input is "0" or "1", the input pulse train is always simple at L level or H level. Also,
The output is connected to the LED via a transistor so that it is turned off at the L level and turned on at the H level. Since the synchronization clock is set to 1000 kHz, the brightness of the LED changes for human eyes according to the pulse density, and the brightest LED part is the answer. A recognition rate of 100% was obtained for a sufficiently learned character.

However, since the final output is also obtained as a pulse train, it may be necessary to convert this into binary data or an analog signal in some cases. In this respect, in the proposed example system, as shown in FIGS. 29 and 30, the output pulse train is counted for a certain period of time to obtain binary data. This is one in which an output pulse train is counted by a synchronous counter 55 for five basic clocks. That is, a clear signal is generated every six pulse trains and is input to the clear terminal (clear at L level) of the counter 55 to clear it. When the basic clock of this system is input to the clock terminal of the counter 55 and the output pulse train is input to the enable terminal (enable at H level) of the counter 55, the counted binary data is obtained. In the case of this method, the timing at which the binary data is obtained is as shown in FIG. 30, and every time a plurality of basic clocks are input (timing A and B shown in FIG. 30, for example, timing B is counted between intervals). Only when the value is obtained) can the data be obtained. Therefore, it cannot be applied to applications such as motion control that require particularly high speed.

[0072]

[Means for Solving the Problems] Coupling coefficient varying means, coupling coefficient generating means for generating a variable coupling coefficient value of the coupling coefficient varying means based on an error signal with respect to a teacher signal, and the coupling coefficient value being fixed in a changed state. A storage means for storing a final output result in a final output stage, which is provided with a signal processing means in which a plurality of neural cell mimicking means having a self-learning means having a switching means for switching to a state are connected to a neural cell mimicking element And a signal extracting means for extracting the result stored in the storing means after a predetermined time has passed, an operation is determined based on the final output result and the output result extracted from the storing means by the signal extracting means, and the binary value is obtained. An up / down counter for outputting data was provided.

[0073]

The fact that the final output result is stored in the storage means and taken out after a predetermined time means that the previous final output result can be obtained at a certain point in time, and this output result and the current final output result can be obtained. By determining the counting operation of the up / down counter using both, up-counting, down-counting, or not counting the final output result, sequential up-down is performed without waiting for multiple input of the basic clock. Binary data can be obtained from the counter, which provides high-speed compatibility.

[0074]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of the present invention will be described with reference to FIG. The present embodiment is based on the above-described proposal example, and the portions shown in FIGS. 2 to 27 are used as they are, and the description thereof will be omitted. First, the memory 56 is provided as a storage unit that temporarily stores the final output result (output signal). As the memory 56, a shift register, a FIFO (first-in / first-out) memory, or a combination of a RAM and an address controller may be used. The memory 56 is provided with a signal extracting means (not shown) for extracting the output signal stored after a predetermined time. The predetermined time for which the signal extracting means takes out the output signal may be taken out after a certain number of basic clocks of the system are input. At this time, the number of basic clocks (= predetermined time) may be a fixed numerical value, a numerical value set externally by a switch or the like, or a numerical value written in the memory 56 or the like. May be

An exclusive OR gate 57 which receives the output signal fetched from the memory 56 and the current final output signal is provided, and the exclusive OR output of the exclusive OR gate 57 is an enable terminal ( Is enabled at H level) and the current final output signal is up /
Synchronous up / down counter 58 input to the down terminal (up operation at H level, down operation at L level)
Is provided. The basic clock is input to the clock terminal. Therefore, the operation of the up / down counter 58 is determined as shown in Table 1.

[0076]

[Table 1]

As described above, using the memory 56 to take out the output signal after a certain period of time means that the previous output signal can be obtained at a certain time (current time). The exclusive-OR with the current output signal enables the up / down counter 58. Therefore, looking at the operation when the fixed time is set to one basic clock, as shown in the timing chart of FIG. 1 (b), at each timing C, D, E, ... in accordance with the basic clock of the system. The binary data in the section ~ will be sequentially obtained. This allows
Even if high-speed operation is required, it can be handled.

The circuit configuration is not limited to that shown in FIG. 1A, but may be any other configuration as long as the logical relationships and functions shown in Table 1 are secured. . Also,
When an analog signal is required, the binary data of the up / down counter 58 may be converted by the D / A converter.

[0079]

As described above, according to the present invention, the function of the neural network including the learning function can be performed in parallel on the hardware, and the final output result is saved in the final output stage. Means and a signal taking-out means for taking out the result stored in the storing means after a lapse of a predetermined time, an operation is determined based on the final output result and the output result taken out from the storing means by the signal taking-out means, and a binary value is obtained. Since the up / down counter that outputs data is provided, binary data can be sequentially obtained from the up / down counter without waiting for a time interval in which a plurality of basic clocks are input, and high speed and high processing capability are provided. At this time, by determining the predetermined time for taking out the signal from the storage means according to the contents of the storage means or an instruction from the outside, Thereby enabling the processing speed of the setting according to.

[Brief description of drawings]

1A and 1B show an embodiment of the present invention, in which FIG. 1A is a block diagram and FIG. 1B is a timing chart.

FIG. 2 is a logic circuit diagram for performing basic signal processing in an already proposed example.

FIG. 3 is a timing chart showing an example of logical operation.

FIG. 4 is a timing chart showing an example of logical operation.

FIG. 5 is a timing chart showing an example of logical operation.

FIG. 6 is a timing chart showing an example of logical operation.

FIG. 7 is a timing chart showing an example of logical operation.

FIG. 8 is a timing chart showing an example of logical operation.

FIG. 9 is a timing chart showing an example of logical operation.

FIG. 10 is a timing chart showing an example of logical operation.

FIG. 11 is a timing chart showing an example of logical operation.

FIG. 12 is a timing chart showing an example of logical operation.

FIG. 13 is a timing chart showing an example of logical operation.

FIG. 14 is a timing chart showing an example of logical operation.

FIG. 15 is a timing chart showing an example of logical operation.

FIG. 16 is a timing chart showing an example of logical operation.

FIG. 17 is a timing chart showing an example of logical operation.

FIG. 18 is a timing chart showing an example of logical operation.

FIG. 19 is a timing chart showing an example of logical operation.

FIG. 20 is a timing chart showing an example of logical operation.

FIG. 21 is a timing chart showing an example of logical operation.

FIG. 22 is a logic circuit diagram showing a configuration example of each unit.

FIG. 23 is a logic circuit diagram showing a configuration example of a modified example thereof.

FIG. 24 is a logic circuit diagram showing a configuration example of a modification thereof.

FIG. 25 is a logic circuit diagram showing a configuration example of a modified example thereof.

FIG. 26 is a logic circuit diagram showing a configuration example of each unit.

FIG. 27 is a logic circuit diagram showing a configuration example of each unit.

FIG. 28 is an explanatory diagram illustrating an example of image reading.

FIG. 29 is a circuit diagram for final output stage processing.

FIG. 30 is a timing chart showing the operation.

FIG. 31 is a conceptual diagram showing a unit configuration of a conventional example.

FIG. 32 is a conceptual diagram of the neural network configuration.

FIG. 33 is a graph showing a sigmoid function.

FIG. 34 is a circuit diagram showing a specific configuration of one unit.

FIG. 35 is a block diagram showing a digital configuration example.

FIG. 36 is a circuit diagram of a part thereof.

FIG. 37 is a different partial circuit diagram.

[Explanation of symbols]

20 Neuron mimetic elements 56 Storage means 58 up-down counter

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Osamu Takehira             1-3-3 Nakamagome, Ota-ku, Tokyo Stocks             Company Ricoh (72) Inventor Shuji Motomura             1-3-3 Nakamagome, Ota-ku, Tokyo Stocks             Company Ricoh

Claims (3)

[Claims] 1. A coupling coefficient varying means, a coupling coefficient generating means for generating a variable coupling coefficient value of the coupling coefficient varying means based on an error signal with respect to a teacher signal, and switching the coupling coefficient value between a changing state and a fixed state. A storage means for storing the final output result in the final output stage, and a storage means for storing the final output result in the final output stage, which is provided with a signal processing means in which a plurality of neuron mimicking means having self-learning means having a switching means attached to the neuron mimic A signal taking-out means for taking out the result stored in the means after a lapse of a predetermined time, an operation is decided based on the final output result and the output result taken out from the storing means by the signal taking-out means, and binary data is outputted. A signal processing device comprising an up-down counter. 2. A predetermined time taken by the signal taking-out means,
The signal processing device according to claim 1, wherein the time is set according to the contents of the storage means. 3. The predetermined time taken by the signal taking-out means,
The signal processing device according to claim 1, wherein the time is given from the outside.