JP3338713B2 - Signal processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for imitating a nerve cell which can be applied to various motion controls such as image and voice recognition, position control of a robot, temperature control of an air conditioner, orbit control of a rocket, and the like. And a signal processing device such as a neural computer.

[0002]

2. Description of the Related Art The function of a nerve cell (neuron), which is a basic unit of information processing of a living body, is imitated, and further, this "neural cell mimic element" (neural cell unit) is networked to perform parallel processing of information. What we aimed at was a so-called neural network. Even if it is very easily performed on a living body, such as character recognition, associative memory, and movement control, there are many that are not easily achieved by a conventional Neumann computer. Attempts have been made to solve these problems by imitating the neural system of a living body, particularly functions unique to the living body, that is, parallel processing, self-learning, and the like, using a neural network.

First, a conventional neural network model will be described. FIG. 32 is a diagram showing a certain neuron unit A, and FIG. 33 is a diagram showing this as a network. A ₁ , A ₂ , and A ₃ each represent a neuron unit. One neuron unit is coupled to many other neuron units and processes signals received therefrom to produce an output. In the case of FIG. 33, the network is of a hierarchical type, and the neuron unit A ₂ receives a signal from the neuron unit A _{1 in} the immediately preceding (left) layer and the neuron unit A in the next (right) layer. Output to ₃ .

[0004] This will be described in more detail. First, in the nerve cell unit A in FIG. 32, a degree of connection between another nerve cell unit and its own unit is referred to as a connection coefficient, and the connection coefficient between the i-th nerve cell unit and the j-th nerve cell unit is referred to. The coupling coefficient is generally denoted by T _ij . The coupling includes an excitatory connection in which the larger the signal from the other unit (a unit sending a signal to the own unit), the larger the output of the own unit, and a larger signal from the other unit, the output of the own unit. Is small, and T _ij > 0 represents an excitatory connection, and T _ij <0 represents a suppressive connection. Now, suppose that the own neuron unit is the j-th unit, and the output of the i-th neuron unit is y.
_{When i} is set, T _ij y _{i obtained} by multiplying this by the coupling coefficient T _ij becomes an input to the own unit. As described above, since one neuron unit is connected to many neuron units, the result of adding T _ij y _i to those units is ΣT _ij y _{i, which} is the value of one neuron unit in the network. Input to the unit. This is called an internal potential and is represented by u _j .

[0005]

(Equation 1)

Next, a threshold value is added to the input (internal potential) to perform nonlinear processing, thereby obtaining an output of the nerve cell unit. The function used at this time is called a nerve cell response function, and a sigmoid function as shown in Equation (2) and FIG. 34 is used as a nonlinear function.

[0007]

(Equation 2)

When such a nerve cell unit is configured in a network as shown in FIG. 33, each coupling coefficient T
By giving _ij and calculating equations (1) and (2) one after another, parallel processing of information becomes possible and a final output is obtained.

In such a hierarchical neural network, a desired neural network is constructed by performing learning such that the coupling coefficient T _ij is updated so that a desired result is output for a certain input. . The most widely used learning method is an error back propagation method, a so-called back propagation method.

FIG. 35 shows an example of such a network realized by an electric circuit. This is disclosed in Japanese Unexamined Patent Publication No. 62-295188. Basically, a plurality of amplifiers 1 having an S-shaped transfer function and the output of each amplifier 1 is connected to the input of an amplifier of another layer by one point. A resistive feedback network 2 is provided for connection as shown by the dashed line. The input side of each amplifier 1 is connected to a grounded capacitor and a grounded resistor.
R time constant circuits 3 are individually connected. Then, the input current _{I 1, I 2, ~,} I n is supplied to the input of the amplifier 1, the output is obtained from a set of these amplifiers 1 output voltage.

Here, the signal strength of the input or output to the network is represented by a voltage, and the strength of the connection between the neuron units is determined by the resistance 4 (the resistance feedback network 2 in the resistive feedback network 2) connecting the input / output lines between the cells. ), And the nerve cell response function is represented by the transfer function of each amplifier 1. That is, in FIG. 35, the plurality of amplifiers 1 have an inverted output and a non-inverted output, and the input of each amplifier 1 has a CR time constant circuit 3 as input current supply means. The feedback network 2 connects each output of the amplifier 3 to the input by a resistor 4 (T _ij ) which is a value of 1 or a second value selected in advance. Resistor 4 represents the transconductance between the i-th amplifier output and the j-th amplifier input, and is selected to create a plurality of local minima where the network is balanced, minimizing an energy function having a plurality of local minima. I try to. In addition, the connection between nerve cells has excitability and inhibition, and is mathematically represented by the sign of the coupling coefficient. However, since it is difficult to realize the sign with a constant on the circuit, Then, by dividing the output of the amplifier 1 into two and inverting one of the outputs,
This is realized by generating two signals and selecting them appropriately. An amplifier is used as a function corresponding to the sigmoid function shown in FIG.

On the other hand, an example in which a neural network is realized by a digital circuit will be described with reference to FIGS. FIG. 36 shows a circuit configuration of a single nerve cell. Each synapse circuit 6 is connected to a cell body circuit 8 via a dendritic circuit 7. FIG. 37 shows an example of the configuration of a synapse circuit 6 in which a magnification a is applied to an input pulse f via a coefficient circuit 9.
There is provided a rate multiplier 10 to which a value multiplied by (multiplied by 1 or 2 by the feedback signal) is input, and a synapse load register 11 storing a weight value w is connected to the rate multiplier 10. . FIG. 38 shows a configuration example of the cell body circuit 8, and the control circuit 1
2, an up / down counter 13, a rate multiplier 14, and a gate 15 are sequentially connected, and an up / down memory 16 is further provided.

[0013] The input and output of the nerve cell unit is represented by a pulse train, and the signal density is represented by the pulse density.
The coupling coefficient is represented by a binary number and stored in the memory 16. By inputting the input signal to the clock of the rate multiplier 14 and inputting the coupling coefficient to the rate value,
The pulse density of the input signal is reduced according to the rate value. This is the T _ij of the back propagation model equation.
y _i . Next, the portion {} of {T _ij y _i } is realized by an OR circuit indicated by the dendrite circuit 7. Since the coupling has excitatory and inhibitory properties, it is grouped in advance and OR is performed for each group. An output is obtained by inputting these two outputs to the up side and the down side of the counter 13 and counting. This output is 2
Since it is a decimal number, it is converted into a pulse density by using the rate multiplier 14 again. By making this unit a network, a neural network can be realized.

[0014]

However, the former analog circuit system has the following problems. Signal strength is represented by analog values such as potential and current,
When the internal calculation is performed in an analog manner, the value changes due to temperature characteristics, drift immediately after turning on the power, and the like. Since it is a network, a large number of elements are required, but it is difficult to make the characteristics of each element uniform. When the accuracy or stability of one element becomes a problem, when it is made into a network, there is a possibility that a new problem may occur, and the movement when viewed as a whole network cannot be predicted. The coupling coefficient T _ij is fixed, and a value learned in advance by another method such as a simulation has to be used, and self-learning cannot be performed.

In the case of the latter network configuration using digital circuits, the up-down counter 1 is actually used.
3. A very complicated and large-scale circuit, such as the use of the rate multiplier 14.

As described above, in the case of the prior art, there is no certainty in the operation of the analog circuit system, and the calculation method is complicated in the learning method by numerical calculation. The circuit configuration is large and complicated.

In order to solve such disadvantages, the applicant has proposed a pulse density type digital neuron model with a self-learning function as Japanese Patent Application No. 2-412448, Japanese Patent Application No. 3-29342, or the like. . However, it is also applicable to a system with higher nonlinearity than the proposed neuron element, a case where the input data needs to have a time dependency, or a case where the value of the input signal is finely quantized. In order to further increase the versatility and flexibility of the neuron, it is desired that the information of the output signal represented by the pulse train be reflected in the output signal value at a later time in the forward process.

This will be further described. The pulse density type neural network includes, besides the one proposed by the present applicant as described above, for example, Japanese Patent Laid-Open No. 1-244.
There are also those disclosed in US Pat. No. 567, US Pat. In any case, the way of signal processing in the pulse density type neural network differs depending on each type.

Here, the signal processing in the pulse density type neural network proposed by the present applicant will be briefly described. A bit train is actually used as the pulse train. That is, a pulse train is represented by a string of bits having a value of “0” or “1”, a pulse having a value of “1” represents a pulse, and a bit having a value of “0” represents no pulse.

The signal value is represented by the pulse density, that is, the bit density of the value "1" in the bit string. In other words, the value of the signal is equal to the probability that the value of the bit in the bit string is "1". For example, when a signal of a value "X" is input to the neural network, a bit of a value "1" is input to the neural network with a probability X, and the probability (1-
X), a bit having a value “0” is input. Also,
The fact that the output value of the neural network is “A” means that the probability that the neural network outputs a bit of value “1” is A.

Each neuron unit in the neural network receives as an input signal a bit sequence sent from outside the neural network or from some other neuron unit in the neural network. A coupling coefficient indicating the strength of coupling between a neuron unit and another neuron unit is also represented by a bit string, and weighting of the input signal by the coupling coefficient is performed by taking a logical product of the input bit string and the coupling coefficient bit string. The weighted input signals are divided into several groups, and the weighted input signals are logically ORed separately for each division. Next, by performing a logical operation on the bit string obtained by these logical sum to generate a new bit sequence, and outputs to the outside of the neuron units that bit string as an output signal. FIG. 39 shows the image. In the figure, reference numeral 17 denotes a neuron unit (neural cell unit).

In such a pulse-density type neural network system, as shown in FIG.
From the bit string which is the input signal to the neuron unit 1
7 to generate a bit string which is an output signal of the input signal 7, 1
One bit of an output bit sequence is generated from only a set of bits.

As described above, from only one bit set that simultaneously enters the neuron unit from each input bit string,
Since one bit of the output bit string is generated, the relationship between the input value and the output value of the neural network formed by combining the neuron units is linear. If this point is described with a one-input one-output neural network, it can be schematically represented as shown in FIG. First, the output when the signal of the value “X” is input to the neural network 18 is the output when the signal of the value “1” is input and the output when the signal of the value “0” is input, X: It becomes a value mixed at the ratio of (1-X). Therefore, the neural network 1 when the input value is "1"
8 is "A", and the output of the neural network 18 when the input value is "0" is "B", the value "X"
The output value "Y" of the neural network 18 at the time of inputting the signal is Y = AX + B (1-X). That is,
The relationship between the input value “X” and the output value “Y” of the neural network 18 is linear as shown in FIG. This means that the one-input one-output neural network 18 cannot learn the nonlinear relationship between the input value “X” and the output value “Y”.

As a method for solving the problem that the nonlinear relationship between the input value and the output value cannot be learned in the one-input one-output neural network, the number of pulses in the output pulse train is counted. A pulse train with the above pulse is regenerated and output. The logical OR of the weighted input signal pulse is taken not only in the spatial direction but also in the time direction. Are proposed, but there is a concern that the signal processing speed is slow.

[0025]

According to the first aspect of the present invention, a logical product calculating means for calculating a logical product of an input signal represented by a pulse train and a coupling coefficient represented by a pulse train,
The operation result of the AND operation means is expressed by the excitatory connection glue.
Grouping means for dividing the group into a group and an inhibitory connection group, a logical sum operation means for calculating a logical sum of the operation results in each group, and an excitatory connection group as an operation result of these logical sum operation means Is "1" and suppressive
Output "1" only when the connection group is "0"
Operation means for performing a logical operation as described above, pulse density changing means for changing the pulse density of a pulse train output from the logic operation means, and connection for changing the coupling coefficient based on an error signal expressed by a pulse train. a signal processing means having a variable coefficient unit is provided, said pulse density change
Means for counting the operation result of the logical operation means
And a pulse train with a probability according to the counting result of this counter
Signal generating means for generating a signal represented by
Signal generated by the signal generation means and the operation result of the logic operation means.
AND operation means for calculating the AND operation with the result .

[0026]

[0027] Instead of the AND operation unit in the pulse density changing means of the invention of claim 1, wherein, in the third aspect of the present invention, the logical sum of the operation result of the generation signal and logic operation means by the signal generating means In the invention according to claim 5 , there is provided a logical operation means for performing a logical operation other than a logical AND operation of a signal generated by the signal generating means and an operation result of the logical operation means. In these inventions, in the inventions according to the second, fourth, and sixth aspects , an up-down counter is provided instead of the counter.

According to the seventh aspect of the present invention, the first aspect of the present invention is provided.
Instead of the pulse density changing means of the present invention, the pulse density changing means, and a plurality of counters for counting the operation result of the logic operation means and adder means for adding the count results of these counters, integer value or a real value Random number generating means for generating randomly, comparing means for comparing the result of addition by the adding means with the random number value generated by the random number generating means, and outputting a pulse in accordance with the comparison result; A logical sum of the result and the output signal from the comparison means is calculated to obtain a new output signal.

Similarly, in the invention described in claim 8 , the claim
Instead of the pulse density changing means in the embodiment described 1, a pulse density changing means, count the operation results of the logic operation means
Counters and, integers or real numbers and a random number generating means for randomly generating, comparing the random number value generated by counting result and the random number generating means according to said counter said random number
Is "1" if is less than or equal to the counting result, otherwise
Comparison means and said logic Starring for outputting a pulse which becomes "0"
The logical AND of the operation result of the calculating means and the output of the comparing means is performed.
The logic operation means and the comparison means simultaneously set "1".
When outputting a different pulse, the logical product result is
A probability improving means for increasing the probability that the comparing means outputs a pulse of "1" by adding the result to the counting result by the counter, and a logical sum of the operation result of the logical operation means and the output signal from the comparing means. OR operation means for calculating a new output signal.

Further, instead of the probability improving means in the pulse density changing means according to the invention described in claim 8 , according to the invention described in claim 9 , the counting result by the counting means is calculated according to the number of pulses output from the comparing means. Claim 1
In the invention described in Item 0, means is provided for reducing the counting result by the counting means when the comparing means outputs a pulse and the logic operation means does not output a pulse.

[0031] In the invention of these claims 7, 8, 9 or 1 0, wherein, in the invention of claim 1 1, wherein the means for increasing the pulses plurality ligated into an output signal.

[0032]

According to the present invention, in a simple digital logic configuration for processing a signal represented by a pulse train, a logical AND operation of an input signal and a coupling coefficient, a logical OR operation in groups, and a logical operation result of these logical OR operations are performed. Since the signal obtained as a result of the operation is not output as it is, the pulse density is changed by the pulse density changing means and output.
The output signal pulse density on the forward process, it shall be varied non-linearly with the input signal pulse density, having high flexibility and versatility in the case of building a network and Do Ri, also, such a pulse density Change means
It is specifically shown that the configuration is made using a counter or the like.
As a result, reliable operation is ensured.

[0033]

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described with reference to FIGS. First, a neural cell unit (neuron element) and a neural network using a pulse density type digital logic circuit having a self-learning function will be described.

First, the neuron and the neural network according to the present embodiment are implemented by hardware using a digital configuration. The basic idea is that input / output signals, intermediate signals,
The coupling coefficient, the teacher signal, and the like are all represented by a pulse train represented by binary values “0” and “1”. The amount of the signal inside the network is represented by the pulse density (the number of “1” within a certain time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed on a memory in the nerve cell unit. Learning is realized by rewriting this pulse train. For learning, an error is calculated based on a given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the calculation of the change in the coupling coefficient are all performed by the logical operation of the pulse train of “0” and “1”. It is like that.

Hereinafter, this concept will be described. First,
Regarding signal processing by a digital logic circuit, signal processing in a forward process will be described. FIG. 2 shows a part corresponding to one neuron unit (neurocyte mimic element) 20, and the entire neural network is of a hierarchical type, for example, as shown in FIG. All inputs and outputs are
A value that is binarized to “1” and “0” and synchronized is used. The value (intensity) of the input signal y _i is represented by the pulse density, for example, by the number of states of “1” within a certain time as in the pulse train shown in FIG. That is, the example of FIG.
4/6, and “1” is 4
And “0” are two. At this time, it is desirable that the arrangement of “1” and “0” be random.

On the other hand, a coupling coefficient T _ij indicating the degree of coupling between the respective neuron units 20 is similarly expressed by a pulse density.
A pulse train of “0” and “1” is prepared on a memory in advance. The example in FIG. 5 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of “1” and “0” be random.

Then, the coupling coefficient pulse train is sequentially read out from the memory in accordance with the synchronous clock, and as shown in FIG. 2, the AND gate 21 takes the logical product with the input signal pulse train (y _i ＴT _ij ). This is an input to the nerve cell j. In the case of the above example, if the input signal is "10
"When entered as calls the pulse train from the memory in synchronization with this, by taking sequential AND, as shown in FIG. 6" 1101 101000 "is obtained, which is input y _i is the coupling coefficient T _ij And the pulse density becomes 2/6.

The pulse density of the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This is because the longer the signal sequence, the more “1”
The more random the arrangement of "" and "0", the closer the function to the product of numerical values. When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no more data to be read, it is sufficient to return to the beginning of the data and repeat the reading.

Since one nerve cell unit 20 has a large number of inputs, there are many "ANDs between input signals and coupling coefficients" described above, and the OR circuit 22 calculates the logical sum of them. Since the inputs are synchronized, for example, the first data is “101000” and the second data is “01000”.
In the case of "0", the OR of both is "111000". If this is calculated and output as multiple inputs (m) simultaneously, for example, it is as shown in Fig. 7. This is the sum of the analog calculation. Calculations and nonlinear functions (sigmoid functions)
Corresponds to the part.

In the case where the pulse density is low, the pulse density obtained by ORing them approximately matches the sum of the respective pulse densities. As the pulse density increases, the OR circuit 22
Since the output becomes increasingly saturated, the output does not match the sum of the pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and furthermore, it is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, there is an excitatory property and an inhibitory property in the connection. In the present embodiment, it is assumed that the connection between the neurons can have both. First, depending on whether the coupling coefficient is an excitatory coupling coefficient T _{ij (+)} or an inhibitory coupling coefficient T _{ij (−)} , each coupling is divided into an excitatory coupling group and an inhibitory coupling group.
Minute. Then, the input signal and A of the pulse train of each coupling coefficient
The OR of the ND outputs is calculated for each group. And
"1" is output only when the OR result of the excitatory connection group is "1" and the OR result of the inhibitory connection group is "0", and "0" is output otherwise. FIG. 8 shows this state.

When expressed by a logical expression, it is expressed by the following expressions (3) to (5).

[0044]

(Equation 3)

Further, the number of pulses of the signal _{xj as} a logical result is counted, and a pulse train having a pulse density corresponding to the counted value is generated and output. However, when the counting by the pulses in the output pulse train, may be corrected counting result of the number of pulses of the signal x _j. The logical product of such a pulse signal z _j and the signal x _j resulting from the above logic is obtained, and this is used as the final output signal y _j from the neuron.
And Note that, instead of the logical product of the pulse signals z _j and x _j , the final output signal y is calculated as a logical sum or another logical operation.
_j may be obtained. The output signal y _j at each time of the neuron by such an operation depends on the output at the previous time.

The network of the neuron unit 20 is
It is a hierarchical type (that is, FIG. 3). Then, if the entire network is synchronized, the calculation can be performed for each layer by the above-described function.

Next, a signal calculation process in learning (back propagation) will be described. Basically, an error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method c.

A. Error Signal in Last Layer In the final layer, an error signal in each neuron unit is calculated from the output signal and the teacher signal. Here, a desired output for the input at that time is given as a teacher signal in a pulse train. In general, when an error is represented by a numerical value, it can take both positive and negative values. However, since the pulse density cannot represent both at the same time, an error signal is represented by using two types of signals, a + component signal and a − component signal. Express. That is, the error signal of the j-th nerve cell unit is shown as in FIG. That is, the + component of the error signal is a pulse existing on the teacher signal side in the part (1, 0) or (0, 1) where the teacher signal pulse and the output pulse are different, while the-component is similarly output. This is the pulse present on the side. In other words, adding an error signal + pulse to the output pulse and removing the error signal-pulse results in a teacher pulse. That is, when these positive and negative error signals δ _{j (+)} and δ _{j (−)} are expressed by logical expressions, they are expressed by the expressions (6) and (7), respectively. Based on such an error signal pulse, the coupling coefficient is changed as described later.

[0049]

(Equation 4)

B. Error Signal in Intermediate Layer First, the above error signal is back-propagated to change not only the coupling coefficient between the final layer and the immediately preceding layer but also the coupling coefficient of the preceding layer. Therefore, it is necessary to calculate an error signal in each neuron unit in the intermediate layer. The error signal in each neuron unit in the next layer is collected in exactly the opposite way that the signal is propagated from the neuron unit in the middle layer to each neuron unit in the next layer. And uses it as its own error signal. This is because the above-described arithmetic expressions (3) to (5) in the nerve cell unit and FIG.
8 can be performed in the same manner as in the case shown in FIG. However, the difference from the above-described processing in the nerve cell unit is that while y is one signal, δ is positive,
This means that there are two signals as negative signals, and both signals need to be considered. Therefore, the coupling coefficient T
Needs to be divided into two cases depending on the sign of

First, the case of excitatory coupling will be described. In this case, for a neuron unit having an intermediate layer, the error signal + at the j-th neuron unit in the next layer (final layer in FIG. 3), the neuron unit and its own (the intermediate layer in FIG. 3) An AND of the coupling coefficient with the neuron unit with the suffix (δ _{j (+)} Ｔ T _ij ) is obtained for each neuron unit, and an OR of these is obtained ｛∪ (δ _{j (+)} {T _ij )}. This is set as the error signal + of this nerve cell unit. That is, the result is as shown in FIG.

Similarly, an AND between the error signal and the coupling coefficient in the neuron unit in the next layer is obtained, and an OR between them is obtained to obtain an error signal of the neuron unit. That is, the result is as shown in FIG.

Next, the case of inhibitory binding will be described. In this case, the AND of the error signal in the neuron unit of the next layer and the coupling coefficient between the neuron unit and the self is obtained, and further, the OR of these is obtained. This is set as the error signal + of this nerve cell unit. That is, the result is as shown in FIG.

Further, an AND between the error signal + and the coupling coefficient, which is one point ahead, and an OR between them are similarly obtained to obtain an error signal − of the nerve cell unit. That is, the result is as shown in FIG.

Further, an OR of the excitatory coupling error signal + and the inhibitory coupling error signal + of this neuron unit is taken as an error signal δ _{i (+)} of this unit. Similarly, the error signal of the excitatory connection − and the error signal of the inhibitory connection − are ORed, and this is used as the error signal δ of this unit.
_{i (-)} .

The above is summarized as shown in equation (8).

[0057]

(Equation 5)

Further, the same or different learning rates (learning constants) may be provided for the input error signals. This can be realized by thinning out the pulse train. For example, considering a counter-like concept, FIGS.
What is necessary is just to be as shown in FIG. In this example, at the learning rate η = 0.5, every other pulse train of the original signal is thinned out. However, even if the pulses of the original signal are not equally spaced, the original pulse train can be thinned out. 14 and 15,
When η = 0.5, every other pulse is thinned out, and η =
In the case of 0.33, every third pulse is left, and η = 0.6
In the case of 7, it indicates that every second pulse is thinned once.

In this way, the function of the learning rate is provided by thinning out the error signal. Such thinning of the error signal can be easily realized by performing a logical operation on the output of a commercially available counter or using a flip-flop. In particular, when a counter is used, the value of the learning constant η can be set arbitrarily and easily, so that the characteristics of the network can be controlled.

C. Method of changing each coupling coefficient based on the error signal Line to which the coupling coefficient to be changed belongs (see FIG. 3)
The output y _i from the previous neuron unit corresponding to
And the error signal δ _{j (+)} or δ _{j (-) of the} own neuron unit
(Δ _j ∩y _i ) (see FIGS. 16 and 17). Each of the two signals thus obtained is represented by ΔT
_{ij (+)} and ΔT _{ij (-)} .

Then, based on this ΔT _ij , a new T
_ij is obtained. Since T _ij is an absolute value component, cases are classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the component of ΔT _{ij (+)} is increased with respect to the original T _ij ,
Reduce the components of _{ij (-)} . That is, the result is as shown in FIG. Conversely, in the case of suppressiveness, the component of ΔT _{ij (+)} is reduced with respect to the original T _ij , and the component of ΔT _{ij (−)} is increased. That is, FIG.
As shown in FIG.

The above is summarized as in equation (9).

[0063]

(Equation 6)

The network is calculated based on the above learning rules.

Next, an actual circuit configuration based on the above algorithm will be described. FIG. 20 and FIG. 1 show examples of the circuit configuration. The configuration of the entire network is the same as that of FIG. FIG. 20 shows a circuit corresponding to a line (connection) in FIG. 2, and FIG. 1 shows a circle (each nerve cell unit 2) in FIG.
2 shows a circuit corresponding to (0). 20 and FIG. 1 can be implemented as a network as shown in FIG. 2 to realize a digital neural network capable of self-learning.

First, FIG. 20 will be described. In the figure, reference numeral 25 denotes an input signal to the nerve cell unit, which corresponds to FIG. The values of the coupling coefficients as shown in FIG. 5 are stored in the shift register 26 as a memory. The shift register 26 has an outlet 26a and an inlet 26b, but may have the same function as a normal shift register. For example, the shift register 26 may be a combination of a RAM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are A
An AND operation is performed by a logic circuit 28 having an ND gate 27 and performing the processing shown in FIG. The outputs of this logic circuit 28 must be grouped according to whether the coupling is excitatory or inhibitory.
It is more versatile to prepare 9 and 30 and switch between them to output. For this reason, in this embodiment, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31, and the switching is performed by the switching gate circuit 32 as the grouping means using the information. The switching gate circuit 32 includes two AND gates 32.
a, 32b and an inverter 32c interposed between one input
And

Further, as shown in FIG. 1, each input process (FIG. 7)
Circuit 33 having a plurality of OR gates
a and 33b are provided. Further, as shown in the figure, A which outputs an output "1" only when the excitatory connection group shown in FIG. 8 is "1" and the inhibitory connection group is "0"
A gate circuit 34 is provided as a logical operation means by the ND gate 34a and the inverter 34b.

Further, a counter 46 for counting the signal output from the gate circuit 34 is provided, and a pulse train generator 47 as a signal generating means for generating a pulse train having a density having a probability corresponding to the count value of the counter 46 is provided. Is provided. AND gate 48 is provided as a preparative Ri theory Riseki calculating means a logical product of the signal outputted signal outputted from the pulse train generator 47 and from the gate circuit 34. The signal from the AND gate 48 is used as the final output of the nerve cell unit 20. here,
The counter 46 may be a conventional counter (claim 1
Alternatively, an up-down counter that performs a subtraction process when a pulse is not output or at a certain interval may be used (the invention described in claim 2 ). Also, the counter 46
May be reset every data. Also, an OR gate (claim) is used instead of the AND gate 48.
3,4 or using equivalent) in the Hare theory Liwa computing means are in the invention described, the gate circuit (claim performing another logical operation
( Equivalent to another logical operation means according to the invention described in the fifth and sixth aspects ).

The counter 46 is an up / down counter that inputs and counts a signal output from the gate circuit 34 and a signal output from the pulse train generator 47 or a signal obtained by performing a logical operation on these two signals. It may be. Further, instead of the counter 46, a circuit combining a plurality of counters, an adding circuit, and a memory may be used. Also, instead of the counter 46, the gate circuit 34
A counter that counts a signal output from the counter, a counter that counts a signal obtained by performing a logical operation on a signal output from the gate circuit 34 and a signal output from the pulse train generator 47, and outputs the values of these two counters. A circuit in which an addition circuit for adding and a memory are combined may be used.

Next, the learning process will be described. Error signals 35 and 36 are input to the circuit shown in FIG. These error signals 35 and 36 are respectively collected by a plurality of gate circuits 37 having an OR gate configuration (the processing of the equation (8)), and then processed by the frequency dividing circuit 38 corresponding to the learning rate (see FIGS. 14 and 15). Processing), the error signal 3 for the immediately preceding layer
It is output as 9, 40. Here, the processing as shown in FIGS. 10 to 13 for calculating the error signal in the intermediate layer is performed by the gate circuit 4 having the AND gate configuration shown in FIG.
1 and error signals 42 and 43 to be output to the neuron unit of the previous layer are obtained according to + and-.

As described above, it is necessary to divide the case depending on whether the connection is the excitatory connection or the inhibitory connection. In this case, the classification is performed by the information stored in the memory 31 indicating whether the connection is excitatory or inhibitory.
In response to the + and − signals 39 and 40 of the error signal, AND,
This is performed by a gate circuit 44 having an OR gate configuration.

FIGS. 16 to 1 for calculating a new coupling coefficient based on such error signals 39 and 40.
9, an AND gate, an inverter, and an O
A coupling coefficient variable circuit 45 having an R gate configuration is provided and connected to the data inlet 26b of each shift register 26. Thereby, the update of the coupling coefficient stored in the shift register 26 is rewritten. In the case of the gate circuit 45 as well, it is necessary to divide the cases depending on the excitability and suppression of the connection. This is performed by the gate circuit 44.

The configuration of the present invention is not limited to the above configuration example, but may be any configuration having the same function.
Further, all or a part of the configuration may be realized by software without configuring the entire hardware.

By the way, according to the above description, in order to make it possible to learn the non-linear relationship between the input value and the output value in the one-input one-output neural network, the counted values are counted while counting the number of pulses in the output pulse train. The result of performing a logical operation on the generated pulse and the output pulse is output again as a signal, so that a pulse train is output simultaneously while outputting a pulse train as compared with the corresponding method described above. Since the pulse train can be changed, there is an advantage that the signal processing speed is high. Here, the contents described above are expanded and embodied, and further, the association between the value of counting the number of pulses in the output pulse train and the probability of generating a pulse is referred to.

First, in order to avoid the conventional disadvantage that the nonlinear relationship between the input value “X” and the output value “Y” cannot be learned in the pulse density type neural network, the relationship between the input and output of the neuron unit is nonlinear. FIG. 21 shows an example of a basic configuration for realizing the idea based on the above-described embodiment. The circuit 51 shown in FIG. 21 is disposed at a position as shown in the nerve cell unit 20, as shown in FIG. In the case of FIG. 1,
It is provided at a portion indicated by reference numerals 46 to 48. This circuit 51
To, first, corresponds to a counter 46 is provided with counter 52 for counting the nu Ron output. A portion corresponding to the signal generating means 47 is constituted by a random number generator 53 and a comparator 54, and an OR gate is provided on the output side as a configuration corresponding to the invention according to claim 3 or 4. (logical OR operation calculating means) 55 is configured provided.

As a result, first, the counter 52 counts the bits of the value “1” in the output bit string from the neuron. The output value of the counter 52 and the output value of the random number generator 53 are compared by a comparator 54. If the output value of the random number generator 53 is smaller than the count value of the counter 52, the comparator 54
Output a bit of value "1", otherwise output a bit of value "0". The logical sum of the output bit from the comparator 54 and the output bit from the neuron is ORed.
The result is taken as a new output bit from the neuron by the gate 55. The counter 52 is a neural cell unit 2
0 is cleared to 0 each time a bit string for one data is processed.

Here, as the random number generator 53, an LFSR (linear feedback shift register) is usually used.
Is used. L forming such a random number generator 53
The FSR has a structure in which a plurality of predetermined output units of a plurality of registers sequentially connected in a linear manner are connected in a feedback manner to the first input unit via an exclusive OR, and the bits set in the registers are read out. By combining exclusive ORs when feeding back from a plurality of locations, pseudo-random numbers are generated by bit strings having extremely long periods. Then, a random number generator 5 composed of such an LFSR
In the case of No. 3, if the number of registers, the position of the feedback connection, and the set initial value are the same, the output bit string is also the same, so that a reproducible random number can be generated.

[0078] For example, as illustrated in FIG. 23, the case of forming the random number generator 53 _{1-53 4} 7 stages of LFSR, 7 two registers 58 _1-5 this is sequentially connected to the linear
8 _7, a predetermined intermediate output portion 59 and a terminal output portion 60 are feedback-connected to the first input portion 62 via an exclusive OR 61, and the same reference clock 6 is connected to each register 58.
3 are connected in parallel. In such a configuration, in the random number generator 53, if the bit stored in the register 58 ₁ at the time t is expressed as At, the register 58 _{(1 + i) at} the time (t + j) has i The bit A _{(t-i + j)} stored in the register 58 ₁ before the time has moved, and the time ( _{t−i + j)}
The bit stored in the register 58 _{(1 + i)} in ₁₎ is A _(ti-1) . For example, as illustrated in FIG. (C), the random number generator 53 ₃ provided intermediate output section 59 to the register 58 ₄ of the fourth stage, the bit A _t of the first stage of the register 58 in _1, the bits of the register 58 _4, 58 in ₇ of the fourth stage and seventh stage at time (t-1) has a synthesized value exclusive OR 61. Here, synthesis of bits by exclusive OR 61 is equal to the addition in "modulo2", bit A _t of the first stage of the register 58 in _{1, A t = A (t-} 4) + A (t- ₇₎ (mod2) ………… (10)

Here, the set value in which predetermined bits are input to each register 58 of the random number generator 53 composed of seven stages of LFSRs as described above is based on whether the bit in each register 58 is “0”. Since it is one of “1”, there are 2 ⁷ = 128. However, if all the seven set values are “0”, all the set values remain “0” even if the bit forwarding is repeated. If at least one “1” is included in the seven set values, the set values change to one of the above combinations at a predetermined cycle by repeating bit forwarding. Here, when the bit A _t in a predetermined register 58 of the random number generator 53 is defined by the equation (10), the bit string of A _t may be a pseudo-random sequence of period (2 ⁷ -1) It is known. That is, the random number generator 53 composed of the LFSR shown in FIG.
^Since the bit string of 7-1) can be generated, the random number generator 53 outputs an integer value of 1 to 127 by reading this bit string as a 7-digit binary number. When the set value of the random number generator 53 composed of such an LFSR is read as a binary number, both a method of setting the head to the lowest and a method of setting the end to the lowest are implemented. Now, the description will be made with the top being the lowest. However, in this case, a method in which either the head or the end of the random number generator 52 is set to the lowest order is established.

Here, c _i (= 1, 2,... P) is set to “0”.
Or an integer of “1” (where c _p = 1), this c _i
Recurrence formula showing a A _{t some, A t = c 1 A (} t-1) + c 2 A (t-2) ... + c p A (tp) (mod2) ...... (11) , and this recurrence formula Is a formula (12).

[0081]

(Equation 7)

Here, the random number sequence generated by the recurrence formula (11) as described above has a period equal to or less than the length “2 to the power of p−1”. Polynomials are particularly called primitive polynomials. Then, a bit string of A _t of the period of such generated by a primitive polynomial "2 of p squared -1", p order linear maximum period sequence (Maximum
-Length Linearly Recurrence Sequence)
It is called a series. For example, bit string A _t generated from equation (10) is a 7-order M sequence corresponding to the primitive polynomial f (x) = 1 + x 4 + x 7. In addition, 7-stage LFSR
In the case of the random number generator 53 composed of the following, since the generation of the 7th-order M-sequence bit string is limited to the four types illustrated in FIG. 23, the primitive polynomial, recurrence formula, and the like of the random number generator 53 are Examples are given below.

.. FIG (a) to the illustrated random number generator 53 ₁ primitive polynomial f (x) = 1 + x + x 7 recurrence formula _{A t = A (t-1} ) + A (t-7) (mod2) bit string of M series tables 1 ( Example in a) Generated random number sequence Example in Table 2 (a). Bit string table of FIG. (B) exemplary random number generator 53 ₂ primitive polynomial f (x) = 1 + x 3 + x 7 recurrence formula _{A t = A (t-3} ) + A (t-7) (mod2) M -sequence 1 (b) Example of generated random number sequence Table 2 (b). Bit string table of FIG. (C) to the illustrated random number generator 53 ₃ primitive polynomial f (x) = 1 + x 4 + x 7 recurrence formula _{A t = A (t-4} ) + A (t-7) (mod2) M -sequence 1 (c) Example of generated random number sequence Table 2 (c). Bit string table of the (d) of FIG exemplary random number generator 53 ₄ primitive polynomial f (x) = 1 + x 6 + x 7 recurrence formula _{A t = A (t-6} ) + A (t-7) (mod2) M -sequence Example in 1 (d) Generated random number sequence Example in Table 2 (d)

[0084]

[Table 1]

[0085]

[Table 2]

That is, in the random number generator 53 composed of such an LFSR, since the cycle of the random number generated by the bit string of the M sequence is the maximum, the random number generated is extremely good. Here, such an M-sequence random number generator 53
Since it is known in advance that the cycle of the output bit string is the maximum, it is possible to generate a signal having an irregular pulse position by modulating this with the pulse density or the number of pulses. For example, in the seventh-order random number generator 53 that generates 127 random numbers as described above, the pulse density is 1
When a signal of 0/127 is requested, the generated random number is 1
If a pulse is output if the value is -10 and if no pulse is output if the random number is 11 to 127, this signal has an irregular pulse position and a density of 10/127. In this way, since the random number generator 53 generates the M-sequence bit string, the generation cycle of the pseudo random numbers can be maximized.

Now, the point that the output of the nerve cell unit 20 becomes non-linear due to the circuit 51 shown in FIG. 21 will be described. Here, an example will be described in which one data is composed of 127 bits, and the output of the nerve cell unit 20 is a pulse density 1/127 when the circuit shown in FIG. 21 is not provided. First, the counter 52 is cleared to zero. The position where the bit having the value "1" exists in the neuron output consisting of 127 bits is completely random. If the first bit in the neuron output is "1", the first bit of the new neuron output from the OR gate 55 becomes "1", and the value of the counter 52 changes to "1". Although the remaining 126 bits in the neuron output are all “0”, the probability that “1” is output from the LFSR random number generator 53 during 126 clocks when the remaining 126 bits are output is 126 /
Since there are 127, the probability that another bit of value “1” is mixed in the new neuron output is also 126/127.

However, if the bit of value "1" exists at the second position in the neuron output, the number of bits output after the value of the counter 52 changes to "1" is 125 Therefore, the probability that another bit of value “1” is mixed in the new neuron output is 126/127.
Instead of 125/127. Further, assuming that the bit having the value "1" exists at the last position in the neuron output, the probability that another bit having the value "1" will be mixed in the new neuron output is zero.

As described above, the new neuron output increases up to twice as large as the Niuron output when the circuit 51 is not added, although it differs depending on where the pulse exists in the neuron output. This means that the circuit 51
The pulse density of the neuron output without adding
The same applies to the case where the value is larger than 127. However, since the pulse density does not exceed 1, when the neuron output increases, the magnification of the new neuron output with respect to the original neuron output gradually decreases.

In FIG. 22, the relationship between the neuron output without the circuit 51 and the new neuron output with the circuit 51 is illustrated in FIG. That is, even if the neuron output without the circuit 51 has the same value, the value of the new neuron output is different if the pulse position is different. Therefore, FIG. Is shown.

From the relationship shown in FIG. 24, the relationship between the input value and the output value of the nerve cell unit 20 is a linear relationship as shown by a broken line when there is no circuit 51, as shown in FIG. If there is, it can be seen that there is a non-linear relationship as shown by the solid line. In this way, by incorporating the circuit 51 shown in FIG. 21 at the position shown in FIG. 22, the relationship between the input value and the output value of the nerve cell unit 20 can be changed nonlinearly. The relationship between the input value and the output value is
This is a preferable form for adding a learning ability to a neural network, such as a sudden rise at a small input value and a saturation at a large input value.

If the bit of the value "1" output from the comparator 54 overlaps the bit of the original neuron output value "1", the bit is the bit of the original neuron output value "1". Useless to increase the number of. The circuit 56 shown in FIG. 26 improves this point, and corresponds to the eighth aspect of the present invention. In the circuit shown in FIG. 26, the AND gate 57 calculates the logical product of the bit output from the comparator 54 and the bit of the neuron output. If the result of the logical product by the AND gate 57 is true, the value of the counter 64 increases by "1". The adder 65 counts a neuron output by a counter 66 (46).
Or 52) and the count value of the counter 64, and the sum is stored in the memory 67 for comparison by the comparator 54. Therefore, when the bit of the value “1” output from the comparator 54 overlaps with the bit of the original neuron output value “1”, it passes through the counter 64 and the adder 65 (which constitutes a means for increasing information). The value of the memory 67 increases by “1”, and a bit of the value “1” may be set later in a new neuron output output from the OR gate 55 instead of the bit of the value “1” which has been superfluous and has become useless. Sex increases by one. In the case where the number of bits of the value “1” in the original neuron output is small and the number of bits of the value “1” output from the comparator 54 and the bit of the original neuron output “1” are small, or conversely, Since the number of bits of the value "1" in the original neuron output is large and close to the number of bits in one data, the bit of the original neuron output value "1" overlaps with the bit of the new neuron output. Except in the case where there are not many bit positions that do not remain, when the circuit 56 shown in FIG. 26 is used, the number of bits of the value “1” increases in a new neuron output as compared with the case where the circuit 51 is used. .

The new neuron output value is compared with the original neuron output value by the circuit 5 shown in FIG.
If it is desired to increase the non-linearity more greatly than in the case of No. 1, the AND gate 57 and the counter 64 are omitted as shown in FIG. 27 instead of FIG. A circuit 69 provided with a plurality of (two in the illustrated example) 66b,...
7 may be stored. According to such a circuit 69 corresponding to the invention of claim 7, the value of the new neuron output increases up to three times the value of the original neuron output.

By the way, in the circuit 51 shown in FIG. 21, it is assumed that the bit of the initial value "1" of the neuron output causes the counter 52 to become "1". Then, it is assumed that a bit of a value “0” continues for a while in the neuron output until the next bit of the value “1” of the neuron output comes. Then, if the value of the random number generator 53 having the LFSR configuration becomes “1” during this period, one bit of the value “1” is generated in a new neuron output. Thereafter, it is assumed that the next bit “1” of the neuron output comes and the value of the counter 52 becomes “2”. Then, when the value of the random number generator 53 becomes “1” or “2”, a bit of the value “1” is generated in a new neuron output. In this case, assuming that the value of the random number generator 53 can be “1” or “2”, two bits of the value “1” are generated, and the bit of the value “1” generated earlier is generated. , A total of three are generated.

However, since the random number generator 53 having the LFSR configuration never takes the same value twice in one cycle, if the LFSR having the same cycle as the pulse length of one data is used, the counter 52 After the value becomes "2", the value of the LFSR does not become "1" until the neuron output for one data has been output, and therefore the number of bits of the value "1" is 2 in total.
Only up to pieces are generated.

On the other hand, when an LFSR having a cycle different from the pulse length of one data is used as the random number generator 53 or a circuit other than the LFSR is used, the value of the counter 52 becomes “1”.
At this time, the random number generator 53 increases the number of bits of the value “1” by one in the output of a new neuron, and then increases the number of bits by two after the value of the counter 52 becomes “2”. obtain. In this way, 1 is set as the random number generator 53.
When a circuit other than the LFSR having the same period as the pulse length of the data is used, the number of bits of the value “1” in the new neuron output is increased by the LF having the same period as the pulse length of one data.
It becomes sharper than when SR is used. This may be undesirable because the nonlinearity is too strong depending on the problem of applying the neural network.

In such a case, in order to suppress the increase in the number of bits of the value “1” in the new neuron output, FIG.
A circuit 70 as shown in FIG. The circuit 70 corresponding to the ninth aspect of the present invention includes the counter 5 in the circuit 51.
2, an up / down (U / D) counter 70 is provided.
It is a digit . As a result, the value “1” is output from the comparator 54.
The value of the U / D counter 70 is decremented by "1" every time this bit is output, so that the number of bits of the value "1" in the new neuron output is suppressed from increasing.

However, in the case of the configuration of FIG. 28, the value of the bit of the original neuron output and the value of the output bit from the comparator 54 are both "1", and the output bit from the comparator 54 is Even if it does not help to increase the number of bits of the value “1” in the neuron output, the value of the U / D counter 70 is reduced by “1”. In consideration of this point, the configuration may be as shown in FIG. In this circuit 72, which corresponds to the invention of claim 1 0, wherein, the AND logical product of the results without the input directly to the U / D counter 71 the output of the comparator 54, the neuron output denied by NOT gate 73
The data is input at the gate 74.

Thus, only when the bit of the original neuron output is “0” and the value of the output bit of the comparator 54 is “1”, that is, the output bit from the comparator 54 becomes the new neuron output Only when it is used to increase the bit of the value “1” in the middle, the value of the U / D counter 71 is set to “1”.
It is intended to be reduced.

[0100] Incidentally, FIG. 21, the circuit 51,56,69,70,72 shown by FIGS. 26 through 29 (corresponding to the invention of claim 1 1 wherein) which may be bound to each other in series. FIG. 30 shows an example in which two circuits 51 shown in FIG. 21 are connected in series. According to this, three or more circuits may be connected in series, and similarly, the circuits 56, 69, 70, and 72 may be connected. Further, the invention is not limited to the series connection of the same type of circuit as shown in FIG.
6, etc., may be combined. In any case, by superposing two or more circuits, the nonlinearity of the new neuron output can be enhanced.

For example, when the relationship between the original neuron output and the new neuron output when only one circuit 51 shown in FIG. 21 is used, the relationship shown by a broken line in FIG. 31 is obtained. The relationship between the original neuron output and the new neuron output when two circuits 51 are connected in series as shown in FIG. 30 is indicated by a solid line in FIG.
Also, by devising the number and types of series-coupled circuits,
The degree of non-linearity of the output value from the nerve cell unit 20 with respect to the input value can be adjusted to be strong or weak over a wide range.

[0102]

According to the present invention, in a simple digital logic configuration for processing a signal represented by a pulse train, an AND operation of an input signal and a coupling coefficient, an OR operation in groups, and a result of these OR operations are described. Since the signal obtained as a result of the logical operation is not output as it is, but is output after changing the pulse density by the pulse density changing means, the output signal pulse density in the forward process is compared with the input signal pulse density. The network can be changed in a non-linear manner, and it is possible to cope with a highly nonlinear system, when it is desired to make the input data time-dependent, or when the input signal value is finely quantized. it is possible to increase the flexibility and versatility in the case of, or
In addition, such a pulse density changing means uses a counter or the like.
It is specifically shown that the
Ru can be secured.

[0103]

[Brief description of the drawings]

FIG. 1 is a logic circuit diagram showing one embodiment of the present invention.

FIG. 2 is a logic circuit diagram.

FIG. 3 is a schematic diagram illustrating a network configuration example.

FIG. 4 is a timing chart showing a logical operation example.

FIG. 5 is a timing chart showing a logical operation example.

FIG. 6 is a timing chart showing a logical operation example.

FIG. 7 is a timing chart showing a logical operation example.

FIG. 8 is a timing chart showing a logical operation example.

FIG. 9 is a timing chart showing a logical operation example.

FIG. 10 is a timing chart showing a logical operation example.

FIG. 11 is a timing chart showing a logical operation example.

FIG. 12 is a timing chart showing a logical operation example.

FIG. 13 is a timing chart showing a logical operation example.

FIG. 14 is a timing chart showing a logical operation example.

FIG. 15 is a timing chart showing a logical operation example.

FIG. 16 is a timing chart showing a logical operation example.

FIG. 17 is a timing chart showing a logical operation example.

FIG. 18 is a timing chart showing a logical operation example.

FIG. 19 is a timing chart showing a logical operation example.

FIG. 20 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 21 is a circuit diagram showing a configuration example of each unit.

FIG. 22 is a schematic diagram showing the arrangement location.

FIG. 23 is a block diagram showing an LFSR scheme for random number generation.

FIG. 24 is a characteristic diagram corresponding to the presence or absence of the circuit shown in FIG. 22;

FIG. 25 is an input value-output value characteristic diagram.

FIG. 26 is a circuit diagram showing a modification.

FIG. 27 is a circuit diagram showing another modification.

FIG. 28 is a circuit diagram showing still another modification.

FIG. 29 is a circuit diagram showing still another modified example.

FIG. 30 is a circuit diagram showing an example of series connection.

FIG. 31 is a characteristic diagram corresponding to the number of connection circuits.

FIG. 32 is a conceptual diagram showing one unit configuration showing a conventional example.

FIG. 33 is a conceptual diagram of the neural network configuration.

FIG. 34 is a graph showing a sigmoid function.

FIG. 35 is a circuit diagram showing a specific configuration of one unit.

FIG. 36 is a block diagram illustrating a digital configuration example.

FIG. 37 is a partial circuit diagram thereof.

FIG. 38 is a circuit diagram of a different part.

FIG. 39 is a schematic diagram showing a neuron unit of the proposed pulse density type neural network.

FIG. 40 is a schematic diagram showing an example of the signal bit processing.

FIG. 41 is an explanatory diagram for explaining a linear relationship.

FIG. 42 is a linear characteristic diagram thereof.

[Explanation of symbols]

 Reference Signs List 20 signal processing means 27 logical product calculating means 32 grouping means 33 logical sum calculating means 34 logical calculating means 45 coupling coefficient variable means 46 counter 47 signal generating means 48 another logical product calculating means 52 counter 53 random number generating means 54 comparing means 55 Another OR operation means

──────────────────────────────────────────────────続 き Continuing on the front page (72) Hironori Mifune 1-3-6 Nakamagome, Ota-ku, Tokyo Inside Ricoh Co., Ltd. (72) Shinichi Suzuki 1-3-6 Nakamagome, Ota-ku, Tokyo (56) References JP-A-3-256184 (JP, A) Takayuki Yamayama et al., “Integration of variable synaptic neural circuit by pulse transmission and its measurement”, IEICE technical report. Research Report, Japan, The Institute of Electronics, Information and Communication Engineers, January 18, 1992, Vol. 91, No. 414 (NC91-82-97), pp. 1-5 Yoshikazu Kondo et al., "Hardware of Artificial Neural Network Using Fluctuation Pulse Sequence", IEICE Technical Report, Japan, IEICE, Japan, 1992 Vol. 91, No. 413 (NC91-65-81), pp. 117-122 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7/60 G11C 11/54 H03K 19/00 H03K 19/173 JST file (JOIS) CSDB ( (Japan Patent Office)

Claims (11)

(57) [Claims] 1. An AND operation means for performing an AND operation of an input signal expressed by a pulse train and a coupling coefficient expressed by a pulse train, and an operation result by the AND operation means is expressed by excitability.
Grouping means for dividing into a connection group and an inhibitory connection group; a logical sum operation means for calculating a logical sum of the operation results in each group; and an excitatory connection as an operation result of these logical operation means Group
Output only when “1” and the inhibitory binding group is “0”
Logic operation means for performing a logical operation so as to output "1"; pulse density changing means for changing the pulse density of a pulse train output from the logic operation means; Signal processing means having coupling coefficient changing means for changing the coupling coefficient , wherein the pulse density changing means operates the logical operation
And a counter for counting the results.
Signal that generates a signal represented by a pulse train with the same probability
Generating means, a signal generated by the signal generating means,
AND operation for calculating the logical AND with the operation result of the logical operation means
Signal processing apparatus characterized by comprising further a unit. 2. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means, or in some time interval counted pulse is not output an operation result of the logic operation means
An up / down counter for performing subtraction processing, a signal generating means for generating a signal represented by a pulse train having a probability according to the counting result of the up / down counter, a signal generated by the signal generating means, an operation result of the logical operation means, signal processing apparatus you <br/> wherein become more that a logical product calculating means for calculating a logical product of. 3. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means includes a counter for counting an operation result of the logic operation means, a signal represented by a pulse train of a probability in accordance with the counting result of the counter a signal generating means for generating, signal processing apparatus you wherein become more that a logical OR operation means for calculating a logical sum of the calculation result of the logic operation means and generating signals by the signal generating means. 4. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means, or in some time interval counted pulse is not output an operation result of the logic operation means
An up / down counter for performing subtraction processing, a signal generating means for generating a signal represented by a pulse train having a probability according to the counting result of the up / down counter, a signal generated by the signal generating means, an operation result of the logical operation means, signal processing apparatus you <br/> wherein become more that a logical OR operation means for calculating a logical sum of. 5. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means includes a counter for counting an operation result of the logic operation means, a signal represented by a pulse train of a probability in accordance with the counting result of the counter Signal generating means for generating a signal, and another logical operation means for performing a logical operation other than a logical product and a logical sum of a signal generated by the signal generating means and an operation result of the logical operation means. > signal processing apparatus you. 6. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means, or in some time interval counted pulse is not output an operation result of the logic operation means
An up / down counter for performing subtraction processing, a signal generating means for generating a signal represented by a pulse train having a probability according to the counting result of the up / down counter, a signal generated by the signal generating means, an operation result of the logical operation means, Theory of
Signal processing apparatus you wherein become more that another logical operation means for performing Riseki and logical operations other than logical sum. 7. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means includes a plurality of counters for counting the operation result of the logic operation means and adder means for adding the count results of the counters, integer Or a random number generating means for randomly generating a real value, a comparing means for comparing a result of addition by the adding means with a random number value generated by the random number generating means, and outputting a pulse in accordance with the comparison result; calculation result signal processing apparatus it wherein become more that a logical sum operation means for calculating a logical sum as a new output signal with an output signal from said comparing means computing means. 8. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means, and Luca counter to count the operation results of the logic operation means, a random number generating means for generating a random integer value or real value, if the random number value by comparing the random number value generated by counting result and the random number generating unit by the counter the counting result than "1", the following
A comparison means for outputting a pulse which is otherwise "0" ;
Of the operation result of the logical operation means and the output of the comparison means
When a logical product is calculated and the logical calculating means and the comparing means simultaneously output pulses of "1" , the logical product result is obtained.
Is added to the result of counting by the counter, thereby increasing the probability that the comparing means outputs a pulse of "1" . signal processing apparatus you <br/> wherein become more that a logical sum operation means for calculating a logical sum as a new output signal of the output signal. 9. An input signal and a pulse represented by a pulse train.
AND operation to calculate the logical AND with the coupling coefficient represented by the column
Calculation means and the result of the logical product
Divided into join groups and inhibitory join groups
Grouping means, and the operation result within each group.
OR operation means for calculating the OR of the results
The excitatory connection group is calculated as the result of the sum operation means.
Output only when “1” and the inhibitory binding group is “0”
A logical operation means for performing a logical operation to output “1”;
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means includes a counter for counting an operation result of the logic operation means, a random number generating means for generating a random integer value or real value, said counter Comparing means for comparing the result of counting with the random number value generated by the random number generating means and outputting a pulse in accordance with the result of the comparison; and counting the result of counting by the counting means in accordance with the number of pulses output by the comparing means. means for reducing, said logical operation means of the operation result and signal processing apparatus you wherein become more that a logical sum operation means for calculating a logical sum as a new output signal with an output signal from said comparison means . 10. An input signal and a pulse represented by a pulse train.
AND that calculates the logical AND with the coupling coefficient represented by the data sequence
Excited by the calculation means and the calculation result by this logical product calculation means
Sex group and inhibitory group.
Grouping means, and the operation within each group
OR operation means for operating the OR of the result,
The excitatory connection group is calculated by the
Output only when “1” and the inhibitory binding group is “0”
And facilities to logical operation means a logical operation to produce a "1", this
The pulse train output from the logical operation means
Pulse density changing means for changing the pulse density and pulse train
Changing the coupling coefficient based on the expressed error signal
A signal processing means having a coupling coefficient varying means provided, the pulse density changing means includes a counter for counting an operation result of the logic operation means, a random number generating means for generating a random integer value or real value, said counter Comparing means for comparing the result of counting with the random number value generated by the random number generating means and outputting a pulse according to the comparison result; the comparing means outputting a pulse; and the logical operation means not outputting a pulse. Sometimes, a means for reducing the counting result by the counting means and a logical sum calculating means for calculating a logical sum of the calculation result of the logical calculating means and the output signal from the comparing means to obtain a new output signal. Characteristic signal processing device. 11. The method of claim 7, characterized in that a plurality connecting means to increase pulse in the output signal, 8, 9 or 1
0. The signal processing device according to 0 .