JPH05197706A - Signal processor - Google Patents

Abstract

PURPOSE:To improve the generality of this signal processor by non-linearly changing the density of output signal pulses in a forward process against that of input signal pulses in simple digital logic constitution for processing a signal expressed by a pulse string. CONSTITUTION:A signal processing means has an AND operation means for operating AND between an input signal expressed by a pulse string and a connection coefficient expressed by a pulse string, a grouping means for grouping the operation result of the AND operation means into plural groups, OR operation means 33a, 33b for operating OR between operation results in each group, a logical operation means 34 for executing the logical operation of operation results from the means 33a, 33b, pulse density changing means 47, 48 for changing the pulse density of a pulse string outputted from the means 34, and a connection coefficiency changing means for changing the connection coefficiency based upon an error signal expressed by pulse density.

Description

Detailed Description of the Invention

[0001]

INDUSTRIAL APPLICABILITY The present invention is applicable to control of various movements such as image and voice recognition, position control of robots, temperature control of air conditioners, orbit control of rockets, etc. The present invention relates to a signal processing device such as a neural computer.

[0002]

2. Description of the Related Art The function of a nerve cell (neuron), which is a basic unit of information processing of a living body, is mimicked, and further, this "nerve cell mimicking element" (nerve cell unit) is connected to a network to process information in parallel. The aim is a so-called neural network. Although it is easy to perform character recognition, associative memory, motion control, etc. in a living body, there are many things that conventional Neumann computers cannot easily achieve. There have been many attempts to solve these problems by imitating a neural system of a living body, in particular, a function peculiar to the living body, that is, parallel processing, self-learning, etc. by a neural network.

First, a conventional neural network model will be described. FIG. 32 is a diagram showing a certain nerve cell unit A, and FIG. 33 is a network thereof. A ₁ , A ₂ , and A ₃ each represent a nerve cell unit. One neuronal cell unit is coupled to a number of other neuronal cell units and processes the signals received from them to produce an output. In the case of FIG. 33, the network is hierarchical, and the nerve cell unit A ₂ receives a signal from the nerve cell unit A _{1 in the} previous (left) layer and the nerve cell unit A in the next (right) layer. Output to ₃ .

A more detailed description will be given. First, in the nerve cell unit A of FIG. 32, the degree of coupling between another nerve cell unit and its own unit is called a coupling coefficient, and the coupling coefficient of the i-th nerve cell unit and the j-th nerve cell unit. The coupling coefficient is generally represented by T _ij . For the coupling, excitatory coupling in which the larger the signal from the other unit (the unit that sends a signal to the own unit) is, the larger the own unit output is, and the larger the signal from the other unit is, the own unit output Is an inhibitory bond, where T _ij > 0 represents an excitatory bond, and T _ij <0 represents an inhibitory bond. Now, let's say that our nerve cell unit is the jth unit, and the output of the ith nerve cell unit is y.
_{If it is i} , T _ij y _{i obtained} by multiplying this by the coupling coefficient T _ij becomes the input to the own unit. As described above, since one nerve cell unit is connected to many nerve cell units, ΣT _ij y _{i, which} is the result of adding T _ij y _i for these units, is It becomes an input to the unit. This is called the internal potential and is represented by u _j .

[0005]

[Equation 1]

Next, a threshold value is added to this input (internal potential) to perform non-linear processing, and the result is output from the nerve cell unit. The function used at this time is called a nerve cell response function, and the sigmoid function as shown in equation (2) and FIG. 34 is used as the nonlinear function.

[0007]

[Equation 2]

When such a nerve cell unit is constructed in a network as shown in FIG. 33, each coupling coefficient T
By providing _ij and sequentially calculating equations (1) and (2), parallel processing of information becomes possible and a final output is obtained.

In such a hierarchical neural network, a desired neural network is constructed by performing learning such that the coupling coefficient T _ij is updated so that a desired result is output for a certain input. .. The most widely used such learning method is the error back-propagation method, so-called back-propagation method.

An example of such a network realized by an electric circuit is shown in FIG. This is disclosed in Japanese Patent Laid-Open No. 62-295188, and basically, a plurality of amplifiers 1 having an S-shaped transfer function, and the output of each amplifier 1 is input to the input of the amplifier of another layer. A resistive feedback network 2 is provided which is connected as shown by the dashed line. The input side of each amplifier 1 has a C connected by a grounded capacitor and a grounded resistor.
The R time constant circuits 3 are individually connected. Then, the input current _{I 1, I 2, ~,} I n is supplied to the input of the amplifier 1, the output is obtained from a set of these amplifiers 1 output voltage.

Here, the signal strength of the input and output to the network is represented by a voltage, and the strength of the coupling between the nerve cell units is the resistance 4 (in the resistive feedback network 2) connecting the input / output lines between the cells. Is represented by the resistance value of each of the amplifiers 1, and the nerve cell response function is represented by the transfer function of each amplifier 1. That is, in FIG. 35, the plurality of amplifiers 1 have an inverting output and a non-inverting output, and the input of each amplifier 1 has a CR time constant circuit 3 as an input current supply means. The feedback network 2 connects each output of the amplifier 3 to the input by a resistor 4 (T _ij ) having a value of 1 or a second value selected in advance. Resistor 4 represents the transconductance between the i-th amplifier output and the j-th amplifier input and is chosen to create multiple minima that the network will balance to minimize the energy function with multiple minima. I am trying to In addition, the coupling between nerve cells has excitability and inhibitory property, and is mathematically represented by the sign of the coupling coefficient, but it is difficult to realize the sign with a constant on the circuit. Then, by dividing the output of the amplifier 1 into two and inverting one output, positive and negative 2
It is realized by generating two signals and selecting them appropriately. Further, an amplifier is used as the one corresponding to the sigmoid function shown in FIG.

On the other hand, an example in which a neural network is realized by a digital circuit will be described with reference to FIGS. 36 to 38. FIG. 36 shows a circuit configuration of a single nerve cell, in which each synapse circuit 6 is connected to a cell body circuit 8 via a dendrite circuit 7. FIG. 37 shows a configuration example of the synapse circuit 6 in which the input pulse f is multiplied by the magnification a via the coefficient circuit 9.
A rate multiplier 10 to which a value obtained by multiplying (a multiplication factor of 1 or 2 by the feedback signal) is input is provided, and the rate multiplier 10 is connected to a synapse weight register 11 that stores a weighting value w. . 38 shows a configuration example of the cell body circuit 8, and the control circuit 1
2, an up / down counter 13, a rate multiplier 14 and a gate 15 are sequentially connected, and an up / down memory 16 is further provided.

In this case, the input and output of the nerve cell unit is represented by a pulse train, and the pulse density represents the amount of signal.
The coupling coefficient is represented by a binary number and stored in the memory 16. By inputting the input signal into the clock of the rate multiplier 14 and inputting the coupling coefficient into the rate value,
The pulse density of the input signal is reduced according to the rate value. This is the T _{ij of the} backpropagation model equation.
It corresponds to the part of y _i . Next, the Σ portion of ΣT _ij y _i is realized by the OR circuit shown by the dendrite circuit 7. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance and OR is taken for each group. An output is obtained by inputting these two outputs to the up side and down side of the counter 13 and counting. This output is 2
Since it is a decimal number, the rate multiplier 14 is used again to convert it into a pulse density. A neural network can be realized by making this unit a network.

[0014]

However, the former analog circuit system has the following problems. The signal strength is represented by an analog value such as electric potential or current,
When the internal calculation is also performed in an analog manner, its value changes due to temperature characteristics, drift immediately after power-on, and the like. Since it is a network, a large number of elements are required, but it is difficult to make the respective characteristics uniform. When the accuracy or stability of one element becomes a problem, it may cause a new problem when it is used as a network, and the behavior when viewed in the whole network cannot be predicted. Since the coupling coefficient T _ij is fixed and the value learned in advance by another method such as simulation is used, self-learning cannot be performed.

Also, in the latter case of the network configuration of digital circuits, the up / down counter 1 is actually used.
3. Use of the rate multiplier 14 results in a very complicated and large-scale circuit.

As described above, according to the prior art, the analog circuit system is not reliable in operation, and the learning method by numerical calculation is complicated in calculation. The circuit configuration is large and complicated.

In order to solve such a drawback, a pulse density digital type neuron model with a self-learning function has been proposed by the present applicant as Japanese Patent Application Nos. 2-412448 and 3-29342. .. However, it is also applicable to systems with higher non-linearity than the neuron element of the proposed example, when it is desired to make input data time-dependent, or when input signal values are finely quantized. In order to increase the versatility and flexibility of the neuron, it is required to reflect the information of the output signal represented by the pulse train in the output signal value at a later time in the forward process.

This point will be further described. The pulse density type neural network is not limited to the one proposed by the applicant as described above, and is disclosed in, for example, JP-A-1-244.
Some are disclosed in Japanese Patent No. 567, US Pat. No. 4,893,255, and the like. In any case, the method of signal processing in the pulse density neural network differs depending on each type.

Here, the signal processing in the pulse density type neural network proposed by the present applicant will be briefly described. A bit train is actually used as the pulse train. That is, a train of bits that takes "0" or "1" as a value represents a pulse train, a bit having a value of "1" represents a pulse, and a bit having a value of "0" represents that there is no pulse.

The value of a signal is represented by the density of pulses, that is, the density of bits having a value "1" in a bit string. In other words, the value of the signal is equal to the probability that the value of the bit in the bit string is "1". For example, when a signal of value "X" is input to this neural network, a bit of value "1" is input to this neural network with probability X, and the probability (1-
In X), the bit having the value “0” is input. Also,
The output value of the neural network being "A" means that the probability that the neural network outputs the bit having the value "1" is A.

Each neuron unit in the neural network receives as an input signal a bit string sent from outside the neural network or from some other neuron units in the neural network. The coupling coefficient representing the strength of coupling between the neuron unit and another neuron unit is also represented by a bit string, and weighting by the coupling coefficient of the input signal is performed by taking the logical product of the input bit string and the coupling coefficient bit string. The weighted input signals are divided into several groups, and the weighted input signals are logically ORed separately for each division. Next, a logical operation is performed on the bit string obtained by these two logical sums to generate a new bit string, and the bit string is output as an output signal to the outside of the neuron unit. FIG. 39 shows an image thereof. In the figure, reference numeral 17 indicates a neuron unit (nerve cell unit).

In such a pulse density type neural network system, a certain neuron unit 17 is provided as shown in FIG.
Neuron unit 1 from the bit string which is the input signal to
When the bit string which is the output signal of 7 is generated, 1 is input from each input bit string to the neuron unit 17 at the same time.
One bit of the output bit string is generated only from the set of bits.

In this way, from only the 1-bit group that has simultaneously entered each neuron unit from each input bit string,
Since one bit of the output bit string is generated, the relationship between the input value and the output value of the neural network configured by combining these neuron units becomes linear. This point will be described with a one-input one-output neural network, which can be schematically represented as shown in FIG. First, the output when the signal of the value “X” is input to the neural network 18 is the output when the signal of the value “1” is input and the output when the signal of the value “0” is input: X: The value is a mixture of (1-X). Therefore, the neural network 1 when the input value is "1"
If the output of 8 is "A" and the output of the neural network 18 is "B" when the input value is "0", the value "X" is obtained.
The output value "Y" of the neural network 18 when the signal of (1) is input is Y = AX + B (1-X). That is,
The relationship between the input value "X" and the output value "Y" of the neural network 18 is linear as shown in FIG. This means that the one-input one-output neural network 18 cannot learn the non-linear relationship between the input value “X” and the output value “Y”.

As a method for solving the problem that the non-linear relationship between the input value and the output value cannot be learned in the one-input one-output neural network as described above, the number of pulses in the output pulse train is counted to obtain the same number. The pulse train with the pulse of is regenerated and output. The weighted input signal pulse is ORed not only in the spatial direction but also in the time direction. However, there is a concern that the signal processing speed is slow.

[0025]

According to a first aspect of the invention, there is provided a logical product calculating means for calculating a logical product of an input signal expressed by a pulse train and a coupling coefficient expressed by the pulse train,
Grouping means for dividing the operation result by the AND operation means into a plurality of groups, OR operation means for operating the OR of the operation results in each group, and operation results of these OR operation means Logical operation means for performing logical operation, pulse density changing means for changing the pulse density of the pulse train output from the logical operation means, and the coupling coefficient is changed based on the error signal expressed by the pulse density. The signal processing means having the coupling coefficient varying means is provided.

Here, as such a pulse density changing means, in the invention described in claim 2, it is represented by a counter for counting the operation result of the logical operation means and a pulse train having a probability according to the count result of this counter. The signal generating means for generating a signal and the logical product calculating means for calculating the logical product of the signal generated by the signal generating means and the calculation result of the logical calculating means.

In place of the logical product calculating means in the pulse density changing means of the invention described in claim 2, in the invention described in claim 4, the logical sum of the signal generated by the signal generating means and the calculation result of the logical calculating means is calculated. An OR operation means for performing the operation is provided, and in the invention according to the sixth aspect, the logic operation means for performing the logical operation of the signal generated by the signal generating means and the operation result of the logical operation means is provided. Regarding these inventions, claims 3, 5, 7
In the invention described, an up / down counter is provided instead of the counter.

According to the eighth aspect of the present invention, the pulse density changing means includes a counter for counting the number of information units, which is a specific value of the two values in the output signal, and an addition for adding the counting result by the counting means. Means, a random number generating means for randomly generating an integer value or a real value, the addition result by the adding means and the random number value generated by the random number generating means are compared, and one of two values is selected according to the comparison result. A comparison means for outputting an information unit having a specific value, and a logical sum operation for calculating a logical sum of the information unit in the output signal and the information unit in the output signal from the comparison means to obtain a new output signal. And means.

Similarly, in the invention described in claim 9, the pulse density changing means is provided with a counter for counting the number of information units which is a specific value of the two values in the output signal, and an integer value or a real value. Randomly generated random number generation means is compared with the counting result by the counting means and the random number value generated by the random number generation means, and an information unit, which is a specific value of two values, is output according to the comparison result. And the information unit in the output signal and the information unit in the output signal from the comparison unit both take a specific value of the two values, the information unit having the specific value is then compared. Probability increasing means for increasing the probability of output by the means, and logical sum calculating means for calculating a logical sum of the information unit in the output signal and the information unit in the output signal from the comparing means to obtain a new output signal. Composed by.

Further, in place of the probability improving means in the pulse density changing means according to the ninth aspect of the invention, in the invention of the tenth aspect, the information unit in the output signal from the comparing means is specified among two values. A means for reducing the counting result by the counting means is provided when taking the value of, and in the invention according to claim 11, the information unit in the output signal from the comparing means takes a specific value out of two values, and Means for reducing the counting result by the counting means when the information unit does not take a specific value out of the two values.

In the invention described in claim 8, 9, 10 or 11, in the invention described in claim 12, a plurality of means for increasing the information unit which is a specific value of the two values in the output signal is connected. did.

[0032]

According to the invention described in claim 1, in a simple digital logic configuration for processing a signal of a pulse train representation, an AND operation of an input signal and a coupling coefficient, a grouped OR operation, and an OR operation of these The signal obtained as a result of the logical operation of the result is not output as it is, but the pulse density is changed and output by the pulse density changing means. Therefore, the output signal pulse density in the forward process is changed to the input signal pulse density. Can be changed in a nonlinear manner with respect to the flexibility of a network.
It is highly versatile.

According to the invention described in claims 2 to 12,
It is specifically shown that such a pulse density changing means is configured by using a counter or the like, and a reliable operation is ensured.

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to FIGS. First, a nerve cell unit (neuron element) and a neural network using a pulse density type digital logic circuit having a self-learning function will be described.

First, the neurons and neural networks of this embodiment are digitalized and implemented as hardware. The basic idea is to input / output signals, intermediate signals,
The coupling coefficient, the teacher signal, etc. are all represented by a binary pulse train of "0" and "1". The amount of signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed on the memory in the nerve cell unit. Learning is realized by rewriting this pulse train. For learning, the error is calculated based on the given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1". It was done like this.

This idea will be described below. First,
Regarding signal processing by a digital logic circuit, signal processing in the forward process will be described. FIG. 2 shows a portion corresponding to one nerve cell unit (nerve cell mimicking element) 20, and the neural network as a whole is of a hierarchical type as shown in FIG. Input and output are all
The binarized and synchronized "1" and "0" are used. The value (intensity) of the input signal y _i is represented by a pulse density, and is represented by the number of states of “1” within a certain fixed time as in the pulse train shown in FIG. 4, for example. That is, the example of FIG.
4/6, and the signal "1" is 4 in 6 sync pulses.
There are two "0" s. At this time, it is desirable that the arrangement of "1" and "0" is random.

On the other hand, the coupling coefficient T _ij indicating the degree of coupling between the nerve cell units 20 is similarly expressed by pulse density,
A pulse train of "0" and "1" is prepared in advance in the memory. The example of FIG. 5 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of "1" and "0" is random.

Then, the coupling coefficient pulse train is sequentially read from the memory in response to the synchronous clock, and the AND gate 21 takes the logical product with the input signal pulse train as shown in FIG. 2 (y _i ∩ T _ij ). This is used as an input to the nerve cell j. In the case of the above example, the input signal is "10.
When input as "1101", a pulse train is called from the memory in synchronization with this and "101000" as shown in FIG. 6 is obtained by sequentially performing AND, which means that the input y _i is the coupling coefficient T _ij. It is shown that the pulse density becomes 2/6 by the conversion.

The pulse density of the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This is because the longer the signal train is, the more "1"
The more random the arrangement of "0" and "0", the closer to the product of numerical values it has. When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no data to be read, the head of the data may be returned to and the reading may be repeated.

Since one nerve cell unit 20 has multiple inputs, there are also many "ANDs of the input signal and the coupling coefficient" described above, and then the OR circuit 22 takes the logical sum of these. Since the inputs are synchronized, for example, the first data is "101000" and the second data is "01000".
In the case of "0", the OR of both results in "111000". When this is simultaneously calculated by multiple inputs (m) and output, the result is as shown in Fig. 7. This is the sum of analog calculation. Calculation and non-linear function (sigmoid function)
It corresponds to the part of.

When the pulse density is low, the pulse density of the OR of the pulse density is approximately equal to the sum of the pulse densities. As the pulse density increases, the OR circuit 22
Since the output of is gradually saturated, it does not match the sum of pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, although the connection has excitability and inhibitory property, it is assumed that the present embodiment can have both of them in connection between neurons. First, depending on whether the coupling coefficient is the excitatory coupling coefficient T _{ij (+)} or the inhibitory coupling coefficient T _{ij (-)} , each coupling is divided into an excitatory coupling group and an inhibitory coupling group.
Share Then, A of the pulse train of the input signal and each coupling coefficient
The OR of ND outputs is calculated for each group. And
"1" is output only when the OR result of the excitatory coupling group is "1" and the OR result of the inhibitory coupling group is "0", and "0" is output otherwise. This is shown in FIG.

When expressed by a logical expression, it is expressed by the following expressions (3) to (5).

[0044]

[Equation 3]

Further, the number of pulses of the signal x _{j which is} the logical result is counted, and a pulse train having a pulse density corresponding to the counted value is generated and output. However, when counting, the counting result of the pulse number of the signal x _j may be corrected by the pulse in the output pulse train. The logical product of such a pulse signal z _j and the signal x _j resulting from the above logical result is calculated, and this is the final output signal y _j from this neuron.
And It should be noted that instead of the logical product of the pulse signals z _j and x _j , the final output signal y is obtained as a logical sum or another logical operation.
You may get _j . The output signal y _j of each time of the neuron by such an operation depends on the output of the previous time.

The network of nerve cell units 20 is
It is hierarchical (ie, FIG. 3). If the entire network is synchronized, each layer can be calculated by the functions described above.

Next, the signal calculation processing in learning (back propagation) will be described. Basically, the error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method of c.

A. Error signal in the final layer In the final layer, the error signal in each nerve cell unit is calculated from the output signal and the teacher signal. Here, a desired output with respect to the input at that time is given by a pulse train as a teacher signal. Generally, if the error is expressed by a numerical value, it can take both positive and negative values, but since it cannot be expressed simultaneously by the pulse density, an error signal can be obtained by using two kinds of signals, one representing a + component and the other representing a − component. Express. That is, the error signal of the j-th nerve cell unit is shown in FIG. That is, the + component of the error signal is a pulse existing on the teacher signal side in the portion (1, 0) or (0, 1) where the teacher signal pulse and the output pulse are different, while the − component is similarly output. It is a pulse that exists on the side. In other words, if the error signal + pulse is added to the output pulse and the error signal−pulse is removed, it becomes a teacher pulse. That is, when these positive and negative error signals δ _{j (+)} and δ _{j (-)} are expressed by logical expressions, the expressions (6) and (7) are obtained. The coupling coefficient is changed based on such an error signal pulse as described later.

[0049]

[Equation 4]

B. Error Signal in Intermediate Layer First, the above-mentioned error signal is back-propagated to change not only the coupling coefficient between the final layer and the layer immediately before it but also the coupling coefficient of the layer before that. Therefore, it is necessary to calculate the error signal in each nerve cell unit in the middle layer. Signals were propagated from the neuron unit with an intermediate layer to each neuron unit in the next layer, which is exactly the reverse of the procedure of collecting error signals in each neuron unit in the next layer. And use it as its own error signal. This means that the above-mentioned arithmetic expressions (3) to (5) in FIG.
~ It can be performed in the same manner as the case shown in FIG. However, the difference from the above-mentioned processing in the nerve cell unit is that y is one signal, while δ is positive,
That is, it is necessary to have two signals as negative signals and to consider both of them. Therefore, the coupling coefficient T
It is necessary to divide into two cases according to the positive or negative of.

First, the case of excitatory coupling will be described. In this case, for a nerve cell unit having an intermediate layer, the error signal + at the jth nerve cell unit of the layer immediately before (the last layer in FIG. 3), the nerve cell unit and the self (the middle layer in FIG. 3) For each nerve cell unit, the AND (δ _{j (+)} ∩ T _ij ) of the coupling coefficient with a certain nerve cell unit) is obtained, and the OR of these is calculated {∪ (δ _{j (+)} ∩ T _ij )}. This is the error signal + of this nerve cell unit. That is, it becomes as shown in FIG.

Similarly, the error signal of the nerve cell unit of the previous layer is ANDed with the coupling coefficient, and the OR between them is taken as the error signal of this nerve cell unit. That is, it becomes as shown in FIG.

Next, the case of inhibitory binding will be described. In this case, the AND of the error signal in the nerve cell unit of the layer one layer ahead and the coupling coefficient of the nerve cell unit and the self is taken, and further the OR between them is taken. This is the error signal + of this nerve cell unit. That is, it becomes as shown in FIG.

Further, the error signal + of the preceding one is ANDed and the coupling coefficient is ANDed, and the OR between them is similarly taken as the error signal − of this nerve cell unit. That is, it becomes as shown in FIG.

Further, the error signal + of the excitatory connection and the error signal + of the inhibitory connection of this nerve cell unit are ORed, and this is taken as the error signal δ _{i (+)} of this unit. Similarly, the error signal of the excitatory coupling and the error signal of the inhibitory coupling are ORed, and this is calculated as the error signal δ of this unit.
_{i (-)}

The above is summarized as shown in equation (8).

[0057]

[Equation 5]

Further, the same or different learning rates (learning constants) may be set for the input error signals. This can be realized by thinning out the pulse train. For example, a counter-like idea is adopted, and FIG.
It may be as shown in. In this example, while the learning rate η = 0.5, the pulse train of the original signal is thinned out every other pulse, but even if the pulses of the original signal are not evenly spaced, they can be thinned out with respect to the original pulse train. 14 and 15,
When η = 0.5, every other pulse is thinned out, and η =
In the case of 0.33, every other pulse is left and η = 0.6.
The case of 7 indicates that every other pulse is thinned once.

In this way, the function of the learning rate is provided by thinning out the error signal. Such thinning out of the error signal can be easily realized by logically operating the output of a counter commercially available or using a flip-flop. In particular, when a counter is used, the value of the learning constant η can be set arbitrarily and easily, so that the characteristics of the network can be controlled.

C. Method of changing each coupling coefficient based on the error signal The line to which the coupling coefficient to be changed belongs (see Fig. 3)
Output y _i from the previous neuron unit corresponding to
And the error signal δ _{j (+)} or δ _{j (-) of the} own nerve cell unit
AND with (δ _j ∩ y _i ) (see FIGS. 16 and 17). The two signals thus obtained are respectively denoted by ΔT
_{Let ij (+)} and ΔT _{ij (-)} .

Then, based on this ΔT _ij , a new T
_ij is obtained. Since this T _ij is an absolute value component, it is classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the component of ΔT _{ij (+)} is increased with respect to the original T _ij , and ΔT
Reduce the components of _{ij (-)} . That is, it becomes as shown in FIG. On the contrary, in the case of the suppressive property, the component of ΔT _{ij (+)} is reduced and the component of ΔT _{ij (−)} is increased with respect to the original T _ij . That is, FIG.
It becomes as shown in 9.

The above can be summarized as in equation (9).

[0063]

[Equation 6]

The network is calculated based on the above learning rule.

Next, an actual circuit configuration based on the above algorithm will be described. 20 and 1 show examples of the circuit configuration, the configuration of the entire network is the same as that of FIG. 20 shows a circuit of a portion corresponding to a line (connection) in FIG. 2, and FIG. 1 shows a circle (each nerve cell unit 2 in FIG. 3).
The circuit of the part corresponding to 0) is shown. A digital neural network capable of self-learning can be realized by making the circuits of the configurations of FIGS. 20 and 1 into a network as shown in FIG.

First, FIG. 20 will be described. In the figure, 25 is an input signal to the nerve cell unit and corresponds to FIG. The value of the coupling coefficient as shown in FIG. 5 is stored in the shift register 26 as a memory. The shift register 26 has an outlet 26a and an inlet 26b, but it may have the same function as a normal shift register, and may be, for example, a combination of a RAM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are A as an AND operation means.
The AND is performed by the logic circuit 28 including the ND gate 27 and performing the processing shown in FIG. The output of this logic circuit 28 must be grouped according to whether the coupling is excitatory or inhibitory.
It is more versatile to prepare 9 and 30 and switch which one of them is output. For this reason, in this embodiment, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31, and the information is used for switching by the switching gate circuit 32 serving as a grouping unit. The switching gate circuit 32 includes two AND gates 32.
a and 32b and an inverter 32c interposed between one input
And consists of.

Further, as shown in FIG. 1, each input process (see FIG.
Gate circuit 33 having a plurality of OR gates
a and 33b are provided. Further, as shown in the figure, the output "1" is output only when the excitatory coupling group shown in FIG. 8 is "1" and the inhibitory coupling group is "0".
A gate circuit 34, which serves as a logical operation means by the ND gate 34a and the inverter 34b, is provided.

Further, a counter 46 for counting the signals output from the gate circuit 34 is provided, and a pulse train generator 47 serving as signal generating means for generating a pulse train having a density of probability corresponding to the count value by the counter 46. It is provided. An AND gate 4 serving as a second AND operation means for calculating the logical product of the signal output from the pulse train generator 47 and the signal output from the gate circuit 34.
8 are provided. The signal from the AND gate 48 is the final output of the nerve cell unit 20. Here, the counter 46 may be a normal counter (corresponding to the invention of claim 2) or an up / down counter that performs a subtraction process when a pulse is not output or at a certain interval (invention of claim 3). ). Further, the counter 46 may be reset every data.
Further, instead of the AND gate 48, an OR gate (corresponding to another OR operation means in the inventions of claims 4 and 5) is used, or a gate circuit for performing another logic operation processing (claims 6 and 7). (Corresponding to another logical operation means referred to in the described invention) may be used.

The counter 46 is an up / down counter that counts by inputting the signal output from the gate circuit 34 and the signal output from the pulse train generator 47, or a signal obtained by performing a logical operation on these two signals. May be Further, instead of the counter 46, a circuit in which a plurality of counters, an adder circuit, and a memory are combined may be used. Further, instead of the counter 46, the gate circuit 34
A counter for counting the signals output from the gate circuit 34, a counter for counting the signals output from the gate circuit 34 and a signal output from the pulse train generator 47, and the output values of these two counters. A circuit in which an adding circuit for adding and a memory are combined may be used.

Next, the learning process will be described. Error signals 35 and 36 are input to the circuit shown in FIG. These error signals 35 and 36 are collected by the gate circuits 37 each having a plurality of OR gates (processing of equation (8)), and then the frequency dividing circuit 38 performs processing corresponding to the learning rate (see FIGS. 14 and 15). Processing) and the error signal 3 for the previous layer
It is output as 9,40. The process shown in FIGS. 10 to 13 for calculating the error signal in the intermediate layer is performed by the gate circuit 4 having the AND gate configuration shown in FIG.
The error signals 42 and 43 to be output to the nerve cell unit of the previous layer are obtained according to + and −.

As described above, it is necessary to classify the connection depending on whether the connection is excitatory connection or inhibitory connection. In this case, information indicating whether the connection is excitatory or inhibitory is stored in the memory 31, and
Depending on the + and-signals 39 and 40 of the error signal, AND,
This is performed by the gate circuit 44 having an OR gate configuration.

16 to 1 for calculating a new coupling coefficient based on the error signals 39 and 40.
AND gate, inverter, O for performing the processing shown in FIG.
A coupling coefficient variable circuit 45 having an R gate configuration is provided and connected to the data inlet 26b side of each shift register 26. As a result, the update and rewriting of the coupling coefficient stored in the shift register 26 is performed. In the case of the gate circuit 45 as well, it is necessary to separate the cases depending on the excitability and inhibitory property of the coupling, but this is performed by the gate circuit 44.

It should be noted that the present invention is not limited to the above-described configuration examples as long as it has equivalent functions.
Further, some or all of them may be realized by software without configuring all by hardware.

By the way, according to the above description, in order to make it possible to learn the nonlinear relationship between the input value and the output value in the one-input one-output neural network, the value counted while counting the number of pulses in the output pulse train. Pulse is generated based on the above, and the result of the logical operation of the generated pulse and the output pulse is used as the output signal again. Since the pulse train can be changed, there is an advantage that the signal processing speed is fast. Here, the contents described above are expanded and embodied, and further, the correspondence between the value obtained by counting the number of pulses in the output pulse train and the probability of generating pulses is also referred to.

First, in order to avoid the conventional defect that the nonlinear relationship between the input value "X" and the output value "Y" cannot be learned in the pulse density type neural network, the relationship between the input and output of the neuron unit is nonlinear. FIG. 21 shows an example of a basic configuration for realizing the idea based on the above-described embodiment that “the above-mentioned is sufficient”. The circuit 51 shown in FIG. 21 is arranged at a position as shown in the nerve cell unit 20 as shown in FIG. In the case of FIG. 1,
It is provided in the portions indicated by reference numerals 46 to 48. This circuit 51
First, a counter 52, which corresponds to the counter 46 and counts the output of the Nitron, is provided. Further, a portion corresponding to the signal generating means 47 is composed of a random number generator 53 and a comparator 54, and the output side is further provided with an OR gate (another OR gate) as a structure corresponding to the invention of claim 4 or 5. Arithmetic means) 55 is provided and configured.

As a result, first, the counter 52 counts the bits of the value "1" in the output bit string from the neuron. The comparator 54 compares the output value of the counter 52 with the output value of the random number generator 53. If the output value of the random number generator 53 is less than or equal to the count value of the counter 52, the comparator 54
Outputs a bit having a value "1", and otherwise outputs a bit having a value "0". The OR of the output bit from the comparator 54 and the output bit from the neuron is ORed.
It is taken by the gate 55 and the result is used as a new output bit from the neuron. The counter 52 is the nerve cell unit 2
Whenever 0 finishes processing the bit string for 1 data, 0 is cleared.

Here, the random number generator 53 is usually an LFSR (linear feedback shift register).
Is used. L forming such a random number generator 53
The FSR has a structure in which a predetermined plurality of output sections of a plurality of linearly connected registers are feedback-connected to the head input section through an exclusive OR, and the bits set in the registers are By combining with an exclusive OR when feeding back from a plurality of places, a pseudo random number is generated with an extremely long-period bit string. Then, the random number generator 5 including such an LFSR
In No. 3, if the number of registers, the position of feedback connection, and the set initial value are the same, the output bit string is also the same, so that a random number having reproducibility can be generated.

For example, as shown in FIG. 23, when the random number generators 53 _{1 to} 53 ₄ are formed by 7 stages of LFSR, these are seven registers 58 _{1 to} 5 8 connected in a linear sequence.
A predetermined intermediate output unit 59 and a terminal output unit 60 of 8 ₇ are feedback-connected to the head input unit 62 via an exclusive OR 61, and the same reference clock 6 is supplied to each register 58.
It is implemented as a structure in which 3 are connected in parallel. In such a configuration, in the random number generator 53, if the bit stored in the register 58 _{1 at} time t is represented by At, the register 58 _{(1 + i) at} time (t + j) contains i The bit A _{(t-i + j)} stored in the register 58 ₁ before the time is moved, and the time (t-
The bit stored in the register 58 _{(1 + i)} in ₁₎ is A _(ti-1) . For example, as illustrated in FIG. (C), the random number generator 53 ₃ provided intermediate output section 59 to the register 58 ₄ of the fourth stage, the bit A _t of the first stage of the register 58 in _1, the bits of the register 58 _4, 58 in ₇ of the fourth stage and seventh stage at time (t-1) has a synthesized value exclusive OR 61. Here, synthesis of bits by exclusive OR 61 is equal to the addition in "modulo2", bit A _t of the first stage of the register 58 in _{1, A t = A (t-} 4) + A (t- ₇₎ (mod2) ………… (10).

Here, the set value in a state where a predetermined bit is input to each register 58 of the random number generator 53 composed of 7 stages of LFSR as described above is whether the bit in each register 58 is "0". Since it is one of "1", there are 2 ⁷ = 128 ways, but if all 7 set values are "0", all the set values remain "0" even if the bit forwarding is repeated, If at least one of the seven set values includes "1", the set value changes to one of the above combinations in a predetermined cycle when the bit forwarding is repeated. Here, when the bit A _t in a predetermined register 58 of the random number generator 53 is defined by the equation (10), the bit string of A _t may be a pseudo-random sequence of period (2 ⁷ -1) It's known. In other words, the random number generator 53 composed of the illustrated LFSR is (2
Since it is possible to generate a bit string of ⁷ −1), the random number generator 53 outputs an integer value of 1 to 127 by reading this bit string as a 7-digit binary number. When reading the set value of the random number generator 53 composed of such an LFSR as a binary number, both the method of setting the head to the lowest and the method of setting the terminal to the lowest are performed. Now, let us say that the beginning is the lowest. However, here, the method of setting either the beginning or the end of the random number generator 52 to the lowest is valid.

Here, c _i (= 1, 2, ... P) is set to “0”
Or an integer of "1" (where c _p = 1), this c _i
Recurrence formula showing a A _{t some, A t = c 1 A (} t-1) + c 2 A (t-2) ... + c p A (tp) (mod2) ...... (11) , and this recurrence formula The characteristic polynomial of is as shown in equation (12).

[0081]

[Equation 7]

Here, the random number sequence generated by the above-mentioned recurrence formula (11) has a period of length “2 to the power of p−1” or less, but the characteristic that produces the maximum period within this range. Polynomials are especially called primitive polynomials. Then, a bit string of A _t of the period of such generated by a primitive polynomial "2 of p squared -1", p order linear maximum period sequence (Maximum
-Length Linearly Recurring Sequence) M
It is called an affiliate. For example, bit string A _t generated from equation (10) is a 7-order M sequence corresponding to the primitive polynomial f (x) = 1 + x 4 + x 7. In addition, 7-stage LFSR
In the case of the random number generator 53 consisting of, the number of generators of the 7th-order M-sequence bit string is limited to the four types illustrated in FIG. 23, and therefore the primitive polynomials and recurrence equations of these random number generators 53 are An example is given below.

.. FIG (a) to the illustrated random number generator 53 ₁ primitive polynomial f (x) = 1 + x + x 7 recurrence formula _{A t = A (t-1} ) + A (t-7) (mod2) bit string of M series tables 1 ( Example a) Generated random number sequence Example table 2 (a). Bit string table of FIG. (B) exemplary random number generator 53 ₂ primitive polynomial f (x) = 1 + x 3 + x 7 recurrence formula _{A t = A (t-3} ) + A (t-7) (mod2) M -sequence 1 (b) Example of generated random number sequence Table 2 (b). Bit string table of FIG. (C) to the illustrated random number generator 53 ₃ primitive polynomial f (x) = 1 + x 4 + x 7 recurrence formula _{A t = A (t-4} ) + A (t-7) (mod2) M -sequence 1 (c) Example generated random number sequence Table 2 (c). Bit string table of the (d) of FIG exemplary random number generator 53 ₄ primitive polynomial f (x) = 1 + x 6 + x 7 recurrence formula _{A t = A (t-6} ) + A (t-7) (mod2) M -sequence 1 (d) illustrated Random number sequence generated Table 2 (d) illustrated

[0084]

[Table 1]

[0085]

[Table 2]

That is, in the random number generator 53 composed of such an LFSR, since the cycle of the random number generated by the bit string of the M series is the maximum, the irregularity of the generated random number is extremely good. Here, such an M-sequence random number generator 53
Since it is known in advance that the cycle of the output bit string is the maximum, it is possible to generate a signal with an irregular pulse position by modulating this with the pulse density or the number of pulses. For example, in the 7th-order random number generator 53 that generates 127 random numbers as described above, the pulse density is 1
If you request a 0/127 signal, the generated random number is 1
If a pulse is output if -10 and a pulse is not output if the random number is 11-127, this signal has irregular pulse positions and a density of 10/127. In this way, the random number generator 53 generates the M-sequence bit string, so that the pseudo random number generation period can be maximized.

Now, the point that the output of the nerve cell unit 20 becomes non-linear by the circuit 51 shown in FIG. 21 will be described. Here, an example will be described in which 127 bits configure one data, and the output of the nerve cell unit 20 when the circuit shown in FIG. 21 is not provided has a pulse density of 1/127. First, the counter 52 is cleared to zero. The position of the bit of the value "1" in the neuron output consisting of 127 bits is totally random. Now, assuming that the first bit in the neuron output is "1", the first bit of the new neuron output from the OR gate 55 becomes "1", and the value of the counter 52 changes to "1". The remaining 126 bits in the neuron output are all "0", but the probability that "1" is output from the LFSR random number generator 53 during 126 clocks when the remaining 126 bits are output is 126 /
Since there are 127, the probability that another bit with the value “1” is mixed in the new neuron output is also 126/127.

However, if the bit having the value "1" is present at the second position in the neuron output, the number of bits output after the value of the counter 52 is changed to "1" is 125. Therefore, the probability that another bit with the value “1” is mixed in the new neuron output is 126/127.
Instead, it becomes 125/127. Further, assuming that the bit having the value "1" exists at the last position in the neuron output, the probability that another bit having the value "1" is mixed in the new neuron output is 0.

As described above, the new neuron output increases up to twice as much as the Nyron output without the circuit 51, although it depends on the position of the pulse in the neuron output. This means that the circuit 51
If the pulse density of the neuron output is 1 /
The same applies when the value is larger than 127. However, since the pulse density never exceeds 1, when the neuron output becomes large, the multiplication factor of the new neuron output with respect to the original neuron output gradually decreases.

In FIG. 22, the relationship between the neuron output when the circuit 51 is not present and the new neuron output when the circuit 51 is present is shown in FIG. That is, even if the neuron output without the circuit 51 is the same value, the new neuron output value is different if the pulse position is different. Therefore, in FIG. 24, a line obtained by averaging the new neuron output values is shown. It is shown.

From the relationship shown in FIG. 24, the relationship between the input value and the output value of the nerve cell unit 20 is a linear relationship as shown by the broken line in the case where there is no circuit 51, as shown in FIG. If there is, it is understood that there is a non-linear relationship as shown by the solid line. As described above, by incorporating the circuit 51 shown in FIG. 21 at the position shown in FIG. 22, the relationship between the input value and the output value of the nerve cell unit 20 can be changed non-linearly. Also, the relationship between the input value and the output value is
This is a preferable form for adding learning ability to the neural network, such that the rise at a small input value rises sharply and that at a large input value becomes saturated.

When the bit of the value "1" output from the comparator 54 overlaps with the bit of the original neuron output value "1", that bit is the bit of the original neuron output value "1". Does not help to increase the number of. The circuit 56 shown in FIG. 26 is an improvement of this point and corresponds to the invention of claim 9. In the circuit shown in FIG. 26, the AND gate 57 takes the logical product of the bit output from the comparator 54 and the bit output from the neuron. If the result of the logical product by the AND gate 57 is true, the value of the counter 64 increases by "1". In the adder 65, a counter 66 (46
Or 52) and the count value of the counter 64, and the sum is stored in the memory 67 for comparison by the comparator 54. Therefore, when the bit of the value "1" output from the comparator 54 overlaps with the bit of the original neuron output value "1", the counter 64 and the adder 65 (constituting means for increasing information) The value of the memory 67 is increased by "1", and the bit of the value "1" may be set in the output of the new neuron output from the OR gate 55 later, instead of the bit of the value "1" which is not useful. The sex increases by one. When there are few bits of the value "1" in the original neuron output and there is little overlap between the bits of the value "1" output from the comparator 54 and the bits of the original neuron output value "1", or vice versa. , The number of bits with the value "1" in the original neuron output is large and close to the number of bits in one data, and the bit string of the new neuron output overlaps with the bit with the value "1" of the original neuron output. Except when there are not many remaining bit positions that do not exist, when the circuit 56 shown in FIG. 26 is used, the number of bits with the value “1” is increased in the new neuron output than when the circuit 51 is used. ..

By the way, the new value of the neuron output is compared with the original value of the neuron output in the circuit 5 shown in FIG.
When it is desired to increase the non-linearity by increasing it more greatly than in the case of 1, the AND gate 57 and the counter 64 are omitted as shown in FIG. 66b is a circuit 69 provided with a plurality (two in the illustrated example), and the values of these counters 66a and 66b are added together to make the memory 6
7 may be stored. In the case of such a circuit 69 corresponding to the invention of claim 8, the value of the new neuron output increases up to three times the value of the original neuron output.

In the circuit 51 shown in FIG. 21, it is assumed that the counter 52 is set to "1" by the bit having the first value "1" of the neuron output. Then, it is assumed that until the next bit of value "1" of the neuron output comes, the bit of value "0" continues for a while during neuron output. Then, if the value of the LFSR-configured random number generator 53 becomes "1" during this period, one bit of the value "1" is generated in the output of the new neuron. Then, it is assumed that the next bit "1" of the neuron output comes and the value of the counter 52 becomes "2". Then, when the value of the random number generator 53 becomes "1" or "2" thereafter, a bit having the value "1" is generated in the new neuron output. In this case, assuming that the value of the random number generator 53 can be either "1" or "2", two bits of the value "1" are generated, and the bit of the value "1" generated previously is generated. In total, a total of 3 will be generated.

However, since the random number generator 53 having the LFSR does not take the same value twice in one cycle, when the LFSR having the same cycle as the pulse length of one data is used, After the value becomes "2", the value of LFSR does not become "1" until the output of the neuron for one data is completed, so that the total number of bits of the value "1" is 2 as well.
Only up to individual items are generated.

On the other hand, when an LFSR having a cycle different from the pulse length of one data is used as the random number generator 53 or a circuit other than the LFSR is used, the value of the counter 52 is "1".
At this time, the random number generator 53 increases the number of bits with the value "1" by one in the output of the new neuron, and then the value of the counter 52 increases by "2", and the number of bits increases by two more. obtain. In this way, the random number generator 53
When a circuit other than the LFSR having the same period as the data pulse length is used, the increase in the number of bits of the value “1” in the new neuron output is the LF having the same period as the pulse length of one data.
It becomes sharper than when using SR. This may occur when the non-linearity is too strong and not desirable depending on the problem of applying the neural network.

In such a case, in order to suppress the increase in the number of bits of the value "1" in the output of the new neuron, in FIG.
The circuit 70 shown in FIG. The circuit 70 corresponding to the invention of claim 10 has an up / down (U / D) counter 70 in place of the counter 52 in the circuit 51. As a result, the value "1" is output from the comparator 54.
Since the value of the U / D counter 70 is decremented by "1" every time the bit of "1" is output, the increase of the bit of value "1" in the new neuron output is suppressed.

However, in the case of the configuration of FIG. 28, the bit value of the original neuron output and the value of the output bit from the comparator 54 are both "1", and the output bit from the comparator 54 has a new value. The value of the U / D counter 70 is decremented by "1" even if it is not useful for increasing the bit of the value "1" in the neuron output. In consideration of this point, the configuration may be as shown in FIG. In the case of this circuit 72 corresponding to the invention described in claim 11, the output of the comparator 54 is not directly input to the U / D counter 71, and the AND of the neuron output and the result negated by the NOT gate 73 is ANDed.
The gate 74 is used for inputting.

As a result, only when the bit of the original neuron output is "0" and the value of the output bit of the comparator 54 is "1", that is, the output bit from the comparator 54 is a new neuron output. The value of the U / D counter 71 is set to "1" only when it is used to increase the bit of the value "1" in
I tried to reduce it.

A plurality of the circuits 51, 56, 69, 70, 72 shown in FIGS. 21, 26 to 29 may be connected in series (corresponding to the invention of claim 12). FIG. 30 shows an example in which two circuits 51 shown in FIG. 21 are connected in series. According to this, three or more may be connected in series, and similarly, the circuits 56, 69, 70 and 72 may be connected. Furthermore, not only the series connection of the same type of circuit as shown in FIG. 30 but also different types, for example, the circuit 51 and the circuit 5.
6 may be combined in a mixed format. In any case, the non-linearity of the new neuron output can be strengthened by stacking two or more circuits.

For example, the relationship between the original neuron output and the new neuron output when the number of the circuits 51 shown in FIG. 21 is only one is as shown by the broken line in FIG. 31, The relationship between the original neuron output and the new neuron output when two circuits 51 are connected in series as shown in FIG. 30 is shown by the solid line in FIG.
In addition, by devising the number and types of circuits for series coupling,
The degree of non-linearity of the output value from the nerve cell unit 20 with respect to the input value can be adjusted in a wide range, strongly or weakly.

[0102]

According to the invention described in claim 1, in a simple digital logic configuration for processing a signal of a pulse train representation, a logical product operation of an input signal and a coupling coefficient, a grouped logical sum operation, and these logics Without outputting the signal obtained as a result of the logical operation of the sum operation result as it is,
Since the pulse density is changed and output by the pulse density changing means, the output signal pulse density in the forward process can be changed non-linearly with respect to the input signal pulse density. It becomes possible to deal with the case where data is required to have time dependency or the case where the input signal value is finely quantized, and the flexibility and versatility in the case of constructing a network can be enhanced.

According to the invention described in claims 2 to 12,
It is specifically shown that such a pulse density changing means is configured by using a counter or the like, and reliable operation can be secured.

[Brief description of drawings]

FIG. 1 is a logic circuit diagram showing an embodiment of the present invention.

FIG. 2 is a logic circuit diagram.

FIG. 3 is a schematic diagram showing a network configuration example.

FIG. 4 is a timing chart showing an example of logical operation.

FIG. 5 is a timing chart showing an example of logical operation.

FIG. 6 is a timing chart showing an example of logical operation.

FIG. 7 is a timing chart showing an example of logical operation.

FIG. 8 is a timing chart showing an example of logical operation.

FIG. 9 is a timing chart showing an example of logical operation.

FIG. 10 is a timing chart showing an example of logical operation.

FIG. 11 is a timing chart showing an example of logical operation.

FIG. 12 is a timing chart showing an example of logical operation.

FIG. 13 is a timing chart showing an example of logical operation.

FIG. 14 is a timing chart showing an example of logical operation.

FIG. 15 is a timing chart showing an example of logical operation.

FIG. 16 is a timing chart showing an example of logical operation.

FIG. 17 is a timing chart showing an example of logical operation.

FIG. 18 is a timing chart showing an example of logical operation.

FIG. 19 is a timing chart showing an example of logical operation.

FIG. 20 is a logic circuit diagram showing a configuration example of each unit.

FIG. 21 is a circuit diagram showing a configuration example of each unit.

FIG. 22 is a schematic view showing the location of the arrangement.

FIG. 23 is a block diagram showing an LFSR method for random number generation.

FIG. 24 is a characteristic diagram corresponding to the presence or absence of the circuit shown in FIG. 22.

FIG. 25 is an input value-output value characteristic diagram.

FIG. 26 is a circuit diagram showing a modified example.

FIG. 27 is a circuit diagram showing another modification.

FIG. 28 is a circuit diagram showing still another modification.

FIG. 29 is a circuit diagram showing still another modification.

FIG. 30 is a circuit diagram showing an example of serial connection.

FIG. 31 is a characteristic diagram corresponding to the number of connection circuits.

FIG. 32 is a conceptual diagram showing one unit configuration showing a conventional example.

FIG. 33 is a conceptual diagram of the neural network configuration.

FIG. 34 is a graph showing a sigmoid function.

FIG. 35 is a circuit diagram showing a specific configuration of one unit.

FIG. 36 is a block diagram showing a digital configuration example.

FIG. 37 is a circuit diagram of a part thereof.

FIG. 38 is a different part of the circuit diagram.

FIG. 39 is a schematic diagram showing a neuron unit of a proposed pulse density neural network.

FIG. 40 is a schematic diagram showing an example of signal bit processing.

FIG. 41 is an explanatory diagram for explaining a linear relationship.

FIG. 42 is a linear characteristic diagram thereof.

[Explanation of symbols]

 20 signal processing means 27 logical product calculating means 32 grouping means 33 logical sum calculating means 34 logical calculating means 45 coupling coefficient changing means 46 counter 47 signal generating means 48 different logical product calculating means 52 counter 53 random number generating means 54 comparing means 55 Alternative OR operation means

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Hironori Mifune Inoue 1-3-6 Nakamagome, Ota-ku, Tokyo Inside Ricoh Co., Ltd. (72) Shinichi Suzuki 1-3-3 Nakamagome, Ota-ku, Tokyo Stock company Ricoh

Claims (12)

[Claims] 1. A logical product calculating means for calculating a logical product of an input signal expressed by a pulse train and a coupling coefficient expressed by a pulse train, and a grouping means for dividing a calculation result by the logical product calculating means into a plurality of groups. An OR operation means for calculating the OR of the operation results in each group, a logic operation means for performing a logic operation on the operation results of these OR operation means, and an output from the logic operation means. A signal processing means having pulse density changing means for changing the pulse density of the pulse train, and coupling coefficient changing means for changing the coupling coefficient based on an error signal expressed by the pulse density. Signal processing device. 2. A pulse density changing means, a counter for counting the operation result of the logical operation means, a signal generating means for generating a signal represented by a pulse train having a probability corresponding to the counting result of the counter, and the signal generating means. 2. The signal processing apparatus according to claim 1, further comprising: a logical product calculating means for calculating a logical product of a signal generated by the means and a calculation result of the logical calculating means. 3. A pulse density changing means, an up-down counter for counting the operation result of the logical operation means, and a signal generating means for generating a signal represented by a pulse train having a probability corresponding to the count result of the up-down counter. 2. The signal processing apparatus according to claim 1, further comprising: a logical product calculating means for calculating a logical product of a signal generated by the signal generating means and a calculation result of the logical calculating means. 4. A pulse density changing means, a counter for counting the operation result of the logical operation means, a signal generating means for generating a signal represented by a pulse train having a probability corresponding to the counting result of the counter, and the signal generating means. 2. The signal processing device according to claim 1, further comprising: a logical sum calculating means for calculating a logical sum of a signal generated by the means and a calculation result of the logical calculating means. 5. The pulse density changing means includes an up / down counter for counting the calculation result of the logical operation means, and a signal generating means for generating a signal represented by a pulse train having a probability corresponding to the count result of the up / down counter. 2. The signal processing apparatus according to claim 1, further comprising: a logical sum calculating means for calculating a logical sum of a signal generated by the signal generating means and a calculation result of the logical calculating means. 6. A pulse density changing means, a counter for counting the operation result of the logical operation means, a signal generating means for generating a signal represented by a pulse train having a probability corresponding to the counting result of the counter, and the signal generating means. 2. The signal processing device according to claim 1, further comprising another logical operation means for performing a logical operation of the signal generated by the means and the operation result of the logical operation means. 7. The pulse density changing means includes an up / down counter for counting the operation result of the logical operation means, and a signal generating means for generating a signal represented by a pulse train having a probability corresponding to the count result of the up / down counter. 2. The signal processing apparatus according to claim 1, further comprising another logical operation means for performing a logical operation between the signal generated by the signal generating means and the operation result of the logical operation means. 8. The pulse density changing means has a function of adding 2 to the output signal.
A counter for counting the number of information units, which is a specific value among the values, and an adding means for adding the counting result by the counting means,
Random number generating means for randomly generating an integer value or a real number is compared with the addition result by the adding means and the random number value generated by the random number generating means, and 2 is obtained according to the comparison result.
Comparing means for outputting an information unit which is a specific value among the values, and a new output signal by calculating the logical sum of the information unit in the output signal and the information unit in the output signal from the comparing means. 2. The signal processing device according to claim 1, further comprising: a logical sum calculating means. 9. The pulse density changing means is a means for changing the pulse density of 2 in the output signal.
A counter that counts the number of information units that are specific values among the values, a random number generation unit that randomly generates an integer value or a real value, a counting result by the counting unit, and a random number generated by the random number generation unit. Comparing means for comparing a numerical value and outputting an information unit which is a specific value out of two values according to the comparison result, an information unit in the output signal and an information unit in the output signal from the comparing means. When both take a specific value out of two values, the probability improving means for increasing the probability that the comparing means outputs the information unit having the specific value thereafter, and the information unit in the output signal and the comparing means. 2. The signal processing device according to claim 1, further comprising a logical sum calculating means for calculating a logical sum of the information unit in the output signal and a new output signal. 10. The pulse density changing means includes a counter for counting the number of information units, which is a specific value out of two values in the output signal, and a random number generating means for randomly generating an integer value or a real value. Comparing means for comparing the counting result by the counting means and the random number value generated by the random number generating means, and outputting an information unit which is a specific value of two values according to the comparison result, and the comparing means. The information unit in the output signal is 2
When taking a specific value among the values, means for reducing the counting result by the counting means, and a logical sum of the information unit in the output signal and the information unit in the output signal from the comparing means are calculated to obtain a new value. 2. The signal processing device according to claim 1, further comprising a logical sum operation means for outputting an output signal. 11. The pulse density changing means includes a counter that counts the number of information units that are specific values of two values in the output signal, and a random number generating means that randomly generates an integer value or a real value. Comparing means for comparing the counting result by the counting means and the random number value generated by the random number generating means, and outputting an information unit which is a specific value of two values according to the comparison result, and the comparing means. The information unit in the output signal is 2
The information unit in the output signal has a value of 2
When a specific value out of the values is not taken, means for reducing the counting result by the counting means, and the logical sum of the information unit in the output signal and the information unit in the output signal from the comparison means are calculated and newly calculated. 2. The signal processing apparatus according to claim 1, further comprising: a logical sum operation means for producing a different output signal. 12. The signal processing device according to claim 8, wherein a plurality of means for increasing the information unit which is a specific value of the two values in the output signal are connected.