JP2612640B2 - Signal processing circuit, signal processing network, signal processing device, and signal processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing circuit, a signal processing circuit network, a signal processing device, and a signal processing method applicable to a neural computer or the like imitating a neural cell network.

[0002]

2. Description of the Related Art The aim was to imitate the functions of nerve cells (neurons), which are the basic units of information processing in living organisms, and to make this "neural cell mimicry element" into a network for parallel processing of information. This is a so-called neural network. Even if it is very easily performed on a living body, such as character recognition, associative memory, and movement control, there are many that are not easily achieved by a conventional Neumann computer. Attempts have been made to solve these problems by imitating the nervous system of a living body, in particular, functions unique to the living body, that is, parallel processing, self-learning, and the like. Many of these trials have been performed by computer simulations, and parallel processing is required to exhibit their original functions, and for this purpose, hardware implementation of a neural network is required. Some have already attempted to implement hardware, but the self-learning function, which is one of the features of neural networks, cannot be realized, which is a major bottleneck. Most of the components are realized by analog circuits, and there is a problem in operation as described later.

[0003] The conventional method will be examined below in order. First, a conventional neural network model will be described. FIG. 34 is a diagram showing a certain neuron unit 1, and FIG. 33 is a diagram showing this as a network.
One neuron unit combines with and receives signals from many other neuron units, processes it, and outputs. FIG.
In the case of 3, the network is hierarchical, and receives a signal from the unit of the immediately preceding (left) layer and outputs it to the unit of the next (right) layer.

[0004] This will be described in more detail. First, in the nerve cell unit 1 of FIG. 34, what is called a coupling coefficient represents the degree of coupling between another neuron unit and its own unit. The coupling coefficient between the i-th unit and the j-th unit is Generally represented by T _ij . The connection includes an excitatory connection in which the larger the signal from the partner neuron, the larger the output is, and a conversely, the larger the signal from the partner neuron, the smaller the output, the inhibitory connection.
_ij > 0 indicates an excitatory connection, and _Tij <0 indicates an inhibitory connection. Now, suppose that your own unit is the j-th unit, i
If the output of the th unit is y _i , the coupling coefficient T
_ij multiplied by the T _ij y _i becomes the input to their unit. As described above, since each unit is connected to many units, ΣT _ij y _i obtained by adding T _ij y _i to those units is an input to its own unit. This is called an internal potential and is represented by u _j .

U _j = ΣT _ij y _i …………………………………….
(1)

Next, the input is subjected to a non-linear process to produce an output. The function at this time is called a nerve cell response function, and a sigmoid function as shown in Equation (2) and FIG. 35 is used as a nonlinear function.

[0007]

F (x) = 1 / (1 + e ^-x ) (2)

When a network is formed as shown in FIG. 33, a final output is obtained by giving each coupling constant T _ij and successively calculating equations (1) and (2).

On the other hand, an example of such a network realized by an electric circuit is shown in FIG. This is disclosed in Japanese Patent Application Laid-Open No. Sho 62-295188. Basically, a plurality of amplifiers 2 having an S-shaped transfer function and the output of each amplifier 2 is connected to the input of an amplifier of another layer by one point. A resistive feedback network 3 is provided which connects as shown in dashed lines. A CR time constant circuit 4 composed of a grounded capacitor and a grounded resistor is individually connected to the input side of each amplifier 2. Then, input currents I ₁ , I ₂ ,..., And _IN are supplied to the inputs of each amplifier 2, and the output is obtained from a set of output voltages of these amplifiers 2.

Here, the strength of the input or output signal is represented by a voltage, and the strength of the connection between nerve cells is determined by the resistance 5 (a grid point in the resistive feedback network 3) connecting input / output lines between the cells. ), And the nerve cell response function is represented by the transfer function of each amplifier 2. As described above, the connection between nerve cells has excitability and inhibition, and is mathematically represented by the sign of the connection coefficient. However, since it is difficult to realize positive and negative with constants on the circuit, here, the output of the amplifier 2 is divided into two, and one output is inverted to generate two positive and negative signals. Is appropriately selected. FIG.
An amplifier is used as a function corresponding to the sigmoid function shown in FIG.

However, these circuits have the following problems. The coupling constant T _ij is fixed, and a value learned in advance by another method such as simulation can only be used, and self-learning cannot be performed. Signal strength is represented by analog values such as potential and current,
When the internal calculation is performed in an analog manner, the value changes due to temperature characteristics, drift immediately after turning on the power, and the like. Since it is a network, a large number of elements are required, but it is difficult to make the characteristics of each element uniform. When the accuracy or stability of one element becomes a problem, when it is made into a network, there is a possibility that a new problem may occur, and the movement when viewed as a whole network cannot be predicted.

On the other hand, as a learning rule used in numerical calculation, there is the following one called back propagation.

First, each coupling coefficient is randomly given first. In this state, if an input is given, the output result is not always desirable. For example, in the case of character recognition,
If a handwritten character "1" is given, it is desirable that the output result is "this character is" 1 "". However, if the coupling coefficient is random, the desired result is not necessarily obtained. Therefore, a correct answer (teacher signal) is given to this network, and each coupling coefficient is changed again so that the correct answer is obtained when the same input is received. At this time, an algorithm for obtaining the amount by which the coupling coefficient is changed is called back propagation.

For example, in the hierarchical network shown in FIG. 33, if the output of the j-th neuron in the final layer is y _j and the teacher signal for the neuron is d _j , then E = Σ (d _j −y _j ) ² ΔT _ij = ∂E / ∂T _ij …………… (4) so that E expressed by (3) becomes minimum. _Is used to change the coupling coefficient T _ij . More specifically, first, when calculating the coupling coefficient between the output layer and the immediately preceding layer, δ _j = (d _j −y _j ) × f ′ (u _j ). (5) is used to obtain δ (error signal), and to obtain the coupling coefficient between the layers before that, δ _j = Σδ _j T _ij × f ′ (u _j ) (6) is used to obtain δ (error signal), and ΔT _ij = η (δjyj) + αΔT _ij (at the time of the previous learning) T _ij = T _ij (at the time of the previous learning) + ΔT _ij ... (7) is obtained and T _ij is changed. Where η is the learning constant,
α is called a stabilization constant. Since each is not logically determined, it is determined empirically. In general, the smaller these values are, the slower the convergence is. The order is about one. Further, f 'is a first derivative function of the sigmoid function f.

Learning is performed in this manner. Thereafter, an input is given again, an output is calculated, and learning is performed. As this operation is repeated many times, a coupling coefficient T _ij that will obtain a desired result for a given input is determined.

If the learning method is to be implemented by hardware in some way, the learning requires a large amount of four arithmetic operations, which is difficult to realize. The learning method itself is not suitable for hardware implementation.

On the other hand, an example in which a neural network is realized by a digital circuit will be described with reference to FIGS. FIG. 37 shows a circuit configuration of a single nerve cell. Each synapse circuit 6 is connected to a cell body circuit 8 via a dendrite circuit 7. FIG. 38 shows an example of the configuration of the synapse circuit 6 in which a magnification a is applied to an input pulse f via a coefficient circuit 9.
There is provided a rate multiplier 10 to which a value multiplied by (multiplied by 1 or 2 by the feedback signal) is input, and a synapse load register 11 storing a weight value w is connected to the rate multiplier 10. . FIG. 39 shows a configuration example of the cell body circuit 8, and the control circuit 1
2, an up / down counter 13, a rate multiplier 14, and a gate 15 are sequentially connected, and an up / down memory 16 is further provided.

The input / output of the nerve cell unit is represented by a pulse train, and the amount of signal is represented by the pulse density.
The coupling coefficient is represented by a binary number and stored in the memory 16. By inputting the input signal to the clock of the rate multiplier 14 and inputting the coupling coefficient to the rate value,
The pulse density of the input signal is reduced according to the rate value. This is the T _ij of the back propagation model equation.
y _i . Next, the portion {} of {T _ij y _i } is realized by an OR circuit indicated by the dendrite circuit 7. Since the coupling has excitatory and inhibitory properties, it is grouped in advance and OR is performed for each group. An output is obtained by inputting these two outputs to the up side and the down side of the counter 13 and counting. This output is 2
Since it is a decimal number, it is converted into a pulse density by using the rate multiplier 14 again. By making this unit a network, a neural network can be realized. The learning is realized by inputting the final output to an external computer, performing a numerical calculation inside the computer, and writing the result to the coupling coefficient memory 16. Therefore, there is no self-learning function. In addition, the circuit configuration is complicated because the signal of the pulse density is once converted into a numerical value by using a counter, and then converted into the pulse density again.

[0019]

As described above, the prior art has a disadvantage that self-learning cannot be performed on hardware.

Further, the operation of an analog circuit is not reliable, and the learning method by numerical calculation is complicated in calculation, and is not suitable for hardware implementation. On the other hand, a digital type which operates reliably has a complicated circuit configuration.

[0021]

[0022]

[0023]

[0024]

According to a first aspect of the present invention, there is provided a signal processing apparatus comprising: an input means for receiving an input signal; a supply means formed by a logic circuit for supplying a coupling coefficient; A logical processing unit for performing a logical operation on each of the signals and outputting a result of the operation, a forward processing unit coupled to the input unit, and an output signal of the logical operation unit and a teacher signal. A coupling coefficient generating unit for generating a new coupling coefficient based on an error signal; and a coupling coefficient changing unit for changing a coupling coefficient supplied by the supplying unit to a new coupling coefficient generated by the coupling coefficient generating unit. It uses a neuron mimic circuit consisting of self-learning means coupled to the means. Here, in the signal processing device according to claim 2, the coupling coefficient generation means in the self-learning means in the signal processing device according to claim 1 is a means for generating a new coupling coefficient based on the error signal and the learning constant. Things.

According to a third aspect of the present invention, there is provided a signal processing apparatus.
Regarding the configuration of the signal processing device described in the above, the input means is the first
A plurality of first input lines for receiving a binary input signal and a plurality of second input lines for receiving a second binary input signal, wherein the supplying means stores the coupling coefficient. A first memory means and a second memory means for performing logical operation on one of the first binary input signals received from the first input line and the first memory means. First gate means for obtaining, for each of the first binary input signals, a logical product of the read coupling coefficient corresponding to the input line and the second gate signal received from the second input line; A second gate for obtaining, for each of said second binary input signals, the logical product of one of said binary input signals and the coupling coefficient corresponding to that input line read from said second memory means Means and said first gate hand A third gate means for obtaining a logical sum of the logical product outputs of the a fourth gate means for obtaining a logical sum of the logical product outputs of the said second gate means,
An inverter for inverting a logical sum output of the fourth gate means; and a gate for obtaining a logical product and a logical sum of the logical sum output of the third gate means and the logical sum output inverted by the inverter. Output means. The signal processing device according to claim 4 is the signal processing device according to claim 3, wherein the first input line, the first memory unit, the first gate unit, and the third gate unit are excitatively coupled. Forming a group, a second input line,
The second memory means, the second gate means, the fourth gate means and the inverter form an inhibitory coupling group, and the third gate means, the fourth gate means, the inverter and the output means comprise the excitatory A majority decision means is provided for determining an output signal of a neuron unit based on a majority decision between an output obtained from a connection group and an output obtained from the inhibitory connection group.

According to a fifth aspect of the present invention, in the hierarchical signal processing device comprising a plurality of sets each having a logical operation means, a final output signal outputted from the logical operation means and the logical operation means A comparison is made with the corresponding teacher signal to generate an error signal in the logical operation means, wherein a signal existing only in the teacher signal is a positive error signal and a signal existing only in the final output signal is a negative error signal. An output means, and an excitatory coupling coefficient signal representing a coupling state between the arithmetic means constituting the other aggregate in a logical arithmetic means within one aggregate which provides an output signal to the arithmetic means constituting another aggregate; and A coupling coefficient signal consisting of at least one of the suppression coupling coefficient signal and an error consisting of a positive error signal and a negative error signal in the arithmetic means constituting the other aggregate. Using the signal, based on the excitatory coupling coefficient signal and the positive error signal of the coupling coefficient signal, and the suppressive coupling coefficient signal and the negative error signal of the coupling coefficient signal Performing a logical operation to generate a positive error signal in the logical operation means in the certain set,
Of the coupling coefficient signals, the positive error signal in the suppression coupling coefficient signal and the other aggregate, and, based on the excitatory coupling coefficient signal and the negative error signal in the coupling coefficient signal, Error signal generating means for performing a logical operation to generate a negative error signal in the logical operation means in the certain set, all input signals input to the logical operation means constituting the other set, and the logical operation means The coupling coefficient signal is controlled on the basis of the positive error signal and the negative error signal in the above, and a coupling coefficient signal indicating a coupling state between the logical means and the arithmetic means which constitutes the certain assembly which provides the output signal to the logical arithmetic means. And a coupling coefficient control means. 7. The signal processing device according to claim 6, further comprising a first aggregate having a logical operation means, a final aggregate, and an output signal received from the first aggregate and supplying an output signal to the final aggregate. An input signal is provided to the first aggregate by mutually transmitting and receiving signals between a logical operation unit in the aggregate that is an aggregate and a logical operation unit in another aggregate that is an aggregate. By comparing a final output signal output from the final assembly with a specific teacher signal and controlling all coupling coefficients between the logical operation means based on the comparison result, the input given In the hierarchical signal processing device configured to converge the final output signal from the logical operation means in the final set obtained for the signal to the teacher signal, before the signal is output from the logical operation means in the final set. The final output signal is compared with a teacher signal corresponding to the logical operation means, and a signal existing only in the teacher signal is regarded as a positive error signal, and a signal existing only in the final output signal is regarded as a negative error signal. A connection state between the comparison output means for generating an error signal in the arithmetic means and the arithmetic means constituting the other aggregate in the logical arithmetic means in one aggregate for providing the output signal to the arithmetic means constituting another aggregate From a coupling coefficient signal composed of at least one of an excitatory coupling coefficient signal and an inhibitory coupling coefficient signal, and a positive error signal and a negative error signal in arithmetic means constituting the other aggregate. Using the following error signal, the excitatory coupling coefficient signal and the positive error signal among the coupling coefficient signals, and the suppressive coupling coefficient signal and the negative error among the coupling coefficient signals. Signal to generate a positive error signal in the logical operation means in the certain set, and a suppressive coupling coefficient signal among the coupling coefficient signals and the positive error signal in another set. And an error signal generating means for performing a logical operation based on the excitatory coupling coefficient signal and the negative error signal among the coupling coefficient signals to generate a negative error signal in the logical operation means in the certain set And all the input signals input to the logical operation means constituting the other aggregate, the positive error signal and the negative error signal in the logical operation means, and the output signal to the logical operation means. Coupling coefficient control means for controlling the coupling coefficient signal based on a coupling coefficient signal indicating a coupling state with the arithmetic means constituting the aggregate.

[0027] The signal processing apparatus according to claim 7 is the first embodiment.
A neural cell mimic circuit in the described signal processing device has learning constant setting means for arbitrarily setting a learning constant from outside the neural cell mimic circuit. In the signal processing device according to the eighth aspect, in the configuration of the signal processing device according to the first aspect, the input unit includes a plurality of input lines for receiving a binary input signal, and the supply unit includes a coupling coefficient. Memory means for indicating one of the excitatory coupling group or the inhibitory coupling group that is present and storing grouping information corresponding to the coupling coefficient,
Logical operation means for obtaining, for each of the binary input signals, a logical product of one of the binary input signals received from the input line and a coupling coefficient corresponding to the input line read from the memory means. And a second gate means for obtaining a logical product of one of the grouping information read from the memory means and a corresponding logical product output output from the first gate means And third gate means for obtaining a logical product of one inversion information of the grouping information read from the memory means and a corresponding logical product output outputted from the first gate means; Fourth gate means for obtaining the logical sum of the logical product outputs of the second gate means, and fifth gate means for obtaining the logical sum of the logical product outputs of the third gate means An inverter for inverting a logical sum output of the fifth gate means, and a gate means for obtaining a logical product or a logical sum of the logical sum output of the fourth gate means and the logical sum output inverted by the inverter And output means including: According to a ninth aspect of the present invention, in the signal processing apparatus according to the first aspect, the input unit includes a plurality of input lines for receiving a binary input signal, and the supply unit stores the coupling coefficient. A first memory means and a second memory means, wherein the logical operation means is configured to output one of the binary input signals received from the input line and the input line read from the first memory means. The logical product with the coupling coefficient corresponding to
First gating means for obtaining each of the binary input signals; one of the binary input signals received from the input line and a coupling coefficient corresponding to the input line read from the second memory means; Second gating means for obtaining the logical product of each of the binary input signals;
Third gate means for obtaining the logical sum of the logical product outputs of the first gate means, fourth gate means for obtaining the logical sum of the logical product outputs of the second gate means, An inverter for inverting the logical sum output of the fourth gate means; and a gate for obtaining a logical product or logical sum of the logical sum output of the third gate means and the logical sum output inverted by the inverter. Output means. According to a tenth aspect of the present invention, there is provided a signal processing apparatus according to the ninth aspect.
The signal processing device according to claim 1, wherein the first memory means, the first memory means,
And the third gate means form an excitatory coupling group, the second memory means, the second gate means, the fourth gate means, and the inverter form an inhibitory coupling group, and A gating means, a fourth gating means, an inverter and an output means for determining an output signal of the neuron unit based on a majority decision on an output obtained from the excitatory connection group and an output obtained from the inhibitory connection group. It forms the majority decision means.

According to another aspect of the present invention, there is provided a signal processing apparatus for processing a plurality of input signals to a neuron unit and outputting a result of the processing as an output signal. Forward processing means for performing respective arithmetic processing with an input signal to be processed and outputting the result of the arithmetic operation as an output signal; and the coupling coefficient based on an error signal representing a difference between an output signal obtained by the forward processing means and a teacher signal. A neural cell mimicking circuit comprising self-learning means for learning by controlling
The difference is a first error component consisting of a pulse present on the teacher signal side when the teacher signal differs from the output signal, and a second error component consisting of a pulse present on the output signal side. A signal processing device according to claim 12, wherein the signal processing device processes a plurality of input signals to a neuron unit and outputs a processing result as an output signal, wherein the plurality of coupling coefficients and an input signal corresponding to the coupling coefficients are provided. And a forward processing means for outputting the result of the calculation as an output signal, and controlling the coupling coefficient based on an error signal representing a difference between the output signal and the teacher signal obtained by the forward processing means. Equipped with a neuron mimicry circuit consisting of self-learning means for learning
The difference is made up of a first error component and a second error component having non-negative values, and the difference between the output signal obtained by the forward processing means and the teacher signal is the first error component and the second error component. 2 is equal to the difference from the error component 2. According to a thirteenth aspect of the present invention, in the signal processing device of the twelfth aspect, the first error signal is calculated by a logical product of an output signal output from the forward processing means and a signal obtained by logically negating the teacher signal. The second error signal is calculated by the logical product of a signal obtained by logically negating the output signal output from the forward processing means and the teacher signal. Further, the signal processing device according to claim 14 relates to the signal processing device according to claim 12,
The signal processed by the forward processing means is synchronized with the signal processed by the self-learning means. Further, the signal processing device according to claim 15 relates to the signal processing device according to claim 1, 5, 6, 11, 12, or 14, and outputs at least an input signal, an output signal, a coupling coefficient signal, a teacher signal, and an error signal. , A signal represented by a pulse train representing the pulse density.

According to a sixteenth aspect of the present invention, in the signal processing method for processing a plurality of input signals to a neuron unit and outputting a result of the processing as an output signal, a plurality of coupling coefficients and an engagement coefficient of the plurality of coupling coefficients are determined. Performing each operation with a corresponding input signal and executing a forward process of outputting the operation result as an output signal;
Controlling the coupling coefficient based on an error signal indicating a difference between the output signal and the teacher signal obtained by the forward process and executing a self-learning process,
The difference is a first error component consisting of a pulse present on the teacher signal side when the teacher signal differs from the output signal, and a second error component consisting of a pulse present on the output signal side. A signal processing method according to claim 17, wherein a plurality of coupling coefficients and an input signal corresponding to these coupling coefficients are processed in a signal processing method for processing a plurality of input signals to a neuron unit and outputting a result of the processing as an output signal. Performing a forward process of performing each of the arithmetic processing of and a calculation result as an output signal,
Controlling the coupling coefficient based on an error signal indicating a difference between the output signal and the teacher signal obtained by the forward process and executing a self-learning process,
The difference comprises a first error component and a second error component having non-negative values, and a difference between an output signal obtained by the forward process and a teacher signal is a difference between the first error component and the second error component. Is made equal to the difference from the error component.
The signal processing method according to claim 18 is the method according to claim 16 or 17.
Regarding the described signal processing method, the first error signal is calculated by the logical product of the output signal output from the forward process and the signal obtained by logically negating the teacher signal, and the second error signal is output from the forward process. The output signal is calculated by the logical product of a signal obtained by logically negating the output signal and the teacher signal. A signal processing method according to claim 19 is the signal processing method according to claim 16 or 17, wherein a signal processed by a forward process and a signal processed by a self-learning process are synchronized. It is. Further, in the signal processing method according to claim 20, in the signal processing method according to claim 16 or 17, at least the input signal, the output signal, the coupling coefficient signal, the teacher signal, and the error signal are represented by a pulse train representing a pulse density. This is the signal to be performed.

[0030]

According to the signal processing circuit, the signal processing circuit network, the signal processing device or the signal processing method, the functions of the neural cell network including the self-learning function can be performed in parallel on the hardware. The self-learning function is exhibited, and the processing speed is remarkably improved as compared with the calculation by the serial processing of the conventional computer simulation. At this time, the operation is assured by the digital circuit configuration. In particular, since all the signals are digital signal processing based on the pulse density expression, there is no inconvenience in the analog system such as the influence of the temperature characteristics of the amplifier.

[0031] In particular, with the configuration according to the fifth or sixth aspect of the present invention, the function of the neuron can be realized by software, and the network configuration can be changed only by changing the software. A flexible and versatile network can be constructed.

At this time, there are two types of binding, excitatory and inhibitory.
There are types, but as in the invention according to claim 8, 9 or 10, after dividing into two groups according to the sign of the coupling coefficient and processing each group, the logical product of one negative result and the other result or The processing may be a logical sum or a flexible processing determined by majority logic based on the ratio of the two, and thus can be realized by a digital circuit as a logic circuit.

The coupling coefficient is also prepared on the memory, and is rewritable and versatile, unlike the case of using a resistor or the like.

Further, since the learning coefficient is made variable, it becomes efficient and easy to use in accordance with the actual application environment.

[0035]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described with reference to FIGS. This embodiment has a self-learning function. To enable self-learning, the coupling coefficient must be variable. FIG. 2 shows a circuit configuration of the coupling coefficient variable circuit 20 for this purpose. That is, a large number of resistors 21 representing the coupling coefficient are prepared, and the coupling coefficient is made variable by appropriately switching the resistance by the switching circuit 22. The switching circuit 22 may be a commercially available circuit that can be switched by inputting a binary number from an external controller.
Here, an A / D converter 23 that converts an analog value such as a voltage value from an external controller into a binary number is used. The switching circuit 24 is also switched and controlled using the sign bit of the A / D converter 23, and switching between the excitatory output and the suppressive output is performed by selecting whether or not to pass through the inverting amplifier 25. As a result, the coupling coefficient becomes variable according to the external signal S, and an output obtained by multiplying the input signal by an arbitrary coupling coefficient is obtained.

FIG. 3 shows equations (1) and (2) using such a coupling coefficient variable circuit 20. Here, the coupling coefficient variable circuit 20 has a function of multiplying the input from the immediately preceding layer by each coupling coefficient, and is input to the addition circuit 26 and added. The addition circuit 26 can be easily realized by using a commercially available operational amplifier 27 as shown in FIG. 4, for example. Here, the operational amplifier 27 is for addition, but has an inverting amplifier configuration, so that the operational amplifier 27 is further inverted using the amplifier 28 and output. A non-linear amplifier 29 having an input / output relationship as shown in equation (2) is connected to the output side of the adder circuit 26. The input signal corresponds to the internal potential of equation (1).

Next, a method of creating the external signal S for determining the coupling coefficient will be described. This is (5) ~
This is equivalent to the equation (7), and requires a circuit for realizing this. First, f 'in the expressions (5) and (6) is a first-order differential function of the sigmoid function f shown in FIG. 35, and has a characteristic as shown in FIG. For example, as shown in FIG.
To 35 are connected in multiple stages, and constituted by a circuit having a nonlinear mountain-shaped characteristic. The input / output characteristics of the f 'signal generation circuit 30 are as shown in FIG. Although this circuit 30 does not necessarily realize the characteristics as shown in FIG. 5 exactly, it can be said that it approximately holds. If an amplifier (not shown) having non-linear input / output characteristics is provided before inputting the signal to the f 'signal generation circuit 30, the input / output characteristics are, for example, as shown in FIG. It will approach the characteristics.

FIG. 9 shows an example of the error signal generating circuit 36 corresponding to the equation (5). The circuit 30 in the figure is as shown in FIG. 6, and performs a function process as shown in FIG. 7 or FIG. 8 on the internal potential (input to the amplifier 29 in FIG. 3). On the other hand, there is provided a subtraction circuit 37 for taking the difference between the neuron unit output of the output layer and the teacher signal. This subtraction circuit 37 is shown in FIG.
In this case, one input may be inverted by an amplifier in advance using a circuit as shown in FIG. The outputs of these circuits 30 and 37 are input to a multiplication circuit 38 and multiplied to obtain a result similar to the equation (5).

FIG. 10 shows an example of the error signal generating circuit 39 corresponding to the equation (6). That is, in addition to the coupling coefficient variable circuit 20, the adding circuit 26, and the f 'signal generating circuit 30 having the above-described circuit configuration, a multiplying circuit 40 for obtaining the product of the outputs of these circuits 26 and 30 is provided. Such a configuration
This is equivalent to equation (6). Therefore, by previously inputting the error signal and the internal voltage generated by the circuit 36 as shown in FIG. 9 or the circuit 39 as shown in FIG. 10 in another layer, (6) An output similar to the expression is obtained. Further, the coupling coefficient generation circuit 4 corresponding to the equation (7)
1 can be realized by a multiplication circuit. FIG. 11 illustrates this,
First, a multiplication circuit 42 is provided. This may be various commercially available ones, and takes the product of the output of the neuron of a certain layer, the error signal created by the above-described circuit, and the constant η.
The output of the multiplication circuit 42 is input to an addition circuit 43, and a new T is generated from T and ΔT using a delay circuit 44. Therefore, the output from the adding circuit 43 is equivalent to the equation (7).

FIG. 1 shows an example of a nerve cell mimic circuit 45 constructed by combining these circuits. That is, the circuit 45 shown in FIG. 1 corresponds to a portion surrounded by a broken line in FIG. 33 on the network. The block B1 corresponds to the circuit shown in FIG. 3, and its operation output is sent to the next neuron. The block B2 corresponds to the error signal generation circuit 39 shown in FIG. That is, the block B2-1 of the next layer and the block B2-2 of the neuron shown in this figure have exactly the same configuration as that of FIG. 10, and similarly, the block B2-1 of the neuron immediately before the block B2-1. The block B2-2 has the same configuration as that of FIG. Since the entire network has a multilayer structure as shown in FIG. 33, it is equivalent even if the block of the error signal generation circuit 39 is cut in the middle and divided into two. Block B3-1,
B3-2 to B3-N correspond to the coupling coefficient generation circuit 41 and the A / D converter 23 shown in FIG.
In the figure, the delay circuit 44 is not shown). Using the coupling coefficients T newly obtained in the blocks B3-1, B3-2,..., B3-N, each coupling coefficient is changed by the coupling coefficient variable circuit 20 shown in FIG. Since the same coupling coefficient is used in two places of the blocks B1 and B2-1, the two are interlocked and variable as shown in FIG. That is, block B in FIG.
The circuits 2-1 and B3 and the portion of the coupling coefficient variable circuit 20 in the block B1 correspond to a self-learning circuit, and the remaining portion of the block B1 and the portion of the block B2-2 correspond to a nerve cell mimic element. In other words, the configuration example of the nerve cell mimic circuit 45 shown in FIG. 1 shows a configuration example of the signal processing circuit according to the first aspect of the present invention.

The blocks constructed as shown in FIG. 1 are connected in a network as shown in FIG. 33 to form a network, and an error signal generating circuit 36 as shown in FIG. 9 is attached to the output portion of the final output layer. Then, a neural network can be realized.

The above circuit configuration will be described using a specific example. First, a commercially available general-purpose operational amplifier is used for each of the addition circuits and the like in each block described above, and a 256-input 2
FIG. 1 shows a neuron mimic circuit 45 as shown in FIG.
A number of coupling coefficient generation circuits 41 such as 1 were produced. Next, the input and output lines of these circuits 45 and 41 were connected to form a network. The network configuration was a three-layer configuration. The first layer was composed of 256 neurons, the second layer was composed of four neurons, and the third layer was composed of five neuron mimic circuit units. Further, the output of the third layer was connected to an error signal generating circuit 36 as shown in FIG. When some input is given to each unit of the first layer of such a network, the output result is not always desired at first, but since the self-learning circuit is provided, the output result eventually becomes the desired result. That is, a teacher signal.

An example in which this network is used to further apply to a self-learning type character recognition device will be described. First, handwritten characters as shown in FIG. 12 were read by a scanner, divided into 16 × 16 meshes, and each mesh was used as an input to each neuron in the first layer of the network. A mesh with a character portion was input as 1V, and a mesh without text was input as 0V. The output was connected to a voltmeter, and the output result was directly displayed. The position of the unit with the largest output among the five units is made to be the recognition result, and therefore,
When a number from "1" to "5" is input, learning is performed so that the output of the number corresponding to the number becomes the largest. The recognition rate is 100% for sufficiently learned characters.

Next, a second embodiment of the present invention will be described with reference to FIG.
17 through FIG. This embodiment aims at digitization. Basically, input / output signals, intermediate signals,
Coupling coefficients, teacher signals, and the like are all represented by pulse trains represented by binary values of “0” and “1”. The amount of the signal inside the network is represented by the pulse density (the number of “1” within a certain time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed on the memory. Learning is realized by rewriting this pulse train. In learning, an error is calculated based on a pulse train from a given teacher signal, and a coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the calculation of the change in the coupling coefficient are all performed by the logical operation of the pulse train of “0” and “1”. It is like that.

Hereinafter, a description will be given based on an example in which this concept is embodied. First, the configuration of the signal calculation section will be described with reference to FIG. FIG. 13 shows a portion corresponding to one neuron mimic circuit 50, and the network configuration is a hierarchical type as in the case of FIG. All inputs and outputs are "1"
The one binarized to “0” and synchronized is used. The signal intensity of the input y _i is represented by a pulse density, for example, by the number of states of “1” within a certain period of time as shown in the following pulse train. That is,

(Equation 2) Is an expression representing 4/6, and four signals “1” and two signals “0” are included in six synchronization pulses. At this time,
It is desirable that the arrangement of “1” and “0” be random as described later.

On the other hand, the coupling coefficient T _ij is similarly represented by a pulse density, and is prepared in advance in a memory as a pulse train of “0” and “1”.

[0047]

(Equation 3) Is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of “1” and “0” be random. How to determine specifically will be described later.

Then, this pulse train is sequentially read from the memory in accordance with the synchronous clock, and as shown in FIG. 13, an AND gate 51 performs a logical product with the input signal pulse train (y _i ∩T _ij ). This is an input to the nerve cell j. In the case of the above example, the input signal is “10110
When input as "1", a pulse train is called out from the memory in synchronization with this, and AND is sequentially taken to obtain

(Equation 4) "101000" is obtained as shown in FIG.
It is shown that _i is converted by the coupling coefficient T _ij and the pulse density becomes 2/6.

The pulse density of the output of the AND gate 51 is
Approximately, it is the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This means that the longer the signal train,
The more random the arrangement of "1" and "0", the closer the function to a product. When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no more data to be read, it is sufficient to return to the beginning of the data and repeat the reading.

Since one nerve cell unit has a large number of inputs, there are many "ANDs between input signals and coupling coefficients" described above, and the logical sum of them is then calculated by an OR circuit 52 which is a logic circuit. Since the inputs are synchronized, for example 1
When the second data is “101000” and the second data is “010000”, an OR of both is “111000”.
1000 ". This is calculated at the same time as the multi-input and output. That is, if the number of inputs is m,

(Equation 5) become that way. This corresponds to the sum calculation and the non-linear function (sigmoid function) in the analog calculation.

When the pulse density is low, the pulse density obtained by ORing the pulses approximately matches the sum of the respective pulse densities. As the pulse density increases, the OR circuit 52
Since the output becomes increasingly saturated, the output does not match the sum of the pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and furthermore, it is a monotonically increasing function, which is approximately similar to the sigmoid function.

By the way, the coupling has excitability and suppression. In the case of numerical calculation, it is represented by the sign of the coupling coefficient. In the case of an analog circuit, when T _ij is negative as described above (suppression coupling). Uses an amplifier to invert the output and couple it to other nerve cells with a resistance value equivalent to T _ij . In this regard, in this embodiment of the digital system, first, each coupling is divided into two groups, an excitatory coupling and an inhibitory coupling, _{depending on} the sign of T _ij . A
The OR of “ND” is calculated for each group. And when only the output of the excitatory connection group is “1”,
It outputs “1” and outputs “0” when only the output of the inhibitory connection group is “1”. When both are “1” or “0”, either “1” or “0” may be output, or “1” may be output with a probability of about ２. In this example, the output “1” is output only when the output of the excitatory connection group is “1” and the output of the inhibitory connection group is “0”. In order to realize this function, AND of (NOT of the output of the inhibitory connection group) and (output of the excitatory connection group) may be obtained. That is,

(Equation 6) Becomes

When expressed by a logical expression,

(Equation 7) Indicated by The network of the neuron unit is of a hierarchical type similar to the back propagation (ie, FIG. 33). And if you keep the whole network synchronized,
Each layer can be calculated by the function described above.

On the other hand, each connection is divided into two groups, excitatory connection and inhibitory connection, _{depending on} the sign of T _ij .
OR of "AND of pulse train of input signal and coupling coefficient"
Is calculated for each group, and then the output of the excitatory connection group is “0” and the output of the inhibitory connection group is “1”.
If an output is to be output other than in the case of (1), the OR of (NOT of the output of the inhibitory connection group) and (the output of the excitatory connection group) may be obtained. That is,

(Equation 8) Becomes

When expressed by a logical expression,

(Equation 9) Indicated by

Next, the processing at the time of learning will be described. a. Error Signal in Last Layer The error signal in each neuron is calculated in the last layer, and the coupling coefficient related to the neuron is changed based on the error signal.
The method of calculating the error signal for that purpose will be described. Here, in the present embodiment, the “error signal” is defined as follows. When the error is expressed by a numerical value, both + and-can be generally taken. However, in the case of the pulse density, since both the positive and the negative cannot be expressed simultaneously, the signal representing the + component and the signal representing the-component The error signal is expressed using the two types. That is, j
The error signal of the neuron is

(Equation 10) Indicated by That is, the + component of the error signal is the pulse existing on the teacher signal side in the part (1, 0) or (0, 1) where the teacher signal pulse and the output pulse are different,
The component is a pulse which is likewise present at the output. In other words, adding an error signal + pulse to the output pulse and removing the error signal-pulse results in a teacher pulse. Based on such an error signal pulse, the coupling coefficient is changed as described later.

B. Error Signal in Intermediate Layer Further, the error signal is back-propagated, so that not only the coupling coefficient between the final layer and the immediately preceding layer but also the coupling coefficient of the preceding layer changes. Therefore, it is necessary to calculate an error signal at each neuron in the intermediate layer. Propagating a signal from a neuron in the hidden layer to each neuron in the next layer further is the same as collecting error signals in each neuron in the next layer in exactly the opposite way, and then calculating its own error. Signal. This means that the arithmetic expression in the neuron unit is
It can be performed in the same manner as in (7) to (10). That is, the connection is first divided into two groups depending on whether it is excitatory or inhibitory, and the multiplication part is represented by AND and the Σ part is represented by OR. However, the difference from the equations (7) to (10) in the nerve cell unit is that while y is one signal, δ has two signals representing positive and negative, Needs to be considered. Therefore, it is necessary to divide into four cases according to the sign of T and the sign of δ.

First, the case of excitatory coupling will be described. In this case, the error signal at the k-th neuron in the next layer +
And the coupling coefficient between the neuron and itself (assumed to be j-th) is ANDed (δ _k (+) ∩T _ik ) for the neuron, and further ORed between them (｛∪ (δ)
_k (+) {T _ik )}. This is converted to the error signal of this layer + (= δ
_j (+)). That is, if the number of neurons in the next layer is n,

[Equation 11] Becomes

Also, by taking the AND of the next error signal − and the coupling coefficient, and further taking the OR of them, the error signal − (= δ _j (−)) of this layer is similarly obtained. .
That is,

(Equation 12) Becomes

Next, the case of inhibitory binding will be described. In this case, AND of the error signal of the neuron of the next layer and the coupling coefficient between the neuron and itself is performed, and further, OR of these is calculated. This is converted to the error signal of this layer +
(= Δ _j (+)). That is,

(Equation 13) Becomes

Further, by taking the AND of the error signal + and the coupling coefficient, which are one step ahead, and by taking the OR of them, the error signal of this layer is similarly obtained as − (= δ _j (−)). .
That is,

[Equation 14] Becomes

Some neurons are connected by excitability from one neuron to another neuron, while others are connected by inhibition. Therefore, the error signal δ _j (+) obtained by the equation (20) is obtained. When
An OR with the error signal δ _j (+) obtained by the equation (22) is taken, and this is used as the error signal δ _j (+) of the own neuron. Similarly, (2
The OR of the error signal δ _j (−) obtained by the equation (1) and the error signal δ _j (−) obtained by the equation (23) is taken as the error signal δ _j (−) of the own neuron.

To summarize the above,

(Equation 15) Becomes

Further, a function corresponding to the learning rate (learning constant) may be provided. When the rate is 1 or less in the numerical calculation, the learning ability further increases. This can be realized by thinning out the pulse train in the calculation of the pulse train. This is based on a counter concept, and is as shown in the following Examples 1) and 2). For example, when η = 0.5, every other pulse train of the original signal is thinned out. Even if the pulses of the original signal are not at equal intervals, a method of thinning out every other pulse of the original pulse train (the method of <Example 2>) was adopted.

[0065]

(Equation 16)

In this way, the function of the learning rate is provided by thinning out the error signal. Such thinning of the error signal can be easily realized by performing a logical operation on the output of a commercially available counter or using a flip-flop. In particular, when a counter is used, the value of the learning constant η can be set arbitrarily and easily, so that the characteristics of the network can be controlled.

Incidentally, it is not necessary to always multiply the error signal by a learning constant, and the error signal may be used only for, for example, an operation for obtaining a coupling coefficient described below. Further, the learning constant used when the error signal is propagated in the opposite direction may be different from the learning constant used in the calculation for calculating the coupling coefficient. This means that the characteristics of the nerve cell units placed on the network can be individually set, and an extremely versatile system can be constructed. Therefore, it is possible to appropriately adjust the performance of the network.

C. Changing Each Coupling Coefficient Based on Error Signal The error signal is obtained by the above-described method, and each coupling coefficient is changed. How to change the coupling coefficient will be described. The AND between the signal flowing through the line to which the coupling coefficient to be changed belongs and the error signal is obtained (δ _j ∩y _i ). However, in this embodiment, since the error signal includes two signals of + and-, each is calculated.

[0069]

[Equation 17]

The two signals thus obtained are represented by ΔT
_ij (+) and ΔT _ij (-).

Next, based on this ΔT _ij , a new T
_ij is obtained. Since T _{ij in the} present embodiment is an absolute value component, it is classified according to whether the original T _ij is excitatory or suppressive. In the case of excitability, the component of ΔT _ij (+) is increased with respect to the original T _ij , and the component of ΔT _ij (−) is reduced. That is,

(Equation 18) Becomes In the case of suppression, the component of ΔT _ij (+) is reduced and the component of ΔT _ij (−) is increased with respect to the original T _ij . That is,

[Equation 19] Becomes

The network is calculated based on the above learning rules.

Next, an actual circuit configuration based on the above algorithm will be described. 14 to 16 show circuit examples. The entire network is the same as in FIG. FIG. 14 shows a circuit corresponding to a line (connection) in FIG. 33, and FIG. 15 shows a circuit corresponding to a circle (neural cell unit 1) in FIG. FIG. 16 shows a circuit for obtaining an error signal in the final layer from the output of the final layer and the teacher signal. By making these configurations in FIG. 14 to FIG. 16 into a network as shown in FIG. 33, a digital neural network capable of self-learning can be realized.

First, FIG. 14 will be described. In the figure, 55 is an input signal to the neuron, which corresponds to equation (8).
The value of the coupling coefficient in equation (9) is stored in the shift register 56. The shift register 56 has an outlet 56a and an inlet 56b, but may have the same function as a normal shift register, and may be, for example, a combination of a RAM and an address controller.
The input signal 55 and the coupling coefficient in the shift register 56 are A
A logic circuit 58 having an ND gate 57 and corresponding to equation (10)
Is ANDed. The outputs of the logic circuit 58 must be grouped according to whether the coupling is excitatory or inhibitory, but it is better to prepare the outputs 59 and 60 for each group in advance and switch between them. It becomes highly versatile. Therefore, in this embodiment, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 61, and the switching is performed by the switching gate circuit 62 using the information. The switching gate circuit 62 includes two AND gates 62a and 62b and an inverter 62c interposed at one input.

As shown in FIG. 15, gate circuits 63a and 63b having a plurality of OR gates corresponding to the equation (11) for processing each input are provided. Further, as shown in the figure, A that outputs an output “1” only when the excitatory connection group of Expression (12) is “1” and the inhibitory connection group is “0”
A gate circuit 64 including an ND gate 64a and an inverter 64b is provided.

Next, the error signal will be described. The error signal in the last layer is generated by the logic circuit 65 based on a combination of AND and exclusive OR shown in FIG.
Corresponds to the equation. That is, error signals 68 and 69 are generated by the output 66 from the last layer and the teacher signal 67. Equations (20) to (23) for calculating the error signal in the intermediate layer are shown in FIG.
4 is performed by a gate circuit 72 having an AND gate configuration, and outputs 73 and 74 corresponding to + and-are obtained. As described above, it is necessary to divide the case depending on whether the coupling is excitatory or inhibitory. In this case, the information is stored in the memory 61 as to whether the coupling is excitatory or inhibitory, and the +/− signals 75 and 7 of the error signal are used.
6, a gate circuit 7 having an AND or OR gate configuration
7. Also, the formula (24) to collect the error signal
Is performed by a gate circuit 78 having an OR gate configuration shown in FIG. Further, equation (25) corresponding to the learning rate is performed by a frequency dividing circuit 79 shown in FIG. Finally, a part for calculating a new coupling coefficient from the error signal, that is, (26) to (2)
The portion corresponding to the expression 9) is performed by a gate circuit 80 having an AND, inverter, and OR gate configuration shown in FIG.
The content of the shift register 56, that is, the value of the coupling coefficient is rewritten. The gate circuit 80 also needs to be classified depending on the excitability and suppression of the connection.

Here, the grouping method and the output determining method shown in FIGS. 14 and 15 are extracted and shown in FIG.
become that way. That is, the present invention corresponds to the twelfth aspect of the present invention. In the input stage, the input register is not divided into groups. A logic result provided by an AND gate (first gate means) 57ij is provided in accordance with the contents of a grouping memory (memory means) 61ij and an AND gate (second gate means) 62ai.
j, an inverter 62cij, and a switching circuit 62 composed of an AND gate (third gate means) 62bij, and divided into two groups. If the group is an excitatory coupling group, the OR gate (fourth gate means) 63a performs a logical OR operation OR gate (fifth gate means)
The logical sum is obtained on the 63b side. Thereafter, the output is determined by a logical product process by the gate circuit (output means) 64.

Now, a specific example will be described with reference to the case of the self-learning type character recognition device using the network described above. First, a handwritten character is read by a scanner, and a 16 × 16
, And the mesh with a character part is "1",
The non-existing mesh is set to “0” (the same as in FIG. 12).
This data (256 data) was input to the network, and the output was learned so that the position of the largest output among the five units became the recognition result. Next, the network configuration is such that the first layer has 256 networks and the second layer has 20 networks.
The third layer is composed of five neuron units. At this time, the input unit not connected is connected to the ground. Initially, if each coupling coefficient is set to be random, the output result is not always desired. Therefore, each coupling coefficient is newly obtained by using the self-learning function of this circuit, and this is repeated several times to obtain a desired result. In this embodiment, since the input is “0” or “1”, the input pulse train is always a simple L level or H level. The output is LED through the transistor
And turned off at L level and turned on at H level. Since the synchronous clock is set to 1000 kHz, the brightness of the LED changes for human eyes according to the pulse density, and therefore, the brightest LED portion is the answer. A recognition rate of 100% is obtained for sufficiently learned characters.

Incidentally, the method of grouping the excitatory connection and the inhibitory connection may be configured as shown in FIG. 18, for example. This corresponds to the thirteenth aspect of the present invention. In the input stage, the excitatory connection group a and the inhibitory connection group b are grouped in advance, and each input (input means) 55aij, 55bij For the coupling coefficient T
shift registers 81ai and 81bj as first and second memory means of at least 2 bits storing ij,
Further, AND gates (first gate means) 82aj, A
An ND gate (second gate means) 82bj is provided. Thereafter, an OR gate (third gate means) 63a and an OR gate (fourth gate means) 63b for each group
And so on.

As shown in FIG. 19, a gate circuit (output means) 64 is replaced by an AND gate 64a instead of an AND gate 64a.
Logical OR processing may be performed as a configuration using the R gate 64c. This is the processing of the aforementioned equations (12 ') to (15').

Next, a third embodiment of the present invention will be described with reference to FIG.
This will be described below. This embodiment corresponds to the invention of claim 7 and is provided with a learning constant setting means 82 for arbitrarily and variably setting a learning coefficient used in a coupling coefficient variable circuit from the outside. That is, the learning constant (learning rate) used at the time of learning shown in the above is added to the basic idea shown in the above embodiment.
Is variable, and a network circuit with performance that is suitable for the application is obtained. Function is added.

That is, the learning constant setting means 82
It is provided in place of the frequency dividing circuit 79 shown in the figure,
A counter 83 to which an error signal is input;
3 is composed of OR gates 84 to 87 for performing a logical operation on the output of 3 and processing the learning constant, and one AND gate 88.
More specifically, when all the switches Sa to Sd provided on the respective input sides of the OR gates 84 to 87 connected to the binary outputs A to D of the counter 83 are set to the H level, η = 1.
When all the switches Sa to Sd are set to the L level, η = 1/16. Therefore, if the number of switches on the H level side is N, η = (2 N) / 16
Becomes Therefore, the learning constant can be arbitrarily set by using a switch (or an external signal instead of the switch). When the pulse density is used as the clock input of the counter 83, an AND gate 89 may be appropriately provided for the input of the error signal. The learning constant setting means 82 is not limited to such a circuit configuration. Further, by providing a plurality of such learning constant setting means 82 or by appropriately controlling the learning constant using an external signal, the value of the learning constant used for calculating the coupling constant and the value of the learning constant used for back-propagation of the error signal can be reduced. Can be different.

Further, a fourth embodiment of the present invention will be described with reference to FIGS. This embodiment corresponds to the eighth aspect of the present invention. That is, two types of coupling coefficients, excitatory and inhibitory, are prepared based on the basic idea shown in 1 to 3 in the second embodiment, and the calculation result for the input signal is calculated using each coupling coefficient. Decide by majority rule from the percentage of results that have been received, and increase the flexibility of the network. Function is added.

That is, one neuron unit has two coupling coefficients, excitatory and inhibitory. The output result by “AND of input signal and coupling coefficient” is different from that of excitatory coupling. In this case, the treatment is performed at the ratio of the case of the inhibitory binding. Here, the processing by the ratio means the number of cases where the output result obtained by using the excitatory coupling coefficient is “1” for a plurality of input signals calculated in synchronization, Is compared with the number when the output result obtained is “1”, and when the latter is larger than the former, “0” is output, otherwise, “1” is output by the neuron unit. Means that. Alternatively, when both are equal, “0” may be output.

FIGS. 21 and 22 show an example of a circuit configuration for this purpose. First, shift registers 90a and 90 as a set of memories are individually provided for each input 55.
b is provided. These shift registers 90a,
A shift register 90b has a data outlet and a data inlet like the shift register 56, but one shift register 90a stores an excitatory coupling coefficient and the other shift register 90b stores an inhibitory coupling coefficient. It is.
The contents sequentially read from these shift registers 90a and 90b by a reading means (not shown) are input to the corresponding AND gates 91a and 91b together with the input 55, and the logical product is obtained. There are two types of such logical results, one having an excitatory connection and the other having an inhibitory connection. Here, the input is input to the majority circuit 92 and the output is determined. That is, the digital signal of the operation group using the excitatory coupling coefficient based on the shift register 90a is added by the amplifier 93a, and the digital signal of the operation group using the suppressive coupling coefficient based on the shift register 90b is similarly amplified. 93b, and the magnitude of both is compared by the comparator 9
4 determines the majority. The majority circuit 92 is not limited to the illustrated example, and may be a general majority circuit.

FIG. 23 shows the extracted grouping method shown in FIG. 21. That is, a set of memories (shift registers) storing the coupling coefficients of the excitatory coupling and the inhibitory coupling for each input is prepared, The process is performed until a logical product is obtained for each of the divided groups.

In the example shown in FIG. 23, instead of the majority decision circuit 92, OR gates 63a and 63b for performing OR operation for each group are shown in the same manner as in FIGS. In this case, the gate circuit 64 may be configured as shown in FIG. 18 or FIG.

In this embodiment, each input 55
Since each has one set of shift registers 90a and 90b,
Rewriting of the coupling coefficient by the self-learning function is also performed for each of the shift registers 90a and 90b. For this reason, as shown in FIG. 21, a self-learning circuit 95 is provided for performing the processing of the equations (21) to (24) for calculating a new coupling coefficient using the error signals of + and-. 90a, 90b
Is connected to the data entry side.

According to this embodiment, since the connection of the nerve cell units is not limited to the excitatory or the inhibitory, the network has flexibility and versatility in practical applications.

The frequency dividing circuit 79 in the case of FIG. 22 may be replaced with the learning constant setting means 82 as shown in FIG.

The output decision method by the majority circuit 92 is not limited to the method having two memories (shift registers 90a, 90b) for each input as shown in FIG. The same can be applied to those having the memory 56. That is, instead of the combination of FIG. 14 and FIG. 15, a combination of FIG. 14 and FIG.

A fifth embodiment of the present invention will be described with reference to FIGS. The second embodiment described above is a nerve cell mimic element constituted by circuits (hereinafter referred to as neuron circuits) as shown in FIGS. 13 to 23 and a network (circuit network) thereof. (8) to (2) even if all of these
The signal processing may be performed by software according to the procedure described in 9). The present embodiment is one example thereof.

That is, in this embodiment, the functions of the neurons constituting the network are realized by software. First, in the case of a network as shown in FIG. 33, signal processing is performed by software in an arbitrary neuron constituting the network. The number of neurons using software may be one or all, or may be determined for each layer forming a network. FIG. 24 shows the configuration of a neuron that does not perform signal processing by a neuron circuit. Here, the input / output device 101 is connected to another neuron using a neuron circuit or a device that inputs / outputs a signal to / from a network, and transmits / receives a signal. The memory 102 has a CPU 10
The software and data for controlling 3 are stored,
The signals are processed by the CPU 103. The signal processing procedure is as described above, but is shown in FIGS. 25 and 26 again. FIG. 25 shows an algorithm in the forward process, and such signal operation processing is performed in a digital circuit or a computer. FIG. 27 shows the anteroposterior relationship of the neurons during the processing shown in FIG. FIG. 26 shows an algorithm in the learning operation process, and such a signal operation process is performed in a digital circuit or a computer. FIG. 28 shows the relationship between the neurons during the processing shown in FIG. 26. Software is created according to the procedure shown in FIGS. 25 and 26 and stored in the memory 102. Here, one of the neurons in FIG. 24 can be made to function as a plurality of neurons by software. However, it is necessary to process the signal in a time-division manner.

By adopting such a configuration, it is possible to change the network configuration only by rewriting the memory 102 without making any hardware changes, thereby constructing a highly flexible and versatile network. Can be.

Further, a sixth embodiment of the present invention will be described with reference to FIG. In this embodiment, a part of the functions of one neuron is executed by software. In the configuration shown in FIG. 24, by storing software based on the signal processing procedure shown in FIG. 25 in the memory 102, it is possible to realize a neuron using software capable of executing a forward process.
In order to realize a neuron having a learning function, a circuit as shown in FIG. 14 or FIG. In any case, the right half of FIG. 15 and the circuit part shown in FIG. 16 are necessary. The circuit illustrated in FIG. 20 may be provided as appropriate. FIG. 29 shows a circuit for providing such a learning function as a learning circuit 104.

According to the present embodiment, as in the case of the fifth embodiment, the network configuration can be changed only by changing the software, and a highly flexible and versatile network can be constructed.

Further, when practically considered, ordinary electronic devices are often provided with a CPU in advance, so that it can be said that it is not necessary to newly provide components as shown in FIG. Furthermore, if the system does not require a learning function,
The amount of hardware can be greatly reduced.

A seventh embodiment of the present invention will be described with reference to FIG. In the present embodiment, the learning process function is realized by software. In the configuration shown in FIG. 24, by storing software based on the signal processing procedure shown in FIG. 26 in the memory 102, it is possible to realize a neuron using software capable of executing a learning process. In order to realize a neuron having a forward process function, the input / output device 101 needs to be configured as shown in FIG.
4 and FIG. 15, or FIG. 14 and FIG.
The circuit shown in FIG. The circuit illustrated in FIG. 19 may be provided as appropriate. FIG. 30 shows a circuit for providing such a forward process function as a forward circuit 105.

According to this embodiment, as in the fifth and sixth embodiments, the network configuration can be changed only by changing the software, and a network having high flexibility and versatility can be constructed. Become. In particular, it is easy to respond to a change in the learning rule. Also in this case,
Paying attention to the fact that ordinary electronic devices often have a CPU mounted in advance, it can be said that it is not necessary to newly provide a component as shown in FIG. Furthermore, if the system does not require a learning function, the amount of hardware can be greatly reduced.

According to the embodiment using the software (corresponding to the invention of claims 20 to 24), the signal processing method can be executed only by digital logic operation, so that the required software is low level. Language, and the software can be executed at high speed.

By the way, as a network structure of the neuron, for example, as shown in FIG.
1 or a structure as shown in FIG. FIG. 31 is a conceptual structural diagram corresponding to the invention according to claim 9 or claim 10, wherein the first aggregate 11
0, the intermediate aggregate 111, and the final aggregate 112 (the aggregate can be classified in this way even in FIG. 33),
This shows a state in which the nerve cell units 1 included in a certain aggregate (circles indicate logical operation means) are not connected to all of the nerve cell units 1 included in another aggregate. FIG.
3, all neural cell units 1 in a certain aggregate
Transmits and receives signals to and from all the neuron units in another assembly. As shown in FIG. 31, the neuron units 1 in each assembly are fully connected between the aggregations. It is not necessary.

FIG. 32 shows a four-layer network using the two-layer intermediate assemblies 113 and 114 between the first assembly 110 and the final assembly 112 according to the fifth aspect of the present invention. is there. Generally, an appropriate number of intermediate assemblies may be provided.

Further, FIG. 31, FIG. 32 and FIG.
In each of the figures, the number of neuron units 1 included in each assembly is shown as four, but these numbers are arbitrary as described in the specific examples in the embodiments, and The number of nerve cell units may be different.

[0104]

According to the present invention, since the signal processing circuit, the signal processing circuit network, the signal processing device and the signal processing method are configured as described above, the functions of the neural cell network including the self-learning function can be implemented on hardware. In parallel, the self-learning function can be demonstrated, and the processing speed can be remarkably improved compared to the calculation by the serial processing of the conventional computer simulation. At this time, the operation is assured by the digital circuit configuration In particular, since everything was digital signal processing by pulse density expression,
The inconvenience of the analog system, such as being affected by the temperature characteristics of the amplifier, does not occur.

In particular, with the configuration according to the fifth or sixth aspect of the present invention, the function of the neuron can be realized by software, and no hardware change is required, and only the software change is required. The configuration of the network can be changed, and a highly flexible and versatile network can be constructed.

There are two types of coupling, excitatory and inhibitory. As in the invention according to the eighth, ninth or tenth aspect, the coupling is divided into two groups depending on the sign of the coupling coefficient and processed for each group. After that, a process of calculating the logical product or the logical sum of one negative result and the other result, or a flexible process determined by majority logic based on the ratio of both, may be used, and therefore, it can be realized by a digital circuit as a logical circuit. Things.

The coupling coefficient is also prepared on the memory, and is rewritable and versatile, unlike the case of using a resistor or the like.

Furthermore, according to the seventh aspect of the present invention,
Since the learning coefficient is variable, it can be adapted to the actual application environment.
It can be efficient and easy to use.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one neuron mimic circuit in a first embodiment of the present invention.

FIG. 2 is a circuit diagram showing a coupling coefficient variable circuit.

FIG. 3 is a circuit diagram of a coupling coefficient multiplying circuit using the variable circuit.

FIG. 4 is a circuit diagram showing an example of the addition circuit.

FIG. 5 is a graph showing first-order differential characteristics of a sigmoid function.

FIG. 6 is a circuit diagram of an f ′ signal generation circuit.

FIG. 7 is a graph showing its input / output characteristics.

FIG. 8 is a graph showing examples of different input / output characteristics.

FIG. 9 is a circuit diagram of an error signal generation circuit.

FIG. 10 is a circuit diagram of an error signal generation circuit.

FIG. 11 is a circuit diagram of a coupling coefficient creation circuit.

FIG. 12 is an explanatory diagram.

FIG. 13 is a logic circuit diagram of a configuration example of a signal operation portion according to a second embodiment of the present invention.

FIG. 14 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 15 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 16 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 17 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 18 is a logic circuit diagram showing a modification.

FIG. 19 is a logic circuit diagram showing a modification.

FIG. 20 is a circuit diagram showing a third embodiment of the present invention.

FIG. 21 is a circuit diagram showing a fourth embodiment of the present invention.

FIG. 22 is a circuit diagram.

FIG. 23 is a circuit diagram.

FIG. 24 is a block diagram showing a fifth embodiment of the present invention.

FIG. 25 is a flowchart showing processing in a forward process.

FIG. 26 is a flowchart showing processing in a learning process.

FIG. 27 is a schematic diagram showing the before-and-after relationship of the Euron.

FIG. 28 is a schematic diagram showing a front-back relationship of a euron.

FIG. 29 is a block diagram showing a sixth embodiment of the present invention.

FIG. 30 is a block diagram showing a seventh embodiment of the present invention.

FIG. 31 is a conceptual diagram showing a modification of the network structure.

FIG. 32 is a conceptual diagram showing a different modification of the network structure.

FIG. 33 is a conceptual diagram of a neural network showing a conventional example.

FIG. 34 is a conceptual diagram showing one unit configuration.

FIG. 35 is a graph showing a sigmoid function.

FIG. 36 is a specific circuit diagram of one unit.

FIG. 37 is a block diagram illustrating a digital configuration example.

FIG. 38 is a partial circuit diagram thereof.

FIG. 39 is a partial other circuit diagram.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Logical operation means 20 Coupling coefficient variable circuit 41,80,95 Coupling coefficient generation circuit 45,50 Nerve cell imitation circuit 51,52,58,62-65,72,77-80
Digital logic circuit 56, 81, 90a, 90b Memory 61 Grouping memory 82 Learning constant setting means 92 Majority decision circuit 110-114 Aggregation

 ──────────────────────────────────────────────────続 き Continued on the front page (31) Priority claim number Japanese Patent Application No. 2-272827 (32) Priority date Hei 2 (1990) October 11 (33) Priority claim country Japan (JP) (56) References JP-A-1-173257 (JP, A) JP-A-1-204171 (JP, A) JP-A-1-244567 (JP, A) JP-A-1-193982 (JP, A)

Claims (20)

(57) [Claims] An input means for receiving an input signal, a supply means formed of a logic circuit for supplying a coupling coefficient, and performing a logical operation processing on each of the coupling coefficient and the input signal, and performing the operation processing Forward processing means coupled to the input means, comprising: logical operation means for outputting a result; coupling coefficient for generating a new coupling coefficient based on an error signal generated from an output signal of the logical operation means and a teacher signal Generating means, and self-learning means coupled to the forward processing means, comprising: coupling coefficient changing means for changing the coupling coefficient supplied by the supplying means to a new coupling coefficient generated by the coupling coefficient generating means. A signal processing device comprising a nerve cell mimicking circuit. 2. A signal processing apparatus according to claim 1, wherein said coupling coefficient generating means in said self-learning means is means for generating a new coupling coefficient based on the error signal and the learning constant. 3. An input means comprising: a plurality of first input lines for receiving a first binary input signal; and a plurality of second input lines for receiving a second binary input signal. Wherein the supply means includes first memory means and second memory means for storing coupling coefficients, and logical operation means comprises: the first binary input received from the first input line. First gating means for obtaining, for each of the first binary input signals, a logical product of one of the signals and a coupling coefficient corresponding to the input line read from the first memory means; The logical product of one of the second binary input signals received from a second input line and a coupling coefficient corresponding to the input line read from the second memory means is input to the second binary input signal. A second gate to obtain for each of the signals. Gate means; third gate means for obtaining a logical sum between logical product outputs of the first gate means; and fourth gate for obtaining a logical sum between logical product outputs of the second gate means. And an inverter for inverting a logical sum output of the fourth gate means, and a logical product and a logical sum of the logical sum output of the third gate means and the logical sum output inverted by the inverter are obtained. The signal processing apparatus according to claim 1, further comprising: output means including a gate for performing the operation. 4. The first input line, the first memory means, the first gate means and the third gate means form an excitatory coupling group, the second input line, the second memory means, A second gating means, a fourth gating means and an inverter forming an inhibitory coupling group, wherein the third gating means, the fourth gating means, the inverter and the output means comprise an output obtained from the excitatory coupling group; 4. The signal processing device according to claim 3, wherein a majority decision means for determining an output signal of the neuron unit based on a majority decision with respect to an output obtained from the inhibitory connection group is formed. 5. A hierarchical signal processing device comprising a plurality of aggregates each having logical operation means, wherein a final output signal output from said logical operation means is compared with a teacher signal corresponding to said logical operation means. A comparison output means for generating an error signal in the logical operation means in which a signal existing only in the leverage teacher signal is a positive error signal and a signal existing only in the final output signal is a negative error signal; In the logical operation means in one set which gives the output signal to the operation means constituting the above, the excitatory coupling coefficient signal and the suppressing coupling coefficient signal representing the coupling state with the operation means constituting the other aggregate A coupling coefficient signal comprising at least one of the signals and an error signal comprising a positive error signal and a negative error signal in the arithmetic means constituting the other aggregate; The logical sum is calculated based on the excitatory coupling coefficient signal and the positive error signal, and the suppressive coupling coefficient signal and the negative error signal among the coupling coefficient signals. Generating a positive error signal in the logical operation means of the above, the positive error signal in the other combination and the suppressing coupling coefficient signal among the coupling coefficient signal, and excitability among the coupling coefficient signal Error signal generating means for performing a logical operation based on the coupling coefficient signal and the negative error signal to generate a negative error signal in the logical operation means in the certain set; and logical operation means constituting the other set And the positive error signal and the negative error signal in the logical operation means, and the combined state of the arithmetic means constituting the certain assembly for providing the output signal to the logical operation means. Conclusion Based on the coefficient signal a signal processing apparatus characterized by comprising a coupling coefficient control means for controlling the coupling coefficient signal. 6. A first set, each having a logical operation means,
An intermediate aggregate that receives an output signal from the final aggregate and the first aggregate and supplies an output signal to the final aggregate; and a logical operation unit in the aggregate that is the aggregate and a logical operation unit in another aggregate. A signal is mutually transmitted and received between the logical operation means, and a final output signal output from the final aggregate when an input signal is given to the first aggregate is compared with a specific teacher signal. By controlling all coupling coefficients between the logical operation means based on the comparison result,
In a hierarchical signal processing apparatus configured to converge the final output signal from a logical operation unit in a final set obtained for the given input signal to the teacher signal, the logical operation unit in the final set Is compared with a teacher signal corresponding to the logical operation means, a signal existing only in the teacher signal is a positive error signal, and a signal existing only in the final output signal is a negative error signal. A comparison output means for generating an error signal in the logic operation means as a signal; and an operation constituting the other set in a logic operation means in one set for providing the output signal to the operation means constituting another set A coupling coefficient signal comprising at least one of an excitatory coupling coefficient signal and an inhibitory coupling coefficient signal representing a coupling state with the means, and forming the other aggregate Using an error signal composed of a positive error signal and a negative error signal in the arithmetic means, an excitatory coupling coefficient signal and the positive error signal among the coupling coefficient signals, and among the coupling coefficient signals, A logical operation is performed based on the suppression coupling coefficient signal and the negative error signal to generate a positive error signal in a logical operation unit in the certain set,
Of the coupling coefficient signals, the positive error signal in the suppression coupling coefficient signal and the other aggregate, and, based on the excitatory coupling coefficient signal and the negative error signal in the coupling coefficient signal, Error signal generating means for performing a logical operation to generate a negative error signal in the logical operation means in the certain set; all input signals input to the logical operation means constituting the other set; and the logical operation means The coupling coefficient signal is controlled on the basis of the positive error signal and the negative error signal in the above, and a coupling coefficient signal indicating a coupling state between the logical means and the arithmetic means which constitutes the certain assembly which provides the output signal to the logical arithmetic means. And a coupling coefficient control means. 7. The signal processing apparatus according to claim 1, wherein the neural cell mimic circuit has learning constant setting means for arbitrarily setting a learning constant from outside the neural cell mimic circuit. 8. The input means comprises a plurality of input lines for receiving a binary input signal, the supply means indicating one of an excitatory coupling group or an inhibitory coupling group to which the coupling coefficient belongs. Memory means for storing grouping information corresponding to coupling coefficients, wherein a logical operation means corresponds to one of the binary input signals received from the input line and the input line read from the memory means First gating means for obtaining a logical product of each of the binary input signals with a coupling coefficient to be obtained, and one of the grouping information read from the memory means and output from the first gating means Second gate means for obtaining a logical product with a corresponding logical product output; and one of the inversion information of the grouping information read from the memory means, Third gate means for obtaining a logical product of the corresponding logical product output output from the first gate means, and fourth fourth means for obtaining the logical sum of the logical product outputs of the second gate means. Gate means; fifth gate means for obtaining a logical sum of the logical product outputs of the third gate means; an inverter for inverting the logical sum output of the fifth gate means; and the fourth gate 2. The signal processing apparatus according to claim 1, further comprising output means including a gate means for obtaining a logical product or a logical sum of the logical sum output of the means and the logical sum output inverted by the inverter. . 9. The input means comprises a plurality of input lines for receiving a binary input signal, the supply means having first and second memory means for storing coupling coefficients, A logical operation means for calculating a logical product of one of the binary input signals received from the input line and a coupling coefficient corresponding to the input line read from the first memory means, for each of the binary input signals; And a logical product of one of the binary input signals received from the input line and a coupling coefficient corresponding to the input line read from the second memory means. Second gate means for obtaining each of the binary input signals; third gate means for obtaining a logical sum of AND outputs of the first gate means; and logic of the second gate means. product Fourth gate means for obtaining a logical sum of forces, an inverter for inverting a logical sum output of the fourth gate means, a logical sum output of the third gate means, and a logic inverted by the inverter 2. The signal processing device according to claim 1, further comprising: an output unit including a gate for obtaining a logical product or a logical sum with the sum output. 10. The first memory means, the first gate means and the third gate means form an excitatory coupling group, the second memory means, the second gate means, the fourth gate means and the inverter. Form an inhibitory coupling group, wherein the third gate means, the fourth gate means, the inverter and the output means determine a majority of an output obtained from the excitatory coupling group and an output obtained from the inhibitory coupling group. 10. The signal processing device according to claim 9, wherein majority decision means for deciding an output signal of the neuron unit based on the majority decision is formed. 11. A signal processing apparatus for processing a plurality of input signals to a neuron unit and outputting a result of the processing as an output signal, wherein each operation of a plurality of coupling coefficients and an input signal corresponding to these coupling coefficients is performed. A forward processing means for performing processing and outputting the operation result as an output signal; and a self processing means for learning by controlling the coupling coefficient based on an error signal representing a difference between the output signal obtained by the forward processing means and the teacher signal A neural cell mimicking circuit comprising learning means, wherein the difference is determined from a first error component consisting of a pulse present on the teacher signal side when the teacher signal and the output signal are different, and a pulse present on the output signal side. And a second error component. 12. A signal processing apparatus for processing a plurality of input signals to a neuron unit and outputting a result of the processing as an output signal, wherein each operation of a plurality of coupling coefficients and an input signal corresponding to these coupling coefficients is performed. A forward processing means for performing processing and outputting the operation result as an output signal; and a self processing means for learning by controlling the coupling coefficient based on an error signal representing a difference between the output signal obtained by the forward processing means and the teacher signal A neuron mimicking circuit comprising learning means, wherein the difference comprises a first error component and a second error component having non-negative values, and an output signal and a teacher signal obtained by the forward processing means. Is equal to the difference between the first error component and the second error component. 13. A first error signal is calculated by a logical product of an output signal output from the forward processing means and a signal obtained by logically negating the teacher signal, and a second error signal is output from the forward processing means. 13. An output signal obtained by logically negating the output signal and a teacher signal.
A signal processing device according to claim 1. 14. The signal processing apparatus according to claim 12, wherein a signal processed by the forward processing means is synchronized with a signal processed by the self-learning means. 15. The apparatus according to claim 1, wherein at least the input signal, the output signal, the coupling coefficient signal, the teacher signal and the error signal are signals represented by a pulse train representing a pulse density. , 12 or 14. 16. A signal processing method for processing a plurality of input signals to a neuron unit and outputting a result of the processing as an output signal, wherein each operation of a plurality of coupling coefficients and an input signal corresponding to the coupling coefficients is performed. Executing a forward process of outputting a result of the operation as an output signal, and controlling the coupling coefficient based on an error signal representing a difference between an output signal obtained by the forward process and a teacher signal, and performing self-learning. Performing the process, wherein the difference is a first error component comprising a pulse present on the teacher signal side when the teacher signal differs from the output signal, and a second error component comprising a pulse present on the output signal side. A signal processing method comprising: 17. A signal processing method for processing a plurality of input signals to a neuron unit and outputting a result of the processing as an output signal, wherein each operation of a plurality of coupling coefficients and an input signal corresponding to the coupling coefficients is performed. Executing a forward process of outputting a result of the operation as an output signal, and controlling the coupling coefficient based on an error signal representing a difference between an output signal obtained by the forward process and a teacher signal, and performing self-learning. Performing the process, wherein the difference comprises a first error component and a second error component having non-negative values, and the difference between the output signal obtained by the forward process and the teacher signal is A signal processing method, wherein the difference is equal to a difference between a first error component and the second error component. 18. A first error signal is calculated by a logical product of an output signal output from a forward process and a signal obtained by logically negating a teacher signal, and a second error signal is an output output from the forward process. 17. The method according to claim 16, wherein the signal is calculated by a logical product of a signal obtained by logically negating the signal and a teacher signal.
7. The signal processing method according to 7. 19. The signal processing method according to claim 16, wherein a signal processed by the forward process and a signal processed by the self-learning process are synchronized. 20. The signal processing according to claim 16, wherein at least the input signal, the output signal, the coupling coefficient signal, the teacher signal, and the error signal are signals represented by a pulse train representing a pulse density. Method.