JP3256553B2 - Learning method of signal processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for learning a signal processing device such as a neural computer which imitates nerve cells.

[0002]

2. Description of the Related Art The aim was to imitate the functions of nerve cells (neurons), which are the basic units of information processing in living organisms, and to make this "neural cell mimicry element" into a network for parallel processing of information. This is a so-called neural network. Here, the nervous system of the living body, especially the functions unique to the living body,
That is, many attempts have been made to imitate parallel processing, self-learning, and the like to achieve character recognition, associative memory, motion control, and the like. Many of these trials have been performed by computer simulations, and parallel processing is required in order to exhibit their original functions, and for this purpose, hardware implementation of a neural network is required. However, at present, there is no hardware realization.

[0003]

When a neural network is to perform some processing, it is necessary to first learn, but in the conventional case, processing is performed using software on a computer. The speed is very slow. In particular, in the case where the signal to the network changes over time, for example, in the case of exercise control, data must be once captured and stored using another method, and then learning must be performed using the data. It is not possible to obtain real-time learning.

[0004]

A memory storing rewritable coupling coefficient indicating a degree of coupling between the neurons mimic elements Means for Solving the Problems], varying the coupling coefficient by inverse propagate an error signal for the teacher signal A self-learning unit including a generating unit that generates a variable coupling coefficient value; and a coupling coefficient varying unit that varies the coupling coefficient stored in the memory by the variable coupling coefficient value generated by the generating unit. a plurality of neuronal cells mimicking means annexed to mimic element is provided a signal processing means connected to the network, to enter the time-varying input signal to said signal processing hand stage, the self-study
Learning by learning means according to the changing speed of the input signal
At predetermined time intervals .

[0005]

[0006]

Generating [action] a memory storing rewritable coupling coefficient indicating a degree of coupling between the neurons mimic elements, a variable coupling coefficient values for varying the coupling coefficient by inverse propagate an error signal for the teacher signal A self-learning means having a coupling coefficient varying means for varying the coupling coefficient stored in the memory by an amount corresponding to the variable coupling coefficient value generated by the generating means. According to the hardware configuration of the signal processing means in which the neuron mimic means are connected in a net-like manner, the processing speed including learning is extremely high. Here, the learning speed
If the change of the input signal is slower than the
That when it reverse learning efficiency may be deteriorated, there is an input signal changes with time also is input the input signal to such a signal processing hand stage, learning by self-learning means
A predetermined time according to the changing speed of the input signal
Improve learning efficiency by performing intermittently at intervals
be able to.

[0007]

[0008]

An embodiment of the present invention will be described with reference to the drawings. First, a learning method for a time-varying system will be described with reference to FIG. FIG. 1A shows a block diagram configuration at the time of learning, and FIG. 2B shows a block diagram configuration after learning. As a basic configuration, a neural network 2 and a teacher signal generation unit 3 which are signal processing units in a configuration described later are connected to a control target 1 which is a system for controlling motion. The teacher signal generating means 3 may be a human.

The control object 1 outputs a signal 4 representing the state. The signal 4 is generally an amount that changes with time, but the number of signals is not limited to one and is not limited to an electrical signal. The signal 4 is converted into a signal 6 suitable for input to the neural network 2 by preprocessing by the conversion means 5. For example, when the signal 4 indicates position information, the signal 6 is converted into a voltage value or a binary number. Similarly, the signal 4 is converted into a signal 8 suitable for input to the teacher signal generating means 3 by preprocessing by the converting means 7. Also in this case, for example, when the signal 4 represents the position information, the signal 6 is converted into a voltage value or a binary number. However, if the teacher signal generating means 3 is a human, the signal for observing the control target 1 is used. A television camera or the like may be used. The teacher signal 9 generated by the teacher signal generation means 3 is converted by the conversion means 10 into a teacher signal 11 suitable for the neural network 2 and is input to the neural network 2. Further, the teacher signal 9 is converted by the conversion means 12 into a control signal 13 suitable for the control target 1, and is input to the control target 1. After learning, the output signal 14 from the neural network 2 is converted into a control signal 16 suitable for the control target 1 by the conversion means 15 and input to the control target 1. These control signals 13 and 16 are generally
And the signal 4 does not always change. The conversion means 5, 7, 10, 12, 1
If 5 is unnecessary, it may be omitted as appropriate.

In such a configuration, for simplifying the description, for example, learning for applying an appropriate voltage to the motor in order to control the position of a certain system will be described. Here, it is assumed that the teacher signal generating means 3 is a human. First, the signal 4 as position information is converted into a signal 6 as a voltage value by the conversion means 5, and the signal 6 is input to the neural network 2.
On the other hand, a human (teacher signal generating means 3) operates the volume by looking at the system and attempts to control the position. The information of this volume is converted into a teacher signal 11 having a voltage value by the conversion means 10 and input to the neural network 2. Further, the information of the volume is converted into a control signal 13 of a voltage value by the conversion means 12 and input to the system (control target 1). The neural network 2 accepts a voltage value as an input signal and a teacher signal, and
If the processing speed is sufficiently high for the system, learning becomes possible.

The basic concept of the neural network 2 and an example of its configuration will be described with reference to FIGS. The neural network 2 of the present embodiment has a connection indicating the degree of connection between the respective neuron mimic elements.
Memory for storing rewritable coefficients and teacher signal generation
Error signal to the teacher signal generated by the means
Variable coupling coefficient that varies the coupling coefficient by sowing
Generating means for generating a value, and
The coupling coefficients stored in the memory for the variable coupling coefficient values
A self-learning means having a coupling coefficient variable means for making the variable is connected to a plurality of nerve cell mimic means attached to the nerve cell mimic element.

First, the neural network 2 in this embodiment is implemented by hardware using a digital configuration. The basic concept is that input / output signals, intermediate signals, coupling coefficients, teacher signals, etc. relating to a nerve cell unit are used. Are represented by pulse trains represented by binary values of “0” and “1”. The amount of the signal inside the network is represented by the pulse density (the number of “1” within a certain time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed on the memory. Learning is realized by rewriting this pulse train. For learning, an error is calculated based on a given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the calculation of the change in the coupling coefficient are all performed by the logical operation of the pulse train of “0” and “1”. In a network constituted by such nerve cell units, analog signals can be handled as input / output signals. It is like that.

Hereinafter, this concept will be described. First, a neural cell unit using a digital logic circuit and signal processing by the network circuit will be described, and then, input and output of an analog signal to and from the network circuit will be described.

First, regarding signal processing by a digital logic circuit, signal processing in a forward process will be described. FIG. 2 shows a portion corresponding to one neuron unit (neurocyte mimic element) 20, and the entire network (neural network 2) is of a hierarchical type, for example, as shown in FIG. All inputs and outputs are binarized to “1” and “0” and synchronized. The intensity of the input signal y _i is represented by a pulse density, for example, by the number of states of “1” within a certain time, such as a pulse train shown in FIG. That is, the example of FIG. 4 represents 4/6, and four signals “1” and two signals “0” are included in six synchronization pulses. At this time, the arrangement of “1” and “0” is desirably random as described later.

On the other hand, a coupling coefficient T _ij indicating the degree of coupling between the respective neuron units 20 is similarly expressed by a pulse density.
A pulse train of “0” and “1” is prepared in the memory in advance. The example in FIG. 5 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of “1” and “0” be random. How to determine specifically will be described later.

Then, the pulse train is sequentially read from the memory in accordance with the synchronous clock, and each pulse train is read out as shown in FIG.
The ND gate 21 takes a logical product with the input signal pulse train (y _i Ｔ T _ij ). This is an input to the nerve cell j. In the case of the above example, the input signal is “10110
When it entered as 1 ", which the call pulse train from the memory in synchronization, by taking sequential AND," 101000 "is obtained as shown in FIG. 6, which is the input y _i is the coupling coefficient T _ij And the pulse density is 2/6
It is shown that it becomes.

The pulse density at the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This means that the longer the signal sequence, the more "1"
The more random the arrangement of "0" and "0", the closer the function to the product of numerical values. Not random
It means that "1" (or "0") is dense (close). When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no more data to be read, it is sufficient to return to the beginning of the data and repeat the reading.

Since one nerve cell unit 20 has a large number of inputs, there are also a large number of "ANDs between input signals and coupling coefficients", and the logical sum of them is then calculated by an OR circuit 22 which is a logic circuit. Since the inputs are synchronized, for example, if the first data is “101000” and the second data is “010000”, ORing the two results in “1”.
11000 ". If this is calculated and output as multiple inputs simultaneously, for example, the result is as shown in FIG. This corresponds to the sum calculation and the non-linear function (sigmoid function) in the analog calculation.

When the pulse density is low, the OR of the pulse densities approximately matches the sum of the respective pulse densities. As the pulse density increases, the OR circuit 22
Since the output becomes increasingly saturated, the output does not match the sum of the pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and furthermore, it is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, coupling has excitability and suppression. In the case of numerical calculation, it is represented by the sign of the coupling coefficient. In the case of an analog circuit, when T _ij is negative (suppression coupling), the amplifier is connected. The output is inverted to _connect to another neuron unit with a resistance value corresponding to T _ij . In this regard, in this embodiment of the digital system, first, each coupling is divided into two groups, an excitatory coupling and an inhibitory coupling, _{depending on} the sign of T _ij . The OR of “AND” is calculated for each group. And when only the output of the excitatory connection group is “1”,
It outputs “1” and outputs “0” when only the output of the inhibitory connection group is “1”. When both are “1” or “0”, either “1” or “0” may be output, or “1” may be output with a probability of about ２. In this example, the output “1” is output only when the output of the excitatory connection group is “1” and the output of the inhibitory connection group is “0”. In order to realize this function, AND of (NOT of the output of the inhibitory connection group) and (output of the excitatory connection group) may be obtained. That is,
As shown in FIG.

When expressed by logical expressions, the following expressions (1) to (3) are obtained.

(Equation 1) Is shown.

The network of the neuron unit 20 is as follows.
Hierarchical type similar to back propagation (ie, FIG. 3)
And Then, if the entire network is synchronized, the calculation can be performed for each layer by the above-described function.

Next, a signal calculation process in learning (back propagation) will be described. Basically, an error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method c.

First, an error signal in the last layer will be described as a. In the final layer, an error signal in each neuron unit is calculated, and based on the error signal, a coupling coefficient relating to the neuron unit is changed. The method of calculating the error signal for that purpose will be described. Here, in the present embodiment, the “error signal” is defined as follows. When the error is expressed numerically,
In general, both + and-can be taken. However, in the case of pulse density, since both positive and negative can not be expressed simultaneously, an error is calculated using two types of signals, ie, a signal representing a + component and a signal representing a-component. Express the signal. That is, the error signal of the j-th nerve cell unit is shown as in FIG. That is, the + component of the error signal is a pulse existing on the teacher signal side in the part (1, 0) or (0, 1) where the teacher signal pulse and the output pulse are different, while the-component is similarly output. This is the pulse present on the side. In other words, adding an error signal + pulse to the output pulse and removing the error signal-pulse results in a teacher pulse. That is, when these positive and negative error signals δ _{j (+)} and δ _{j (−)} are expressed by logical expressions, they are expressed by the expressions (4) and (5), respectively. In the formula, XOR represents exclusive OR. Based on such an error signal pulse, the coupling coefficient is changed as described later.

[0025]

## EQU2 ## δ _{j (+)} ≡ (y _j EXOR _dj ) AND _dj ... (4) δ _{j (−)} ≡ (y _j EXOR _dj ) AND y _j ......... (5)

Next, a method for obtaining an error signal in the intermediate layer as b will be described. First, the above error signal is back-propagated, so that not only the coupling coefficient between the final layer and the immediately preceding layer but also the coupling coefficient of the preceding layer changes. Therefore, it is necessary to calculate an error signal in each neuron unit in the intermediate layer. The error signal in each neuron unit in the next layer is collected in exactly the opposite way that the signal is propagated from the neuron unit in the middle layer to each neuron unit in the next layer. And uses it as its own error signal. This can be performed in the same manner as in the above-described arithmetic expressions (1) to (5) and the cases shown in FIGS. 4 to 8 in the nerve cell unit. However, the difference from the above-described processing in the nerve cell unit is that while y is a single signal, δ has two signals as positive and negative signals, and both signals are considered. It is necessary. Therefore, it is necessary to divide into four cases according to the sign of the coupling coefficient T and the sign of the error signal δ.

First, the case of excitatory coupling will be described. In this case, with respect to a neuron unit having an intermediate layer, an error signal + at the k-th neuron unit in the next layer (final layer in FIG. 3) and the neuron unit and its own (the intermediate layer in FIG. 3) The AND of the coupling coefficient with the neuron unit with the difference (δ _{k (+)} ∩ T _jk ) is obtained for each neuron unit, and the OR between them is calculated as ｛∪ (δ _{k (+)} {T _jk )}. This is taken as the error signal + of this layer. That is, the result is as shown in FIG.

Similarly, an AND between the error signal and the coupling coefficient in the neuron unit in the next layer is obtained, and an OR between them is obtained to obtain an error signal in this layer. That is, the result is as shown in FIG.

Next, the case of inhibitory coupling will be described. In this case, the AND of the error signal in the neuron unit of the next layer and the coupling coefficient between the neuron unit and the self is obtained, and further, the OR of these is obtained. This is defined as an error signal + of this layer. That is, the result is as shown in FIG.

Further, the AND of the error signal + and the coupling coefficient, which is one step ahead, is taken, and further, the OR of these is taken, whereby the error signal of this layer is similarly obtained. That is, FIG.
It becomes as shown in.

Since there are some excitatory couplings from one neuron unit to another neuron unit and some couplings with an inhibitory coupling, the error signal δ obtained as shown in FIG. 10 is obtained. _{j (+)} and the error signal δ obtained as shown in FIG.
OR with _{j (+)} and use it as the error signal δ _{j (+)} of the own nerve cell unit. Similarly, the error signal [delta] _j determined as in FIG. 11 _(-) and the error signal [delta] _j determined as in FIG. 13 _(-)
And the result is taken as the error signal δ _{j (−)} of the own nerve cell unit.

To summarize the above, as shown in equation (6),

(Equation 3) Becomes

Further, a function corresponding to a learning rate (learning constant) may be provided. When the rate is 1 or less in the numerical calculation, the learning ability further increases. This can be realized by thinning out the pulse train in the calculation of the pulse train. In the present embodiment, the concept of a counter is used, and the configuration is as shown in FIGS. For example, at the learning rate η = 0.5, every other pulse train of the original signal is thinned out, but even if the pulses of the original signal are not at equal intervals, the original pulse train can be thinned out. 14 and 15, when η = 0.5, every other pulse is thinned out, when η = 0.33, every other pulse is left, and when η = 0.67, two pulses are left. Indicate that it is thinned once every other time.

In this way, the function of the learning rate is provided by thinning out the error signal. Such thinning of the error signal can be easily realized by performing a logical operation on the output of a commercially available counter or using a flip-flop. In particular, when a counter is used, the value of the learning constant η can be set arbitrarily and easily, so that the characteristics of the network can be controlled.

Further, a method of changing each coupling coefficient by using such an error signal as c will be described. The AND between the signal flowing through the line (see FIG. 3) to which the coupling coefficient to be changed belongs and the error signal is obtained (δ _j ∩y _i ). However, in this embodiment, since there are two error signals, + and-, the error signals are calculated and obtained as shown in FIGS.

The two signals thus obtained are denoted by ΔT _{ij (+)} and ΔT _{ij (−)} , respectively.

Next, based on this ΔT _ij , a new T
_ij is obtained. Since T _{ij in the} present embodiment is an absolute value component, cases are classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the component of ΔT _{ij (+)} is increased with respect to the original T _ij , and the component of ΔT _{ij (−)} is reduced. That is, the result is as shown in FIG. Conversely, in the case of inhibitory ΔT to the original T _ij
The component of _{ij (+)} is reduced and the component of ΔT _{ij (-)} is increased. That is, the result is as shown in FIG.

The network is calculated based on the above learning rules.

Next, the input and output of analog signals will be described. First, an input to the network 2 will be described. As described above, since the signal handled by the network 2 is a digital signal (= pulse train), in order to input an analog signal to the signal processing circuit, the analog data is converted into a pulse train using density as information. This is network 2
Of the neural cell unit 20 belonging to the input layer (the layer A _{1 on} the left side of FIG. ₃ ) and the teacher signal of the neural cell unit 20 belonging to the output layer of the network 2 (the layer A ₃ on the right side of FIG. ₃ ). This is realized by providing a conversion unit in each of the input units (corresponding to the conversion units 7 and 10 in FIG. 1).

As a first example, an analog signal is input to a comparator, a random number is input as the other input of the comparator, and the comparison result is input to a network. As the random number value, a voltage value generated by thermal noise of a transistor or the like is used. By performing this for a time corresponding to the number of reference clocks, a pulse train proportional to the input analog voltage value and existing at random intervals can be obtained.

As a second example, a memory may be used. First, pulse train data corresponding to an analog value is stored in a memory in advance. Next, the analog signal is converted into a binary digital signal by a normal A / D converter or the like. By reading data using this result as an address signal of the memory, a pulse train proportional to the input analog value and at random intervals can be obtained.

As a third example, the output of the A / D converter may be converted into a serial pulse train. This can be easily realized by using a pseudo random pulse generation circuit.

Next, the output from the network 2 will be described. As in the case of the input, the output from the network 2 is a pulse train using the pulse density as information.
This is converted to an analog signal. This is realized by providing a conversion unit in each of the signal output units of the neuron unit 20 belonging to the output layer of the network 2 (the right layer A _{3 in} FIG. 3) (corresponding to the conversion means 15 in FIG. 1). .

A first example for that will be described. First, the output from the network 2 is a pulse generated at random time intervals. Therefore, the pulse is converted into binary digital data by inputting the pulse to the counter for the reference time interval. An analog signal can be easily obtained by using the data with a normal D / A converter or the like. Normally, this operation is repeated

As a second example, the pulse density output of the network 2 may be used as it is. That is, since the pulse density output corresponds to frequency modulation in other words, this can be easily converted to an analog signal by using a normal F / V converter.

Next, an actual circuit configuration based on the above algorithm will be described. 20 to 30 show examples of the circuit configuration, and the configuration of the entire network 2 is the same as that of FIG. FIG. 20 shows a circuit corresponding to a line (connection) in FIG. 3, and FIG. 21 shows a circuit corresponding to a circle (each nerve cell unit 20) in FIG. FIG.
Shows a circuit for obtaining an error signal in the final layer from the output of the final layer and the teacher signal. These FIGS. 20 to 2
By forming the three circuits of the two configurations into a network as shown in FIG. 3, a self-learning digital neural network can be realized. Further, by providing such an input section as shown in FIGS. 23 to 29 and an output section as shown in FIG. 30 in such a network 2, a signal processing circuit capable of handling analog signals can be realized.

First, FIG. 20 will be described. In the figure, reference numeral 25 denotes an input signal to the nerve cell unit, which corresponds to FIG. The values of the coupling coefficients as shown in FIG. 5 are stored in the shift register 26. The shift register 26 has an outlet 26a and an inlet 26b, but may have the same function as a normal shift register.
It may be a combination of an AM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are ANDed by a logic circuit 28 having an AND gate 27 and performing the processing shown in FIG. The outputs of the logic circuit 28 must be grouped according to whether the coupling is excitatory or inhibitory. However, it is better to prepare the outputs 29 and 30 for each group in advance and switch to which output. It becomes highly versatile. Therefore, in the present embodiment, a bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31 and the switching is performed by the switching gate circuit 32 using the information. The switching gate circuit 32 includes two AND gates 32a and 32b
And an inverter 32c interposed at one input.

As shown in FIG. 21, gate circuits 33a and 33b having a plurality of OR gates for processing each input (corresponding to FIG. 7) are provided. Further, as shown in FIG. 8, the excitatory connection group shown in FIG.
A gate circuit 34 including an AND gate 34a that outputs an output "1" only when the suppressive coupling group is "0" and an inverter 34b is provided.

Next, the error signal will be described. The error signal in the last layer is generated by the logic circuit 35 using a combination of AND and exclusive OR shown in FIG. 16, and corresponds to the equations (4) and (5). That is, error signals 38 and 39 are generated by the output 36 from the last layer and the teacher signal 37. The processing as shown in FIGS. 10 to 13 for calculating the error signal in the intermediate layer is performed by the gate circuit 42 having the AND gate configuration shown in FIG.
3, 44 are obtained. As described above, it is necessary to divide the case depending on whether the coupling is excitatory or inhibitory. In this case, information on whether the coupling is excitatory or inhibitory and the +/− signals 45 and 46 of the error signal are stored. AND, OR, depending on
This is performed by a gate circuit 47 having a gate configuration. The equation (6) for collecting error signals is performed by a gate circuit 48 having an OR gate configuration shown in FIG. 14 and 15 corresponding to the learning rate are performed by the frequency dividing circuit 49 shown in FIG. Finally, the part for calculating a new coupling coefficient from the error signal, that is, the part corresponding to the processing in FIGS. 16 to 19 is performed by the gate circuit 50 having the AND, inverter, and OR gate configuration shown in FIG. The content of the shift register 26, that is, the value of the coupling coefficient is rewritten. The gate circuit 50 also needs to be classified according to the excitability and the suppression of the connection.

Next, the means for inputting and outputting analog signals to and from the network 2 will be described. FIG. 23 shows an analog signal input circuit, and FIG. 23 (a) shows a circuit 51 using semiconductor thermal noise. That is, the thermal noise output 52 of the transistor or the like is input to the comparator 53 and compared with the input signal 54 to the signal processing circuit, and the output 55 of the comparator 53 is input to the network 2. FIG. 6B shows a case where a conversion circuit 56 using an A / D converter 57 and a memory 58 is used. First, input signal 5
4 is input to the A / D converter 57, and the converted binary data is used as an address signal of the memory 58. Then, the value of the input signal 54 and the address of the memory 58 have a one-to-one correspondence, and pulse train data corresponding to the input value stored in the memory 58 in advance can be obtained. An amplifier 59 may be provided before the A / D converter 57.

FIG. 24 shows another example of an analog signal input circuit. This is achieved by an A / D converter (not shown).
Input data 61 converted to binary data of bits
And N-bit binary data output from the random number generator 62 are compared by an N-bit magnitude comparator 63, and when the input data is larger, an N-level output is output from the N-bit magnitude comparator 63. It is. Since the random number generator 62 generates a random number every time the periodic clock is input, a random pulse train proportional to the analog input signal can be obtained.

The random number generator 62 can also be realized as follows. First, a random number generator generation table 64 as shown in FIG. 25 is prepared. This is for explanation and need not always be prepared. In FIG. 25, the primitive polynomial determines a random number (M sequence) to be generated, and specifically defines the number of delays of a signal to be fed back. For example, 1 + x + x ⁴ indicates that an exclusive OR (EXOR) of the output of delay 1 and the output of delay 4 is input. τ _n defines how each bit of the generated random number is taken from the M-sequence. For example, a 3-bit random number R (= D2 (MSB), D1, D0
(LSB)) For, D1 is the output of the delay by tau ₁ from D2, D0 means that the output of the delay by tau ₂ from D1. Here, the random number is obtained from one primitive polynomial, but each bit of the random number may be generated using a plurality of primitive polynomials. FIG. 26 shows how the random number sequences R0, R1, R2, R3,... Are generated in this manner.

[0053] Figure 27 is a block diagram illustrating an exemplary implementation of the M-sequence, in this case, a 1 + x + x ^4. That is,
Four-stage delay elements 65 to 68 having a flip-flop configuration
And an exclusive OR element 69.

FIG. 28 shows a cycle 7 and a bit length 7 (values 1 to
7 shows a circuit configuration example of a random number generator 62 for generating a random number.

By the way, the input value to the network 2 is
As described above, it is represented by the number of pulses within a certain time, that is, the pulse density. This is expressed as the number of reference synchronous clocks = the ratio of the number of input data pulses to the number of reference clocks. When the above-described random number generator 62 is used,

(Equation 4) FIG. 29 shows an example in which the number of output pulses after the conversion into the pulse density completely matches the value of the input data and the synchronization is maintained. First, the synchronous clock is OR
The signal is input to the random number generator 62 through the gate 71. The generated random number is an N-bit magnitude comparator 63
Is compared with the input data 61. If the input data is larger, the AND gate 73 takes the logical product of its output and the synchronous clock that has passed through the delay element 72 to obtain an output. If the generated random number is larger than the reference clock number, the N-bit magnitude comparator 74 outputs the H level. Reference numeral 76 denotes a continuous pulse generator which operates at a frequency at least twice as high as the synchronous clock. Therefore, the output of the AND gate 75 becomes H level earlier than the synchronous clock, and prompts generation of the next random number. The above object is achieved by repeating this operation.

FIG. 30 shows an example of an analog signal output circuit. FIG. 7A shows a circuit 80 using a counter 81 and a D / A converter 82. First, the number of output pulses is counted by the counter 81 and converted into binary data. By inputting the binary data to the D / A converter 82, an analog signal 83 can be obtained. An amplifier 84 may be provided at the subsequent stage of the D / A converter 82 as appropriate. FIG. 9B shows an example of a circuit 85 using an F / V converter 86. The F / V converter 86 generates a voltage output according to the input frequency, whereby an analog signal 83 according to the pulse density can be obtained. Again, F
An amplifier 84 may be provided downstream of the / V converter 86.

As for these hardware configurations, they may be incorporated in one computer entirely, or only a part thereof may be incorporated.
Further, it is a matter of course that the components individually configured with a single function may be combined and configured as a whole.

With such a configuration of the neural network 2, it is possible to learn in real time even an input that changes with time. After sufficient learning is performed, as shown in FIG.
The output signal 14 is converted by the conversion means 15 into a control signal 16.
, It becomes controllable.

In order to efficiently perform the learning of the neural network 2, it is better to give the set of the input signal and the teacher signal as randomly as possible. For example, in the case of character recognition learning, the number "1"
If learning from "9" to "9", the learning of "1" is performed 10 times, the learning of "2" is performed 10 times, and so on.
It is better to repeat the cycle of “1” learning once, “2” learning once, and so on 10 times. Therefore,
In the case of real-time learning of a time-varying input signal as in the method of the present embodiment, if the input signal changes slowly compared to the learning speed, similar signals are continuously learned. , Learning efficiency may be reduced. In such a case, as shown in FIG. 31, at predetermined time intervals t ₁ and t ₂ , which are preset according to the changing speed of the input signal,
If the learning / unlearning state is repeated and the learning is executed intermittently, the efficiency of the learning is increased. The learning time t ₁ and the unlearned time t ₂ may be controlled using a timer or the like.

In order to provide the function of switching between the learning / unlearning state, that is, the state of changing the coupling coefficient and the state of keeping the coupling coefficient fixed, as shown in FIG. May be added. That is, the terminal 9 0
By supplying the external switching signal Sa to the input signal a, the input to the shift register 26 that stores the coupling coefficient is changed to the original coupling coefficient value (the output value of the shift register 26) or a newly obtained value (the gate circuit 50). (Output value). By performing this switching based on a signal Sa from outside the network 2, the entire network can be controlled.

Further, if the switching between the learning / non-learning state is performed in accordance with the state of the input signal or the teacher signal, the learning efficiency is further improved. For example, in a system whose time change is relatively slow, it is determined whether the amount of change in the amount representing the state of the system has become larger than a predetermined amount,
If the learning is performed only when the size becomes large, the learning efficiency increases. If the teacher signal 9 obtained by the teacher signal generation means 3 is clearly wrong, for example, if the teacher signal generation means 3 is a human and there is an operation error or an error due to lack of skill, this is determined separately. In such a case, by switching from the learning state to the non-learning state, the learning efficiency is improved.

[0062]

According to the present invention, as described above, a memory for storing the coupling coefficient indicating a degree of coupling between the neurons mimic elements rewritable, said by back propagation of error signals for the teacher signal Self-learning having a generating means for generating a variable coupling coefficient value for varying a coupling coefficient, and a coupling coefficient varying means for varying the coupling coefficient stored in the memory by the variable coupling coefficient value generated by the generating means Hardware as a signal processing means in which a plurality of neuron mimicking means attached to the neuron mimic element are connected in a net-like manner
According to the hardware configuration, the processing speed including learning is extremely fast.
Can be entered here, compared to the learning speed
When the change of the force signal is slow, the real-time learning is reversed.
Although the learning efficiency may decrease, the input signal
This input signal, even if it changes,
Input to the processing means, and
At intervals set in advance according to the speed at which the signal changes.
The efficiency of learning can be improved by doing it intermittently.
Wear.

[0063]

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention.

FIG. 2 is a logic circuit diagram.

FIG. 3 is a schematic diagram illustrating a network configuration example.

FIG. 4 is a timing chart showing a logical operation example.

FIG. 5 is a timing chart showing a logical operation example.

FIG. 6 is a timing chart showing a logical operation example.

FIG. 7 is a timing chart showing a logical operation example.

FIG. 8 is a timing chart showing a logical operation example.

FIG. 9 is a timing chart showing a logical operation example.

FIG. 10 is a timing chart showing a logical operation example.

FIG. 11 is a timing chart showing a logical operation example.

FIG. 12 is a timing chart showing a logical operation example.

FIG. 13 is a timing chart showing a logical operation example.

FIG. 14 is a timing chart showing a logical operation example.

FIG. 15 is a timing chart showing a logical operation example.

FIG. 16 is a timing chart showing a logical operation example.

FIG. 17 is a timing chart showing a logical operation example.

FIG. 18 is a timing chart showing a logical operation example.

FIG. 19 is a timing chart showing a logical operation example.

FIG. 20 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 21 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 22 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 23 is a logic circuit diagram illustrating a configuration example of each unit.

FIG. 24 is a logic circuit diagram showing a configuration example of each unit.

FIG. 25 is an explanatory diagram showing a configuration example of a random number generator generation table.

FIG. 26 is an explanatory diagram showing how a random number sequence is generated.

FIG. 27 is a logic circuit diagram showing a configuration example of each unit.

FIG. 28 is a logic circuit diagram showing a configuration example of each unit.

FIG. 29 is a logic circuit diagram showing a configuration example of each unit.

FIG. 30 is a logic circuit diagram showing a configuration example of each unit.

FIG. 31 is a timing chart showing an example of switching between a learning state and an unlearned state.

[Explanation of symbols]

 2 signal processing means 3 teacher signal generation means 4 input signal 9 teacher signal

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-4-100101 (JP, A) Eguchi et al., A pulse-density neuron model with a learning function and its hardware implementation, IEICE technical report, Japan, The Institute of Electronics, Information and Communication Engineers, published, October 25, 1990, Vol. 90, no. 273 (CPSY90-64 to 74), pp. 273-143. 63-70. (Patent Office CSDB Document No .: CS NT199900846009) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06N 1/00-7/00 G06G 7/60 G05B 13/02 JICST file (JOIS) CSDB ( (Japan Patent Office)

Claims (1)

(57) [Claims] 1. A memory and a variable coupling coefficient value for varying the coupling coefficient by inverse propagate an error signal for the teacher signal to store rewritable coupling coefficient indicating a degree of coupling between the neurons mimic elements Generating means for generating
A plurality of neural cell mimic means provided with a self-learning means attached to the neural cell mimic element, the variable means comprising a coupling coefficient variable means for varying the coupling coefficient stored in the memory by the variable coupling coefficient value generated by the generating means; the provided signal processing means connected to the network, to enter the time-varying input signal to said signal processing hand stage, Manabu by the self-learning unit
Learning is preset according to the changing speed of the input signal
A learning method for a signal processing device, wherein the learning is performed at predetermined time intervals .