JPH05217010A - Signal processor - Google Patents

Abstract

PURPOSE:To obtain a signal processor with high versatility by mutually connecting respective nerve cell simulating units with learning functions through a programmable connecting means. CONSTITUTION:Each neuron 20 has input signal lines 61 to the neuron 20 itself, error signal line (substituted by the line 61) reversely propagated from the neuron 20, output signal lines 62 from the neuron 20 itself, and error signal lines (substituted by the line 62) reversely propagated to the neuron 20. A matrix circuit 63 to be a programmable connecting means capable of programming the connection/non-connection of input and output signal lines 61, 62 is connected between mutual neurons 20. The circuit 63 includes input signal lines 64 for receiving input signals from the outside of a device, output signal lines 65 for outputting output signals to the outside of the device and error signal input lines (substituted by the lines 65) from the outside.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing device such as a neural computer imitating a neural network, which is applicable to character and figure recognition, motion control of robots, and associative memory.

[0002]

2. Description of the Related Art The aim of parallel processing of information is to imitate the function of nerve cells (neurons), which are the basic units of information processing in the living body, and to use this "nerve cell mimicking element" as a network to process information in parallel. This is a so-called neural network. Although it is easy to perform character recognition, associative memory, motion control, etc. in a living body, there are many things that conventional Neumann computers cannot easily achieve. Therefore, many attempts have been made to solve these problems by imitating the nervous system of the living body, particularly the functions peculiar to the living body, that is, parallel processing, self-learning and the like. However, many of these attempts are carried out by computer simulation, and parallel processing is required to realize the original function, and for that purpose, the neural network must be implemented as hardware. Although some attempts have already been made to implement the hardware, the self-learning function, which is one of the features of neural networks, cannot be realized, which is a major obstacle. Also, most of them are realized by analog circuits, and there is a problem in operation.

These points will be examined in more detail. First, a conventional neural network model will be described. FIG. 42 shows a certain nerve cell unit (nerve cell mimicking element) 1, and FIG. 43 shows this as a network. That is, one nerve cell unit 1 is combined with many other nerve cell units 1 to receive a signal, process the signal, and output the signal. In the case of FIG. 43,
The network is hierarchical, and receives a signal from a unit in the previous (left) layer and outputs it to a unit in the next (right) layer.

Here, in the nerve cell unit 1 of FIG. 42, the degree of coupling between another nerve cell unit and its own nerve cell unit is called a coupling coefficient. The i-th unit and the j-th unit The coupling coefficient with the unit is generally represented by T _ij . There are two types of coupling: excitatory coupling, in which the output of the partner unit increases as the signal from the partner unit increases, and conversely, inhibitory coupling, in which the output of the partner unit decreases, the output decreases. _ij > 0 represents excitatory coupling, and T _ij <0 represents inhibitory coupling. When the input from the i-th unit is y _i when it is the j-th unit, T _ij y _{i obtained} by multiplying this by the coupling coefficient T _ij becomes the input to the own unit. As described above, since each unit is connected to a large number of units, ΣT _ij y which is the result of adding up T _ij y _i for those units
_i becomes the input to your unit. This is called the internal potential and is represented by u _j as in the equation (1).

[0005]

[Equation 1]

Next, the input is subjected to non-linear processing and output. The function at this time is called a nerve cell response function, and the sigmoid function as shown in equation (2) and FIG. 44 is used as the nonlinear function.

[0007]

[Equation 2]

When the network is formed as shown in FIG. 43, each coupling coefficient T _ij is given and the formulas (1) and (2) are calculated one after another to obtain the final output.

On the other hand, as an example of such a network realized by an electric circuit, there is one as shown in FIG. This is disclosed in Japanese Patent Laid-Open No. 62-295188, and basically, a plurality of amplifiers 2 having an S-shaped transfer function, and the output of each amplifier 2 is connected to the input of an amplifier of another layer. And a resistive feedback network 3 connected as shown by the chain line. A CR time constant circuit 4 composed of a grounded capacitor and a grounded resistance is individually connected to the input side of each amplifier 2. And
Input currents I ₁ , I ₂ , ..., _IN are supplied to the inputs of each amplifier 2 and the output is obtained from the set of output voltages of these amplifiers 2.

Here, the strength of the input or output signal is represented by a voltage, and the strength of the coupling between nerve cells is determined by the resistance 5 (the grid point in the resistive feedback network 3) connecting the input / output lines between the cells. ) And the nerve cell response function is represented by the transfer function of each amplifier 2. Further, the coupling between nerve cells includes excitatory coupling and inhibitory coupling as described above, and is mathematically represented by the sign of the coupling coefficient. However, since it is difficult to realize positive and negative with a constant on the circuit, here, the output of the amplifier 2 is divided into two and one output is inverted to generate two positive and negative signals. Is properly selected. Further, an amplifier is used as the one corresponding to the sigmoid function shown in FIG.

However, in these circuits, the signal strength inside the network is represented by an analog value such as a potential or a current, and the internal calculation is also performed in an analog manner. ,
The value will change. Since it is a network, a large number of elements are required, but it is difficult to make the respective characteristics uniform. When the accuracy or stability of one element becomes a problem, if it is used as a network, a new problem may occur, and the behavior of the entire network cannot be predicted. Since the value of the coupling coefficient T _ij is fixed and the value learned in advance by another method such as simulation is used, self-learning cannot be performed. There is a problem such as.

On the other hand, as a learning law used in numerical calculation, there is the following one called back propagation.

First, each coupling coefficient is first randomly given. If input is given in this state, the output result is not always desirable. For example, in the case of character recognition, if a handwritten "1" character is given, the output result "This character is" 1 "" is a desirable result, but the coupling coefficient is not always random. Not the desired result. Therefore, a correct answer (teaching signal) is given to this network, and each coupling coefficient is changed so that the correct answer is given when the same input is given again. At this time, the algorithm to find the amount to change the coupling coefficient is
This is called back propagation.

For example, in the hierarchical network shown in FIG. 43, the output of the _jth nerve cell unit in the final layer is y _j , and the teacher signal for that nerve cell unit is d.
_{When j} is set, the coupling coefficient T _ij is changed using the equation (4) so that E represented by the equation (3) is minimized.

[0015]

[Equation 3]

[0016]

[Equation 4]

More specifically, first, when the coupling coefficient between the output layer and the layer immediately before it is obtained, the error signal δ is obtained by using the equation (5), and the layer further before that is obtained. When obtaining the coupling coefficient between the two, the error signal δ is obtained by using the equation (6), the equation (7) is obtained, and T _ij is changed.

[0018]

[Equation 5]

[0019]

[Equation 6]

[0020]

[Equation 7]

Here, η is a learning constant, and α is a stabilizing constant. Since it is not possible to obtain each logically, we ask empirically. Further, f ′ is a first-order differential function of the sigmoid function f, and ΔT _ij ′ and T _ij ′ are values at the time of previous learning.

Learning is carried out in this manner, and thereafter, the input is given again and the output is calculated and the learning is carried out. By repeating this operation many times, the coupling coefficient T _ij is finally determined so as to obtain a desired result for a given input.

However, if such a learning method is to be implemented as hardware by some method, a large amount of four arithmetic operations are required for learning, which is difficult to realize. The learning method itself is not suitable for hardware implementation.

On the other hand, an example in which a neural network is realized by a digital circuit will be described with reference to FIGS. 46 to 48. FIG. 46 shows a circuit configuration of a single nerve cell, in which each synapse circuit 6 is connected to a cell body circuit 8 via a dendrite circuit 7. FIG. 47 shows a configuration example of the synapse circuit 6 therein, and a rate multiplier to which a value obtained by multiplying the input pulse f by a factor a (a factor for multiplying a feedback signal by 1 or 2) is input via a coefficient circuit 9. 10
The synapse weight register 11 storing the weighting value w is connected to the rate multiplier 10. 48 shows a configuration example of the cell body circuit 8,
A control circuit 12, an up / down counter 13, a rate multiplier 14 and a gate 15 are sequentially connected,
Further, an up / down memory 16 is provided.

In this case, the input and output of the nerve cell unit are represented by a pulse train, and the pulse density represents the amount of signal.
The coupling coefficient is represented by a binary number and stored in the memory 16. By inputting the input signal into the clock of the rate multiplier 14 and inputting the coupling coefficient into the rate value,
The pulse density of the input signal is reduced according to the rate value. This is the T _{ij of the} backpropagation model equation.
It corresponds to the part of y _i . Next, the Σ portion of ΣT _ij y _i is realized by the OR circuit shown by the dendrite circuit 7. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance and OR is taken for each group. An output is obtained by inputting these two outputs to the up side and down side of the counter 13 and counting. This output is 2
Since it is a decimal number, the rate multiplier 14 is used again to convert it into a pulse density. A neural network can be realized by making this unit a network. Learning is realized by inputting the final output to an external computer, performing numerical calculation inside the computer, and writing the result in the memory 16 of the coupling coefficient. Therefore, there is no self-learning function. Also, the circuit configuration is complicated because the pulse density signal is once converted into a numerical value (binary number) using a counter and then converted into the pulse density again.

As described above, according to the conventional technique, the analog circuit system is not reliable in operation, and the learning method by numerical calculation is also complicated in calculation, and is not suitable for hardware implementation. The circuit configuration is complicated. There is also a drawback that self-learning cannot be performed on hardware.

In order to solve such a defect, a pulse density type neuron model with a learning function is disclosed in Japanese Patent Application No. 2-41.
No. 2448, Japanese Patent Application No. 3-154244, Japanese Patent Application No. 3-1
No. 54245 and Japanese Patent Application No. 3-154246 are proposed by the present applicant.

[0028]

When a network having a plurality of neurons integrated in one package is formed by using the improved proposed example or the known neuron with a learning function, the number of layers is , The number of neurons in each layer must be determined. Here, since the network structure differs depending on the application, once the network structure is fixed, it can be applied only to a limited purpose, and the network structure has no versatility.

[0029]

According to a first aspect of the present invention, in a signal processing device in which a plurality of neural cell mimicking units with learning functions are accommodated in one package to form a circuit network, each neural cell mimicking is performed. The units are connected by programmable connecting means that can be connected or disconnected.

According to the second aspect of the present invention, with respect to the one in which the circuit network is formed by the nerve cell mimicking unit having the improved structure, each nerve cell mimicking unit is connected by the programmable coupling means which can be connected / disconnected.

According to the third aspect of the present invention, not only a plurality of neural cell mimicking units with learning functions, but also an error signal generating circuit for generating an error signal by its output signal and a teacher signal is contained in one package. In the signal processing device configured to form a net, the neural cell mimicking units and the error signal generating circuits are connected by programmable connecting means that can be connected / disconnected.

According to a fourth aspect of the present invention, a plurality of nerve cell mimicking units with a pulse density type learning function using a pulse train as a signal transmitting means, and an error signal generating circuit for generating an error signal from its output signal and a teacher signal are provided. In a signal processing device in which a pulse train converting circuit for converting a decimal value into a pulse train and a binary number converting circuit for converting a pulse train into a binary number are housed in one package to form a circuit network, between the nerve cell mimicking units and The respective circuits are connected by programmable connecting means which can be connected / disconnected freely.

On the other hand, in the fifth aspect of the present invention, a plurality of pulse density type nerve cell mimicking units having a pulse train as a signal transmission means, a pulse train conversion circuit for converting a digital signal or an analog signal into a pulse train signal, and the pulse train signal as a digital signal. Alternatively, in a signal processing device in which a conversion circuit for converting into an analog signal is housed in one package to form a circuit network, programmable coupling means capable of connecting / disconnecting each neural cell mimicking unit and each circuit. Connected by.

In these inventions, the programmable coupling means is erasable by irradiation of ultraviolet rays in the invention described in claim 6, and is erasable by electrical processing in the invention described in claim 7.

[0035]

According to the first aspect of the present invention, since each neural cell mimicking unit with a learning function is connected by the programmable connecting means, it is possible to select or change the connecting point in the preformed circuit network structure. If so, the structure can be changed to a desired structure, and the signal processing device has high versatility.

In particular, according to the second aspect of the present invention, the nerve cell mimicking unit itself processes all signals digitally, and there is no problem of temperature characteristics, drift, etc. as in the analog system. Since the information as the coupling coefficient is also stored in the memory, it can be easily rewritten and has versatility, and the signal processing device is excellent in combination with the versatility of the circuit network structure.

According to the third aspect of the invention, not only the nerve cell mimicking unit but also the error signal generating circuit in the output layer is included, and the location by the programmable coupling means is selected or changed in the preformed network structure. If so, the structure can be changed to a desired structure, and the signal processing device has high versatility.

In addition, according to the invention described in claim 4, not only the nerve cell mimicking unit but also the error signal generating circuit, the pulse train converting circuit and the binary number converting circuit in the output layer are formed in advance. By selecting or changing the location by the programmable coupling means in the circuit network structure, the structure can be changed to a desired structure, and the signal processing device has high versatility.

On the other hand, according to the fifth aspect of the present invention, a circuit network formed in advance including the pulse density type nerve cell mimicking unit and the signal conversion circuit between the digital signal or the analog signal and the pulse train signal is formed. By selecting or changing the position of the programmable coupling means in the structure, the structure can be changed to a desired structure, and the signal processing device has high versatility.

Further, such programmable coupling means is realized by utilizing ultraviolet irradiation or electrical processing as in the case of the invention of claim 6 or 7, as in the case of the erasable programmable ROM or the electrical erasable programmable ROM. It becomes possible and it becomes more versatile.

[0041]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described with reference to FIGS. As a single neuron mimicking unit (neuron) having a learning function in the present invention, a known one may be used, but a unit having the configuration and action according to the already proposed example as described above is particularly preferable. 33
The configuration and operation will be described below. A neural network with a neuron element configuration using a digital logic circuit with a self-learning function according to the proposed example uses a coupling coefficient variable circuit and a variable coupling coefficient value of this coupling coefficient variable circuit based on positive and negative error signals for a teacher signal. And a coupling coefficient generating circuit for generating a coupling coefficient generating circuit.

First, the neural network in the already proposed example is implemented by hardware by a digital configuration, but the basic idea is to input / output signals, intermediate signals,
The coupling coefficient, the teacher signal, etc. are all represented by a binary pulse train of "0" and "1". The amount of signal inside the network is represented by the pulse density (the number of "1" s within a certain fixed time). The calculation in the nerve cell unit is represented by a logical operation between pulse trains. The pulse train of the coupling coefficient is placed in the memory. Learning is realized by rewriting this pulse train. For learning, an error is calculated based on the given teacher signal pulse train, and the coupling coefficient pulse train is changed based on the error. At this time, the calculation of the error and the change of the coupling coefficient are all performed by the logical operation of the pulse train of "0" and "1". It was done like this.

The idea will be described below. First, regarding signal processing by a digital logic circuit, signal processing in the forward process will be described. FIG. 4 shows a portion corresponding to one neuron (nerve cell mimicking unit) 20, and the entire neural network is of a hierarchical type similar to the case shown in FIG. 43, for example. All inputs and outputs are binarized to “1” and “0” and synchronized. The intensity of the input signal y _i is represented by the pulse density, and is represented by the number of states of “1” within a certain fixed time as in the pulse train shown in FIG. 5, for example. That is, the example of FIG. 5 represents 4/6, and the signal is 4 "1" and 2 "0" in 6 sync pulses. At this time, "1" and "0"
It is desirable that the arrangement of is random.

On the other hand, the coupling coefficient T _ij indicating the degree of coupling between the neurons 20 is also expressed by a pulse density, and is "0".
And "1" as a bit string are prepared in advance in the memory. The example of FIG. 6 is an expression representing “101010” = 3/6. Also in this case, it is desirable that the arrangement of "1" and "0" is random.

Then, this bit string is sequentially read from the memory in response to the synchronous clock, and as shown in FIG.
The ND gate 21 takes the logical product of the input signal pulse train (y _i ∩ T _ij ). This is used as an input to the nerve cell j. In the case of the above example, the input signal is “10110
When "1" is input, a pulse train is called from the memory in synchronization with this and "101000" as shown in FIG. 7 is obtained by sequentially performing AND, which means that the input y _i is the coupling coefficient T _ij. And the pulse density is 2/6
It shows that it becomes.

The pulse density of the output of the AND gate 21 is
It is approximately the product of the pulse density of the input signal and the pulse density of the coupling coefficient, and has the same function as the analog coupling coefficient. This is because the longer the signal train is, the more "1"
The more random the arrangement of "0" and "0", the closer to the product of numerical values it has. When the pulse train of the coupling coefficient is shorter than the input pulse train and there is no data to be read, the head of the data may be returned to and the reading may be repeated.

Since one neuron 20 has multiple inputs, there are many "ANDs of the input signal and the coupling coefficient" described above, and the OR circuit 22 then takes the logical sum of these.
Since the inputs are synchronized, for example, the first data is "101000" and the second data is "010000".
In the case of, the OR of both results in “111000”. If this is calculated for m inputs for multiple inputs simultaneously and used as outputs, for example, as shown in FIG. This corresponds to the sum calculation and the non-linear function (sigmoid function) part in the analog calculation.

When the pulse density is low, the pulse density of its OR is approximately equal to the sum of the pulse densities. As the pulse density increases, the OR circuit 22
Since the output of is gradually saturated, it does not match the sum of pulse densities, and nonlinearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and is a monotonically increasing function, which is approximately equivalent to the sigmoid function.

By the way, the coupling has excitability and inhibitory ability. In the case of numerical calculation, it is represented by the sign of the coupling coefficient. In the case of an analog circuit, when T _ij is negative (inhibitory coupling), an amplifier is used. It is used to invert the output and _connect it to another neuron with a resistance value corresponding to T _ij . In this respect, in the already-proposed example of the digital system, first, each coupling is divided into two groups of excitatory coupling and inhibitory coupling according to the positive or negative of T _ij , and then “the pulse train of the input signal and the coupling coefficient is divided. AN
The OR of "D" s is calculated for each group. The result F _j of the excitatory group and the result I _{j of the} inhibitory group thus obtained are set.

Alternatively, the coupling coefficient T _{ij (+) indicating} excitability and the coupling coefficient T _{ij (-) indicating} inhibition for one input y _i .
And both are prepared, and each is ANDed (y _i ∩
T _{ij (+)} , y _i ∩T _{ij (-)} ). Furthermore, O between these
Take each R (∪ (y _i ∩T _{ij (+)} ), ∪ (y _i ∩T
_{ij (-)} ), the excitatory group result F _j , and the inhibitory group result I _j .

In summary, when one input has only one of the excitatory and inhibitory coupling coefficients, the equations (8) and (9) are obtained.

[0052]

[Equation 8]

When the coupling coefficient has both excitability and inhibitory property with respect to one input, the equations (10) and (11) or (12) and (13) are obtained.

[0054]

[Equation 9]

[0055]

[Equation 10]

However, in the equations (12) and (13), when the coupling coefficient has only one of excitability and inhibition with respect to one input, y _Fij and y _Iij are (14) (15) In the case where the coupling coefficient has both excitability and inhibitory property with respect to one input, y _Fij and y _Iij are expressed by equations (16) and (17).

[0057]

[Equation 11]

If the result F _j of the excitatory group and the result I _j of the inhibitory group thus obtained do not match, the result of the excitatory group is output. That is, the result F _j of the excitatory group is “0” and the result I _j of the inhibitory group is
Is “1”, “0” is output and the excitatory group result F _j is “1” and the inhibitory group result I _j is “0”.
If so, “1” is output. Excitability group result F
_{When j} and the result I _j of the inhibitory group match,
Either "0" or "1" may be output, or
It may be allowed to output a second input signal E _j which are separately prepared, or, with the contents of the memory provided to the second input signal E _j of such second input signal E _j Toko You may make it output what calculated the logical product. Similar to the coupling coefficient for the input signal, this memory can be read from the beginning again after all the data have been read.

In order to realize this function, in the case of outputting “0”, the output of the excitatory group and the negation of the output of the inhibitory group may be ANDed.
FIG. 9 shows this example, which is expressed by equation (18).

[0060]

[Equation 12]

In the case of the example of outputting "1", the negation of the output of the excitatory group and the output of the inhibitory group may be ANDed. FIG. 10 shows this example, which is expressed by equation (19).

[0062]

[Equation 13]

In the case of the example of outputting the second input signal, it becomes as shown in FIG.
It becomes like a formula.

[0064]

[Equation 14]

Further, in the case of the example of the fourth method, the contents (coefficients) of the memory provided for the second input signal E _j
When T ′ _j is represented by T ′ _j , the result is as shown in FIG.

[0066]

[Equation 15]

The network of the nerve cell unit 20 is
Hierarchical type similar to backpropagation (ie, FIG.
3). If the entire network is synchronized, each layer can be calculated by the functions described above.

Next, the signal calculation processing in learning (back propagation) will be described. Basically, the error signal may be obtained by the following a or b, and then the value of the coupling coefficient may be changed by the method of c.

A. Error signal in the final layer The error signal in each neuron is calculated in the final layer, and the coupling coefficient related to the neuron is changed based on the error signal calculated.
The calculation method of the error signal for that is described. Here, the "error signal" is defined as follows. When the error is expressed by a numerical value, generally, both + and − can be taken, but in the case of the pulse density, both positive and negative cannot be expressed at the same time. Therefore, there are a signal representing a + component and a signal representing a − component. The error signal is expressed using two types. That is, the error signal of the j-th neuron is shown in FIG. That is, the + component of the error signal is a pulse existing on the teacher signal side in the portion (1, 0) or (0, 1) where the teacher signal pulse and the output signal pulse are different, and the − component is the same. This is a pulse existing on the output signal side. In other words, the output signal y _j
When the error signal + pulse is added to and the error signal−pulse is removed, the teacher signal d _j is obtained. That is, when these positive and negative error signals δ _{j (+)} and δ _{j (-)} are expressed by logical expressions, they are respectively expressed by the expressions (22) and (23). The coupling coefficient is changed based on such an error signal pulse as described later.

[0070]

[Equation 16]

B. Error Signal in Intermediate Layer First, the above-mentioned error signal is back-propagated to change not only the coupling coefficient between the final layer and the layer immediately before it but also the coupling coefficient of the layer before that. Therefore, it is necessary to calculate the error signal in each neuron in the hidden layer. The error signal of each neuron of the next layer is collected in the same way as the signal is propagated from the neuron of the intermediate layer to each neuron of the next layer, and the error of its own is collected. Signal. This can be performed in a manner similar to the above-described arithmetic expressions (8) to (10) in the neuron and the cases shown in FIGS. However, what is different from the above-described processing in the neuron is that y _j is one signal that is always positive, whereas δ _j has two signals as positive and negative signals, and both signals are It is necessary to consider. Therefore, it is necessary to divide into four cases depending on whether the coupling coefficient T _ij is positive or negative and the error signal δ _j is positive or negative.

First, the case of excitatory coupling will be described. In this case, for a neuron in the intermediate layer, the AND of the error signal δ _{k (+)} at the k-th neuron in the next layer and the coupling coefficient T _jk between the neuron and self (δ
_{k (+)} ∩ T _jk ) is obtained for each neuron, and
The OR of these is taken {∪ (δ _{k (+)} ∩ T _jk )}. This is the error signal δ _{j (+)} of this neuron. That is, 1
If there are n neurons in the next layer, the result is as shown in FIG. If these are shown in order by mathematical expressions, (24) to (26)
It becomes like a formula.

[0073]

[Equation 17]

Similarly, the error signal δ _{j of} this neuron is taken by _ANDing the error signal δ _{k (-)} and the coupling coefficient T _{jk in} the neuron of the layer immediately before and ORing them. _(-) That is, as shown in FIG.
If these are shown in order by mathematical formulas, they become like the formulas (27) to (29).

[0075]

[Equation 18]

Next, the case of inhibitory binding will be described. In this case, the AND of the error signal δ _{k (-) in} the neuron of the next layer and the coupling coefficient T _jk of the neuron and self is taken,
Furthermore, the OR of these is taken. This is the error signal δ _{j (+) of} this neuron. That is, it becomes as shown in FIG. 16, and if these are expressed in order by mathematical expressions, they become as in Expressions (30) to (32).

[0077]

[Formula 19]

Similarly, the error signal δ _{j (-) of} this neuron is similarly obtained by _{ANDing the} preceding error signal δ _{k (+)} and the coupling coefficient T _jk and further ORing them.
And That is, it becomes as shown in FIG. 17, and when these are expressed in numerical order in order, they become as in Expressions (33) to (35).

[0079]

[Equation 20]

Since some neurons are excitatoryly coupled to another neuron and some are inhibitoryly coupled, the error signal δ obtained as shown in FIG. 14 is used.
_The error signal δ _{j (+)} obtained as shown in FIG. 16 is ORed with _{j (+),} and this is taken as the error signal δ _{j (+) of} its own neuron. Similarly, the error signal [delta] _j determined as in FIG. 15 _(-) and the error signal [delta] _j determined as in FIG. 17 _(-) takes the OR of, it their neuronal error signal [delta] _{j (- )} .

Summarizing the above, equation (36) or
It becomes as shown in equation (37).

[0082]

[Equation 21]

[0083]

[Equation 22]

Next, in the case of having both excitatory and inhibitory coupling coefficients for one input, only the mathematical formulas are shown as follows:
It becomes as shown in the equation (38) or the equation (39).

[0085]

[Equation 23]

[0086]

[Equation 24]

Further, a function corresponding to the learning rate (learning constant) may be provided. When the rate is 1 or less in the numerical calculation, the learning ability is further enhanced. This can be realized by thinning out the pulse train in the pulse train calculation. Here, a counter-like idea is adopted and the one shown in FIGS. 18 and 19 is adopted. For example, at the learning rate η = 0.5, every other pulse train of the original signal is thinned out, but even if the pulses of the original signal are not evenly spaced, the original pulse train can be thinned out. 18 and 19, in the case of η = 0.5, every other pulse is thinned out, in the case of η = 0.33, every two pulses are left, and in the case of η = 0.67, two pulses are left. It indicates that one thinning is performed every other time.

C. Method of changing coupling coefficient The signal flowing through the line (see FIG. 43) to which the coupling coefficient to be changed belongs and the error signal are ANDed (δ _j ∩
y _i ). However, since there are two signals, + and −, in the error signal here, they are respectively calculated and obtained as shown in FIGS. That is, δ _{j (+)} ∩y _i and δ _{j (-)} ∩y _i are positive and negative coupling coefficient change signals, respectively.

The two signals thus obtained are designated as ΔT _{ij (+)} and ΔT _{ij (-)} , respectively. Next, this ΔT
_A new T _ij is obtained based on _ij . Since this T _ij is an absolute value component, it is classified depending on whether the original T _ij is excitatory or inhibitory. In the case of excitability, the ΔT _{ij (+)} component is increased and the ΔT _{ij (-)} component is decreased with respect to the original T _ij . That is, it becomes as shown in FIG. On the contrary, in the case of the suppressive property, the component of ΔT _{ij (+)} is reduced and the component of ΔT _{ij (−)} is increased with respect to the original T _ij . That is, it becomes as shown in FIG. These FIG. 22,
When the contents of FIG. 23 are expressed by mathematical expressions, they are expressed by Expressions (40) and (41).

[0090]

[Equation 25]

The network is calculated based on the above learning rule.

Next, an actual circuit configuration according to the already proposed example system based on the above algorithm will be described. 24 to 29 show examples of the circuit configuration, the configuration of the entire network is the same as that of FIG. 24 to 27 are shown in FIG.
3 shows a circuit of a portion corresponding to a line (connection) in the hierarchical network, and FIG. 28 shows a circuit of a portion corresponding to a circle (each neuron 20 in the proposed example) in FIG. Further, FIG. 29 shows a circuit of a portion for obtaining an error signal in the final layer from the output of the final layer and the teacher signal. A digital neural network having a self-learning function can be realized by forming a network of these three circuits having the configurations of FIGS. 24 to 29 as in the case of FIG.

First, description will be made with reference to FIG. In the figure, 25 is an input signal to the neuron as shown in FIG. The value of the coupling coefficient as shown in FIG. 7 is stored in the shift register 26. This shift register 26 has an outlet 26
Although it has a and the entrance 26b, it may have the same function as a normal shift register, and may be, for example, a combination of a RAM and an address controller. The input signal 25 and the coupling coefficient in the shift register 26 are ANDed by a logic circuit 28 having an AND gate 27 and performing the processing shown in FIG. The output of the logic circuit 28 must be divided into groups depending on whether the coupling is excitatory or inhibitory. However, it is better to prepare the outputs 29 and 30 for each group in advance and switch which one is output. It is highly versatile. For this reason,
In the already proposed example, the bit indicating whether the coupling is excitatory or inhibitory is stored in the grouping memory 31 and the information is used to switch by the switching gate circuit 32. The switching gate circuit 32 includes two AND gates 32a and 32b and an inverter 32c interposed at one input.

When it is not necessary to switch, they may be fixed. For example, FIG. 25 shows the case of excitability and FIG. 26 shows the case of inhibition. Further, for one input, both a memory for a bit indicating excitability and a memory for a bit indicating inhibitory property may be prepared. FIG. 27 shows this example. In the figure, 26A is a bit memory for a coupling coefficient representing excitability, and 26B is a bit memory for a coupling coefficient representing inhibition.

Further, as shown in FIG. 28, a gate circuit 3 having a plurality of OR gates for performing each input processing (corresponding to FIG. 8).
3a and 33b are provided. Further, as shown in the figure, a gate formed by an AND gate 34a and an inverter 34b which outputs an output "1" only when the excitatory coupling group shown in FIG. 9 is "1" and the inhibitory coupling group is "0". A circuit 34 is provided. Similarly, even when the processing results illustrated in FIGS. 10 to 12 are obtained, it can be easily realized by the logic circuit.

Of course, the gate circuit 34 shown in FIG.
In the case of the 0 system, a combination of the inverter 34b and the OR gate 34c is used as shown in FIG.
In the case of the system, the exclusive O as shown in FIG.
It is assumed that the R gate 34d, the inverter 34e, the two AND gates 34f and 34g, and the OR gate 34h are configured to input the second input E _j to the AND gate 34g. ), A memory 34i storing the coefficient T'for the second input E _j and an AND gate 34j may be added to the configuration shown in FIG.

Next, the error signal will be described. The error signal in the final layer is generated by the logic circuit 35 formed by the combination of the AND gate and the exclusive OR gate shown in FIG. 29, which corresponds to the equations (6) and (7). That is, the error signals 38 and 39 are produced by the output 36 from the final layer and the teacher signal 37. Calculate error signal in middle layer (37)
The process of _obtaining E _{j (+)} and E _{j (-)} in the equation is performed by the gate circuit 42 having an AND gate configuration shown in FIG.
Outputs 43 and 44 corresponding to + and-are obtained. here,
Although the coefficient after learning is used as the coupling coefficient, the coefficient before learning may be used, and the circuit before learning may be easily formed into a circuit.
In addition, it is necessary to classify the connection depending on whether the coupling is excitatory or inhibitory. In this case, the information indicating whether excitatory or inhibitory is stored in the memory 31, and the + and-signals 45 of the error signal. , 46 according to the AND and OR gates. In addition, in the system of FIG. 25 and FIG. 26 in which the distinction between excitability and inhibitory property of binding is fixed,
The circuit is equivalent to one in which the contents of the memory 31 are fixed to "0" and "1", respectively. On the other hand, in the system of FIG. 27 which uses both the memory 26A representing excitatory coupling and the memory 26B representing inhibitory coupling for one input, (39)
The circuit corresponding to the equation is shown as the gate circuit 42 in FIG.

Further, the calculation equation (8) for collecting error signals, that is, the rest of the equation (37) is performed by the gate circuit 48 of the OR gate structure shown in FIG. Further, the processing of FIGS. 18 and 19 corresponding to the learning rate is performed by the frequency dividing circuit 49 shown in FIG. This can be easily realized by using a flip-flop or the like. However, the frequency dividing circuit 49 may be omitted if it is not necessary. Even if the frequency dividing circuit 49 is provided, the frequency dividing circuit 49 is not limited to the example shown in FIG.
It may be provided at an appropriate position as indicated by reference numerals 49, 49 ', 49 "in FIG.

Finally, the part for calculating a new coupling coefficient from the error signal, that is, the part corresponding to the processing of FIGS. 22 to 23, is the gate circuit (self-operation) of AND, inverter and OR gates shown in FIG. The contents of the shift register 26, that is, the value of the coupling coefficient T _ij is rewritten. This gate circuit 50 also needs to be divided into cases depending on the excitability and inhibition of the coupling, but it is performed by the gate circuit 47. In the case of FIG. 25 and FIG. 26, the excitability and the inhibitory property are fixed, so that the circuit corresponding to the gate circuit 47 is unnecessary. In the case of the method of FIG. 27, both excitability and inhibitory property with respect to one input are provided, which corresponds to the case where the gate circuit 50A is excitable and the gate circuit 50B is inhibitory.

When forming the neuron 20,
The present invention is not limited to the illustrated example, and the above-mentioned Japanese Patent Application No. 3-
No. 154244, Japanese Patent Application No. 3-154245, Japanese Patent Application No. 3
It may be as shown in No. 154246 or the like. For example, in Japanese Patent Application No. 3-154244, in order to improve learning ability, "logical means for prohibiting positive error signal and negative error signal from being 1 at the same time is provided", for example, as shown in FIG. The logic means is provided in the output section of the gate circuit 42 in FIG. 27, in the front or rear stage of the frequency divider circuit 49 in FIG. 28, or in the input side of the gate circuit 48. In addition, Japanese Patent Application No. 3-154245 also discloses that in order to improve the learning ability, "a match between a positive error signal and a negative error signal in the output layer is detected, and if they match, learning by the self-learning means is prohibited. It has a learning prohibition means. "
Specifically, the learning inhibiting means is provided in the gate circuit 50. In Japanese Patent Application No. 3-154246, in order to improve the learning ability, "a correction means for using an input signal before a preset time is provided in a part or all of the coupling coefficient change signal for generating a variable coupling coefficient value. Specifically, a correction means is added to the gate circuit 50.

By the way, the pulse train representing the coupling coefficient is stored in the shift register (memory) 26 in the form as shown in FIG. In such a pulse train, the learning ability may be improved by sometimes changing the arrangement of the pulse trains with the same pulse density. To do that, see Figure 3.
The processing may be performed by a circuit as shown in FIGS. First, in FIG. 31, the contents of the memory 26 are read out once, and the pulse train is changed and written to the memory 26 again. Therefore, a counter 51 for counting the number of pulses of the pulse train read from the memory 26 is provided. Note that an integrator may be used instead of the counter 51 to obtain a voltage value according to the number of pulses of the pulse train read from the memory 26. Further, a random number generator 52 for generating and outputting a uniform random number from 0 to this maximum value with the maximum number of bits read from the memory 26, that is, the number of synchronization clocks shown in FIG. 6, is provided. The output of the counter (or integrator) 51 and the output of the random number generator 52 are compared by the comparator 53, and "1" is output when the number of pulses is larger, and "0" is output when the number of pulses is smaller. Is obtained. Therefore, the output of the comparator 53 may be input to the memory 26 in accordance with the switching of the switch 54 to change the writing. However, the memory 26
During the rewriting of, the original operation of the neural network is disabled.

FIG. 32 shows the pulse density of the pulse train representing the coupling coefficient in the memory 2 in the form of a numerical value, for example, a binary number or a voltage value.
6 is stored in advance and converted into a pulse train for each calculation. That is, the pulse train representing the learned coupling coefficient is counted by using the counter 55 (or converted into a voltage value by using an integrator instead of the counter 55), and is shown at fixed intervals, that is, in FIG. The contents of the memory 26 are updated every fixed number of synchronization clocks. Such a memory 26
The pulse train signal similar to that in the case of FIG. 31 is obtained by comparing the output of 1) with the output of the random number generator 52 by the comparator 53. The maximum value of the random numbers at this time is also the number of synchronous clocks for updating the contents of the memory 26.

In FIG. 33, the pulse density of the pulse train representing the coupling coefficient is stored in the up / down counter 56 in the form of a numerical value, for example, a binary number, and the random number generator 52 and the comparator 53 are stored.
Is used to convert to a pulse train for each calculation. That is, the counter 56
By inputting a pulse representing the learned coupling coefficient to the up side of the counter 56.
1 and the same coupling coefficient rewriting effect as in the case of FIG. 32 can be obtained.

Here, when the counters 51, 55 and 56 are used, digital ones may be used as the random number generator 52 and the comparator 53, and when an integrator is used instead of these counters. May be an analog type. This can be easily realized by using a known technique. For example, thermal noise of a transistor may be used as the analog random number generator, and an M-sequence pseudo random number generator may be used as the digital random number generator.

As described above, the method of expressing a signal with a pulse density is useful not only for actual circuits but also for simulation on a computer. On the computer, the calculation is performed serially, but compared with the calculation using the analog value, only the binary logical calculation of "0" and "1" is performed, so that the calculation speed is significantly improved. Generally, real-valued four arithmetic operations require many machine cycles for one calculation, but the number of logical operations is small. In addition, it is also easy to use low-level languages for high-speed processing when only logical operations are performed.

When a plurality of neurons 20 (or known neurons) as described above are integrated on one device (package) to form a network structure as shown in FIG. 43, the network structure, that is, The number of layers and the number of neurons in each layer must be determined. Therefore, in this embodiment, each connection is programmable, and the network structure itself is programmable (corresponding to the invention described in claims 1 and 2).

For example, in FIG. 2, four neurons 20 (#
1 to # 4) is a conceptual diagram showing a device in which the coupling between the neurons 20 is programmable by being mounted. First, each neuron 20 has an input signal line for that neuron and an error signal line (represented by an input signal line) 61 that is back-propagated from that neuron, and the output signal line from each neuron and that neuron And an error signal line (represented by an output signal line) 62 that is propagated back. Input signal line 61 between mutual neurons 20
A matrix circuit 63 is provided as a programmable coupling means capable of programming connection / disconnection between the output signal line 62 and the output signal line 62 at each lattice point. The matrix circuit 63 has an input signal line 64 for receiving an input signal from the outside of the device.
And an output signal line for outputting an output signal to the outside of the device and an error signal input line from the outside (represented by an output signal line) 6
5 and are included. Here, each lattice point in the matrix circuit 63 is composed of, for example, a fuse,
In the initial state, all are connected.

Therefore, a special signal (not shown) is applied from the outside.
Is given and the fuse in the unnecessary portion is blown, an arbitrary coupling state can be programmed. this is,
It can be implemented in the same manner as a conventional programmable ROM and can be easily realized. For example, if a similar one to a conventionally known erasable programmable ROM is used (corresponding to the invention of claim 6), it becomes erasable by ultraviolet rays, and the network structure programmed by ultraviolet irradiation is erased and reprogrammed. Can be converted. Further, if a similar one to the electrical erasable programmable ROM is used (corresponding to the invention of claim 7), it can be erased by electrical processing, and the network structure can be changed and set more easily.

Therefore, in the device shown in FIG. 2, for example, in the matrix circuit 63 of FIG. 1A, if the lattice points indicated by the points x are connection points and the other unmarked lattice points are non-connection points. As shown in FIG. 1B, the network shown in FIG. 1A is a neuron 2 in the input layer, the intermediate layer, and the output layer.
This is equivalent to a three-layer network structure with two 0s (though the neuron 20 in the input layer has 1 input, it is unnecessary as a device). In the figure, a,
b is an input signal to the network, A and C are output signals from the network, and B ₍₊₎ , B _(-) , D ₍₊₎ and D _(-) are error signals to the network.

When each connection is programmed as shown in FIG. 3 (a), as shown in FIG. 3 (b), there are three neurons 20 in the input layer and three neurons in the intermediate layer, and there are three neurons 20 in the output layer. One 3-layer structure network will be formed. In the figure, c indicates an input signal to the neural network, similar to a and b.

Therefore, the same device can have various network structures, and the device is very versatile. Note that the matrix circuit 63 does not necessarily have to be a complete matrix as shown in FIG. 2 and the like, and a part thereof may be omitted assuming a possible coupling range.

Further, when implementing the above-mentioned method, it is not necessary to form all the circuits, and some or all of them may be executed by software. Further, the circuit configuration itself is not limited to the illustrated one, and it may be replaced with another circuit having an equivalent logic, or further replaced with a negative logic.

Now, a specific example will be described. First, only four neurons 20 having the structure shown in FIG.
Fabricated on one chip. Next, the matrix circuit 63 was formed for the input / output signal lines of each neuron 20 as shown in FIG. 2, and the lattice point portion thereof had a normal erasable programmable ROM (EPROM) structure. Here, the shift register 26 of FIG. 24 is of 128 bits and the contents are rotated and used. Further, an input signal line 64 and an output signal line 65 as shown in FIG. 2 are provided outside the chip. Further, a power supply and a clock input terminal for shifting the shift register are provided. Such a chip is manufactured by a normal LSI process.

Subsequently, the second embodiment of the present invention will be described with reference to FIG.
Through FIG. 36. The same parts as those shown in the above-mentioned embodiments are designated by the same reference numerals (the same applies to the following embodiments). Although the programmable coupling between the neurons 20 is taken into consideration in the above-described embodiment, in the present embodiment, an error signal generation circuit that generates an error signal from the output signal and the teacher signal not only between the neurons 20 but also in the output layer is provided. A programmable combination is included, and claim 3,
This corresponds to the invention described in 6 and 7.

For example, FIG. 34 shows four neurons 20.
(# 1 to # 4) and four error signal generation circuits 70
FIG. 8 is a conceptual diagram showing a device in which (# 11 to # 14) are mounted and the coupling between the neurons 20 and between the error signal generation circuits 70 is programmable. Here, the signal line 6
Regarding a matrix circuit 63 capable of programming connection / disconnection with 1, 62 at each lattice point, an input signal line 71 for inputting an output signal y and a teacher signal d from the network to the error signal generation circuit 70. , The output signal y _i from the network and the output signal line 72 for outputting the error signals E _{(+) i} and E _{(-) i} to be propagated back to the network can be similarly programmed to be connected / disconnected at each grid point. It is added in a matrix configuration. Furthermore, an output signal line for outputting an output signal to the outside of the device and a teacher signal input line (hereinafter referred to as an output teacher signal line) 73 from the outside are similarly added in a matrix configuration. Also in this case, the matrix circuit 6
Each lattice point in 3 is composed of, for example, a fuse, and all are connected in the initial state.

Therefore, also in the case of this embodiment, an arbitrary coupling state can be programmed by applying a special signal (not shown) from the outside and cutting the fuse at an unnecessary portion. This may be exactly the same as the conventional programmable ROM and can be easily realized. For example, if the same one as the erasable programmable ROM is used (corresponding to the invention of claim 6), it becomes erasable by ultraviolet rays, and the network structure programmed by the ultraviolet irradiation can be erased and reprogrammed. Further, if a similar one to the electrical erasable programmable ROM is used (corresponding to the invention of claim 7), it can be erased by electrical processing, and the network structure can be changed and set more easily.

Therefore, in the device as shown in FIG. 34, for example, the matrix circuit 63 of FIG.
Assuming that the grid points indicated by X are connection points and the other unmarked grid points are non-connection points, the network shown in FIG. 35 (a) has an input layer, an intermediate layer and an output layer as shown in FIG. This is equivalent to a network structure having a three-layer structure in which two layer neurons 20 each are provided. In the figure, a and b are input signals to the network, and A to D are output signals from the network.

Further, when each connection is programmed as shown in FIG. 36 (a), as shown in FIG. 36 (b), there are 3 neurons 20 in the input layer and 3 neurons in the intermediate layer, and 20 neurons in the output layer. One 3-layer structure network will be formed. In the figure, c indicates an input signal to the neural network, similar to a and b.

Therefore, the same device can have various network structures, which is a very versatile device. Incidentally, also in the case of the present embodiment, the matrix circuit 6
Reference numeral 3 does not necessarily have to be a complete matrix as shown in FIG. 34 or the like, and a part thereof may be omitted assuming a possible coupling range.

In implementing the above-mentioned method, it is not necessary to make all of them into circuits, and some or all of them may be performed by software. Further, the circuit configuration itself is not limited to the illustrated one, and it may be replaced with another circuit having an equivalent logic, or further replaced with a negative logic.

Now, a specific example will be described. First, only four neurons 20 having the structure shown in FIG.
Fabricated on one chip. Next, a matrix circuit 63 was formed for the input / output signal lines of each neuron 20 as shown in FIG. 34, and the lattice points thereof had a normal erasable programmable ROM (EPROM) structure. Here, the shift register 26 of FIG. 24 is of 128 bits and the contents are rotated and used. Further, signal lines 64 and 65 as shown in FIG. 34 are provided outside the chip. Further, a power supply and a clock input terminal for shifting the shift register are provided. Such a chip is manufactured by a normal LSI process.

Further, a third embodiment of the present invention will be described with reference to FIGS. 37 to 39. In the above-mentioned embodiment, the programmable coupling between the neuron 20 and the error signal generating circuit 70 was taken into consideration, but in the present embodiment, considering that the neuron 20 has the pulse density type structure in which the pulse train is the signal transmitting means, In order to be able to deal with the case where the signal from the outside is a digital signal, a digital-pulse train conversion circuit is included and these are made to be mutually programmable, and correspond to the inventions of claims 4, 6 and 7. To do.

That is, in comparison with FIG. 34, as shown in FIG. 37, instead of the input signal line 64 from the outside of the device,
An input teacher signal line 75 for receiving an input signal and a teacher signal from the outside of the device is provided, an output signal line 76 for outputting an output signal to the outside of the device is provided instead of the output teacher signal line 73 to the outside of the device, and the matrix circuit is provided. 63
Is configured as a matrix. here,
A pulse train conversion circuit 77 for converting a digital signal (binary value) into a pulse train signal is provided on the input teacher signal line 75, and a pulse train signal is converted into a digital signal (binary value) on the output signal line 76. To binary conversion circuit 7
8 are provided. For these conversion circuits 76 and 78, for example, a combination configuration of a random number generator and a comparator, a counter, or the like as shown in FIGS. 31 to 33 may be used.

Therefore, in the device as shown in FIG. 37, for example, the matrix circuit 63 of FIG.
Assuming that the grid points indicated by x points are connection points and the other unmarked grid points are non-connection points, the network shown in FIG. 38 (a) has an input layer, an intermediate layer and an output layer as shown in FIG. 38 (b). This is equivalent to a network structure having a three-layer structure in which two layer neurons 20 each are provided. In the figure, a and b are digital input signals to the network, and A to D are digital output signals from the network.

Further, if each connection is programmed as shown in FIG. 39 (a), as shown in FIG. 39 (b), there are three neurons 20 in the input layer and three neurons in the intermediate layer, and there are three neurons 20 in the output layer. One 3-layer structure network will be formed. In the figure, c indicates a digital input signal to the neural network, similar to a and b.

Therefore, the same device can have various network structures, which is a very versatile device. In particular, in the case of the present embodiment, even if the neuron 20 or the network itself handles the pulse density, the digital signal can be handled as the input / output signal, and the versatility is further increased.

Next, a fourth embodiment of the present invention will be described with reference to FIGS. Also in this embodiment, the neuron 20
Is configured as a pulse density type, the purpose is to be able to handle a digital signal or an analog signal as an input / output signal for the network, but unlike the case of the above-described embodiment, learning is performed. Whether or not the function is present, only the forward process is specifically taken into consideration and corresponds to the inventions according to claims 5, 6 and 7.

For example, FIG. 40 shows six neurons 20.
FIG. 3 is a conceptual diagram showing a device in which (# 1 to # 6) are mounted and the coupling between the neurons 20 is programmable. First, each neuron 20 has an input signal line 81 for that neuron and an output signal line 82 from each neuron. A matrix circuit 83 is provided as a programmable coupling means capable of programming connection / disconnection between the input signal line 81 and the output signal line 82 between the neurons 20 of each other at each lattice point. The matrix circuit 83 includes an input signal line 84 for receiving an input signal from the outside of the device and an output signal line 85 for outputting an output signal to the outside of the device. Here, for the input signal line 84, a D / P converter (pulse train conversion circuit) 86 for converting a digital signal into a pulse train signal and an A / P converter (pulse train conversion circuit) 87 for converting an analog signal into a pulse train signal 87. The signal lines 88 interposing the and are arranged in a matrix form a part of the matrix circuit 83. Similarly, for the output signal line 85, a P / D converter (signal conversion circuit) that converts a pulse train signal into a digital signal
Signal lines 91 interposing 89 and a P / A converter (signal conversion circuit) 90 for converting a pulse train signal into an analog signal are arranged in a matrix and form a part of a matrix circuit 83. Here, each lattice point in the matrix circuit 83 is composed of, for example, a fuse, and all are connected in the initial state.

Therefore, also in the case of the present embodiment, an arbitrary coupling state can be programmed by applying a special signal (not shown) from the outside and cutting the fuse in the unnecessary portion.
This may be exactly the same as the conventional programmable ROM and can be easily realized. For example, if a similar one to a conventionally known erasable programmable ROM is used (corresponding to the invention of claim 6), it becomes erasable by ultraviolet rays, and the network structure programmed by ultraviolet irradiation is erased and reprogrammed. Can be converted. Further, if a similar one to the electrical erasable programmable ROM is used (corresponding to the invention of claim 7), it can be erased by electrical processing, and the network structure can be changed and set more easily.

Thus, in such a device,
For example, in the matrix circuit 83 of FIG. 40 (a), if the grid points indicated by x are connection points and the other unmarked grid points are non-connection points, the network shown in FIG. 40 (a) is shown in FIG. 40 (b). Thus, this is equivalent to a network structure of a three-layer structure in which four neurons 20 in the input layer and four neurons 20 in the intermediate layer and two neurons 20 in the output layer (in this case, the neuron 20 in the input layer also has one input). So it is not needed as a device). In the figure, a to d represent input signals, but the signals a,
b is a digital signal, c is an analog signal, and d is a pulse train signal. A and B indicate output signals, both of which are output as digital signals.

Further, in the same device configuration, as shown in FIG.
Programming each bond as shown in 1 (a)
As shown in (b), a network having a three-layer structure is formed, with three neurons 20 in each of the input layer, the intermediate layer, and the output layer. In the figure, a to c are digital input signals, and A to C are digital output signals.

Therefore, the same device can have various network structures, which is a very versatile device. In particular, it has versatility regardless of the type of input / output signal (pulse train / digital / analog). That is, not only is the interface between the neuron 20 using the pulse train as a signal transmission means and the external signal improved, but the connection of the converters 86, 87, 89, 90 is programmable, so that, for example, a part of the input signal This makes it possible to easily construct a network that uses pulse train signal inputs and the rest as digital signal inputs. Further, when the same device is recombined into a different network structure, this time, it becomes possible to flexibly deal with a configuration in which all input signals are analog signals.

Also in this embodiment, the matrix circuit 83
Does not necessarily have to be a complete matrix as shown in FIG. 40 and the like, and may be partially omitted in consideration of a possible coupling range. Further, when implementing the above-mentioned method, it is not necessary to make all of them into circuits, and some or all of them may be performed by software. Further, the circuit configuration itself is not limited to the illustrated one, and it may be replaced with another circuit having an equivalent logic, or further replaced with a negative logic.

[0134]

Since the present invention is configured as described above, according to the invention described in claim 1, since each neural cell mimicking unit with a learning function is connected by the programmable coupling means, it is formed in advance. The desired network structure can be formed or modified by simply selecting or changing the connection point in the circuit structure that is being programmed.
The signal processing device can be highly versatile.

In particular, according to the invention described in claim 2, since the nerve cell mimicking unit itself digitally processes signals, there is no problem such as temperature characteristics and drift as in the analog system. Since the information as the coupling coefficient is also stored in the memory, it can be easily rewritten and has versatility, and in combination with the versatility of the circuit network structure by the programmable coupling means, an excellent signal processing device can be obtained. it can.

According to the third aspect of the invention, not only the nerve cell mimicking unit, but also the error signal generating circuit in the output layer is selected or changed in the preformed circuit structure by the programmable coupling means. Since this is possible, the signal processing device can be changed to a desired structure and can be a versatile signal processing device.

In addition, according to the invention described in claim 4, not only the nerve cell mimicking unit but also the error signal generating circuit, the pulse train converting circuit and the binary number converting circuit in the output layer are formed in advance. Since the location by the programmable coupling means can be selected or changed in the network structure, it can be changed to a desired structure, in particular, a binary value as an external input / output signal to the nerve cell mimicking unit using a pulse train as a signal transmission means. Can handle
The signal processing device can be more versatile.

On the other hand, according to the fifth aspect of the present invention, a circuit network formed in advance including a pulse density type nerve cell mimicking unit and a signal conversion circuit between a digital signal or an analog signal and a pulse train signal is formed. Since it is possible to select or change the location by the programmable coupling means in the structure, it is possible to change to a desired structure, in particular, in addition to the pulse train signal as an external input / output signal to the nerve cell mimicking unit using the pulse train as the signal transmission means. It becomes possible to handle a digital signal and an analog signal, and it is possible to provide a signal processing device with higher versatility.

Further, according to the invention of claim 6 or 7, since such programmable coupling means is based on the use of ultraviolet irradiation or the use of electrical processing, it is possible to use an eraseable programmable ROM or an electrically erasable programmable ROM. Similarly, it can be realized easily, can be easily rewritten, and the network structure can be reprogrammed, which makes it more versatile.

[Brief description of drawings]

FIG. 1 shows a first embodiment of the present invention, in which (a) is a device conceptual diagram and (b) is an equivalent connection diagram.

FIG. 2 is a device conceptual diagram showing an initial state.

3A is a conceptual diagram of a device, and FIG. 3B is an equivalent connection diagram.

FIG. 4 is a logic circuit diagram for performing basic signal processing in an already proposed example.

FIG. 5 is a timing chart showing an example of logical operation.

FIG. 6 is a timing chart showing an example of logical operation.

FIG. 7 is a timing chart showing an example of logical operation.

FIG. 8 is a timing chart showing an example of logical operation.

FIG. 9 is a timing chart showing an example of logical operation.

FIG. 10 is a timing chart showing an example of logical operation.

FIG. 11 is a timing chart showing an example of logical operation.

FIG. 12 is a timing chart showing an example of logical operation.

FIG. 13 is a timing chart showing an example of logical operation.

FIG. 14 is a timing chart showing an example of logical operation.

FIG. 15 is a timing chart showing an example of logical operation.

FIG. 16 is a timing chart showing an example of logical operation.

FIG. 17 is a timing chart showing an example of logical operation.

FIG. 18 is a timing chart showing an example of logical operation.

FIG. 19 is a timing chart showing an example of logical operation.

FIG. 20 is a timing chart showing an example of logical operation.

FIG. 21 is a timing chart showing an example of logical operation.

FIG. 22 is a timing chart showing an example of logical operation.

FIG. 23 is a timing chart showing an example of logical operation.

FIG. 24 is a logic circuit diagram showing a configuration example of each unit.

FIG. 25 is a logic circuit diagram showing a configuration example of a modified example thereof.

FIG. 26 is a logic circuit diagram showing a configuration example of the modification.

FIG. 27 is a logic circuit diagram showing a configuration example of a modified example thereof.

FIG. 28 is a logic circuit diagram showing a configuration example of each unit.

FIG. 29 is a logic circuit diagram showing a configuration example of each unit.

FIG. 30 is a logic circuit diagram showing a modified example.

FIG. 31 is a circuit diagram for varying the coupling coefficient.

FIG. 32 is a circuit diagram for varying the coupling coefficient.

FIG. 33 is a circuit diagram for varying the coupling coefficient.

FIG. 34 is a device conceptual diagram showing an initial state of the second embodiment of the present invention.

FIG. 35 (a) is a device conceptual diagram, and FIG. 35 (b) is an equivalent connection diagram.

36A is a conceptual diagram of a device, and FIG. 36B is an equivalent connection diagram.

FIG. 37 is a device conceptual diagram showing the initial state of the third embodiment of the present invention.

38A is a device conceptual diagram, and FIG. 38B is an equivalent connection diagram.

FIG. 39 (a) is a device conceptual diagram, and FIG. 39 (b) is an equivalent connection diagram.

FIG. 40 shows a fourth embodiment of the present invention, (a) is a device conceptual diagram, and (b) is an equivalent connection diagram.

41A is a conceptual diagram of a device, and FIG. 41B is an equivalent connection diagram.

FIG. 42 is a conceptual diagram showing one unit configuration of a conventional example.

FIG. 43 is a conceptual diagram of the neural network configuration.

FIG. 44 is a graph showing a sigmoid function.

FIG. 45 is a circuit diagram showing a specific configuration of one unit.

FIG. 46 is a block diagram showing a digital configuration example.

FIG. 47 is a circuit diagram of a part thereof.

FIG. 48 is a different part of the circuit diagram.

[Explanation of symbols]

 20 nerve cell mimicking unit 50 self-learning means 63, 83 programmable coupling means 70 error signal generation circuit 77 pulse train conversion circuit 78 binary number conversion circuit 86, 87 pulse train conversion circuit 89, 90 conversion circuit

Front page continuation (72) Inventor Hirotoshi Eguchi 1-3-6 Nakamagome, Ota-ku, Tokyo Inside Ricoh Co., Ltd. (72) Inventor Osamu Takehira 1-3-6 Nakamagome, Ota-ku, Tokyo Inside Ricoh Co., Ltd.

Claims (7)

[Claims] 1. In a signal processing device in which a plurality of neural cell mimicking units with learning functions are housed in one package to form a circuit network, programmable coupling means capable of connecting / disconnecting each neural cell mimicking unit. A signal processing device characterized by being connected by. 2. Self-learning means having coupling coefficient varying means and coupling coefficient generating means for generating variable coupling coefficient values of the coupling coefficient varying means based on a positive error signal and a negative error signal with respect to a teacher signal. A signal processing device characterized in that a plurality of nerve cell mimicking units attached thereto are connected in a net-like manner to form a circuit network, and the nerve cell mimicking units are connected by programmable connecting means which can be connected / disconnected freely. 3. A signal processing in which a plurality of neural cell mimicking units with learning functions and an error signal generating circuit for generating an error signal from its output signal and a teacher signal are contained in one package to form a circuit network. In the device, the signal processing device is characterized in that the respective nerve cell mimicking units and the error signal generating circuits are connected by programmable connecting means which can be freely connected / disconnected. 4. A plurality of pulse density type nerve cell mimicking units with a learning function using a pulse train as a signal transmission means, an error signal generating circuit for generating an error signal by its output signal and a teacher signal, and converting a binary value into a pulse train. In a signal processing device in which a pulse train conversion circuit for converting and a binary number conversion circuit for converting a pulse train into a binary number are housed in one package to form a circuit network, the neural cell mimicking units and the circuits are connected. A signal processing device characterized by being connected by a non-connectable programmable coupling means. 5. A plurality of pulse density type nerve cell mimicking units using a pulse train as a signal transmission means, a pulse train conversion circuit for converting a digital signal or an analog signal into a pulse train signal, and a conversion circuit for converting the pulse train signal into a digital signal or an analog signal. In a signal processing device in which and are housed in one package to form a circuit network, the neural cell mimicking units and the circuits are connected by programmable connecting means that can be connected / disconnected. Signal processing device. 6. The signal processing apparatus according to claim 1, wherein the programmable coupling means is erasable by irradiation of ultraviolet rays. 7. The signal processing device according to claim 1, wherein the programmable coupling means is erasable by electrical processing.