JP3276367B2 - Neural cell network - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing method and apparatus, and more particularly, to a neural computer that imitates a neural network, for example, character and figure recognition, motion control of a robot or the like, and associative memory. It is suitable for application to the like. 2. Description of the Related Art Parallel processing of information is achieved by imitating the functions of nerve cells (neurons), which are the basic units of information processing in living organisms, and by configuring this "neural cell mimic element" (neural cell unit) in a network. The goal is a so-called neural network. Even if it is very easily performed on a living body, such as character recognition, associative memory, and motion control, there are many that cannot be easily achieved by a conventional Neumann computer. Attempts to solve these problems by imitating the nervous system of a living body, particularly, a function unique to the living body, that is, parallel processing, self-learning, and the like, have been actively made mainly by computer simulation. Figure 6 is a diagram for explaining a model of a conventional neural network, in the figure, A is represents one neuron unit, Figure 7 is obtained by constituting it to the network, A _1, A ₂ and A ₃ each represent a neuron unit. One neuron unit is coupled to a number of other neuron units and processes and outputs signals received from them. In the case of FIG. 7, the network is of a hierarchical type, and the neural processing unit receives a signal from the neural cell unit of the immediately preceding layer (left side in the figure) and the neural cell in the next layer (right side in the figure). Output to the unit. First, the neuron unit A shown in FIG. 6 will be described. The degree of coupling between one neuron unit and another neuron unit is called a coupling coefficient (T). Yes, i-th neuron unit and j
The coupling coefficient of the th neuron unit is generally denoted by T _ij . There are two types of coupling: excitatory coupling, in which the larger the signal from the partner's neuron, the larger the output of the partner, and conversely, the larger the signal of the partner, the smaller the output, in which the own output is smaller, and T _ij > 0. An excitatory connection and T _ij <0 indicate an inhibitory connection, respectively. The input to the j-th neuron unit, and the output from the i-th neuron units and y _i, determined as T _ij y _i obtained by multiplying the T _ij in the y _i.
As described above, since one neuron unit is connected to many neuron units, the sum of T _ij y _i for all neuron units, that is, ΣT _ij
y _i is an input to one neuron unit in the network. This is called an internal potential and is represented by u _j . u _j = ΣT _ij y _i (1) Next, this input (= internal potential) is subjected to non-linear processing to obtain the output of the nerve cell unit. The nonlinear function used here is called a nerve cell response function, and a sigmoid function f (x) as shown below is used. f (x) = 1 / (1 + e− ^x ) (2) FIG. 8 is a diagram showing the sigmoid function. The above-mentioned nerve cell unit is configured into a network as shown in FIG.
By calculating (2) one after another in parallel, parallel processing of information becomes possible, and a final output is obtained. FIG. 9 is a diagram showing an example of the above-mentioned network realized by an electric circuit (Japanese Patent Laid-Open No. 62-295188), in which the signal strength of input and output to the network is represented by a voltage, The value of the coupling coefficient between the units is realized by a resistance value. That is, in FIG. 9, a plurality of amplifiers 33 have an inverting output 33a and a non-inverting output 33b, and each of the amplifiers 33 has a means 32 for supplying an input current to an input of the amplifier 33. An interconnect matrix 31 connects the output of each of the amplifiers to the input with a conductance (T _ij ) of a first value or a preselected second value. T _ij represents the transconductance between the output of the i-th amplifier and the input of the j-th amplifier, wherein the conductance T _ij is chosen to create a plurality of local minima where the network is balanced; The energy function with multiple minima is minimized. When the coupling coefficient T _ij is negative, a negative resistance value cannot be realized, and this is realized by inverting the output using the amplifier 33. An amplifier 33 is used as a function corresponding to the sigmoid function shown in FIG. Next, the learning function of the network will be described. As a learning rule used in numerical calculation, there is the following one called back propagation. First, the coupling coefficient between each neuron unit is first randomly given. In this state, if an input is provided to the network, the output result is not always desirable. Therefore, the correct answer (= teacher signal)
And again, when the same input is given, the answer is correct (=
Each coupling coefficient is changed so that a desired output result is obtained. For example, in the hierarchical network shown in FIG. 7, the output of the j-th neuron unit in the final layer (right layer in the figure) is y _j, and the teacher signal for the neuron unit is d _j. , E = Σ (d _j −y _j ) ² (3) so that E represented by _Is used to change T _ij . More specifically, first, δ (error signal) is obtained as follows. δ _j = (d _j −y _j ) × f ′ (u _j ) [in the case of the output layer] (5) δ _j = Σδ _i T _ij × f ′ (u _j ) [the previous layer (intermediate layer) (6) where f ′ is the first derivative of f. Using this, T _ij is changed by setting ΔT _ij = η (δ _j y _i ) + αΔT _ij ′ T _ij = T _ij ′ + ΔT _ij (7). Where ΔT _ij ',
T _ij 'is a value at the time of the previous learning. Where η
Is a learning constant and α is a stabilization constant. Since each is not theoretically determined, it is determined empirically. An algorithm for obtaining the amount by which the coupling coefficient is changed by such a method is called back propagation. By repeating the above process, i.e. the learning, eventually, T _ij such desired results for a given input is obtained is determined. As a result, a network learning function can be realized. 10 to 12 are diagrams showing an example in which such a neural network is realized by a digital circuit.FIG. 10 shows an example of a single nerve cell circuit configuration, in which 35 is a synapse circuit, 36 indicates a dendritic circuit, and 37 indicates a cell body circuit.
FIG. 11 is a configuration example of the synapse circuit 35 shown in FIG.
FIG. 12 is a diagram showing a configuration example of the cell body circuit 37 shown in FIG. 10. In FIG. 11, f is an input signal, w is a weighting value, and a is a magnification (1 or 2) by which the feedback signal is multiplied. It is. This means that input / output signals of the nerve cell unit are represented by pulse trains, and signal values are represented by pulse densities. Coupling coefficients are handled in binary numbers and stored in memory. The signal operation is performed as follows. First, the input signal is input to the clock of the rate multiplier, and the coupling coefficient is input to the rate value, thereby reducing the pulse density of the input signal according to the rate value. This corresponds to the expression T _ij y _i in the above-described back propagation model. Next, the part of {T _ij y _i } is realized by an OR circuit indicated by the dendrite circuit 36. Since the coupling has excitatory and inhibitory properties, it is divided into groups in advance, and OR is performed for each group. In Figure 10, F ₁ is excitatory, F ₂ denotes the inhibitory output. These two outputs are input to the up side and the down side of the counter shown in FIG. 12, respectively, and counted to obtain outputs. Since this output is a binary number, it is converted into a pulse density again using a rate multiplier. By constructing a network using a plurality of the nerve cell units, a neural network can be realized. The learning function is realized by inputting the final output of the network to an external computer, performing a numerical calculation inside the computer, and writing the result to a memory for coupling coefficients. However, many trials of neural networks have been performed by computer simulation, and parallel processing is required to exhibit their original functions, and hardware implementation of the network is indispensable. Attempts to implement a neural cell unit in hardware are mostly made with analog circuits, and these circuits have the following problems. The strength of a signal inside the network is represented by an analog value such as a voltage or a current, and the internal calculation is also performed in an analog manner. Many elements are required to construct a network, but it is difficult to make the characteristics of each element uniform. If the accuracy or stability of one element becomes a problem, a new problem may occur when it is made into a network, and the operation in the entire network cannot be predicted. The value of the coupling coefficient is fixed, and a value learned in advance by another method such as simulation has to be used, and self-learning cannot be performed. On the other hand, if the learning method using back propagation is realized by hardware by some means, a large amount of four arithmetic operations are required for learning, and it is difficult to realize.
Also, the learning method itself is not suitable for hardware implementation. A network using a conventional digital circuit also performs learning by an external computer, and thus does not have a self-learning function. In addition, since the signal of the pulse density is once converted into a numerical value (binary number) using a counter, and then converted into the pulse density again, the configuration of the circuit is complicated. In summary, the prior art has the following disadvantages. Analog circuits that perform calculations inside the network do not operate reliably. The learning method by numerical calculation is also complicated in calculation and is not suitable for hardware implementation. The circuit configuration of a conventional digital circuit is complicated. You cannot self-learn on hardware. Therefore, the present applicant first employed a digital circuit with a reliable operation for the operation inside the network, and provided a simple signal processing and learning method that was easy to implement in hardware. A signal processing circuit capable of realizing hardware self-learning was proposed. Object The present invention has been made in view of the above-mentioned circumstances, and in particular, further improves the signal processing circuit previously proposed by the present applicant so that the input signal pulse train and the teacher signal pulse train are fixed to one pattern. The purpose of the present invention is to realize a network having a higher learning ability without realizing a more versatile signal processing circuit. ConfigurationIn order to achieve the above object, the present invention provides a neural cell network in which a neural cell mimic element is configured in a hierarchical network, and an input signal to the neural cell mimic element of an input layer of the neural cell network is This signal is a pulse train signal. In this neural cell network, calculation is performed using the pulse density represented by the pulse train, and each neural network is backpropagated from an error signal between the teacher signal and an output signal from the output layer of the neural cell network. In a neural cell network that performs learning by changing a coupling coefficient between cell mimic elements, a pulse train pattern of input signal data or teacher signal data input to the neural cell network performs learning by the neural cell network. It is characterized by a different pulse train pattern obtained by changing the pulse arrangement order without changing the pulse density every time. It is. Hereinafter, a description will be given based on an example of the present invention. First, the basic idea of the present invention will be described. The basic idea of the present invention is that input / output signals, intermediate signals, coupling coefficients, teacher signals, etc. relating to a nerve cell unit are all binary values of 0 and 1. It is represented by the expressed pulse train. The value of the signal inside the network is represented by the pulse density (the number of “1” within a certain time). The calculation in the nerve cell unit is performed by a logical operation between pulse trains. The pulse train of the coupling coefficient is stored in a memory in the nerve cell unit. Learning is realized by rewriting this pulse train. An error is calculated based on the given teacher signal, and the coupling coefficient is changed based on the error. In this case, the calculation of the error and the calculation of the change in the coupling coefficient are all performed by the logical operation of the 0 and 1 pulse trains. In the case where the supervised learning is performed by using the network circuit configured by the neural cell unit described above, the pulse train of the input signal and the supervisory signal is not limited to one pattern, and the pulse train having the same pulse density is used. By using the pulse train at any time, the learning ability of the network is enhanced. This will be described in detail below based on an embodiment that embodies this. The description will be given first on signal processing by a nerve cell unit using a digital logic circuit and a network circuit configured using the same, and then on an example of a circuit for performing the above processing. I. Signal processing by digital logic circuit

[Signal operation in forward process]

FIG. 1 shows a portion corresponding to one nerve cell unit, which is formed into a network as shown in FIG. All inputs and outputs of the nerve cell unit are binarized to 0 and 1, and further synchronized. The value (= strength) of the signal at input j is
It is expressed by a pulse density, for example, by the number of states of one within a certain time as shown in the following pulse train. This shows a signal representing 4/6, and the sync pulse 6
The input signal indicates that there are four 1s and two 0s. At this time, it is desirable that the arrangement of 1s and 0s is random as described later. On the other hand, the coupling coefficient T _ij is also represented by a pulse density,
And a bit string of 1 are prepared in advance in the memory. This indicates that the value of the coupling coefficient is 3/6, and in this case, it is desirable that the arrangement of 0s and 1s is random. How to determine specifically will be described later. And
The coupling coefficient pulse train is sequentially read from the memory in accordance with the synchronous clock, and ANDed with the input pulse train by the AND circuit of FIG. 1 (y _i ∩T _ij ). This is set as an input to the neuron unit j. To explain using the above example, when the signal “101101” is input, the pulse train of the coupling coefficient is called from the memory in synchronization with the signal, and by sequentially taking the AND, A pulse train (bit train) “101000” as shown in FIG. This is because the pulse train y _i of the input signal is the pulse train of the coupling coefficient
It is converted by T _ij, which indicates that the input pulse density to the neuron unit is 2/6. The pulse density of the output of the AND circuit is approximately the product of the “pulse density of the input signal” and the “pulse density of the coupling coefficient”, and has the same function as the signal product in the analog system.
This is a function closer to a product of numerical values as the signal train (pulse train) is longer and the arrangement of 1s and 0s is more random. Non-random means that 1 (or 0) is dense (close). When the length of the pulse train of the coupling coefficient is shorter than that of the input pulse train and there is no more data to be read, it can be dealt with by returning to the head of the pulse train of the coupling coefficient again and repeating the reading. Since one neuron unit has many inputs, there are many "AND of input signal and coupling coefficient" described above. Next, these ORs are taken. Since the inputs are synchronized, if the first data is "101000" and the second data is "010000", the OR of both is "111000". This is calculated at the same time as the multi-input and output. Becomes This part is the calculation to find the sum of the signals and the nonlinear function (sigmoid function) in the case of the analog system.
Corresponds to the part. In a general pulse calculation, when the pulse density is low, the OR of the pulse density approximately matches the sum of the respective pulse densities. As the pulse density increases, the output of the OR gradually saturates, so the sum of the pulse densities does not match the result,
Non-linearity appears. In the case of OR, the pulse density does not become larger than 1 and does not become smaller than 0, and since it is a monotonically increasing function, it becomes approximately equivalent to the sigmoid function. By the way, coupling has excitability and inhibition, and in the case of numerical calculation, it is represented by the sign of the coupling coefficient. In the case of an analog circuit,
As described above, when T _ij becomes negative (inhibitory coupling), the output is inverted using an amplifier and coupled to another neuron unit with a resistance value corresponding to T _ij . On the other hand, in the present invention, each connection is divided into two groups, excitatory connection and inhibitory connection, _{depending on} the sign of T _ij , and then OR of “AND of pulse train of input signal and coupling coefficient” is performed for each group. I do. As a result, 1 is output only when “the output of the excitatory group is 1” and “the output of the inhibitory group is 0”. In order to realize this function, AND of “NOT of the output of the suppressive group” and “output of the excitatory group” may be performed. That is, When this is expressed by a logical expression, a = ∪ (y _i ∩T _ij ) (T = excitatory) (13) b = ∪ (y _i ∩T _ij ) (T = inhibitory) (14) y _j = A∩ (15) The configuration of the network using this neuron unit is a hierarchical type similar to back propagation (Fig. 7).
And If the entire network is synchronized, it is possible to perform calculations in parallel with the functions described above for each layer.

[Signal operation in learning (back propagation)]

Learning is performed by obtaining an error signal as follows or by changing the value of the coupling coefficient using the method described below. Error Signal in Last Layer A method for obtaining an error signal for each neuron unit belonging to the last layer (the right layer in FIG. 7) will be described. In the present invention, the error signal is defined as follows. That is, when an error is represented by a numerical value, it can generally take both positive and negative values. However, since such a representation cannot be made with a pulse density, the error is expressed by using two signals, a signal representing a + component and a signal representing a-component. Express the signal. In other words, of the difference between the teacher signal pulse and the output signal pulse, the pulse existing on the teacher signal side is used as the error signal.
⁺ A pulse, conversely, the pulse present at the output signal side error signal ^- a pulse. In other words, adding the error signal ⁺ pulse to the output signal pulse, the error signal ^- when removing the pulse, the teacher signal pulse. The error signal obtained as described above is subjected to processing using a learning constant, which will be described in detail in the next section. Error Signal in Intermediate Layer The error signal δ for each neuron unit belonging to the intermediate layer (the central layer in FIG. 7) is obtained as follows. That is, error signals in each neuron unit of the next layer (final layer in FIG. 7) connected to the neuron unit are collected and used as an error signal. This can be performed in a manner similar to the arithmetic expressions (8) to (19) in the nerve cell unit. However, equations (8) to (1
Unlike 9), y is one signal, while δ has two signals to represent positive and negative, and both signals must be considered. Therefore, it is necessary to divide into four cases, that is, positive and negative of T (coupling coefficient) and positive and negative of δ (error signal). First, the case of excitatory coupling will be described. For the neuron unit having the intermediate layer, the error signal ⁺ in the neuron unit in the next layer (the last layer in FIG. 7), the neuron unit and the self (the neuron unit having the intermediate layer) AND of the coupling coefficients (= δ ⁺ ∩
The T _ij)), obtained for each neuron unit of one previous layer, further taking OR of each other (= ∪ (δ ⁺ _i ∩
T _ij ). The result is taken as the error signal ^{+ of} this layer. That is, it is expressed as follows. Similarly, the error signal at the neuron unit in the next layer
^- By using the error signal of this layer ^- it can be a determined in the following manner. Next, the case of inhibitory coupling will be described. Error signal in neurons units of one previous layer ^- and the neuron unit and takes an AND of the coupling coefficient between themselves, the results of further taking these simultaneous OR, the error signal of this layer ⁺ . That is, Similarly, the error signal at the neuron unit in the next layer
By using ⁺ , the error signal ⁻ of this layer for the suppressive coupling can be determined. Coupling from one nerve cell unit to another neuron units, because although the two cases inhibitory as for excitatory certain, [delta] was calculated by the formula (20) in the obtained [delta] ⁺ and Equation (22) ⁺ OR
And set it as δ ^{+ of} this nerve cell unit. Similarly,
Takes the OR, [delta] of the neuron unit ^{- -} [delta] obtained in the equation (23) ^- equation (21) in the obtained [delta] to. To summarize the above, Becomes Here, processing using a learning constant is performed. This is realized by multiplying the error signal by a learning constant (learning rate) in numerical calculation and the like. When the value of the learning rate is 1.0 or less, the learning ability of the network is further enhanced. Since this corresponds to lowering the pulse density in the operation of the pulse train in the present invention, it can be realized by "thinning out" the pulse train. For example, Example 1 shows a case where the pulse train of the original signal is at regular intervals, and Example 2 shows a case where the pulse train of the original signal is not at regular intervals. In both cases, at the learning rate (η) = 0.5, the pulse train of the original signal is 1 Erase (remove) every other time. When y = 0.33, every third pulse is left, and when y = 0.67, every third pulse is erased. Such “thinning out” of the error signal can be easily realized by performing a logical operation on the output of a commercially available counter or using a flip-flop. In particular, when a counter is used, the value of η can be set arbitrarily and easily, so that the characteristics of the network circuit can be controlled. Changing Each Coupling Coefficient According to Error Signal A method of changing each coupling coefficient using the error signal obtained as described above or above will be described. Line to which the coupling coefficient to be changed belongs (see Fig. 7)
(= Input signal to neuron unit)
And the error signal (AND) (δ∩y). However, in this embodiment, since there are two error signals, + and-, the error signals are calculated and obtained. The two signals obtained in this way are represented by ΔT ⁺ ,
ΔT ^- to. A new coupling coefficient T is calculated based on these values. Since the value of T in this embodiment is an absolute value, cases are classified depending on whether T is excitatory or suppressive. ● For the case the original T of excitability, increase the component of [Delta] T ^+, [Delta] T ^- reducing components. ● For the case the original T inhibitory, reducing the component of [Delta] T ^+, [Delta] T ^- Increase of components. The calculation of the entire network is performed based on the above learning rules.

[Network learning operation]

As described above, in order to train the neural network as shown in FIG. 7, a set of input signals to be input to the input layer (the left layer in FIG. 7) and the corresponding output layer (the seventh layer) are input to the input layer. A set of desired output results (= teacher signal) to be output from the right layer in the figure) is required. Moreover, the data used for them is a pulse train whose value is represented by density. As described above, it is preferable that the position of the bit that is 1 in the pulse train be random for the operation of the nerve cell unit. This means that a pulse train signal having the same value but a different pattern must be provided at each learning. Accordingly, the present invention provides a system for generating a pulse train of input signal data and a teacher signal data of the network circuit, in which the pulse train patterns of various data are made different at each learning to improve the learning ability of the network. is there. Hereinafter, the configuration and operation will be described. An appropriate set of input signal data and a set of teacher signal data corresponding to the input data are prepared for learning the network circuit for a certain purpose. Alternatively, as the teacher signal data, data corresponding to the input signal data may be appropriately given. In the case of learning only, these sets may be stored in an external memory of the network. These are converted into a pulse train so as to match the input of the nerve cell unit. Alternatively, it may be stored as a numerical value and converted into a pulse train as needed. The set of input signal data is paired with the set of teacher signal data,
At the time of reading, each pair is read. After preparing the above, learning of the network is started.
Now, a system for generating a pulse train is disclosed in Japanese Patent Application No. 2-67942.
As shown in the figure, if noise existing in the natural world (such as thermal noise of a transistor) is used, the pulse train used for learning the network has complete randomness. It will be different when learning. However, when a random number generator is used as described in the specification of a patent application separately filed by the present applicant, the generated bits circulate with a certain period because the random numbers are pseudo-random numbers. Will be. Therefore,
If the pulse train length of the data matches the cycle of the random number, the same pulse train pattern is always generated for the same value. Therefore, by changing the random number generation cycle,
Change the phase of random numbers. This can be realized by appropriately inputting more basic clocks for generating random numbers to the random number generator than necessary. As a result, different pulse train patterns can be obtained at each learning, and the network can perform highly effective learning. II. Circuit Example FIGS. 2 to 6 show circuit examples based on the above algorithm, and the configuration of the neural network is the same as that of FIG. The circuit corresponding to the "line" in FIG. 7 is shown in FIG. 2, and the circuit corresponding to the "circle" (neural cell unit) in FIG. 7 is shown in FIG. FIG. 4 shows a circuit for obtaining an error signal in the final layer from the output of the final layer and the teacher signal. By constructing the circuits shown in these three figures into a network as shown in FIG. 7, a digital neural network capable of self-learning can be realized. First, FIG. 2 will be described. Signal 1 is an input signal to the nerve cell unit and corresponds to equation (8). The coupling coefficient of equation (9) is stored in the shift register 8. Terminal 8A is a data takeout port, and terminal 8B is a data entry port. As long as it has a function similar to that of the shift register, another one such as a RAM + address controller may be used. The circuit 9 is a circuit corresponding to the equation (10), and ANDs the input signal and the coupling coefficient. This output must be grouped according to whether the coupling is excitatory or inhibitory, but the output to each group
It is more versatile to prepare a group and switch between groups. For this reason, a bit indicating whether the coupling is excitatory or inhibitory is stored in the memory 14, and the circuit
Switch the signal with 13. The OR circuit corresponding to the equation (11) for processing each input is the circuit 16 in FIG. Further equation (12)
The circuit that outputs “1” only when “excitable group is 1” and “suppressive group is 0” is circuit 17 in FIG. Next, the error signal will be described. FIG. 4 shows a circuit for generating an error signal in the last layer. This is the equation (16)-(19)
Is equivalent to The error signals 6, 7 are generated from the output 1 from the last layer and the teacher signal 20. A circuit for realizing the equations (20) to (23) for obtaining the error signal in the intermediate layer is shown as a circuit 10 in FIG. The error signal used differs depending on whether the coupling is excitatory or inhibitory.
It is 12. For this circuit, bits set in the memory 14 in advance are used. The operation of collecting error signals (Equation (24)) is performed by the circuit 18 in FIG. Equation (25) corresponding to the learning rate is realized by the frequency dividing circuit 19 in FIG. Next, a portion for calculating a new coupling coefficient using the error signal will be described. This is represented by equations (26)-(29) and is performed by the circuit 11 of FIG. However, since it is necessary to divide the cases depending on the excitability and suppression of the connection, this is realized by the circuit 12 in FIG. Finally, a case where a system using the above network circuit is learned will be described. In order to input data not only to learning but also to a network, it is necessary to generate a pulse train that represents the value of data in terms of density. An example of the generation circuit is shown in FIG. 5 (corresponding to FIG. 11 of a patent application separately filed by the present applicant). This is realized by inputting the value of data to be converted into a random number into the input 21 of FIG. ,
A random pulse train is obtained from the output 22. The random number generator 23 is a random number generator with a period T. The reference clock of the network is used as the basic clock in FIG. Now, if a random pulse train is generated continuously, the phase of the generated pulse train will be the same every time, and the pulse train input to the network will always have the same pulse pattern for those with the same value. I will.
Therefore, without directly connecting the basic clock of FIG. 5 and the reference clock of the network, during the period from the end of one cycle T to the start of the next cycle, the blanking for random number sequence generation is performed. The clock circuit 24 supplies one or more clocks to the basic clock. Then, the phase of the random number sequence of the previous cycle and the phase of the current random number sequence are shifted by at least one. By repeating this at each learning, a different random number sequence is obtained every time.
Random pulses can be input to the network. By providing this system in the input unit and / or the teacher signal input unit of the network, learning of the entire network is performed. The above circuit is not limited to the above embodiment, and it is needless to say that a part of the circuit may be substituted by a computer or other computing device. Effects As is clear from the above description, according to the present invention, it is possible to avoid the fixation of the coupling coefficient and increase the learning ability.

[Brief description of the drawings]

1 is a diagram showing one of the neural circuit units, FIGS. 2 to 4 are diagrams showing an example of a circuit configuration of each part of the neural circuit unit, and FIG. 5 is a diagram showing an example of a pulse train generating circuit. 6 to 8 are diagrams for explaining the principle of operation of the neural circuit unit, and FIGS. 9 to 12 are diagrams showing examples of a conventional circuit configuration. 1 ... input signal, 2, 3, 6, 7 ... error signal, 4 ... excitatory signal, 5 ... suppressive signal, 8 ... shift register, 20 ...
... teacher signal, 21 ... input, 22 ... output, 23 ... random number generator, 24 ... random-hit clock circuit for random number sequence generation.

Continuation of the front page (56) References JP-A-1-271888 (JP, A) JP-A-2-33655 (JP, A) JP-A-2-238721 (JP, A) JP-A-2-199576 (JP) , A) Japanese Patent Laid-Open No. 1-244567 (JP, A) Translated by Shunichi Amari, "PDP model-cognitive science and search for neuron networks-", Japan, Sangyo Tosho Co., Ltd., February 27, 1989, First edition, pp. 321-365 Yoshida, Horiguchi, Eguchi, "Pulse Density Neuron Model and Hardening of Self-Learning Function", Proc. Of the 1990 IEICE Spring Conference, Japan, The Institute of Electronics, Information and Communication Engineers, Japan. Published, March 5, 1990, 6th volume, pp. 56 Masahiko Koike, "Hardware for Neurocomputing", Transactions of the Institute of Electronics, Information and Communication Engineers, Japan, published by the Institute of Electronics, Information and Communication Engineers, August 25, 1990, vol. J73-D-II, no. 8, p.p. 1132-1145, Patent Office CSDB Literature No .: CNT199800681003 Joshua Alspector, et. al. , "Generating multiple analog sources from a single linear linear feedback shift ...", Proc. of IEEE Int. Symp. on Circuits and Systems 1990. United States, May 3, 1990, vol. 2, pp. 1058-1061, INSPEC Accession Number: 3853488 Yuzo Hirai et al., "Complete Digital Neurochip Configuration," IEICE Technical Report, Japan, IEICE Technical Report, Japan, 1988, vol. 88, no. 323 (ICD88-130) pp. 89-96, JICST Document No .: S0532B (58) Field surveyed (Int. Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7/60 JICST file (JOIS) CSDB (Japan Patent Office)

Claims (2)

(57) [Claims] An input signal to a neuron mimic element in an input layer of the neural cell network is a pulse train signal. In the cell network, the calculation is performed using the pulse density represented by the pulse train, and the coupling coefficient between each neuron mimic element is calculated by back propagation from the error signal between the teacher signal and the output signal from the output layer of the neuron network. In the neural cell network for learning by changing, the pulse train pattern of the input signal data input to the neural cell network has the same pulse density for the same input signal value. Is a pulse train pattern in which a pulse rises at a different random bit position each time learning is performed. 2. A neural cell network comprising a neural network mimic element in a hierarchical network, wherein an input signal to the neural cell mimic element in an input layer of the neural cell network is a pulse train signal. In the cell network, the calculation is performed using the pulse density represented by the pulse train, and the coupling coefficient between each neuron mimic element is calculated by back propagation from the error signal between the teacher signal and the output signal from the output layer of the neuron network. In a neural cell network that performs learning by changing, the pulse train pattern of teacher signal data input to the neural cell network has the same pulse density for the same teacher signal value, but the neural cell network Is a pulse train pattern in which a pulse rises at a different random bit position each time learning is performed.