JPH06131317A - Neuron element - Google Patents

Abstract

PURPOSE:To provide the neuron element with teacher free learning function for enabling self-organization as one of big features of a neural network. CONSTITUTION:This neuron element for forming a pulse density type neural network by outputting one output signal in a pulse sequence expression corresponding to plural input signals respectively connected in the shape of a network in the pulse expression is provided with learning means 20 and 21 for deciding the pulse value of a new coupling coefficient based on the pulse of a present coupling coefficient, the pulse of an input signal, the pulse of an exciting output and the pulse of a suppressing output.

Description

Detailed Description of the Invention

[0001]

The present invention relates to image recognition, voice recognition,
The present invention relates to a neuron element which is a constituent element of a neurocomputer imitating a neural network applied to various fields such as robot movement control, stock price and earthquake prediction, associative memory, and other fields.

[0002]

2. Description of the Related Art The function of a nerve cell (neuron), which is a basic unit of information processing of a living body, is mimicked, and further, the nerve cell mimicking element (nerve cell unit) is networked to perform parallel processing of information. What we aimed for was a so-called neural network. Character recognition, associative memory,
Even if it is done easily in the living body such as movement control,
Many of the conventional Neumann computers cannot easily achieve this. The nervous system of the living body, especially the functions peculiar to the living body,
That is, attempts to solve these problems by imitating parallel processing, self-learning, etc. are being actively made centering on computer simulation.

As a system for realizing such a neural network, for example, Japanese Patent Laid-Open No. 4-549,
The applicant has proposed a pulse density type digital neural network as disclosed in Japanese Patent Laid-Open No. 4-111185.

[0004]

Considering the learning rule of the pulse density type neural network of the proposed example, it is the error back propagation type, that is, the supervised learning type. Although this learning rule is effective for the function approximation problem, unsupervised learning is necessary to perform self-organization, which is a characteristic of neural networks. In addition, since such a neuron element is one of the major features of its simple structure, the learning rule must be easy and small-scale hardware implementation is possible.

Therefore, the present invention is based on Hebb's law of strengthening the coupling between a certain neuron element and another neuron element when they are in an excited state at the same time. The purpose is to enable.

[0006]

According to a first aspect of the present invention, a pulse density type neural network is provided which is connected in a mesh and outputs one output signal in pulse train representation for a plurality of input signals in pulse train representation. In the neuron element forming the, the learning means for determining the pulse value of the new coupling coefficient based on the current coupling coefficient pulse, the input signal pulse, the excitatory output pulse, and the inhibitory output pulse is provided.

In addition, according to the second aspect of the invention, the rule designating means for designating the learning rule of the learning means is provided.

In this case, the invention according to claim 3 is provided with a learning rule changing means for periodically changing the learning rule of the rule specifying means, and in the invention according to claim 4, the learning rule of the rule specifying means is random. The learning rule changing means to change to is provided.

Further, in these inventions, the invention according to claim 5 is provided with a control means for deciding whether or not to permit the transition to the next process according to the update state of the coupling coefficient. A control means for deciding whether or not to shift to the next process is provided according to the number of times of data input.

[0010]

According to the first aspect of the invention, the pulse value of the new coupling coefficient is determined based on the current pulse of the coupling coefficient, the pulse of the input signal, the pulse of the excitatory output, and the pulse of the inhibitory output. Self-organization, which is one of the major features of neural networks, can be performed, and a neuron element with an unsupervised learning function can be realized with a simple circuit scale.

According to the second aspect of the invention, since the learning rule of the learning means can be designated by the rule designating means, the learning rule can be easily changed.

According to the third or fourth aspect of the present invention, since the learning rule is changed in this manner periodically or randomly, it is possible to prevent the learning from converging on the local minimum.

Further, in the invention according to claim 5 or 6, the transition to the next process is determined according to the update state of the coupling coefficient or the number of times of data input, so that the end time of the learning process becomes clear, The transition to the process of is smoothly performed.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described with reference to FIGS. First, FIG. 2 shows a configuration example of the pulse density type neuron element 1 which is the premise of the present embodiment. There is a plurality of inputs to a certain neuron element 1, and there are excitatory inputs and inhibitory inputs. When an input signal in the form of a pulse train in the form of pulse density is input from the excitatory input port 2, the AND coefficient is ANDed by the AND coefficient from the combination coefficient in the form of pulse train in the form of pulse density from the coupling coefficient pulse generator 3. The coupling coefficient pulse generator 3 may be a shift register that stores a pulse train, or may be a combination of a memory that stores a binary train and a pulse generator that generates a pulse train from binary data. There are a plurality of excitatory input circuits 5 including the excitatory input port 2, the coupling coefficient pulse generator 3 and the AND gate 4, and the signals related to the excitatory coupling group from these excitatory input circuits 5 are OR gates. A logical product is taken by 6 and an excitatory output is produced.

Similarly, for the suppressive group, there are a plurality of suppressive input circuits 10 each having the suppressive input port 7, the coupling coefficient pulse generator 8 and the AND gate 9, and the suppressive input circuits 10 suppress the same. The signals for the sex-coupled groups are logically AND'ed by the OR gate 11 to give an inhibitory output.

Then, a logical product of the excitatory output from the OR gate 6 and the negated inhibitory output from the OR gate 11 is taken by the operation gate 12 to output a final output signal of the neuron element 1.

The learning rules used in this embodiment for such a presupposed structure will be described. The coupling coefficient is updated in units of one pulse. That is, it is determined whether to store the pulse of the coupling coefficient used for the pulse of the input signal currently input at the H level, the L level, or the current level. However,
As a storage method, the pulse train may be stored as it is, or may be counted and stored in a binary value state.

First, basically, the coupling coefficient for the input may be updated only when the input pulse to the neuron element 1 is at the H level, and is not updated when it is at the L level. When the coupling is excitatory, the coupling coefficient is at the H level when the excitatory output is at the H level and the inhibitory output is at the L level, and conversely, when the excitatory output is at the L level and the inhibitory output is at the H level. Is set to the L level, and the coupling coefficient is not updated in other cases. Similarly, when the coupling is inhibitory, the excitatory output is at the H level and the inhibitory output is at the L level.
When the level is the level, the coupling coefficient is at the L level, and conversely, when the excitatory output is at the L level and the inhibitory output is at the H level, the coupling coefficient is at the H level, and in other cases, the coupling coefficient is not updated. Table 1 summarizes such learning rules, and shows that the "x" part is not updated (though,
It is also possible to update some of the portions indicated by "X").

[0019]

[Table 1]

FIG. 1 shows the configuration of a neuron element 1 to which learning means based on such learning rules is added. First, FIG. 1A shows the configuration of one excitatory input circuit 5 in the neuron element 1, and three AND gates each having a pulse of the current input signal inputted from the excitatory input port 2 as one input 13, 14, and 15 are provided, and the coupling coefficient pulse from the coupling coefficient pulse generator 3 is input to the other inputs of the AND gates 13 and 15. AND gate 1
For 4, the coupling coefficient pulse is inverted. Also, the output pulses from these AND gates 13 and 14 are used as one input, and three-input AND gates 16, 17 and 18 that are supplied with excitatory output and inhibitory output are also input.
Is provided. The suppressive output is inverted and input to the AND gate 18. Furthermore, an OR gate 19 which receives the output pulses from these AND gates 15, 16, 17, and 18 is provided so that the pulse value of the new coupling coefficient is determined and output to the coupling coefficient pulse generator 3. It is configured. These gates 13-1
9 constitutes a learning means 20 for determining a new pulse value of the coupling coefficient based on the current pulse of the coupling coefficient, the pulse of the input signal, the pulse of the excitatory output, and the pulse of the inhibitory output.

FIG. 2B shows the structure of one inhibitory input circuit 10 in the neuron element 1, and basically, FIG.
The learning means 21 similar to the learning means 20 is added. The difference from the learning means 20 is that the inputs of excitatory output and inhibitory output are reversed (AND
The excitatory output is inverted to the gate 18).

According to the configuration of the neuron element 1 constructed by combining the excitatory input circuit 5 and the inhibitory input circuit 10 as described above, it is possible to perform self-organization, which is a major feature of the neural network. Therefore, a pulse density type neuron element with an unsupervised learning function can be realized with a simple circuit scale.

Next, a second embodiment of the present invention will be described with reference to FIG. The same parts as those shown in the above-mentioned embodiment are designated by the same reference numerals. The present embodiment is not limited to the example shown in Table 1, and is configured in consideration of the fact that it may be set arbitrarily. For example, one excitatory input circuit 5 shown in FIG. A configuration example of is shown. First, the pulse of the current input signal from the excitatory input port 2 and the coupling coefficient pulse generator 3
A learning circuit (learning means) 22 is provided which inputs the current pulse of the coupling coefficient, the excitatory output pulse and the inhibitory output pulse from the above to output a pulse of the new coupling coefficient to the coupling coefficient pulse generator 3. ing. This learning circuit 2
A control circuit (rule designating means / learning rule changing means) 23 for designating the learning rule is connected to 2. The control circuit 23 is for specifying what should be output according to the input pulse to the learning circuit 22. More specifically, since the data (pulse) input to the learning circuit 22 is 4 bits, 16 signal lines are connected from the control circuit 23 to the learning circuit 22. Specify what kind of pulse should be output for the input pattern. The 16 signals output from the control circuit 23 to the learning circuit 22 may use preset values, or periodically using a counter or the like, or randomly (randomly using a random number circuit or the like. ) May be changed. Even if the learning converges to the local minimum, if it is changed periodically or randomly, it is possible to escape from it. That is, the convergence of learning can be prevented.

A control circuit 23 may be similarly added to the suppressive input circuit 10 side shown in FIG. 1 (b).

When the learning of the neural network is performed by the means described in these embodiments, the learning may be repeated with the same input data until the steady state is reached, or the next learning is performed after a certain time elapses. You may make it shift to a process. In the former case, the control means for counting the number of updates of the coupling coefficient when a pulse train having one data length is input, and determining whether or not the transition is permitted according to the update state that the transition to the next process is performed if the number is less than a certain threshold value. Should be provided. In the latter case, the number of times the pulse train with one data length is input is counted,
A control means may be provided to determine whether or not to permit the transition in accordance with the number of data input repetitions, that is, the process is moved to the next process when the value reaches a certain value.

[0026]

According to the first aspect of the present invention, the pulse of a new coupling coefficient is generated by the learning means based on the pulse of the current coupling coefficient, the pulse of the input signal, the pulse of the excitatory output, and the pulse of the inhibitory output. Since the value is determined, it is possible to perform self-organization, which is one of the major features of the neural network, and it is possible to provide a neuron element with an unsupervised learning function with a simple circuit scale.

According to the second aspect of the invention, since the learning rule of the learning means is designated by the rule designating means, the learning rule can be easily changed.

According to the third or fourth aspect of the present invention, the learning rule changing means changes the learning rule periodically or randomly, so that the convergence of learning to the local minimum can be prevented. .

Further, according to the invention of claim 5 or 6, the control means determines the transition to the next process in accordance with the update state of the coupling coefficient or the number of times of data input, so that the learning process is performed. The end time is clarified, and the transition to the next process can be smoothly performed.

[Brief description of drawings]

FIG. 1 shows a first embodiment of the present invention, in which (a) is a circuit diagram of an excitatory input circuit and (b) is a circuit diagram of an inhibitory input circuit.

FIG. 2 is a circuit diagram showing a basic configuration of a neuron element.

FIG. 3 is a circuit diagram showing a second embodiment of the present invention.

[Explanation of symbols]

 20-22 Learning means 23 Rule designating means and learning rule changing means

Claims (6)

[Claims] 1. A neuron element that is connected in a mesh and outputs one output signal of a pulse train representation to a plurality of input signals of a pulse train representation to form a pulse density type neural network. A neuron element characterized by comprising learning means for determining a pulse value of a new coupling coefficient based on a pulse, an input signal pulse, an excitatory output pulse and an inhibitory output pulse. 2. The neuron element according to claim 1, further comprising rule specifying means for specifying a learning rule of the learning means. 3. The neuron element according to claim 2, further comprising learning rule changing means for periodically changing the learning rule of the rule specifying means. 4. The neuron element according to claim 2, further comprising learning rule changing means for randomly changing the learning rule of the rule specifying means. 5. The neuron element according to claim 1, 2, 3 or 4, further comprising control means for deciding whether or not to permit a transition to a next process according to an updated state of the coupling coefficient. 6. The neuron element according to claim 1, further comprising control means for determining whether or not to permit a transition to the next process in accordance with the number of times of data input.