JPH0934858A - Artificial neuron - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase an expression ability of a neural network, flexibility, versatility and a memory capacity by controlling a coupling strength change of a synapse on the opportunity of an input of the artificial neuron receiving the input. SOLUTION: The neuron has a structure of grouped synapses. The artificial neuron is made up of an axon 102 extended from one or plural input artificial cell bodies 101, a presynapse part 103 in existence at the end of each axon, a postsynapse part 104 corresponding to each presynapse part 103, a dendrite 105 connecting to the postsynapse part 104, a postsynapse cell body 106 to which the dendrite 105 reaches, and an axon 107 extended from the postsynapse cell body 106. Furthermore, a couple of presynapse part 103 and a postsynapse part 104 form a unit synapse 108. According to a characteristic of an input to the artificial neuron, a group synapse (k) to which the unit synapse belongs is decided automonously and according to a characteristic of input to the artificial neuron, the relation between the group synapses (k) with respect to the synapse binding strength change is automonously decided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a simulation of an information processing function in a nervous system of an organism, and more particularly to an information processing method simulating an information processing function such as learning or recognition realized by self-organization.

[0002]

2. Description of the Related Art The properties of artificial neurons used in conventional neural network models are simulated by simulating the properties of actual nerve cells based on knowledge of physiological research and the like. The structure of this simplified conventional artificial neuron is as simple as shown in FIG. FIG.
Is an axon 102 extending from one or more input artificial neurons 101, a synapse 501 existing at each axon terminal, an artificial neuron 502, and an axon 10 extending from the artificial neuron 502.
Composed of 7.

The artificial neuron is a multi-input, single-output device. The input / output relationship of the artificial neuron that receives m inputs is given by the following (Equation 1).

[0004]

[Equation 1]

Here, u _i is an input from the i-th artificial neuron i, v is an output of the artificial neuron, and w _i is a synaptic connection degree applied to an input from the i-th artificial neuron i. For input,

[0006]

[Number 2] w _i (t) ≧ 0 and (number 2) and the excitatory input that is represented,

[0007]

There is an inhibitory input such that w _i (t) <0 (Equation 3). The artificial neuron receives inputs from m artificial neurons i, and its synaptic connection degree w _i
A non-linear characteristic f is applied to the weighted addition result by to output v. Where f is

[0008]

[Equation 4] 0 ≤ f (x) ≤ 1 (Equation 4)
It is a monotonically increasing function that satisfies. Also, in general the amount of change δw _i of synaptic coupling degree w _i is,

[0009]

Δw _i (t) = g (t, v (t), u _i (t), w
_i (t)) (Equation 5) (Equation 5) is a means for updating the degree of synaptic connection called a learning hypothesis. The synaptic connectivity is calculated using the amount of change in synaptic connectivity,

[0010]

W _i (t + 1) = w _i (t) + c ₀ δw _i (t)
It is updated by (Equation 6). Here, c ₀ is a constant.

The Hebb rule (associative action hypothesis) is one of the methods for changing the degree of synaptic connection. This is a method of increasing the degree of synaptic connection when two nerve cells adjacent to each other across the synapse have high activity, which is a self-generated learning method different from the so-called "supervised learning method". Is the way. Expressing the Hebb law by an equation,

[0012]

(7) δw _i (t) = c ₁ + c ₂ (v (t) −c ₃ ) (u
_i (t) -c ₄₎ (7) _{where, c 1, c 2, c} 3, c 4 are constants.

As described above, the conventional neural network composed of simple artificial neurons has its own expressive power and flexibility, and therefore 1) the applications to which the neural network is applied are limited. 2) There is a problem that the memory capacity of the neural network is limited, and when increasing the memory capacity, the only method is to substantially increase the number of artificial neurons.

[0014]

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and its object is to increase the expressive power and flexibility of a neural network by incorporating the properties of actual nerve cells into artificial neurons. The present invention is to provide a method of increasing diversity in practical application of the neural network, a method of increasing memory capacity of the neural network, and the like.

[0015]

The feature of the artificial neuron of the present invention is that it has the property of learning from the properties of actual nerve cells and simulating the structure and properties of the actual dendrites of nerve cells. Therefore, the expressive power and flexibility of the neural network become rich, and the diversity and memory capacity can be increased.

As shown in FIG. 6, the actual nerve cell has a much more complicated structure than the artificial neuron shown in FIG.
Actual nerve cells are roughly divided into dendrites 601, cell bodies 60
2. It consists of three parts, axon 603. A large number of signals from other nerve cells mainly input to the dendrite 601 via the synapse 604 (the synapse 604 is an example). The signal propagates like a cable on the dendrites to the cell body. The cell body integrates multiple inputs and outputs according to the "all or none" law. The axon transmits this output signal to the synaptic part of the terminal without attenuation.

It was thought that the signal processing in nerve cells was basically as described above, but it has become clear that there is another mechanism in recent years. This is called a dendrite spike. The dendrite portion is a portion that originally receives an input and passively transmits a signal, but under certain conditions, a spike (action potential) propagates to this portion. This dendrite spike mainly occurs in the cell body
It propagates retrogradely toward the terminal dendrites according to the law of "e" (Nature, 367, 69-72, 1994).

On the contrary, when there is a synaptic input accompanied by a strong depolarization in this dendrite part, a dendrite spike is generated in the synapse part and this spike may propagate toward the cell body. Yes (Science, 268, 297-300,
1995, J. Physiol., 305, 171-195, 1980). In this case, the dendrite spike simultaneously propagates in the peripheral direction starting from the synapse.

From the above findings, when a synapse is plastically strengthened in a nerve cell in which a dendrite spike is generated in the cell body and propagates toward the terminal end of the dendrite, thereafter, a synapse is input to the synapse to cause the degeneration It is conceivable that a dendrite spike originating from the synapse part may be generated and propagated toward the cell body, and the propagation direction may be reversed. That is, it is considered that a kind of switching mechanism due to plastic strengthening of synapses is inherent in nerve cells.

It is conceivable that the propagation range of the dendrite spikes may change due to the reversal of the propagation direction of the dendrite spikes. Generally, when a spike propagates from one thin branch to a thick trunk at a branch point where the thick trunk divides into two thin branches, it is difficult for the spike to enter in the direction of the other thin branch because of high impedance. . for that reason,
When the dendrite spike occurs in the cell body, even in the branch where the spike propagates, when the dendrite spike occurs in the synapse part, the spike may not propagate at the branch part and the spike may not propagate. That is, with the "branch" of the dendrite as a unit, a portion where the dendrite spike propagates and a portion where the dendrite spike does not propagate occur. When expressing the properties of this dendrite spike from the side of the synapse existing on the dendrite, the synapse forms a group with the “branch” of this dendrite as a unit, and the propagation direction of the dendrite spike Reversal results in synaptic groups in which dendrite spikes propagate (hereinafter group synapses) and group synapses that no longer propagate.

The information processing function of this dendrite spike is considered as follows. The dendrite spike causes a strong transient depolarization in the dendrite part. Therefore,
If there is a synaptic input synchronized with the occurrence of this dendrite spike, even a relatively weak input causes plastic synapsis of the synapse according to the above-mentioned Hebb rule (associative action hypothesis). Here, the plastically reinforced synapse may be an excitatory synapse or an inhibitory synapse. Therefore, when the direction of propagation of dendrite spikes is reversed, there is a change in information processing function. First, in group synapses where this dendrite spike does not propagate, transient depolarization due to dendrite spikes is no longer present. It is thought that the synaptic plastic strengthening triggered by is not generated. However, this does not rule out synaptic plastic strengthening due to other causes. Synaptic enhancement by Hebb's rule triggered by an individual strong input such as so-called tetanus stimulus (high frequency stimulus) may occur independently, which causes a synaptic plastic change.

As a change in the information processing function due to the reversal of the propagation direction of the dendrite spike, secondly, since the propagation path of the dendrite spike changes, the timing of input to the synapse synchronized with the dendrite spike. Is likely to change. This can be regarded as one means of spatiotemporal dynamic control of information processing in nerve cells. Also,
It can also be considered as a means for establishing a hierarchical relationship between the group synapses. Here, as the effect of the dendrite spike on the synapse, in addition to the direct depolarization associated with the passage of the dendrite spike, the direct depolarization causes a potential dependence on the dendrite membrane. It is considered that there is an indirect effect of mobilizing the second messenger system triggered by the increase of the intracellular divalent Ca ion concentration by opening the divalent Ca ion channel.

Learning from the properties of actual nerve cells as described above,
The features of the artificial neuron of the present invention simulating that property are:
First, synapses have a grouped structure (see FIG. 1). Secondly, the feature of the artificial neuron of the present invention is that, secondly, according to the characteristic of the input to the artificial neuron, the group synapses to which the synapse belongs and the relationship between the group synapses regarding the change in synaptic connection strength are autonomously determined. That is the point. Thirdly, the feature of the artificial neuron of the present invention is that the group synapse has a branching structure with the group synapse as a unit as a higher-order structure (see FIG. 2). The fourth feature of the artificial neuron of the present invention is that the grouped synapse group forms a branched structure, but the input from the input artificial neuron may span multiple group synapses, hierarchies or trunks. (See FIG. 3).

Due to the above features, the expression and flexibility of the neural network having the artificial neuron of the present invention as a constituent element become rich, and the versatility and memory capacity can be increased.

[0025]

The present invention provides an artificial neuron having a property of simulating the structure and properties of an actual nerve cell dendrite, and its practical use. By using the artificial neuron of the present invention, it becomes possible to increase the expressive power and flexibility of the neural network constituted by the artificial neuron, and increase the diversity and the memory capacity.

[0026]

【Example】

First Embodiment Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing the structure of an artificial neuron according to the first embodiment of the present invention.

The first structural feature of the artificial neuron of the present invention has a structure in which synapses are grouped as shown in FIG. The artificial neuron of FIG. 1 has an axon 102 extending from one or more input artificial cell bodies 101, a presynaptic portion 103 existing at each axon terminal, and a postsynaptic portion 104 corresponding to each presynaptic portion 103, It consists of dendrites 105 connected to the postsynaptic portion 104, postsynaptic cell bodies 106 reached by the dendrites 105, and axons 107 extending from the postsynaptic cell bodies 106.

A pair of presynaptic part 103 and post-synaptic part 104
Constitutes a unit synapse 108. Further, one or more unit synapses 108 form group synapses 109, 110, 111. The number of synapses forming the group synapse may be one or plural. Moreover, the number of the group synapses may be one or more. In the case of FIG. 1, a (109), b (11
There are three group synapses, 0) and c (111).

The characteristic of the behavior of the group synapse is that the unit synapse 108 within the group synapse is related to the relationship between the group synapses.
Are equivalent to each other, in other words, all unit synapses in the group synapse share a behavior. For example, when the group synapse a and the group synapse b do not behave at a certain time t ₀ , all the unit synapses 10 of the group synapse a are
8 does not behave with all unit synapses 108 of group synapse b. However, if the group synapse a and the group synapse b behave together at another time t ₁ , all the unit synapses 108 of the group synapse a behave together with all the unit synapses 108 of the group synapse b.

The relationship between the group synapses is as follows. When one or more unit synapses 108 within a group synapse are plastically strengthened, two classes of plastic strengthening of unit synapses within each group synapse occur.
One is a group synapse group in which plastic enhancement is triggered by dendrite spikes, and the other is a group synapse group in which plastic enhancement is no longer triggered by dendrite spikes. This can be described by the following (Equation 8).

[0032]

(Equation 8)

Here, p is a variable determined by the relationship between group synapses k and the history of synapses j in the group synapses k, and in the initial state,

[0034]

## EQU00009 ## p (k, j, t) = 0
(Equation 9) After one or more unit synapses j in the group synapse α are plastically enhanced,

[0035]

[Mathematical formula-see original document] p (α, j, t) = 1
(Equation 10)

[0036]

## EQU00009 ## p (k, j, t) = 1 or 0
(Equation 11) can be described. Here, whether (Equation 11) takes a value of 1 or 0 is determined according to the relationship between the group synapses defined at the beginning.

The different features of the artificial neuron in the embodiment of FIG. 1 are that the group synapse k to which the unit synapse belongs is autonomously determined according to the characteristics of the input to the artificial neuron, and the input to the artificial neuron is further determined. According to the characteristics, the relationship between the group synapses k regarding the change in synaptic connection strength is autonomously determined. In other words,
It is the spatiotemporal pattern of inputs to the artificial neuron that determines to which group synapse k a unit synapse belongs and how to set the relationship between the group synapses k. When the spatio-temporal pattern of is dynamically changed, the attributes of the unit synapses and the relationships between group synapses change accordingly. Due to this feature, the flexibility and the expressiveness of the neural network composed of the artificial neurons of the present invention are dramatically enhanced.

Embodiment 2 FIG. 2 is a diagram showing the structure of an artificial neuron according to another embodiment of the present invention.

The second structural feature of the artificial neuron of the present invention is that, as shown in FIG. 2, a group synapse has a branching structure with a group synapse as a unit as a higher-order structure. Here, the number of unit synapses forming the group synapse may be one or more. Moreover, the number of the group synapses may be one or more.

This branched structure has a hierarchy based on the postsynaptic cell body 106 described in FIG. Each group synapse may be on any floor. In the case of FIG. 2, d
There are four group synapses (203), e (204), f (205), and g (206), and regarding their branching structures, d, e, and f are one trunk 201 and g is another trunk 202. include. In addition, the trunk 201
Within the group synapses d and e are in the same layer (third layer), and the group synapse f is in a layer (two layers) one step below the group synapses d and e. Furthermore, the group synapse g is located in the lowest floor (1st floor).

The relationship between the group synapses is as follows. When one or more unit synapses 210 in a group synapse α are plastically strengthened, a unit synapse in the group synapse that belongs to the same trunk as the group synapse α and is at the same level as or higher than the group synapse α Then, no longer the plastic enhancement triggered by the dendrite spike. In addition, the relationship between the two trunks is such that when synapse 210 is plastically strengthened within one or more group synapses in a trunk A, no further plastic strengthening is triggered by dendrite spikes. Stem B occurs.

This is

[0043]

(Equation 12)

Can be described as Here, q is a variable determined by the relationship between the group synapses k belonging to the n-th layer of the stem h and the history of the unit synapses j in the group synapses k, and in the initial state,

[0045]

[Mathematical formula-see original document] q (h, n, k, j, t) = 0
(Equation 13) After one or more unit synapses j in the group synapse α belonging to the X layer of the trunk A is plastically enhanced,

[0046]

## EQU00004 ## q (A, X, .alpha., J, t) = 1
(Equation 14)

[0047]

(15) q (A, n, k, j, t) = 0
(N ≧ X) (Equation 15)

[0048]

## EQU16 ## q (A, n, k, j, t) = 1
(N <X) (Equation 16)

[0049]

## EQU00007 ## q (h, n, k, j, t) = 1 or 0
(H ≠ A) (Equation 17) can be described. Here, whether (Equation 17) takes a value of 1 or 0 is determined according to the relationship between the trunk h, the layer n, and the group synapse k defined at the beginning.

Further, a different feature of the artificial neuron of the present invention as shown in FIG. 2 is that the group synapse k to which the synapse belongs is autonomously determined according to the characteristics of the input to the artificial neuron, and further the input to the artificial neuron. The relationship between the trunk h, the layer n, and the group synapse k with respect to the change in synaptic bond strength is autonomously determined according to the characteristics of 1. In other words, which group synapse k a certain unit synapse belongs to and how to set the relationship between the stem h, the layer n, and the group synapse k is determined at the time of input to the artificial neuron. It is a spatial pattern, and when the spatiotemporal pattern of the input changes dynamically, the attributes of the synapses and the relationships between the trunks, layers, and group synapses also change. Due to this feature, the flexibility and the expressiveness of the neural network composed of the artificial neurons of the present invention are dramatically enhanced.

Embodiment 3 FIG. 3 is a diagram showing the structure of an artificial neuron according to another embodiment of the present invention.

The third structural feature of the artificial neuron of the present invention is that, as shown in FIG. 3, the grouped synapse group has a branched structure, but the input from the input artificial neuron is a group of a plurality of groups. It may span synapses, hierarchies or trunks. In FIG. 3, one having a plurality of input artificial neurons 301 is a unit synapse 309 of the group synapse 303, and one having another input artificial neurons 301 is a unit synapse 310, 314 of the group synapses 303 and 304. , Or another input artificial neuron 301 exists in each of the group synapses 304 and the unit synapses 315 and 316 of the group synapse 305 having different layers. Things are in different layers Group 3 synapse
05 and group synapses 306 are connected to unit synapses 317 and 318, respectively.

Embodiment 4 FIG. 4 is an example of a neural network composed of artificial neurons of the present invention. In FIG. 4, the entire circuit is filled with the artificial neurons of the present invention. However, in an actual neural network model,
The entire circuit need not be populated with such artificial neurons. There may be coexistence with artificial neurons having other characteristics.

[0054]

By using the artificial neuron of the present invention, it becomes possible to increase the expressive power and flexibility of the neural network composed of the artificial neuron, and increase the variety and the memory capacity.

A practical advantage of the artificial neuron of the present invention is, for example, that the grouped synapses have a hierarchical structure in advance, so that classified memory or learning can be performed quickly. This is because only synapses on a particular path are likely to be strengthened while other paths on the same trunk are not strengthened and ignored. For example, there are dogs, cats, monkeys, etc. as animals of the animal kingdom Mammalia, but when learning the knowledge having such a tree-like system using the neural network of the present invention, learning progresses, The system of dogs → mammals → animal kingdom gradually becomes 3 on artificial neurons.
It is fixed in the form of weighting of the synapse of layer → 2 layer → 1 layer. On the contrary, knowledge learning using the neural network of the present invention is effective in automatically layering and organizing miscellaneous knowledge.

Another practical advantage of the artificial neuron of the present invention is that, for example, among grouped synapses, a synapse group (hold) that holds the synapse connection degree as a memory and a group synapse that learns and modifies synapses. The posterior cells are prepared separately. In other words, it is a new concept that the post-synaptic cell positively controls how to receive an input to an artificial neuron.

Another practical advantage of the artificial neuron of the present invention is that it is also a method of resetting the strength of synaptic transmission over a wide range depending on the timing of passage of dendrite spikes, for example.

[Brief description of drawings]

FIG. 1 is a schematic diagram showing an embodiment of the present invention of an artificial neuron having a structure in which synapses are grouped.

FIG. 2 is a schematic diagram showing an embodiment of the present invention of an artificial neuron having a branching structure with a group synapse as a unit.

FIG. 3 is a schematic diagram showing an embodiment of the present invention in the case where an input from an input artificial neuron spans a plurality of group synapses, hierarchies or trunks in an artificial neuron having a branch structure with a group synapse as a unit.

FIG. 4 is a schematic diagram showing an embodiment of the present invention of a neural network composed of artificial neurons.

FIG. 5 is a schematic diagram showing an example of a structure of a conventional artificial neuron.

FIG. 6 is a schematic diagram showing an example of an actual nerve cell.

[Explanation of symbols]

101: one or more input artificial neurons, 102: an axon extending from 101, 103: a presynaptic portion existing at each axon terminal, 104: a postsynaptic portion corresponding to each presynaptic 103, 105: Dendrites, 10 connected to the postsynaptic part 104
6: Post-synaptic cell body reached by dendrite 105, 107: axon extending from post-synaptic cell body 106, 108: unit synapse composed of a pair of presynaptic part 103 and postsynaptic part 104, 109, 110, 111: One or more unit synapses 108
Group synapses composed of, 201, 202, 203: trunk, 203, 2
04, 205, 206: group synapse, 210: unit synapse, 301: input artificial neuron, 303, 304, 305, 306: group synapse, 3
09, 310, 314, 315, 316, 317, 318: Unit synapse, 40
1: Artificial neuron having group synapse, 501: Synapse existing at axon terminal, 502: Artificial neuron, 601: Actual nerve cell dendrite, 602: Actual nerve cell cell body, 603:
Axons of real nerve cells, 604: Synapses of real nerve cells.

Claims (6)

[Claims] 1. An artificial neuron, which is an element constituting a neural network, has one or a plurality of unit synapses, receives an input at the unit synapse, performs an integrated operation on the input, and outputs an output. An artificial neuron characterized in that a neuron controls a change in synaptic connection strength triggered by the input. 2. The unit synapse constitutes a group synapse that is a group of synapses that behave in association with a change in synaptic bond strength, and a relationship between group synapses regarding a change in bond strength of the unit synapse is defined in advance. Item 1
The described artificial neuron. 3. The group synapse, which is a group of synapses in which the unit synapses behave in association with changes in synaptic connection strength, constitutes a group synapse, and the group synapse to which the unit synapse belongs autonomously according to the characteristics of the input to the artificial neuron. Determined, and according to the characteristics of the inputs to the artificial neuron,
The artificial neuron according to claim 1, wherein the relationship between group synapses regarding the change in synaptic connection strength is autonomously determined. 4. The group synapses are group synapses that are groups of synapses that behave in association with changes in synaptic bond strength, the group synapses form a branched hierarchical structure, and the unit synapses are connected. The artificial neuron according to claim 1, wherein relationships among group synapses, hierarchies, and trunks regarding intensity changes are defined in advance. 5. The unit synapse constitutes a group synapse, which is a group of synapses that behave in association with a change in synaptic connection strength, and the group synapse forms a branched hierarchical structure, and the input of the input to the artificial neuron. A relationship between group synapses regarding a change in synaptic bond strength is autonomously determined according to a characteristic, and a relationship among group synapses, between hierarchies and trunks regarding a change in bond strength of a unit synapse is autonomously determined. The artificial neuron according to 1. 6. The artificial neuron according to claim 1, wherein the input is coupled to a plurality of unit synapses belonging to a plurality of group synapses.