JPH0410181A - Neural cell model - Google Patents

Abstract

PURPOSE:To execute a processing depending on input signals in the past as well as well by realizing a neural cell model adding a self-retroactive movement average type filtering function to a synapse coupling part. CONSTITUTION:The neural cell model is equipped with a synapse coupling part 1 applied the self-retroactive movement average type filtering function, cell body part 2 and axon hillock 3. In this neutral cell model, an input signal Oi(t) is not used as it is but a signal temporarily filtered to a form, which is suitable for the processing requested to the neural cell, by the synapse coupling part 1a is used so as to decide an internal cell state Xj(t) and a cell output value Oj(t). In this case, a filter coefficient can be effectively learnt by using an error propagation method and a probably maximum grade method. A coupling coefficient Wi,j(Z) showing the weight of coupling between cells is the transmission function, which is to be applied to the synapse coupling part 1a, of a digital filter for realizing the self-retroactive movement average type filtering function.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention relates to a neuron model, and in particular, it applies advanced information processing capabilities such as time-series information processing of the brain to an artificial neural network (neurocomputer). This paper concerns the neural cell model as an engineering model of neurons, which is the basic element for realization.

[Conventional technology]

Neuron models, which are engineered models of neurons that artificially embody the brain's advanced information processing capabilities, have recently become well known.

As shown in Fig. 5, the conventional neuron model uses coupling coefficients W, j (j-1 to nk) for input signals 0i(t) (j-1 to nk) from other neuron groups. a synapse coupling unit 1 that receives a weighted signal WItj·01(tl), and a cell body that adds the weighted signals to generate the cell k internal state Xj(t)−ΣVv,,j−0,(t). 2 and an axon hillock 3 that converts the cell internal state X J (t) into a neuron output value 0j(t).

As mentioned above, the conventional neuron model uses the weighted linear "k sum ΣW1.j -0□(t) of input signals 0i(t) from other neurons as its internal cell state XJ
(t), and the output conversion function to obtain the neuron output value is the sigmoid function f (xj(t))-1/
(1+exp(x,(t)) is often used.

In addition, by adding a function to the conventional neuron model in which the cell internal state X A feedback neuron model shown in FIG. 6, which is generated by the following equation taking into account past inputs, has also been proposed.

k Xj(t)-, r, Wi, j-0I(t)+aj
-Xj (t-1) where ☆ is the above-mentioned coupling coefficient, and ar is a feedback coupling coefficient indicating the weight of the feedback coupling when feeding back from the cytoplasmic body part 2 to the synaptic coupling part 1.

, [Problem to be solved by the invention] The output value Oj (11) in the first example of the conventional neuron model described above is determined only by the input signal 0i(t) from all other neurons at the current time t. Therefore, the input signal 07 (
The temporal change of t) is not used at all in the neuronal model. In other words, what input signal O in the past
i (Even if tJ exists, it has no effect on the output of the cell. On the other hand, the human brain receives not only instantaneous signals but also their temporal changes, as can be seen from the examples of vision and hearing. Therefore, the conventional neuron model, as a neuron model of the brain, does not reflect very important information such as temporal changes in input signals in its output 0j (tl). There is a drawback.

In addition, in the second example of the conventional neuron model, synaptic connections that are actually indirectly connected via chemical transmitters such as acetylcholine are represented by a scalar coupling coefficient W,
Since it is expressed as j, it has the disadvantage that it is a model with a reduced signal processing function of nerve cells.

Furthermore, although the feedback neuron model has the ability to handle temporal changes in the input signal, since the feedback section has a first-order filtering function, the signal transfer function of the cell can only have a real plate, so it has free transfer characteristics. The disadvantage is that it cannot be expressed.

[Failure to solve the problem]

A first aspect of the neuron model of the present invention is a synaptic connection unit that weights input signals from other neuron groups and outputs a weighted signal, and a cell that adds the weighted signals to generate a cell internal state. In a neuron model having a body part and an axon hillock that converts the internal state of the cell into a cell output value, the synaptic connection part receives input signals from other neurons and processes them as required by the neuron. It is configured with an autoregressive moving average type filtering function that filters in an appropriate form.

Further, a second invention of the neuron model of the present invention includes a synaptic connection unit that weights input signals from other neuron groups and outputs a weighted signal, and adds the weighted signals to generate a cell internal state. In a neuron model having a cell body and an axon hillock that converts the cell internal state into a cell output value, a moving average that feeds back the past self cell internal state from the cell body to the synaptic connection part FIG. 1 is a block diagram of an embodiment of the first invention of the neuron model of the present invention, in which a synaptic coupling unit 1 is provided with an autoregressive moving average type filtering function. , a cell body 2a, and an axon hillock 3.

In this neuron model, the input signal 0i(t) is not used as it is, but the cell internal state , cell output value Oj it
), and the filter coefficients in this case can be effectively learned using the error propagation method and the stochastic steepest gradient method.

In FIG. 1, the coupling coefficient Wi indicating the weight of intercellular coupling
.

Here, 2- is a delay operator and Z
-X(t)-X(t-1) Furthermore, the synaptic coupling unit 1a transmits a weighted signal Wi,j(Z)・0i(tl The cell internal state Xj(t) which is output by adding up is expressed by the following equation.

X・(1)-n, Lii, voice) 01 here. j (t) is LL - (t) - Wi, j (Z) - 0i (t) 1, J - Σ Renfou + J, /"1.j (t /) +, Mi." + Jw
Z・0. (t-/) 1 layer! Here, (t-/)(/-1 to n) indicates the past state.

Neuron output value 0j(t)F output from axon hillock 3
It is expressed by a linear equation.

FIG. 2 is a configuration diagram of a hierarchical neural network constructed using the neuron model of FIG. 1.

In FIG. 2, symbols 11 to 15Fi respectively constitute an input layer (first layer) 11, a second layer 12, a second layer 13, . . . (k+1)th layer 14, -・・・
This is an output layer (M-th layer) 15. Further, symbol 101 indicates cells included in each layer, and Xo(su~XnM(
t) indicates the internal state of each cell. Furthermore, 0□(1
)~0 nM (1) is the neuron output value output from each layer.

In addition, +' in θ shown in Fig. 2 is the threshold value given to the (k+1)th layer, the second layer, the third layer, etc.
. . . It is set with the aim of stably forming neurons to be hierarchically configured together with the cell internal state -1 included in the (k+1)th layer.

Next, the operation of the embodiment shown in FIG. 2 will be explained.

In this embodiment, the input signal 1(t) is a scalar, but there is no particular problem in using it as a multidimensional vector. The neural network of this embodiment is composed of M layers, and each layer has n
Contains k (k-1 to M) cells. Connections between neurons occur only between adjacent layers, and there are no connections within layers.

Signals from other neurons that have gathered at a certain neuron are filtered at the synaptic junction 1a using the past values of the signals so that the signals are suitable for realizing the function required of that neuron. It is then input to that neuron. This filtering is applied to the layer i cell and the (K+
1) The weight of the connection between layer j cells is w7′! +1 (4 synaptic connections, J ensured by passing through the autoregressive moving average filter in section 1a. Input signal 1(t) input to input layer 1
flows toward the output layer 15 while being subjected to such filtering processing in each cell forming the hierarchy, and obtains a neuron output value 0j(t) (j-1 to nm). In this way, the output value 0j(tt) at the current time becomes the past input signal 1(t
), and the signal processing ability of the model has also been improved.

In addition, in this hierarchical neural network, the coupling coefficients Wi, j (ZJ) and the filter coefficient "Jm" with other cells are efficiently calculated using the well-known error backpropagation method and the stochastic steepest gradient method, respectively. can be learned.

FIG. 3 is a block diagram of an embodiment of the second invention of the neuron model of the present invention, in which the input signals from other neuron groups are weighted by the coupling coefficient Wi, j''K+j to generate a weighted signal. a synaptic coupling unit 1b that outputs the synaptic coupling unit 1b, and a cell body unit 2b that adds weighted signals to generate a cell internal state, performs moving average filtering on the cell internal state one time before, and feeds it back to the synaptic coupling unit 1b.
and an axon hillock 3 that generates a neuron output value from the output of the cell body 2b.

In this neuron model, the feedback coupling coefficient Aj (Z?), which is the weight of the feedback coupling from the cell body 2b to the synaptic junction 1b, is realized by a moving average filter, and this is used to calculate the Internal state Xj(1
What is shown by the following equation and Z −x(t)−x(t−
4) It is shown as.

FIG. 4 is a configuration diagram of an example of a hierarchical neural network constructed using the neuron model of FIG. 2.

The hierarchical neural network shown in FIG. 4 includes an input layer (first layer) 21,
Second layer 22. ...Layer 23. (k+1)th layer 24,
...is configured with an output layer (M-th layer) 25, each layer is equipped with cells 201, and each cell is each equipped with an AI (
b) A feedback coupling coefficient of ~AnM<7j is fed back as a feedback coupling weight, and this is realized by a moving average filter. In Figure 4, the same symbols as in Figure 3 have the same content, so
A detailed explanation of each of these will be omitted.

Next, the operation of the hierarchical neural network shown in FIG. 4 will be explained.

Although the input signal is a scalar, there is no particular problem in using it as a multidimensional vector. This neural network is composed of M layers, and each layer includes nk (k-1 to M) neurons. Connections between neurons occur only between adjacent layers, and there are no connections within layers.

The signal 1(t) input to the input layer 1 is based on the past intracellular state 11x(t-z)(z-1 to n) of each neuron.
The signal flows unidirectionally through the neural network from the input layer 1 to the output layer 5 while taking into account information about past input signals obtained by filtering the signal, and 0j(j
−1, 2, . . . , nM) from the output layer 5 as neuron output values. In this way, input signal 1 (tl is
This neural network calculates the past input signal 1(t-/)(j-
1 to n) is taken into account and converted into an output value.

In addition, in this hierarchical neural network, the coupling coefficients W t and J with other cells and the coefficient a of the filter in feedback 3. z (1-1+ 2 +...n)
The learning to determine can be performed efficiently using the well-known error backpropagation method and stochastic steep gradient method, respectively.

〔Effect of the invention〕

As explained above, the first aspect of the present invention is to provide a neuron model with a self-return migration (average filtering function) added to the synaptic connection part, and to create a hierarchical type structure using this neuron model. Neural networks have the advantage of being able to perform processing that depends on past input signals as well as very sophisticated processing.

In addition, the second invention of the present invention provides a neuron model with a feedback mechanism for internal cell states filtered by a moving average filtering function, and a hierarchy configured using this neuron model. In a type neural network, the current output value of a neuron depends on signals input in the past, so it has the effect of efficiently performing advanced information processing similar to that of the human brain. .

[Brief explanation of drawings]

FIG. 1 is a configuration diagram of an embodiment of the first invention of the neuron model of the present invention, and FIG. 2 is a configuration diagram of an embodiment of a hierarchical neural network constructed using the neuron model of FIG. 1. 3 is a configuration diagram of an embodiment of the second invention of the neuron model of the present invention, and FIG. 4 is an embodiment of a hierarchical neural network constructed using the neuron model of FIG. 3. FIG. 5 is a block diagram showing a first example of a conventional neuron model, and FIG. 6 is a block diagram showing a second example of a conventional neuron model. 1, la, lb...synaptic junction, 2, 2a, 2b
...Cell body, 3...Axon hillock, 11.21...
Input layer, 22...second layer, 23...th layer, 14°24...(k+1)th layer, 201...cell. 15.25...Output layer, 101°

Claims (2)

[Claims] 1. a synaptic coupling unit that weights input signals from other neuron groups and outputs a weighted signal; a cell body that adds the weighted signals to generate a cell internal state;
In a neuron model having an axon hillock that converts the internal state of the cell into a cell output value, input signals from other neurons are sent to the synaptic junction in a form suitable for the processing required of the neuron. A neuron model characterized by being equipped with an autoregressive moving average type filtering function. 2. a synaptic coupling unit that weights input signals from other neuron groups and outputs a weighted signal; a cell body that adds the weighted signals to generate a cell internal state;
In a neuron model having an axon hillock that converts the internal state of the cell into a cell output value, a moving average filtering function that feeds back the past internal state of the cell from the cell body to the synaptic connection part is provided. A neuron model characterized by the following features: