RU2269155C2 - Neuron model, realizing logical nonequivalence function - Google Patents

Abstract

FIELD: neuro-cybernetics, possible use as functional unit for various artificial neuron networks.

SUBSTANCE: device has two inputs, signals from which are multiplied with appropriate weight coefficients and are sent to adder, total outputted by which is further transformed in activation block by modular activation function, and then by threshold function at the output of neuron.

EFFECT: prevented problem of impossible realization of logical nonequivalence function by known neuron models.

7 dwg, 1 tbl

Description

The invention relates to neurocybernetics and can be used as a functional unit of various artificial neural networks.

Known technical and mathematical model of an artificial neuron [1-4]. The first model was proposed in [1]. Later [2], on the basis of such neuron models, the first artificial neural networks (perceptrons) were implemented to solve letter image recognition problems. Such artificial neurons are used as a functional unit of all modern artificial neural networks [3, 4] and are devices with several inputs and one output. The input signals arriving at the inputs are multiplied with the corresponding weighting coefficients, summed and transformed by the transfer function, which can be used as a threshold, linear, sigmoidal, hyperbolic, etc. [3, 4]. Such a model of a neuron allows, although roughly enough, to reproduce some of the functions of a natural neuron. Combining artificial neurons into layers and networks, together with learning algorithms, it is possible to achieve an acceptable solution by a neural network of complex problems that cannot be adequately formalized [3, 4].

The closest in technical execution to the proposed neuron model is the model of the simplest perceptron neuron with two inputs, a pair of weighting factors, an adder and an activation unit [3, 4]. The advantage of this model is the ability to implement most of the logical Boolean functions. So, for two variables, according to [5], there is a set of 16 functions of the algebra of logic. The well-known model of a neuron with two inputs implements 14 of them [4], ie all, with the exception of logical Boolean functions of unequality (or “sum modulo 2”) and equivalence (equivalence), which are considered, for example, in [6]. The impossibility of realizing two of these logical functions is a significant drawback of such a neuron model and is called the EXCLUSIVE OR problem, which was first discovered in [7]. This problem is fundamental in the theory of artificial neural networks and is clearly stated in a number of modern scientific publications [4, 8, 9].

The proposed model is aimed at eliminating the considered drawback (the problem of the unrealizability of the logical function of unequality) of the known neuron model.

The problem of the unrealizability of the logical function of unequality by a neuron is as follows [4, 8, 9].

Consider the well-known model of a neuron, a functional diagram of which is shown in figure 1. The input signals x ₁ and x ₂ upon receipt of blocks 1 and 2 are multiplied with the corresponding weighting factors w ₁ and w ₂ . Blocks 1 and 2 carry out multiplication taking into account the sign bits of the signals [10]. Then the multiplied signals x ₁ w ₁ and x ₂ w ₂ are fed to the input of the adder with sign inputs / output [10], which is indicated by block 3 in figure 1. From the output of the adder, the signal s = (x ₁ w ₁ + x ₂ w ₂ ) goes to activation block 4 [10], in which it is converted by the threshold transfer function [4, 8, 9]:

where P is the threshold value. The output of block 4 is the output of a neuron model. The required output values of the neuron model for the implementation of the ambiguity function, depending on x ₁ and x ₂ , are shown in table 1.

Table 1 Input x ₁ Input x ₂ Output y (x ₁ , x ₂ ) 0 0 0 0 one one one 0 one one one 0

Consider the sum of s as a function of two variables (surface) of x ₁ and x ₂ over the plane of input values Ox ₁ x ₂ , as shown in figure 2 [4]. Each point of this surface is located above the corresponding point of the plane Ox ₁ x ₂ at a distance equal to the value of s at this point. Points at which the value of s is equal to the threshold value P set in block 4 form a straight line in the Px ₁ x ₂ plane parallel to Ox ₁ x ₂ (in Fig. 2 Px ₁ x ₂ - in a translucent tone; P is 0.5) and projected into some dividing one-dimensional hyperplane (straight line) on the plane of the input values Ox ₁ x ₂ . This dividing line on the plane Ox ₁ x _{2 is} shown in figure 3, where the arrows to the corners (possible values of x ₁ and x ₂ ) indicate the required output values of the neuron to implement the logical function of unequality.

All points on one side of the dividing line are projected into s values greater than the threshold, and points on the other side give smaller s values, i.e. the threshold line divides the plane Ox ₁ x ₂ into two regions. In one of them, the activity of the neuron is unity, in the other - zero. Thus, it is impossible to draw a straight line so that a two-input neuron in accordance with table. 1 divide zeros and ones on the plane Ox ₁ x ₂ , i.e. implement a logical function of disambiguation.

Consider the essence of the proposed neuron model that implements the logical function of unequality.

Functional diagram of the device shown in figure 4.

, а затем с выхода блока 4 поступает на вход блока 5, где преобразуется пороговой функцией (1). В качестве блоков 4 и 5 могут быть использованы описанные в [10] технические устройства, реализующие соответствующие математические функции. Выход блока 5 предлагаемой модели является ее выходом. Таким образом, выход модели нейрона у(х₁,х₂) является результатом подстановки модуля суммы s в пороговую функцию (1). Полная функция передачи модели будет выглядетьThe proposed neuron model, as well as the known one, contains two multipliers (blocks 1 and 2) that multiply the input signals with weight coefficients taking into account their signs. Any of the well-known multipliers of this type described in [10] can be used in the model. As in the known model, the multiplied signals x ₁ w ₁ and x ₂ w ₂ are fed to the input of the adder (block 3) with significant inputs / outputs [10]. As a symbolic adder, any devices of a similar type can be used, for example, those given in [10]. The proposed model differs from the prototype model in that the activation unit consists of two series-connected units, as shown in figure 4. In this case, the sum s from the output of the adder first goes to the input of block 4 (Fig. 4), in which it is transformed by a modular activation function of the form

, and then from the output of block 4 goes to the input of block 5, where it is converted by a threshold function (1). As blocks 4 and 5, technical devices described in [10] that implement the corresponding mathematical functions can be used. The output of block 5 of the proposed model is its output. Thus, the output of the neuron model y (x ₁ , x ₂ ) is the result of substituting the modulus of the sum s into the threshold function (1). The full model transfer function will look

The mathematical model of the neuron in this case will be described

the following expression:

The sum s = (x ₁ w ₁ + x ₂ w ₂ ) should be zero if the values of x ₁ and x _{2 are} equal, which is achievable with w ₁ = 1, w ₂ = -1, as shown in figure 2. In this case, the modulus of the sum (the signal at the output of block 4 of Fig. 4) as a function of two variables x ₁ and x ₂ will have the form, in contrast to the prototype, not one surface, but two subsurfaces at some angle between them, as shown figure 5. If, in this case, the modulus of the sum is transformed by the threshold function (2) in block 5 of Fig. 4, i.e. limit the plane Px ₁ x ₂ , which is shown in figure 5 in a translucent tone, the intersection of the surface of the module with the plane Px ₁ x ₂ forms not one, as in the prototype, but two dividing lines.

, откуда, в соответствии со свойством модуля, образуются два уравнения разделяющих прямыхMathematically, the equation of the dividing line from expression (3) will look

, where, in accordance with the module property, two equations of dividing lines are formed

where x ₁ ⁽¹⁾ is the first line; x ₁ ⁽²⁾ is the second straight line, i.e. two projections of the intersection of the surface s with the plane Px ₁ x ₂ , obtained on the plane Ox ₁ x ₂ , which divide it into three areas as shown in Fig.6.

A full view of the dependence of the neuron output y (x ₁ , x ₂ ) as a function of two variables as x ₁ and x ₂ in the case under consideration is shown in Fig. 7.

In figure 2, 3, 5-7 the dimensions of all coordinate axes are given in hundredths.

Figure 5-7 shows that for x ₁ = x ₂ (or x ₁ ≈ x ₂ ), the output neuron will give zero, and for x ₁ ≠ x ₂ it will give one. Thus, separating the zero input values from the unit values on the Ox ₁ x ₂ plane, unlike the prototype, not one, but two straight lines, the neuron implements a logical function of unequality.

To confirm the feasibility of the claimed invention, we consider the operation of the proposed neuron model.

Weighting factors have constant values in all cases: w ₁ = 1, w ₂ = -1. The threshold value is taken equal to, for example, 0.5. The result of substituting a modular function into a threshold will look similar to (2):

Since the mathematical model of a neuron has the form

then when applying to its inputs x ₁ and x ₂ signals equal to zero, the sum (output of block 3 of FIG. 4) will be zero. Transformed by the modular activation function in block 4, the signal at its output will also be zero, as after block 5 at the output of the y (x ₁ , x ₂ ) neuron. When entering the inputs x ₁ = 1, x ₂ = 0, the sum will be equal to one; module (block 4 output) - also one, which will exceed the threshold P = 0.5 in block 5 and there will be one at the output of the neuron. With the input combination x ₁ = 0, x ₂ = 1, the sum will be equal to minus one; the module is unity, which will also exceed the threshold P = 0.5 in block 5 and will give unity at the output of the neuron. In the case x ₁ = 1, x ₂ = 1, the sum will become zero; the modulus of the sum will also be zero, which will give zero at the output of the neuron.

Thus, the proposed neuron model, in accordance with table 1, implements the logical function of unequality.

The input signals of the neuron x ₁ and x ₂ can be not only discrete, but also continuous (from zero to unity), since the surface of the modulus of the sum s, shown in Fig. 5, exists for all continuous x ₁ and x ₂ .

Thus, the proposed neuron model, realizing the logical function of unequality, overcomes the EXCLUSIVE OR problem.

Information sources

1. McCulloch W.S., Pitts W. A logical calculus of the ideas imminent in nervous activity. // Bulletin of Mathematical Biophysics, vol. 5, 1943, pp. 115-133. (Russian translation: McCullock, W.S., Pitts, W., The Logical Calculus of Ideas Related to Nervous Activity. Automata. / Ed. By K.E. Shannon, J. McCarthy. - M.: IL, 1956).

2. Rosenblatt F. Principles of neurodinamics. Perceptrons and the theory of brain mechanisms. Spartan Books, Washington, 1962. (Russian translation: Rosenblatt F. Principles of neurodynamics. Perceptrons and theory of brain mechanisms. / Under the editorship of S.M. Osovets. - M: Mir, 1965 - 480 p.).

3. Galushkin A.I. Theory of neural networks. Book 1: Textbook. manual for universities / General ed. A.I. Galushkina. - M .: IPRZhR, 2000 .-- 416 p.

4. Wasserman P. Neurocomputing. Theory and practice, Nostram Reinhold, 1990. (Russian translation: Wassermen F. Neurocomputer technology: theory and practice. / Transl. - 240 p.).

5. Kurinnoy G.Ch. Mathematics: Reference. - Kharkov: Folio; Rostov n / a: Phoenix, 1997 .-- 463 p.

6. Zalmanzon L.A. Conversations about automation and cybernetics. -M .: Science. Ch. ed. Phys.-Math. lit., 1981. - 416 p.

7. Minsky M., Papert S. Perceptrons. An introduction to computational geometry, MIT Press, 1969. (Russian translation: Minsky M., Peypert S. Perceptrons. / Ed. By V.A. Kovalevsky. - M .: Mir, 1971. - 261 p.).

8. Kruglov VV, Borisov VV Artificial neural networks. Theory and practice. - M .: Hot line - Telecom, 2001 .-- 382 p.

9. Dyakonov V., Kruglov V. Mathematical extension packages MATLAB. Special reference. - St. Petersburg: Peter, 2001 .-- 480 p.

10. Galushkin A.I. Neurocomputers. Book 3: Textbook. manual for universities / General ed. A.I. Galushkina. - M .: IPRZhR, 2000 .-- 528 s.

Claims (1)

A neuron model that implements a logical function of ambiguity, containing two inputs, signals x ₁ and x ₂ , taking values 0 or 1, from which are multiplied with the corresponding weighting factors ω ₁ and ω ₂ , where ω ₁ = 1, and ω ₂ = -1 , and arrive at the adder, moreover, s∈ (-1, 0, 1), where s is the sum from the output of the adder, s = x ₁ ω ₁ + x ₂ ω ₂ , characterized in that the sum s is transformed in the activation unit by a modular function view activation

a then threshold function

 where P is the threshold value, P = 0.5, f (x) is the function at the output of the neuron.

Images