RU2597496C1 - Single-layer perceptron, simulating real perceptron properties - Google Patents

Abstract

FIELD: simulation of neural structures.

SUBSTANCE: invention can be used on the neural computers, in the technical systems based on neural networks for pattern recognition, analysis and processing of the images. Device comprises recording impulse neurons, inputs for input signals-impulse flows, internal communication channels from the inputs, individual clusters from part of communication channels in each of recording neurons, which pass the part of input signals; formation of clusters and adjustment of channels number in the clusters is carried out in accordance with the code combinations and structure of periods of partial components of input impulse flows; inertialess summation of all partial impulse flows at the outputs of clusters is performed; obtained signal is converted by liminal excitable elements, which generate single impulses or impulse sequences at exceeding of amplitude of impulse flows of liminal excitable elements which are used as the input signals.

EFFECT: technical result is provision of an opportunity to achieve selective detection of the input objects without using a balance measurement of the input signals, possibility of encoding the input object of a certain type by the channel number or by the number of recording neuron, compression of input information, performance increase, objects recognition reliability increase.

1 cl, 13 dwg, 2 tbl

Description

The invention relates to the modeling of neural structures and can find application in the development of neurocomputers, technical systems based on neural networks for pattern recognition, creating artificial intelligence, image analysis and processing, and many other applications. The aim of the invention is to expand the scope of neural networks, the main element of which is a single-layer perceptron, neural network technologies, the creation of artificial intelligence.

A typical block diagram of a known single-layer pulse perceptron using five input signals as an example is shown in FIG. 1 [1, 2, 4, 5, 9, 14, 17, 22, 28, 29].

FIG. 1. The block diagram of a well-known single-layer pulsed perceptron on the example of five input signals.

Requirements for a pulsed or spike neuron used in a known single-layer pulsed perceptron

We formulate the requirements for a pulsed or spike neuron used in the well-known single-layer pulsed perceptron. The well-known single-layer pulsed perceptron uses a pulsed or spike neuron more adequate to a real biological neuron than the McCallock-Pitts neuron, the block diagram of which is shown in FIG. 2 [1, 2, 3, 5, 7, 10, 11, 13, 15, 16, 18, 26-36].

FIG. 2. A block diagram of a known pulsed (spike) neuron.

In FIG. 2 designations are accepted: (x ₁ , x ₂ , ..., x _n ) - the vector of input signals specified in the form of pulse flows; w ₁ , w ₂ , ..., w _N - weighting factors; Σ is a synoptic device for summing input actions with weights; F is a threshold excitable element that generates an output pulse sequence or a single pulse when the signal at the output of the adder exceeds a threshold value; Y is the output signal. The synaptic summation mechanism is conditionally reflected by a symbol generally accepted in neurobiology for displaying the synapse under the sign of the sum Σ. The mechanism for generating an output pulse sequence or a single pulse when the signal at the output of the adder exceeds a threshold value is conditionally reflected by a symbol generally accepted in neurobiology to display a neural pulse. In the future, we will use the block diagram of a pulsed neuron in the form shown in FIG. 3

FIG. 3. A block diagram of a known impulse (spike) neuron and its functions.

In FIG. 3 designations are accepted: KS - communication channels (dendrites); W - memory devices where weight coefficients are stored; P - input signal multipliers by weight coefficients; Σ is a synaptic device for summing input actions with weights; F is a threshold excitable element that generates an output pulse sequence or a single pulse when the signal at the output of the adder exceeds a threshold value; Y is the output signal.

The well-known single-layer impulse perceptron uses the well-known impulse neuron model. Let us consider in more detail the properties and disadvantages of this neuron model. At the moment, a large number of models of single neurons have been developed: from threshold models to biologically plausible models [1, 2, 3, 5, 7, 10, 11, 13, 15, 16, 18, 26-36]. The known model of a neuron consists of the following main blocks: 1) a block for storing or generating weighting factors; 2) a multiplier; 3) a synaptic device for summing impulse fluxes of influences with weighting factors; F is a threshold excitable element that generates an output pulse sequence or a single pulse when the signal at the output of the adder exceeds a threshold value; Y is the output signal.

To study the phenomena observed in neural networks, an understanding of the principles of functioning of neurons is necessary. From the point of view of realization, neurons are complex generators of pulsed signals, the mechanisms of operation of which are currently studied to a certain extent.

A pulsed neuron as an element of a neural network must meet a number of requirements [1, 2]. The main ones: 1) the ability to generate an action potential that is close in form to the action potential of a real biological neuron; 2) the ability to generate packs of neurons that are close in characteristics (frequency, structure, etc.) to packs of a real biological neuron; 3) must have threshold properties, the property of refractoriness, as in a real biological neuron; 4) must sum the impulses coming to the input from other neurons, i.e. the neuron must include a device called a synapse; 5) the synapses themselves can be exciting and inhibitory and can markedly differ in other properties; 6) have the ability to adapt with respect to external influences or ductility; 7) possess a number of other properties that are less significant in the first approximation, but which may be significant in applications.

Usually, they are not specified by the fulfillment of these requirements, but they immediately use a mathematical model of a neuron that is suitable from the user's point of view. For example, mathematical models of the Hodgkin-Huxley neuron, Van der Paul-Fitzhugh, Hindemarsh-Rose, etc. [29-36]. The second important criterion is the surmountable complexity of the calculations when using the selected model. A short list of currently used mathematical models of neurons is given in the table [1, 2, 7, 11, 15, 29-36].

The limited computing resource has given rise to many simplified models that allow, on the one hand, to satisfactorily model the dynamics of individual elements, and on the other, to efficiently use the computing resource. These models began to be called phenomenological.

The mechanisms for summing pulsed flows in a pulsed neuron are fundamentally different from the summation of binary signals in a McCallock-Pits artificial neuron. Consider this process in more detail.

Methods for summing impulse flows in a pulsed neuron

The comparative spatial distribution of dendrites for various types of neurons according to neurobiological studies is illustrated in FIG. four.

FIG. 4. Comparative spatial distribution of dendrites for various types of neurons according to neurobiological studies

In FIG. Figure 4 shows three types of neurons, which, depending on their purpose, have various organizations of a cluster of dendrites, which are the input communication channels between the body of the neuron and sensor systems that are suppliers of input signals. The summation of pulsed streams emerging from communication channels in a real neuron occurs in a specific way, namely, as a spatio-temporal summation. It is fundamentally different from the integral synaptic summation in the known pulsed spike neuron. Specificity follows from a real experimentally known device of dendrites. Dendrites in the form of thin fibers as they approach the body of the neuron are combined by attaching to a thicker fiber or with each other. As a result, thin multiple fibers of dendrites evolve into a smaller number (up to one) of thicker dendrites [19, 20].

Thicker fibers can again combine into even thicker ones, forming electrically powerful current conductors from pulsed currents.

When dendrites are combined, the currents of impulse flows in the initial dendrite fibers are summed. Thus, inertialess summation of the currents of all the initial pulse flows occurs. In this case, hierarchical dendritic trees are formed. An analysis of the functional properties of dendrites in connection with their structure was studied in a fundamental work [19, 20]. The work studied: the input resistance of dendrites, the potentials arising on them during the flow of pulsed currents of electric current in them, the convergence of dendrites. Dendrite convergence is especially present in neurons of the human and animal brain. The convergence of dendrites in the stellate cell of the cerebellar cortex of a cat was studied by V.V. Smolyaninov [19, 20].

The role of synapses in the summation of impulse flows

Pulse flows enter the neuron directly from the dendrites due to the connection of their biological membranes, that is, a common biological membrane arises for dendrites and soma of the neuron [1, 2]. Another part of the impulse flows into the neuron soma through the synapses connecting the dendrite to the neuron soma. The synapse serves as a converter of the electric potential of the dendrite to a chemical mediator and then again to the electric potential [1, 2]. A remarkable feature of chemical synapses is their one-sided conductivity. They can conduct electric current from the dendrite to the body of the neuron in only one direction. In the opposite direction, electric current and potential are not conducted. The summation of the electric potentials of impulse flows after converting them by synapses occurs in the neuron soma. Thus, the synapses themselves do not actually participate in the summation of pulse flows. Synapses carry out unilateral conduction from dendrites to catfish. Thus, they protect the weak electric potentials of dendrites from the effects of potential arising in the trigger zone near the axon. There is a well-reasoned hypothesis that impulse flows entering through chemical synapses play a subtle control role in generating a neuron impulse response.

In a real biological neuron, the number of synapses on the som of the neuron, as a rule, is much less than the number of dendrites ending in the soma due to common membranes. The number of synapses on the body of a neuron varies from units to many thousands. They are attached to the catfish along with dendrite bases. Neurobiological studies of the properties of synapses show that in the overall picture of the spatiotemporal summation of pulse sequences, synapses play the role of practically inertialess converters of electrical pulse sequences into chemical ones and then back into electrical ones.

The fundamental property and purpose of synapses is, apparently, their one-sided conductivity with respect to incoming pulse sequences. The time constant of synapses when exposed to rectangular pulses during their transformation of the trailing edge according to the law

has an order of 10 (1 / ms). Therefore, such pulses during summation can be considered as inertialess one-way converters.

Often the synapses are located not only on the som of the neuron, but also on the dendrites themselves and the impulse flows from them enter the neuron body along with the impulse sequences from the dendrites. This is illustrated in FIG. 5. In a real biological neuron, the number of synapses in the som of the neuron, as a rule, is much less than the number of dendrites ending in the soma due to common membranes. The number of synapses on the neuron soma varies from units to hundreds of thousands, which is also illustrated in FIG. 5.

FIG. 5. The location of the main types of synapses on the body of a neuron: 1 - axodendritic synapse; 2 - axosomatic synapse; 3 - axoaxon synapse; 4 - dendrite; 5 - catfish; 6 - axon knoll; 7 - axon. In FIG. 5 top right and figure 5 below shows neurons with a large number of synapses

General conclusions on the methods of summing pulsed flows in a neuron

The foregoing allows us to state that in most cases, pulsed flows are summed in an inertialess manner:

1) In the dendrites themselves in the zones of their convergence or in the neuron catfish during the natural fusion of the dendrite with catfish with a common membrane. This summation occurs sequentially by convergence due to the increasing thickening of dendrites in the distributed region where the dendritic tree is located. Since the summation occurs on the dendrites themselves, which are electrically conductive resistive channels, this summation is inertialess almost instantaneous summation on a resistive load. This property is fundamentally necessary for distinguishing the maximum values of terminal impulse flows suitable for the neuron soma;

2) Part of the impulse flows into the neuron soma through the synapses connecting the dendrite to the neuron soma. The synapse serves as a converter of the electric potential of the dendrite to a chemical mediator and then again to the electric potential [1, 2]. The summation of the electric potentials of impulse flows after converting them by synapses occurs in the neuron soma. Thus, the synapses themselves do not actually participate in the summation of pulse flows. Synapses carry out unilateral conduction from dendrites to catfish. Thus, they protect the weak electric potentials of dendrites from the effects of potential arising in the trigger zone near the axon. The above described mechanisms for summing pulsed flows are reflected in the present invention.

Disclosure of invention

The closest technical solution to the claimed model of a single-layer perceptron is a single-layer perceptron based on impulse neurons, shown in FIG. 1. The disadvantages of a single-layer perceptron based on an artificial impulse neuron are known in sufficient detail [1, 2, 29-36]: 1) a single-layer perceptron based on artificial impulse neurons is inadequate to a real biological perceptron; 2) in a real biological neuron, a weighted gradation of input signals is not performed, weighted gradation is an artificial technique invented by engineers; 3) neural network algorithms using single-layer perceptrons based on an artificial pulse neuron with a large number of input signals require significant computational resources; 4) an increase in input signals even by one requires a complete recalculation of weighting factors. The list of shortcomings can be continued.

Due to the shortcomings of a single-layer perceptron based on artificial impulse neurons, a single-layer perceptron model is proposed, which is a device based on impulse neurons containing clusters of communication channels that more adequately reflects the functions of a real single-layer perceptron, a block diagram of which is shown in FIG. 6.

FIG. 6. The block diagram of the proposed single-layer perceptron.

In FIG. 6, the following designations are accepted: K — formed cluster of communication channels; Σ - adder, F - threshold transducer. Triangles indicate blocked communication channels from the input, which are not essential characteristic vectors for the object at the input of the perceptron.

A block diagram of a pulsed neuron used in the proposed single-layer perceptron containing clusters of communication channels that more adequately reflects the functions of a real neuron is shown in FIG. 7.

FIG. 7. The block diagram of a pulsed neuron used in the proposed single-layer perceptron. A neuron contains clusters of communication channels that are consistent with the input object. This neuron more adequately reflects the functions of a real neuron.

Let us explain the technology of cluster formation from communication channels. Let us give an example of a single-layer perceptron with pulsed neurons having formed clusters. An explanation of the formation of clusters of communication channels depending on the properties of the input signal is illustrated in FIG. 8.

FIG. 8. An explanation of the formation of clusters from communication channels in individual perceptron neurons depending on the properties of the input signal.

Three code combinations of the input signal are shown on the left, according to which a cluster of communication channels of each neuron is formed. The black color on the code combination corresponds to various, arbitrary periods of partial pulse flows, and the white color corresponds to the absence of partial pulse flows. The frame highlights clusters of communication channels formed in accordance with three characteristic code combinations of the input signal, that is, in accordance with the properties of the input signal.

Neuron communication channels corresponding to non-informative cells are blocked by a triangle symbol, as shown in FIG. 8. Thus, the proposed single-layer perceptron is tuned to the characteristic input code combinations shown on the left.

Can the system of dendrites at the input of a neuron be so flexible and labile as to adequately reflect the input information by changing the number of exciting and inhibitory dendrites. This issue has not been studied enough before. Recently, however, a number of discoveries have been made that have shown that dendrites have completely unusual growth opportunities, changes in quantity and quality using a special spike system [1, 2, 6]. It was also established that dendrites have active properties similar to those of axons [1, 2, 6]. All this taken together proves the possibility of adapting to the reflection of any input information not due to the weight functions of dendrites, but due to the clustering of dendrites, changes in their quantity and quality. Various properties of dendrites are understood to mean their properties: dendrites exciting and inhibitory, different in diameter and electric power, etc .; the possibility of self-organization through the growth and death of individual dendrites. In real conditions, in the human or animal organism, the formation of effective bonds in a cluster occurs naturally due to feeding working dendrites and destroying idle dendrites by stopping nutrition. Dendrites are fed when electrical impulses pass through them. At the same time, building materials in the form of proteins, ATP, which provide dendrites with energy resources, and other organelles enter the dendrite. Regulation of nutrition occurs due to the supply of ATP and protein components during the passage of electrical nerve impulses. If the electrical nerve impulses do not pass through the dendrite, then power is not supplied, and it dies.

In connection with the foregoing, the following claims are proposed.

The perceptron is single-layer, including several recording impulse neurons, inputs for input signals-impulse flows, internal communication channels from inputs, in which the input signals are multiplied by weights and sent to adders associated with threshold nonlinear excitable elements generating single output pulses or pulse sequences when the signals at the outputs of the adders exceed the threshold value of the excitable nonlinear element, which are used as output signals cash;

Characterized by individual clusters from a portion of communication channels in each of the recording neurons that pass part of the input signals; the formation of clusters and equalization of the number of channels in the clusters is carried out in accordance with code combinations and the structure of the periods of the partial components of the input pulse flows; then they produce inertialess summation of all partial pulse flows at the outputs of the clusters; this is realized for the signal in the form of total pulse flows at the outputs of clusters of ε-near-periods with a maximum sum of amplitudes of all partial pulse flows; after inertialess summation, the received signal is converted by threshold excitable elements that generate single pulses or pulse sequences when the maximum sum of the amplitudes of pulse flows exceeds the threshold value of excitable elements that are used as output signals.

Technical results to which the invention is directed

The technical results to which the invention is directed are as follows:

1) achieving selective recognition of input objects without using weighting of input signals;

2) the possibility of coding by the proposed selective single-layer perceptron of an input object of a certain type with a channel number or a recording neuron number;

3) compression of input information due to the preservation of only that information about objects that falls into a given channel or a recording neuron;

4) improving the performance of the proposed single-layer perceptron;

5) improving the reliability of recognition of objects with a large number of them;

6) the achievement of significantly greater adequacy of the proposed single-layer perceptron to a real biological single-layer perceptron.

The set of achieved technical results specified in paragraphs 1-6, is achieved due to the following set of features contained in an independent claim:

1) by individual clusters of communication channels in each of the recording neurons, clusters form individual neurons from part of the input communication channels;

2) the formation of clusters and equalization of the number of channels in the clusters is carried out in accordance with the code combinations of the input signals;

3) clusters from part of the communication channels are associated with adders.

The effectiveness of the proposed perceptron is actually proven in the developed neural network systems for recognition:

1. Recognition of numbers;

2. Recognition of the letters of the Russian alphabet;

3. Recognition of texts of fiction;

4. License plate recognition;

5. Recognition of electrocardiograms (ECG);

6. Recognition of black and white portraits of famous Russian writers and poets.

The systems specified in paragraphs 3, 5, 6, were developed for the first time and have no analogues.

Another justification for the effectiveness of the proposed single-layer perceptron

An indirect justification of the effectiveness of the proposed single-layer perceptron follows from the remarkable work of the outstanding neurophysiologist and neuropsychologist Professor E.N. Sokolov, who developed the vector theory of perception of sensory information and brain memory [9, 21-24].

Consider the basic provisions of the vector theory of perception, formulated in the work of E.N. Sokolov, which can be used to confirm the effectiveness of the proposed single-layer perceptron. In the works of E.N. Sokolov as the input objects are the previous layer of neurons, called E.N. Falcon preceptor. Signals from neurons in the pre-detector layer are fed to the inputs in the neuron layer, which should be considered a single-layer perceptron. Here are some of the points in the paper that contain the main results obtained by E.N. Sokolov and his school [9, 21-24].

1. The excitation vector is a combination of excitations of a limited number of neurons of pretectors forming a neural ensemble.

2. The excitation vector of the pretectors is characterized by the constancy of its length for all stimuli acting on the given ensemble of pretectors.

5. All vectors of synaptic connections of detectors receiving inputs from a given ensemble of pre-detectors are equal in length.

10. The maximum excitation of a particular detector as an element of the detector card corresponds to a specific sensation.

Theory of vector perception E.N. Sokolova is supported by a series of neurophysiological experiments implemented by students of prof. E.N. Sokolova [9, 21-24]. We note some basic similarities and differences in the provisions of the vector theory of perception to confirm the effectiveness of the proposed material single-layer perceptron.

1. The presence of characteristic properties, vectors of the input object (p. 1). An excitation vector is a combination of excitations of a limited number of precursor neurons forming a neural ensemble and located in the layer preceding the layer in which the considered single-layer perceptron is located. Thus, prof. E.N. Sokolov asserts the presence for the object at the input of a single-layer perceptron of individual characteristic vectors. In the proposed single-layer perceptron, characteristic vectors can be considered characteristic code combinations of input objects.

2. The constancy of the length of the characteristic excitation vectors (items 2, 5). In the proposed single-layer perceptron, the equality of the length of the characteristic excitation vectors is reflected by the equality of the number of channels in the specific clusters of each of the neurons of the single-layer perceptron.

3. The maximum excitation of a certain input vector corresponds to a specific sensation (p. 10).

An essential element of the novelty of the proposed single-layer perceptron is, in contrast to the neural networks studied in the works of E.N. Sokolov, the formation of perceptron neurons not of excitation vectors, but clusters of communication channels, which are dendrites in real neurons that connect the inputs of a neuron with its soma (body of a neuron). Clusters from communication channels in a single-layer perceptron are formed in accordance with the output characteristic clusters of the object at the input. It is essential that, in order to normalize the intensity of input actions, the condition of constancy or practical equality of the number of communication channels in individual clusters of the proposed single-layer perceptron should be fulfilled.

Thus, we can say that the proposed single-layer perceptron is a specific original device representing the material embodiment of the vector theory of prof. E.N. Sokolova.

Disclosure of the mathematical theory of the invention. The first mathematical theory of invention

Consider the mathematical theory of the proposed single-layer perceptron. We denote possible characteristic code combinations of objects at the input of a neuron in the form of vectors

x ₁ = (x _l1 , ..., x _1n ); ...; x _m = (x _m1 , ..., x _mn ),

where n is the number of elements of the code combination; m is the number of possible objects. We consider all components of the vectors equal x _ij = A, or x _ij = 0. Thus, there are m vectors x _i = (x _i1 , ..., x _in ) with dimension n. In the proposed neuron, the cluster of links is one of these code combinations. Generally speaking, all possible code combinations form a matrix A, which can be represented as

Let the specific neuron we are considering contain a cluster of bonds characterized by a code combination in the form

x _i = (x _i1 , ..., x _in ).

When this neuron with the number i of the code combination of the input pattern x _k = (x _k1 , ..., x _kn ) is fed to the input (from the set of possible at the input of the neuron) we get the sum

In total we get m sums S _ik k = (1, ..., m). The largest will be the sum of S _ii

where N _i A is the number of units in the code combination x _i = (x _i1 , ..., x _in ). The value of N _i A is taken by the threshold value of the neurons. Exceeding this value leads to the appearance of a pulse or a sequence of pulses at the output of a neuron. We note the following property of the sums S _ik

S _ik <S _ii .

This strict inequality is true, since the sum S _ik contains a larger number of terms equal to the products 1 · 0 than the sum S _ii . This property can be qualitatively compared with the property of orthogonal vectors or orthogonal functions. It is not strict orthogonality, it can be considered orthogonality in a qualitative sense. The property can be useful when using the proposed dynamic neuron for object recognition, channel number coding, and a number of other applications. In general, all possible values of the sums S _ij are equal to the elements of the matrix B equal to

B = AA ^T ,

where A ^T is the transposed matrix to A.

As follows from the mathematical analysis of the proposed single-layer impulse perceptron, clustering of communication channels in perceptron neurons provides the following fundamental functions of the perceptron: 1) selectivity to certain properties of the input object patterns; 2) the selective distribution of patterns across the channels at the output of the perceptron or what is called coding the channel number.

Thus, the first mathematical theory used to disclose the invention allows: 1) to describe the selective properties of a single-layer pattern; 2) to normalize the clusters from communication channels by equalizing the number of channels in the cluster; 3) evaluate the magnitude of artifacts in the allocation of specific channels; 4) carry out coding by channel number; 5) remove redundant input information by sifting out information that channel clusters are not configured for. We note right away that the fundamental properties of almost-periodic functions (which will be discussed in more detail below) do not allow selective selection of input patterns and selective distribution over channels at the output of a single-layer perceptron, that is, encoding with a channel number. All of the above properties can be achieved precisely due to the controlled clustering of communication channels of each of the neurons that make up the single-layer perceptron.

Disclosure of the second mathematical theory of the invention in terms of the mathematical theory of the conversion of pulsed flows by a neuron using the fundamental properties of almost-periodic functions

. Импульсные последовательности нейронов для исследования процессов суммации в первом приближении можно аппроксимировать последовательностями прямоугольных импульсов ν(t) с периодом Т=1/ƒ (ƒ - частота). На интервале периодаThe pulsed electrical activity of neurons creates impulse flows in the nervous system. To a first approximation, these flows represent the sums of periodic impulse functions. The periodic impulse function is given on the interval [0, ∞), has a period T _t , i.e.

. Pulse sequences of neurons for studying summation processes in a first approximation can be approximated by sequences of rectangular pulses ν (t) with a period T = 1 / ƒ (ƒ is the frequency). In the period interval

The total impulse flow is equal to the sum of the partial impulse flows, which are periodic functions

where u _i (t) are individual partial impulse flows (i = 1, ..., N) with periods T _i . The resulting impulse flow is a uniform almost periodic function [7–9]. The main property of uniform almost-periodic functions is the existence for any arbitrarily small number ε corresponding to an ε-almost-period T _ε having the property

Another fundamental property of an almost-periodic function: ε-almost-periods lie on the time axis [0, ∞) is dense everywhere. A set E of real numbers is called everywhere dense if there exists a number l> 0 such that in each interval of the real axis of length l (α <x <α + l) there is at least one number of the set E [7-9]. We consider the Kronecker theorem, which is a consequence of the basic theorem for the theory of an almost periodic function [8, 10, 11].

Theorem. Let T ₁ , T ₂ , ..., T _N be arbitrary real numbers. Whatever the number ε is a positive number, one can indicate a positive number L = L (ε) such that in each interval of length L there is at least one number τ satisfying the system of inequalities

where n _i are natural numbers. According to the Kronecker theorem, there exists an ε-almost-period of an almost-periodic function satisfying relation (1). According to the fundamental property of almost-periodic functions, there exists t such that

; t-ε<t<t+ε.then

; t-ε <t <t + ε.

Moreover, there exists an ε-almost-period of the function

. После прохождения нелинейного преобразователя с характеристикой F получим на его выходе импульсную последовательностьsuch that

. After passing through a nonlinear converter with characteristic F, we obtain the pulse sequence at its output

moreover, | u _ε (t + T _ε ) -u _ε (t) | <ε.

The sum of several pulse sequences is a uniform almost periodic function [8, 10, 11]. According to the Kronecker theorem, such a function has condensations and discharges of pulses that follow with intervals of ε-almost-periods. Consider the illustration of ε-almost-periods. The signals at the inputs and outputs of neurons can be represented as the sum of almost periodic functions, as shown in FIG. 9 to the left, the total impulse flow is shown above the time axis in FIG. 9 on the right.

показаны справа.FIG. 9. A schematic representation of the sequence of electrical impulses of a neuron in the form of rectangular pulses on the left. Four pulse sequences with periods T ₁ -T ₄ , ε-almost-period is approximately equal to T _ε = 8T ₁ = 4T ₂ = 3T ₃ = 2T ₄ . The ε-near-period is enclosed between two maxima. Thickening and rarefaction for pulsed flows with incommensurate pulse repetition periods

shown on the right.

происходит суммирование амплитуд импульсов, происходящее через каждые ε-почти-периоды. Следствие из теоремы аппроксимации показывает возможность использования почти периодических функций в задачах нейроинформатики. Известно, что для возбуждения нейрона необходимо, чтобы сумма импульсов, пришедших по дендритам, превышала порог возбуждения. Специфическое свойство почти-периодических функций - существование ε-почти-периодов - позволяет осуществить это требование.The maximum amplitude of the sum of the pulses is reached through the ε-near-period and is equal to 4A = 2, where A is the pulse amplitude of 0.5. Within

a summation of the amplitudes of the pulses occurs, occurring every ε-almost-periods. A corollary of the approximation theorem shows the possibility of using almost periodic functions in neuroinformatics problems. It is known that in order to excite a neuron, it is necessary that the sum of pulses arriving at dendrites exceed the excitation threshold. A specific property of almost-periodic functions — the existence of ε-almost-periods — allows us to fulfill this requirement.

To disclose the invention from the above second mathematical theory, it follows that the fundamental properties of almost-periodic functions allow for inertialess summation of partial impulse flows and the formation of maximum amplitudes following the almost-periods of an almost-periodic function, which is the resulting impulse stream.

The second mathematical theory, used to disclose the invention, allows to describe the mechanisms of inertial summation of impulse flows coming into the neuron soma for the subsequent threshold excitation of the generator of individual pulses or sequences. It is very important that no other effective mechanism for summing the maximum values of impulse flows exists in biology, physics, and mathematics. There is also no other more effective medium of information in neurobiology, except for pulsed flows. There is a natural way to generate pulsed flows using an excitable tissue that responds to exceeding the threshold of excitation with one or a series of pulses. The combination of such excitations forms a pulsed flow.

Engineers, specialists, scientists know that binary sequences are good for analyzing the operation of neural networks on high-speed computers, but are categorically unsuitable as information carriers in neurobiology.

Based on experimental facts in neurobiology, the same value of impulse flows coming directly from dendrites and impulse flows coming through synapses has been established to a first approximation. It has been established that the maximum value of the pulse flow based on the fundamental properties of almost periodic functions is due to the maximum value of the sum of the pulse flows coming directly from dendrites and through synapses. Naturally, the role of excitatory (positive) and inhibitory (negative) synapses is taken into account.

The mechanisms of inertial summation of impulse flows arriving in the neuron soma for the subsequent threshold excitation of a generator of individual impulses or sequences have invariance properties that make it possible to realize inertialess summation stably and reliably.

It is significant that the difference in the periods of the partial pulse flows arriving at the inputs, to a first approximation, does not affect the maximum value of their sum, which is automatically obtained due to the fundamental properties of almost-periodic functions. This is due to the fact that the maximum values of the almost-periodic function representing the resulting pulse flow do not depend on how much of the periods of the partial pulse flows that make up the entire pulse flow have certain periods. That is, the maximum value does not depend on the structure of the periods of the components of the pulse flow. This basic property of almost-periodic functions makes the mechanism of maximum value formation stable, and the system of inertialess summation based on the fundamental properties of almost-periodic functions reliable in real conditions. The conclusion is this: all components of partial flows can have either the same repetition period or different periods, in the extreme case, all partial impulse flows can have different repetition periods, forming a mosaic of periods. The total maximum value of the sum of the maximum values of the partial periods following through the ε-almost period will be the same, it is equal to

where A _i , B _j are the maximum amplitudes of the exciting and inhibitory partial pulse flows.

Let us consider in more detail the invariance properties of the inertialess summation of partial impulse flows. The invariance of the total impulse flow takes place in two main parameters of the impulse flow: in the mosaic of the periods of partial impulse flows and in the number of individual partial components in the total flow. We show it mathematically. The value of the maximum sum of the total pulse flow in the case of partial pulse flows with periods T ₁ , T ₂ , ..., T _l is

where N is the total number of partial pulse flows. The independence of the maximum amplitude of the total impulse flow from the mosaic of periods of partial impulse flows follows from the fundamental properties of almost-periodic functions, namely, ε-almost-period depends on partial periods, and A _max does not depend on ε-almost-period. This implies the invariance of A _max from the mosaic of periods of partial impulse flows.

Disclosure of the invention in a computational experiment

Consider the process of cluster formation in the proposed single-layer perceptron in a computational experiment. Consider 10 digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9. On a television screen with a 4 × 6 raster, there are numbers. For numbers, the Euclid system is taken. The Euclid system makes it possible to obtain an acceptable coding of numbers using the characters 0 and 1. The image of the numbers in the Euclid system is shown in FIG. 10.

FIG. 10. Image of numbers in the Euclid system.

The coding of numbers, that is, the values of the input signals - vectors x ₁ = (x _l1 , ..., x _ln ); ...; x _m = (x _m1 , ..., x _mn ), are given in table 1. In the future, we assume that the value of one corresponds to the period of the pulse sequence T ₁ = 0.8, the value zero - T ₂ = 1.2

Table of code combinations for the recognition of the numbers 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 when they are located in a two-dimensional 4 × 6 field.

Let us consider a single-layer perceptron, in which clusters tuned to the numbers 2 and 8 are formed in 2 neurons at the output. A more detailed description of the formation of clusters is given below. Cluster formed neurons are shown in FIG. 11. On the left side of the image of neurons, binary code combinations of numbers 2 and 8 are shown, in accordance with which a cluster of communication channels of each neuron is formed. Each black cell reflects a partial impulse flow with a certain repetition period. The black cell population reflects the resulting impulse flow.

FIG. 11. An example of the proposed impulse neurons having formed clusters according to the code combinations of the numbers 2 - from above, according to the code combinations of the numbers 8 - from the bottom. Each black cell reflects a partial impulse flow with a certain repetition period. The black cell population reflects the resulting impulse flow. White color corresponds to the absence of partial pulse flows.

We believe that partial impulse flows with periods of 0.8 correspond to black, the number of these flows is 7, and impulse flows with periods of 1.2, the number of these flows is 7. The total number of flows is 14, the amplitude of each of the partial flows is A = 1. The interaction of the pulse sequences with periods T ₁ = 0.8 and T ₂ = 1.2 is illustrated in FIG. 12 on top. In FIG. 12 below shows the interaction of pulse sequences with periods T ₁ = 0.9 and T ₂ = 1.2.

FIG. 12. Illustration of the invariance of inertialess summation, based on the fundamental properties of almost-periodic functions, with respect to the periods of partial impulse flows. It is shown in the figure above that the interaction of 7 pulse sequences with periods T ₁ = 0.8 and amplitudes A = 1 and 7 pulse sequences with periods T ₂ = 1.2 and amplitudes A = 1 ε-almost-period is 2.4. In FIG. 12 below shows the interaction of 7 pulse sequences with periods T ₁ = 0.9 and amplitudes A = 1 and 7 pulse sequences with periods T ₂ = 1.2 and amplitudes A = 1; ε-almost-period is 3.6.

FIG. 12 illustrates the invariance properties of inertialess summation, based on the fundamental properties of almost-periodic functions, with respect to the periods of partial impulse flows. It is seen that the maximum value of the sum of the partial pulse flows remains unchanged and equal to 14.

The invariance of inertialess summation, based on the fundamental properties of almost-periodic functions, with respect to the number of pulses in partial pulse flows is illustrated in FIG. 13.

FIG. 13. Illustration of the invariance of inertialess summation, based on the fundamental properties of almost-periodic functions, with respect to the number of pulses in partial pulse flows. On Fig left shows the interaction of 7 pulse sequences with periods T ₁ = 0.8 and amplitudes A = 1 and 7 pulse sequences with periods T ₂ -1.2 and amplitudes A = 1. ε-almost-period is 2.4. Pulse flows with a period of 0.8 are shown by a dashed line, pulsed flows with a period of 1.2 are shown by a solid line. On Fig to the right shows the interaction of 9 pulse sequences with periods T ₁ = 0.8 and amplitudes A = 1 and 5 pulse sequences with periods T ₂ = 1.2 and amplitudes A = 1. ε-almost-period is 2.4.

We believe that the total number of pulses characterizing the input pattern is 14. Pulses with a shorter repetition period are shown by a dashed line. On Fig left shows the interaction of 7 pulse sequences with periods T ₁ = 0.8 and amplitudes A = 1 and 7 pulse sequences with periods T ₂ = 1.2 and amplitudes A = 1. ε-almost-period is 2.4. On Fig to the right shows the interaction of 9 pulse sequences with periods T ₁ = 0.8 and amplitudes A = 1 and 5 pulse sequences with periods T ₂ = 1.2 with amplitudes A = 1. ε-almost-period is 2.4. It is seen that the maximum value of the sum of the partial pulse flows remains unchanged and equal to 14.

In conclusion, let us consider an example of how the recognition of the digit 2 of 10 possible digits by a pulsed neuron, the cluster of communication channels of which is configured to number 2. A pulsed neuron, a cluster of communication channels of which is configured to number 2, responds to figure 2, as shown in FIG. 12 on top. In this case, 7 pulse sequences interact with periods of T ₁ = 0.8 and amplitudes A = 1 and 7 pulse sequences with periods of T ₂ = 1.2 and amplitudes A = 1, and the ε-near-period is 2.4. Note that for code combinations of digits 2 and digits 8, the relations

Thus, if the threshold is U _pore = 14, then the neuron whose cluster is formed by the code combinations of the number 2 identifies this number and does not respond to the code combination of the number 8, since S ₂₈ <S ₂₂ or 9 <10. On the other hand, when a neuron tuned to number 2 reacts to this number, other neurons, the total number of which is 10, do not respond to this number, since for them S _2i <14 (i ≠ 2). We have given the simplest example of using the proposed single-layer perceptron for digit recognition. The proposed single-layer perceptron was used to solve more complex problems, for example, for recognizing the letters of the Russian alphabet, for recognizing words, for recognizing phrases. The proposed single-layer perceptron can also be used to solve various other applied problems.

Conclusion

We can conclude that two mechanisms for processing pulse flows based on controlled clustering of input communication channels and on the basis of the fundamental properties of almost periodic functions allow us to generally implement the basic functions of a single-layer perceptron: 1) selectivity to certain properties or patterns of the input object; 2) the selective distribution of patterns across the channels at the output of the perceptron or what is called coding the channel number. The proposed single-layer perceptron allows the inertia-free, stable, independent of the periods of partial pulsed flows summation of the amplitudes of the partial pulsed flows and the formation of maximum amplitudes following the almost-periods of the almost-periodic function, since the resulting pulsed flow is an almost-periodic function.

 LITERATURE

1. Alexandrov Yu.I., Anokhin K.V., Sokolov E.N., Grechenko T.N. et al. Neuron. Signal processing. Plastic. Modeling. Fundamental leadership // Publishing house of the Tyumen State University. 2008.548 s.

2. Anokhin K.V., Burtsev M.S., Ilyin V.A., Kiselev I.I., Kukin K.A., Lakhman K.V., Paraskevov A.V., Rybka RB, Sboev A.G., Tverdokhlebov N.V. Modern approaches to modeling the activity of cultures of neurons in vitro / Mathematical biology and bioinformatics. 2012.V. 7. №2. S. 372-397.

3. Borisov VV Model of a neuron. Patent No. RU 2034332, G06G 7/60 1995.

4. Byzov A.L. Electrophysiological studies of the retina. - M .: Science 1966, 196 p.

5. Wasserman F. Neurocomputer technology: Theory and practice // M .: "World", 1992.

6. Gutman A. Nerve cell dendrites: theory of electrophysiology, function. 1984

7. Kazantsev V. B., Nekorkin, V. I., Velarde M. G. A model of a neuron with oscillatory activity below the threshold of excitation // News of universities. Radiophysics. 1998. No. 41.S. 1623-1635.

8. Levitan B.M. Almost-periodic functions.// GITTL. 1953,396 p.

9. Lebedev A.N. Psychophysiology of memory. // Psychophysiology. 2nd edition / ed. Yu.I. Alexandrova / M.: Infra-M, 2001. S. 128-141.

10. Mazurov M.E. Solution of Diophantine inequalities in the problems of synchronization of relaxation oscillations // ZhVM and MF AN SSSR. 1991.Vol. 31. No. 11. S. 1619-1636.

11. Mazurov M. E. Identification of mathematical models of nonlinear dynamical systems // M .: - Izhevsk. RHD. 2008.284 s.

12. McCallock W., Pitts V. The logical calculus of ideas related to nervous activity // Neurocomputer. -1992. - No. 3/4. - S. 40-50.

13. Markin V.S., Pastushenko V.F., Chizmadzhev Yu.A. Theory of Excitable Media // M .: Science. 1981. 276 p.

14. Minsky M., Peypert S. Perceptrons = Perceptions. - M.: “World”, 1971.

15. Nekorkin, V.I., Dmitrichev, A.S., Shchapin, D.S., Kazantsev V.B. Dynamics of a neuron model with complex threshold excitation // Mathematical Modeling 2005. No. 6. S. 75-91.

16. Prokin I.S., Kazantsev V.B. Analysis of the transmission of pulsed signals in a system of interacting neural generators with frequency-dependent connections // News of universities. Radiophysics. 2011. V. 54, No. 11. S. 848-858.

17. Rosenblatt F. Principles of neurodynamics: perceptrons and theory of brain mechanisms. Principles of Neurodynamic: perceptions and the theory of brain mechanisms. M: The World, 1965.

18. Samoilin E. A. The method of modeling a neuron. Patent RU 2402813, G06N 3/06 -2010.

19. Smolyaninov VV On some features of the organization of the cerebellar cortex. An article in the collection "Models of the structural and functional organization of some biological systems" // M .: Nauka, 1966, p. 203-262.

20. Smolyaninov V.V. Mathematical models of biological tissues. // M: Nauka, 1980, 368 pp.

21. Sokolov E.N. The principle of vector coding in psychophysiology. Bulletin of Moscow State University. Series 14. Psychology, 1995, No. 4.

22. Sokolov E.N., Vaitkevichus G.G. Neurointelligence: from a neuron to a neurocomputer. M .: Nauka, 1989.

23. Sokolov E.N. Neural mechanisms of memory and learning. M., 1981.

24. Sokolov E.N. Perception and conditioned reflex: a new look. M., 2003.

25. Subbotin S.A. Analytical review of inventions and patents. Neurocybernetics in the USSR-CIS. Zaporizhzhya National Technical University.

26. Tikiji-Khamburyan R.A. Analysis of the influence of the spike generator on the dynamic properties of a modified impulse neuron // Proc. doc 5th All-Russian Scientific and Technical Conference Neuroinformatics. 2003, M .: MEPhI, 2003, p. 134-141.

27. Tikiji-Khamburyan R.A. Modified impulse neuron as a basic model for realistic neural networks // Neurocomputers: development and application, No. 7, 2002.

28. Khaikin Simon. Neural Networks: Full Course = Neural Networks: A Comprehensive Foundation. - 2nd ed. - M .: "William", 2006. - S. 1104. - ISBN 0-13-273350-1

29. Shepard G. Neurobiology. In 2 T.T. 1. Trans. from English // M .: Mir, 1987.445 s.

30. Fitzhugh R. Impulses and Physiological States in Theoretical Models of Nerve Membrane. // Biophysical journal. 1961. V. 1, No. 6. P. 445-466.

31. Hindmarsh J.L., Rose R.M. A model of the nerve impulse using two first-order differential equations. // Nature. Nature Publishing Group, 1982. V. 296, No. 5853. P. 162-164.

32. Hodgkin A.L., Huxley A.F. A quantitative description of membrane current and its application to conduction and excitation in nerve // The J. of physiology. 1952. V. 117, No. 4. P. 500-544.

33. Izhikevich E.M. Simple model of spiking neurons. // IEEE transactions on neural networks / a publication of the IEEE Neural Networks Council. 2003. V. 14, No. 6. P. 1569-1572.

34. Izhikevich E.M., FitzHugh R. FitzHugh-Nagumo model // Scholarpedia, 3193, 2006.

des Nerfs

comme une Polarisation [Quantitative studies on electric excitation of nerves treated as polarization] // Journal de Physiologie et Pathologie General. 1907. V. 9. P. 620-635.35. Lapicque ML Recherches quantitatives sur l′excitation

des nerfs

comme une Polarisation [Quantitative studies on electric excitation of nerves treated as polarization] // Journal de Physiologie et Pathologie General. 1907. V. 9. P. 620-635.

36. Morris C., Lecar H. Voltage oscillations in the barnacle giant muscle fiber. // Biophysical journal. 1981. V. 35, No. 1. P. 193-213.

Claims (1)

A single-layer perceptron, including several recording impulse neurons, inputs for input signals - impulse flows, internal communication channels from inputs, in which the input signals are multiplied by weights and sent to adders associated with threshold nonlinear excitable elements generating single output pulses or pulse sequences when the signals at the outputs of the adders exceed the threshold value of the excitable nonlinear element, which are used as output s persecutors; characterized by individual clusters from part of the communication channels in each of the recording neurons, which pass part of the input signals; the formation of clusters and equalization of the number of channels in the clusters is carried out in accordance with code combinations and the structure of the periods of the partial components of the input pulse flows; then they produce inertialess summation of all partial pulse flows at the outputs of the clusters; this implements for a signal in the form of total pulse flows at the outputs of clusters of ε-near-periods with a maximum sum of amplitudes of all partial pulse flows; after inertialess summation, the received signal is converted by threshold excitable elements that generate single pulses or pulse sequences when the maximum sum of the amplitudes of pulse flows exceeds the threshold value of excitable elements that are used as output signals.

Images