CN110991602A - Event-driven pulse neuron simulation algorithm based on single exponential kernel - Google Patents
Event-driven pulse neuron simulation algorithm based on single exponential kernel Download PDFInfo
- Publication number
- CN110991602A CN110991602A CN201910845258.4A CN201910845258A CN110991602A CN 110991602 A CN110991602 A CN 110991602A CN 201910845258 A CN201910845258 A CN 201910845258A CN 110991602 A CN110991602 A CN 110991602A
- Authority
- CN
- China
- Prior art keywords
- neuron
- pulse
- time
- current
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000002569 neuron Anatomy 0.000 title claims abstract description 88
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 22
- 239000012528 membrane Substances 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 16
- 210000005215 presynaptic neuron Anatomy 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 9
- 230000000946 synaptic effect Effects 0.000 claims description 6
- 238000012421 spiking Methods 0.000 claims description 3
- 210000000225 synapse Anatomy 0.000 claims description 3
- 230000001537 neural effect Effects 0.000 claims description 2
- 238000000034 method Methods 0.000 abstract description 19
- 238000004364 calculation method Methods 0.000 abstract description 5
- 210000004556 brain Anatomy 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000010304 firing Methods 0.000 description 2
- 230000003518 presynaptic effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003930 cognitive ability Effects 0.000 description 1
- 230000036992 cognitive tasks Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Abstract
The invention provides an event-driven pulse neuron simulation algorithm based on a single exponential core. Compared with the traditional pulse neuron model, the method disclosed by the invention uses the single-exponential kernel function, the efficiency of the pulse neuron model is greatly improved, the delay degree of the pulse neuron model to input information is reduced, the identification accuracy and the robustness of the pulse neuron model are improved, and the method is more suitable for developing and applying a software and hardware platform of a brain-simulated architecture. Meanwhile, an event-driven-based method is adopted, wherein the calculation is driven by pulses, and the method can efficiently process the pulse mode.
Description
Technical Field
The invention belongs to the field of brain-like computation and neuron models, in particular relates to a technology for improving the computation performance of a pulse neuron model, and particularly relates to an event-driven pulse neuron simulation algorithm based on a single-index kernel.
Background
The human brain shows remarkable ability in various cognitive tasks such as recognition, decision, learning and memory, and consumes very low resources. The remarkable performance of the human brain has stimulated an increasing number of researchers to try to understand its principles of operation and hopefully apply these principles to artificial intelligence systems with similar capabilities to the human brain for processing information.
Driven by deep learning techniques, artificial neural networks have enjoyed great success in solving problems in various fields, including image and speech recognition, natural language processing, autopilot, and the like. Despite the great success of deep learning, biological rationality is still lacking. Deep artificial neural networks rely on specialized graphics processing units and supercomputers, and running these networks on low-power devices is nearly impossible. There is still a great gap in the efficiency of deep learning compared to the brain's superior cognitive abilities. Therefore, efficient and biologically rational neuron models are urgently needed to be developed.
In the nervous system, neurons communicate by impulses. In order to simulate the way that the human brain processes information through pulses, researchers develop a pulse neuron model with more biological reasonableness, and the model is called as a third-generation artificial neural network. Among all the impulse neuron models, the leaky integrated discharge model, Leak Integrated and Fire (LIF) and impulse response model, Spike Response Model (SRM), are the most commonly used impulse neuron models due to their simple and easy-to-implement forms.
Disclosure of Invention
The invention provides an event-driven pulse neuron simulation algorithm based on a single exponential kernel, aiming at overcoming the defects of the prior art. Based on the single exponential kernel function, a more efficient LIF pulse neuron model with biological rationality is provided. Compared with the traditional LIF pulse neuron model based on the bi-exponential kernel function, the method has the advantages that the efficiency of the pulse neuron model is greatly improved, the delay degree of the pulse neuron model to input information is reduced, the identification accuracy and the robustness of the pulse neuron model are improved, and the method is more suitable for development and application of a software and hardware platform of a brain-simulated architecture. Meanwhile, an event-driven-based method is adopted, wherein the calculation is driven by pulses, and the method can efficiently process the pulse mode.
The technical scheme of the invention is as follows: the invention specifically comprises the following steps:
1) initializing model parameters;
2) inputting a pulse space-time pattern diagram;
3) initializing neuron weight;
4) calculating V (t);
5) and (5) learning and adjusting.
The present invention performs integrated performance comparison.
The invention carries out efficiency comparison.
The event-driven pulse neuron simulation algorithm based on the single exponential kernel comprises the following neuron models:
τ represents the time constant of the neuronal membrane potential;
Iinand IoutRespectively representing the input current of a presynaptic neuron and the reset current after the neuron transmits a pulse, wherein the neuron has corresponding reset dynamic response every time after the neuron transmits the pulse;
Iinand IoutIs defined as follows:
δ (t) is a unit pulse function, and its value is 1 only when t is 0, and the values at other times are all 0;
n and wiRepresenting the number of pre-synaptic neurons and corresponding synaptic weights;
θ represents a threshold of the neuron, the neuron emitting a pulse only if its membrane potential is greater than the threshold;
the above formula shows that the pulse received by the current neuron comes from the presynaptic neuron or the output of the current neuron, and the model in the form of SRM can be obtained by integrating the formula (1-3):
the formula shows that the current neuron integrates the input current to obtain a membrane potential V (t); wherein
Is a single exponential kernel function, which is obtained by the formula (1) and represents the influence of the input pulse on the membrane potential through an integral formula
Two kernels can be approximately equivalent, where k (t) is a bi-exponential kernel function in the conventional spiking neuron model, which is formulated as follows:
V0is a constant factor used to normalize k (t);
τmtime constant, τ, representing the membrane potentialsRepresenting the time constant of the synaptic current.
Advantageous effects
The invention provides a more efficient LIF pulse neuron model with biological rationality based on a single exponential kernel function. Compared with the traditional LIF pulse neuron model based on the bi-exponential kernel function, the method has the advantages that the efficiency of the pulse neuron model is greatly improved, the delay degree of the pulse neuron model to input information is reduced, the identification accuracy and the robustness of the pulse neuron model are improved, and the method is more suitable for development and application of a software and hardware platform of a brain-simulated architecture.
Drawings
FIG. 1.A is the presynaptic input current IinAn example of (t) involves 10 pre-synaptic neurons firing pulses (each pulse represented by a dot) in a time series;
FIG. 1.B shows the weights w of presynaptic neuronsi；
FIG. 1.C shows the locus of the neuron model of the present invention after integration of the input current of FIG. A;
FIG. 1.D is a kernel function used in the present invention
FIG. 2 is a kernel function
V (t) from different values of τ, compared to the membrane potential obtained from a conventional neuron model, where neurons are assigned ultra-high membrane potential thresholds. The intermediate overlapping gray curves represent V, τ from a conventional neuron modelmAnd τs20ms and 5ms respectively; other curves are obtained from the neuron model of the present invention, and three curves are obtained from 1.5 τ, and 0.5 τ, respectively, from top to bottom.
Fig. 3 is a comparison of the computational efficiency of the event-driven single-exponential core and the double-exponential core, where the vertical axis represents the time required for integrating the input pulse space-time diagram cpu, and the horizontal axis represents the number of input pulse space-time diagram patterns.
Detailed Description
The use of the invention is explained in detail below with reference to the drawings.
The invention provides a more efficient LIF pulse neuron model with biological rationality based on a single exponential kernel function. The neuron model is shown below.
τ represents the time constant of the neuron membrane potential. I isinAnd IoutRespectively representing the input current of a pre-synaptic neuron and the reset current after the neuron emits a pulse, and the neuron has corresponding reset dynamic response after each pulse emitted by the neuron. I isinAnd IoutIs defined as follows.
δ (t) is a unit pulse function, and its value is 1 only when t is 0, and all other time values are 0.
Is the time to reach the jth pulse of the ith synapse,
representing the time of the jth output pulse of the current neuron. N and wiRepresenting the number of pre-synaptic neurons and the corresponding synaptic weights. θ represents the threshold of the neuron, which will emit a pulse only if the membrane potential of the neuron is greater than the threshold. The above formula shows that the pulse received by the current neuron is from the presynaptic neuron or its own output. By integrating the formulas (1-3), the model in the form of SRM can be obtained.
The above formula shows that the current neuron integrates the input current to obtain the membrane potential V (t). Wherein
Is a single exponential kernel function, which is derived from equation (1) and represents the effect of the input pulse on the membrane potential. By integral formula
Two kernels can be approximately equivalent, where k (t) is a bi-exponential kernel function in a conventional spiking neuron model, which is formulated as follows.
V0Is a constant factor used to normalize k (t). Tau ismTime constant, τ, representing the membrane potentialsThe time constants, which represent the synaptic currents, are set to 20ms and 5ms in the present invention.
In order to better simulate the way that neurons efficiently process the pulse space-time diagram, the invention uses a neuron simulation algorithm based on event driving. This algorithm is superior to algorithms based on time steps. Firstly, the algorithm does not depend on the input time step, so that the response of the neuron can be calculated more accurately, for example, the time when the neuron outputs the pulse can be 14.35ms by using the analysis method in the invention, but the time when the time step is 1ms is not possible. Secondly, if the number of input pulses is N, the time complexity based on the event-driven algorithm is linearly related to the number of input pulses, i.e., O (N), and the time complexity based on the time-step algorithm is O (N × T/dt), assuming that typical values of T ═ 1s and dt ═ 1ms are selected, the efficiency of the algorithm of the present invention is 3 orders of magnitude faster than the algorithm based on the time-step. Therefore, the invention adopts a calculation method based on event driving.
Since the kernel function has almost no delay to the input current, that is, the local maximum of the membrane potential can only occur at the moment when there is a pulse input, when calculating the membrane potential, there is no need to set a time step to calculate v (t), only v (t) at the moment when there is an input pulse needs to be calculated, i.e., the membrane potential calculating method using an event as a drive. At the same time, adopt the eventThe driving method can obtain an accurate solution without being limited by the time resolution. Suppose there is an input pulse train with time t1≤t2…≤tnThe corresponding pre-synaptic neuron weight is w1,w2…wnThen the membrane potential of the neuron can be written as:
V(tk)＝V(tk-1)exp((-Δk-1/τ))+wk(6)
Δk-1＝tk-tk-1representing the kth interval of the input pulse. Thus, a v (t) is obtained, and the subsequent learning adjustment step is performed.
The following algorithm flow describes a single exponential core based event-driven impulse neuron simulation algorithm:
(1) initializing model parameters
The model parameters are first initialized before using the present invention. All subsequent text is based on τm20ms and τsThe explanation is given for 5 ms.
(2) Input pulse space-time pattern diagram
Using the present invention requires providing a pulse spatiotemporal pattern diagram, as illustrated in FIG. 1.A, with presynaptic neuron coding on the vertical axis and pulse firing time on the horizontal axis. In practical applications, the source input needs to be encoded first using a pulse encoding algorithm before subsequent operations can be performed with the present invention.
(3) Initializing neuron weights
Taking fig. 1.B as an example, the weights of the neurons need to be initialized first, so that the subsequent calculation operation can be performed.
(4) Calculation V (t)
The input current is integrated according to the formula (6), and the membrane potential V (t), i.e. the response locus of the pulse space-time pattern diagram of the invention to the input, is calculated, as shown in FIG. 1. C.
(5) Learning adjustment
Due to the response V (t) of the neuron by the weight wiAnd determining that if the neuron is expected to have different output responses to different inputs, selecting a proper learning algorithm to adjust and learn the weight of the neuron.
(6) Integrated performance comparison
Comparing the present invention with conventional neurons, as shown in fig. 2, different values of τ have a greater difference in performance. Under the conditions of improving the efficiency of the pulse model and reducing the delay degree of the model to input information, the track of V (t) is similar to that of the traditional model, and the effectiveness of the method is shown.
(7) Comparison of efficiency
The neuron integration efficiency of the event-driven single-exponential kernel and the neuron integration efficiency of the double-exponential kernel are compared, as shown in fig. 3, the vertical axis is cpu time required to operate when the input pulse space-time diagram is calculated and processed, and the horizontal axis is the number of the input pulse space-time diagrams (patterns), so that the efficiency is doubled compared with that of the traditional method. This is because when the neuron model of the bi-exponential core is computed in an event-driven manner, it needs to perform a complicated computation to solve the precise time for issuing a pulse by predicting whether the membrane potential will exceed the threshold value according to the current state, whereas for the neuron model of the mono-exponential core, since there is no time delay between the input pulse and the issuing pulse, when the membrane potential exceeds the threshold value, the time point at this time is the precise time for issuing a pulse. In addition, the single-index model is simple and efficient, and is more beneficial to implementation on software and hardware platforms.
Claims (4)
1. The event-driven pulse neuron simulation algorithm based on the single exponential kernel is characterized by comprising the following steps of:
1) initializing model parameters;
2) inputting a pulse space-time pattern diagram;
3) initializing neuron weight;
4) calculating V (t);
5) and (5) learning and adjusting.
2. The single exponential-core based event-driven impulse neuron simulation algorithm of claim 1, wherein an integrated performance comparison is performed.
3. The single exponential-core based event-driven impulse neuron simulation algorithm of claim 1, wherein an efficiency comparison is performed.
4. The single exponential-core based event-driven impulse neuron simulation algorithm of any one of claims 1 to 3, wherein the neuron model is as follows:
τ represents the time constant of the neuronal membrane potential;
Iinand IoutRespectively representing the input current of a presynaptic neuron and the reset current after the neuron transmits a pulse, wherein the neuron has corresponding reset dynamic response every time after the neuron transmits the pulse;
Iinand IoutIs defined as follows:
δ (t) is a unit pulse function, and its value is 1 only when t is 0, and the values at other times are all 0;
n and wiRepresenting the number of pre-synaptic neurons and corresponding synaptic weights;
θ represents a threshold of the neuron, the neuron emitting a pulse only if its membrane potential is greater than the threshold;
the above formula shows that the pulse received by the current neuron comes from the presynaptic neuron or the output of the current neuron, and the model in the form of SRM can be obtained by integrating the formula (1-3):
the formula shows that the current neuron integrates the input current to obtain a membrane potential V (t); wherein
Is a single exponential kernel function, which is obtained by the formula (1) and represents the influence of the input pulse on the membrane potential through an integral formula
Two kernels can be approximately equivalent, where k (t) is a bi-exponential kernel function in the conventional spiking neuron model, which is formulated as follows:
V0is a constant factor used to normalize k (t);
τmtime constant, τ, representing the membrane potentialsRepresenting the time constant of the synaptic current.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910845258.4A CN110991602A (en) | 2019-09-08 | 2019-09-08 | Event-driven pulse neuron simulation algorithm based on single exponential kernel |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910845258.4A CN110991602A (en) | 2019-09-08 | 2019-09-08 | Event-driven pulse neuron simulation algorithm based on single exponential kernel |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110991602A true CN110991602A (en) | 2020-04-10 |
Family
ID=70081777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910845258.4A Pending CN110991602A (en) | 2019-09-08 | 2019-09-08 | Event-driven pulse neuron simulation algorithm based on single exponential kernel |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110991602A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022267385A1 (en) * | 2021-06-22 | 2022-12-29 | 中国科学院深圳先进技术研究院 | Neuronal signal processing method and processing apparatus, and readable storage medium |
| CN114723009B (en) * | 2022-04-12 | 2023-04-25 | 重庆大学 | Data representation method and system based on asynchronous event stream |
-
2019
- 2019-09-08 CN CN201910845258.4A patent/CN110991602A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022267385A1 (en) * | 2021-06-22 | 2022-12-29 | 中国科学院深圳先进技术研究院 | Neuronal signal processing method and processing apparatus, and readable storage medium |
| CN114723009B (en) * | 2022-04-12 | 2023-04-25 | 重庆大学 | Data representation method and system based on asynchronous event stream |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Tdsnn: From deep neural networks to deep spike neural networks with temporal-coding | |
| Lawrence et al. | What size neural network gives optimal generalization? Convergence properties of backpropagation | |
| Pham et al. | Control chart pattern recognition using spiking neural networks | |
| Zhang et al. | Supervised learning in spiking neural networks with noise-threshold | |
| Lin et al. | Supervised learning in multilayer spiking neural networks with inner products of spike trains | |
| Lin et al. | Relative ordering learning in spiking neural network for pattern recognition | |
| WO2018156314A1 (en) | Method and apparatus for multi-dimensional sequence prediction | |
| KR102011788B1 (en) | Visual Question Answering Apparatus Using Hierarchical Visual Feature and Method Thereof | |
| CN109102000A (en) | A kind of image-recognizing method extracted based on layered characteristic with multilayer impulsive neural networks | |
| CN110991602A (en) | Event-driven pulse neuron simulation algorithm based on single exponential kernel | |
| CN111310816B (en) | Method for recognizing brain-like architecture image based on unsupervised matching tracking coding | |
| Ahmed et al. | Computing with spiking neuron networks: a review | |
| Yu et al. | Improving multispike learning with plastic synaptic delays | |
| Coenen et al. | Parallel fiber coding in the cerebellum for life-long learning | |
| Li et al. | Pattern recognition of spiking neural networks based on visual mechanism and supervised synaptic learning | |
| Guo et al. | An overhead-free max-pooling method for SNN | |
| Hamed et al. | An extended evolving spiking neural network model for spatio-temporal pattern classification | |
| Gardner et al. | Supervised learning with first-to-spike decoding in multilayer spiking neural networks | |
| Lagorce et al. | Event-based features for robotic vision | |
| CN114372563A (en) | Robot control method and system based on hybrid pulse reinforcement learning network structure | |
| CN114952791A (en) | Control method and device for musculoskeletal robot | |
| CN110717590A (en) | Efficient multi-pulse algorithm based on single-exponential kernel and neural membrane potential states | |
| CN115880324A (en) | Battlefield target image threshold segmentation method based on pulse convolution neural network | |
| Humaidi et al. | Recognition of arabic characters using spiking neural networks | |
| KR20220075025A (en) | Neuromorphic computing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200410 |
|
| WD01 | Invention patent application deemed withdrawn after publication |