CN106779056B - Spiking neuron hardware structure for AER feed forward classification system - Google Patents
Spiking neuron hardware structure for AER feed forward classification system Download PDFInfo
- Publication number
- CN106779056B CN106779056B CN201611190874.3A CN201611190874A CN106779056B CN 106779056 B CN106779056 B CN 106779056B CN 201611190874 A CN201611190874 A CN 201611190874A CN 106779056 B CN106779056 B CN 106779056B
- Authority
- CN
- China
- Prior art keywords
- weight
- psp
- clk
- aer
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Neurology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to AER imaging sensor field of image recognition, to meet the event-driven feed forward classification system based on AER imaging sensor for real-time, parallel processing and the requirement of miniaturization, the present invention is intended to provide a kind of SNN spiking neuron hardware realization structure based on FPGA platform.The technical solution adopted by the present invention is that, spiking neuron hardware structure for AER feed forward classification system, by weight storage area, weight read-write gate, membrane potential parallel multiplication, the PSP function generator of more vairable, triggering determining device, control and timing and modified weight module, amounts to 7 parts and constitute.Present invention is mainly applied to imaging sensor image recognition occasions.
Description
Technical field
The present invention relates to AER imaging sensor field of image recognition, more particularly to one kind is for AER image recognition classification arteries and veins
Rush the neuron hardware realization in neural network.
Background technique
AER (Address-Event Representation, AER, address-event expression) imaging sensor real-time detection
The variation of target scene can filter out static background picture compared to the conventional image sensor based on " frame scan " imaging mode
Prime number evidence, is greatly reduced redundant data.AER only exports the asynchronous digital flow of event of relevant information, this makes subsequent processing system
System can be designed as complete event-driven.
Event-driven feed forward classification system based on AER imaging sensor, it is similar to other categorizing systems, two can be divided into
Point: feature extraction and classification.Characteristic extraction part uses Gabor anisotropic filter and maximum operation, and classifies and then use arteries and veins
Rush neural network (Spiking Neuron Networks, SNN).First with the difference hardware in AER pixel, reach threshold value
When output address flow of event, and be advanced into the anisotropic filter based on Gabor function, output includes the convolution knot of characteristic information
Fruit completes feature extraction.Each output includes appropriate address coordinate, within the scope of specified acceptance region, utilizes maximum operation
Non-significant information is filtered out, then by TFS (Time-to-first spike) unit, impulse response is converted to, is supplied to SNN
Classify.
For the requirement of AER output information and categorizing system, the LIF (Leakage in the numerous models of SNN is selected
Integrate-and-fire) model.The input of LIF generates postsynaptic potential (postsynaptic by TFS
Potential, PSP), with rapid increase and the shape slowly declined.It is strong that with weight cynapse is represented between each neuron
Degree, the membrane potential of neuron are superimposed to obtain by the weighted sum of the PSP of all input pulses.By being compared with threshold value, determine
Whether neuron triggers, meet classification task of the real world based on stimulation.If SNN training stage neuron should trigger,
And it is practically without triggering, then the weight at membrane potential peak value (tmax) can be improved.If should not trigger, and actually touch
Hair, then can reduce the weight at membrane potential peak value.If errorless, amendment is not had to.When test, by the touching for counting corresponding neuron
Hair-like state realizes classification.
The SNN hardware configuration of the categorizing system is as shown in Figure 1.Hardware configuration includes feature input, pulse coder, control
It is exported with timing, spiking neuron and triggering state, amounts to 5 parts.Input comprising characteristic information enters pulse code
Device, into spiking neuron, carries out cumulative wait of corresponding weight and operates, then carry out threshold value under the action of control and timing
Compare, judges the output of triggering state.The feed forward classification system of AER imaging sensor, at the information of AER chip output
Reason completes feature extraction by Gabor convolution sum maximum operation and reduces non-significant feature, and output information enters using LIF
The SNN network of neuron, wherein including address information and temporal information, address information can know the location of pixels that event generates
And respective filter type, temporal information can know the time that event occurs.Pulse coder working principle are as follows: if net
The input quantity of network is N, and a pulse can be regularly activated when inputting effective, does not generate pulse when without effectively input.
SNN algorithm relies primarily on software realization at present, and speed is slow, and degree of concurrence is low, can not be handled in real time.And it is soft
Part needs biggish computer supported also to cannot achieve the miniaturization of equipment other than higher cost.AER is based in order to meet
Requirement of the event-driven feed forward classification system of imaging sensor to miniaturization and real-time, it is real for used SNN hardware
Now become very necessary, and most important in SNN hardware is exactly high number, the spiking neuron with reproducibility.Scene
Programmable gate array (Field-Programmable Gate Array, FPGA) holds very much in the development for having obtained high speed in recent years
Easily realize high speed, parallel computation.It realizes that the spiking neuron of SNN has the performance of the categorizing system using FPGA extremely to weigh
The meaning wanted.
Summary of the invention
In order to overcome the deficiencies of the prior art, meet the event-driven feed forward classification system based on AER imaging sensor for
Real-time, parallel processing and the requirement of miniaturization, the present invention is intended to provide a kind of SNN spiking neuron based on FPGA platform is hard
Part realizes structure.The technical solution adopted by the present invention is that the spiking neuron hardware structure for AER feed forward classification system, by
Weight storage area, weight read-write gate, membrane potential parallel multiplication, the PSP function generator of more vairable, triggering determining device, control and when
Sequence and modified weight module amount to 7 parts and constitute, and each section is specific as follows:
(1) control and timing and modified weight module: for providing two internal clockings, CLK and SLOW_CLK, wherein
The CLK clock cycle is much smaller than SLOW_CLK, and proportional region is 1:10000 to 1:500000, for pulse code sampling, membrane potential
Compare, determines the speed of network processes；SLOW_CLK is used to coordinate the work of inside neurons, in less than one SLOW_CLK
Period in can obtain the membrane potential accumulation results of all inputs；RST signal is additionally needed for resetting zero setting and RE
Read signal and WE write signal are read and write for weight；
(2) weight reads and writes gate: the mode of operation by controlling signal behavior weight, in read phase, connection weight storage
Storage provides weight appropriate, and in write phase, connection to input terminal realizes right value update by outside port；
(3) weight reservoir: it is used to store weight in inside neurons, each neuron weight reservoir space is input
Quantity is multiplied by data width；
(4) the PSP function generator of more vairable: PSP is with the shape for rising rapidly and slowly declining, and (1) is PSP function formula:
Wherein, K is PSP functional value, and V0 control rises maximum value, when τ m and τ s is the control raising and lowering gradient here
Between constant, two constant relationships be τ s=τ m/4, PSP waveform continue 30 SLOW_CLK periods altogether, can be used 30 according to formula
A discrete point stores corresponding PSP functional value, forms the look-up table of 30 data widths；
(5) membrane potential parallel multiplication: including a multiplier and an adder, in each SLOW_CLK period, root
It is weighted according to formula (2), wherein n is the quantity of all inputs in accumulation period, and Δ t is current time and last pulse
Time difference, by the direct evaluation of PSP look-up table, ω is corresponding weight, read by weight reservoir:
(6) trigger determining device: in each clk cycle, the result of membrane potential parallel multiplication all can be with membrane potential threshold value
It is compared, if threshold value reaches, generates triggering state, otherwise do not trigger；Triggering determining device includes a comparator, is generated
The output of SNN triggering state；
(7) when for the study of SNN network training, amendment modified weight module: is carried out to the neuron weight of erroneous trigger.
SNN network training rule is as follows: should trigger and not trigger or should not trigger and trigger, read the PSP function generator of more vairable
Response numerical value is updated multiplied by the former weight position that is added to after learning rate；It is real using internal PSP generator and multiplier
Existing modified weight operation.
7 part processes are as follows: first by read-write control signal, come the update and reading of weight；Read the stage,
Namely test phase obtains weight from weight storage area according to the address of input, in addition obtains PSP from the PSP function generator of more vairable
Functional value enters in membrane potential parallel multiplication jointly, is weighted；Calculated result can be with touching in each clock cycle
Hair threshold value is compared, and the triggering state of neuron is judged using comparator；In the study stage, repaired if necessary to carry out weight
Just, read-write control signal is in write phase at this time, using weight correction module, reads the response numerical value of the PSP function generator of more vairable, right
The weight of weight storage element is modified.
In the integrated programmable gate array FPGA on site in 7 parts, weight storage device is comprehensive at storage RAM.
The features of the present invention and beneficial effect are:
The present invention is based on the hardware configurations that FPGA platform realizes SNN spiking neuron, meet based on AER image sensing
The event-driven feed forward classification system software of device support under speed is slow, the disadvantages such as degree of parallelism is low, at high cost, volume is big, favorably
In the miniaturization and real-time of equipment.
Detailed description of the invention:
Fig. 1 SNN network hardware architecture.
Fig. 2 neuron hardware configuration.
Tetra- input pulse of Fig. 3 coding.
Specific embodiment
Neuron hardware configuration proposed by the present invention for categorizing system SNN is as shown in Figure 2.Spiking neuron can have
Body is divided into weight storage area, weight read-write gate, membrane potential parallel multiplication, the PSP function generator of more vairable, triggering determining device, control
System and timing and modified weight module amount to 7 parts, and functions are as follows:
(1) it control and timing: in order to meet the coordination of inside neurons and the speed of network processes, needs using in two
Portion's clock, CLK and SLOW_CLK, wherein the CLK clock cycle be much smaller than SLOW_CLK, with sampled with pulse code, membrane potential ratio
Compared with determining the speed of network processes.SLOW_CLK is used to coordinate the work of inside neurons, less than SLOW_CLK's
The membrane potential accumulation result of all inputs is obtained in period.RST signal is additionally needed for resetting zero setting and RE
It is read and write with WE for weight.
(2) weight reads and writes gate: the mode of operation by controlling signal behavior weight.In read phase, connection weight storage
Storage provides weight appropriate, and in write phase, connection to input terminal realizes right value update by outside port.
(3) weight reservoir: in inside neurons for storing weight.Each neuron weight reservoir space is input
Quantity is multiplied by data width.Storage RAM is integrated into FPGA.
(4) the PSP function generator of more vairable: PSP is with the shape for rising rapidly and slowly declining, and (1) is PSP function formula:
Wherein, K is PSP functional value, and V0 control rises maximum value, when τ m and τ s is the control raising and lowering gradient here
Between constant, two constant relationships be τ s=τ m/4, PSP waveform continue 30 SLOW_CLK periods altogether, can be used 30 according to formula
A discrete point stores corresponding PSP functional value, forms the look-up table of 30 data widths.
(5) membrane potential parallel multiplication: including a multiplier and an adder, in each SLOW_CLK period, root
It is weighted according to formula (2), wherein Δ t is the time difference of current time and last pulse, can be direct by PSP look-up table
Evaluation, ω are corresponding weight, are read by weight reservoir.
(6) trigger determining device: in each clk cycle, the result of membrane potential parallel multiplication all can be with membrane potential threshold value
It is compared, if threshold value reaches, generates triggering state, otherwise do not trigger.Triggering determining device includes a comparator, is generated
The output of SNN triggering state.
(7) modified weight module: when for the study of SNN network training, carrying out amendment to the neuron weight of erroneous trigger,
Learning rules are as follows: should trigger and not trigger or should not trigger and trigger, read the PSP function generator of more vairable and respond numerical value, multiplied by
Be added to former weight position after habit rate, is updated.Modified weight behaviour is realized using internal PSP generator and multiplier
Make.
First by read-write control signal, come the update and reading of weight.In the stage of reading, that is, test rank
Section obtains weight from weight storage area according to the address of input, in addition generates unit from PSP function and obtain PSP functional value, jointly
Into in membrane potential parallel multiplication, it is weighted.Calculated result can be carried out in each clock cycle with activation threshold value
Compare, the triggering state of neuron is judged using comparator.In the study stage, if necessary to carry out modified weight, Read-write Catrol
Signal is in write phase at this time, using weight correction module, reads the response numerical value of the PSP function generator of more vairable, stores to weight single
The weight of member is modified.
The hardware configuration of the SNN spiking neuron based on FPGA proposed according to the present invention, can satisfy based on AER image
The requirement of the event-driven feed forward classification system of sensor.The parameters such as input and output quantity, sample clock frequency and data width
It is adjustable.PSP function durations are 30 SLOW_CLK, include 30 equidistant discrete points.The CLK clock cycle is much smaller than
The ratio of 1:10000 may be selected in SLOW_CLK.The pulse coder of SNN when input is 4 principle as shown in figure 3, its
In with white piece represent effectively input, black patch represents invalid input.Weight storage area size is related with input quantity and data width.
Since AER imaging sensor filters out the operation of non-significant feature in the advantage in redundant data of reducing and feed forward classification system,
Practical SNN input quantity can input quantity much smaller than required for full connection.Data width is determined by the required accuracy, is defaulted as
16 bit width.The present invention is based on FPGA hardware platforms, thus have extremely strong flexibility and low-cost advantage, pass through parameter
Modification, the spiking neuron structure of dedicated SNN neural network can be obtained, handled by parallel data, solve software processing
Platform low speed problem.
Claims (4)
1. a kind of spiking neuron hardware structure for AER feed forward classification system, characterized in that by weight storage area, weight
Read and write gate, membrane potential parallel multiplication, the PSP function generator of more vairable, triggering determining device, control and timing and modified weight mould
Block amounts to 7 parts and constitutes, and each section is specific as follows:
1) control and timing and modified weight module: for providing two internal clockings, CLK and SLOW_CLK, wherein when CLK
The clock period is less than SLOW_CLK, and proportional region is 1:10000 to 1:500000, compares for pulse code sampling, membrane potential, certainly
The speed of network processes is determined；SLOW_CLK is used to coordinate the work of inside neurons, in the period less than a SLOW_CLK
The interior membrane potential accumulation result that can obtain all inputs；RST signal is additionally needed for resetting zero setting and RE read signal
It is read and write with WE write signal for weight；
2) weight reads and writes gate: the mode of operation by controlling signal behavior weight, in read phase, connection weight reservoir,
Weight appropriate is provided, in write phase, connection to input terminal realizes right value update by outside port；
3) weight reservoir: it is used to store weight in inside neurons, each neuron weight reservoir space is input quantity
Multiplied by data width；
4) the PSP function generator of more vairable: PSP is with the shape for rising rapidly and slowly declining, and (1) is PSP function formula:
Wherein, K is PSP functional value, and V0 control rises maximum value, and τ m and τ s is that the time of the control raising and lowering gradient is normal here
Number, two constant relationships be τ s=τ m/4, PSP waveform continue 30 SLOW_CLK periods altogether, according to formula can be used 30 from
Scatterplot stores corresponding PSP functional value, forms the look-up table of 30 data widths；
5) membrane potential parallel multiplication: including a multiplier and an adder, in each SLOW_CLK period, according to formula
(2) it is weighted, wherein n is the quantity of all inputs in accumulation period, and Δ t is the time of current time and last pulse
Difference, by the direct evaluation of PSP look-up table, ω is corresponding weight, is read by weight reservoir:
6) trigger determining device: in each clk cycle, the result of membrane potential parallel multiplication can all be compared with membrane potential threshold value
Compared with generating triggering state, otherwise do not trigger if threshold value reaches；Triggering determining device includes a comparator, generates SNN touching
Send out state output；
7) when for the study of SNN network training, amendment modified weight module: is carried out to the neuron weight of erroneous trigger.
2. being used for the spiking neuron hardware structure of AER feed forward classification system as described in claim 1, characterized in that SNN net
Network training rules are as follows: it should trigger and not trigger or should not trigger and trigger, read the PSP function generator of more vairable and respond numerical value, multiplied by
Be added to former weight position after learning rate, is updated；Realize that modified weight is grasped using internal PSP generator and multiplier
Make.
3. being used for the spiking neuron hardware structure of AER feed forward classification system as described in claim 1, characterized in that 7 portions
Split flow are as follows: first by read-write control signal, come the update and reading of weight；In the stage of reading, that is, test rank
Section, according to the address of input from weight storage area obtain weight, in addition from the PSP function generator of more vairable obtain PSP functional value, jointly into
Enter in membrane potential parallel multiplication, is weighted；Calculated result can be compared in each clock cycle with activation threshold value
Compared with judging the triggering state of neuron using comparator；In the study stage, if necessary to carry out modified weight, Read-write Catrol letter
Number it is in write phase at this time, utilizes weight correction module, the response numerical value of the PSP function generator of more vairable is read, to weight storage element
Weight be modified.
4. being used for the spiking neuron hardware structure of AER feed forward classification system as described in claim 1, characterized in that 7 portions
For diversity in programmable gate array FPGA on site, weight storage device is comprehensive at storage RAM.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611190874.3A CN106779056B (en) | 2016-12-21 | 2016-12-21 | Spiking neuron hardware structure for AER feed forward classification system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611190874.3A CN106779056B (en) | 2016-12-21 | 2016-12-21 | Spiking neuron hardware structure for AER feed forward classification system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106779056A CN106779056A (en) | 2017-05-31 |
| CN106779056B true CN106779056B (en) | 2019-05-10 |
Family
ID=58893571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611190874.3A Active CN106779056B (en) | 2016-12-21 | 2016-12-21 | Spiking neuron hardware structure for AER feed forward classification system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106779056B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110046695B (en) * | 2019-04-09 | 2021-04-23 | 中国科学技术大学 | Configurable high-parallelism pulse neuron array |
| CN110378469B (en) * | 2019-07-11 | 2021-06-04 | 中国人民解放军国防科技大学 | SCNN inference device based on asynchronous circuit, PE unit, processor and computer equipment thereof |
| CN110674928B (en) * | 2019-09-18 | 2023-10-27 | 电子科技大学 | Online learning method integrating artificial neural network and nerve morphology calculation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469039A (en) * | 2015-11-19 | 2016-04-06 | 天津大学 | Target identification system based on AER image sensor |
| CN105611114A (en) * | 2015-11-02 | 2016-05-25 | 天津大学 | Full-digital multi-convolution core-convolution processing chip for AER (Address-Event Representation) image sensor |
| CN105760930A (en) * | 2016-02-18 | 2016-07-13 | 天津大学 | Multilayer spiking neural network recognition system for AER |
| CN106127800A (en) * | 2016-06-14 | 2016-11-16 | 天津大学 | Real-time many object tracking methods based on AER imageing sensor and device |
-
2016
- 2016-12-21 CN CN201611190874.3A patent/CN106779056B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105611114A (en) * | 2015-11-02 | 2016-05-25 | 天津大学 | Full-digital multi-convolution core-convolution processing chip for AER (Address-Event Representation) image sensor |
| CN105469039A (en) * | 2015-11-19 | 2016-04-06 | 天津大学 | Target identification system based on AER image sensor |
| CN105760930A (en) * | 2016-02-18 | 2016-07-13 | 天津大学 | Multilayer spiking neural network recognition system for AER |
| CN106127800A (en) * | 2016-06-14 | 2016-11-16 | 天津大学 | Real-time many object tracking methods based on AER imageing sensor and device |
Non-Patent Citations (1)
| Title |
|---|
| 《Asynchronous Event-Based Multikernel Algorithm for High-Speed Visual Features Tracking》;Xavier Lagorce 等;《IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS》;20150831;第26卷(第8期);第1710-1720页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106779056A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111417983B (en) | Deformable object tracking based on event camera | |
| CN106779056B (en) | Spiking neuron hardware structure for AER feed forward classification system | |
| EP3087538A2 (en) | Neural watchdog | |
| CN109086873A (en) | Training method, recognition methods, device and the processing equipment of recurrent neural network | |
| Tapiador-Morales et al. | Neuromorphic LIF row-by-row multiconvolution processor for FPGA | |
| WO2015167765A2 (en) | Temporal spike encoding for temporal learning | |
| US20150278641A1 (en) | Invariant object representation of images using spiking neural networks | |
| CN107239727A (en) | Gesture identification method and system | |
| WO2015127110A2 (en) | Event-based inference and learning for stochastic spiking bayesian networks | |
| EP3000076A2 (en) | Efficient hardware implementation of spiking networks | |
| WO2015069614A1 (en) | Implementing synaptic learning using replay in spiking neural networks | |
| CN115393597B (en) | Semantic segmentation method and device based on pulse neural network and laser radar point cloud | |
| CN110147163B (en) | Eye movement tracking method and system driven by multi-model fusion for mobile equipment | |
| WO2015156989A2 (en) | Modulating plasticity by global scalar values in a spiking neural network | |
| WO2015153150A2 (en) | Probabilistic representation of large sequences using spiking neural network | |
| EP3063707A2 (en) | Evaluation of a system including separable sub-systems over a multidimensional range | |
| WO2014172025A1 (en) | Method for generating compact representations of spike timing-dependent plasticity curves | |
| WO2015148254A2 (en) | Invariant object representation of images using spiking neural networks | |
| CN108875500A (en) | Pedestrian recognition methods, device, system and storage medium again | |
| CA2926034A1 (en) | Dynamically assigning and examining synaptic delay | |
| CN108875501A (en) | Human body attribute recognition approach, device, system and storage medium | |
| CN102890822B (en) | Device with function of detecting object position, and detecting method of device | |
| Blott et al. | Scaling neural network performance through customized hardware architectures on reconfigurable logic | |
| US9342782B2 (en) | Stochastic delay plasticity | |
| CN110110589A (en) | Face classification method based on FPGA parallel computation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220713 Address after: 266000 No. 39, Shandong Road, Shinan District, Qingdao, Shandong Province Patentee after: Qingdao Zhencheng Technology Co.,Ltd. Address before: No.92 Weijin Road, Nankai District, Tianjin 300072 Patentee before: Tianjin University |