CN116205784B - Optical flow recognition system based on event time triggering neuron - Google Patents
Optical flow recognition system based on event time triggering neuron Download PDFInfo
- Publication number
- CN116205784B CN116205784B CN202310482995.9A CN202310482995A CN116205784B CN 116205784 B CN116205784 B CN 116205784B CN 202310482995 A CN202310482995 A CN 202310482995A CN 116205784 B CN116205784 B CN 116205784B
- Authority
- CN
- China
- Prior art keywords
- time
- event
- cell body
- sequence
- neuron
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Abstract
The invention discloses an optical flow recognition system based on an event time triggering neuron, which relates to the field of optical flow recognition, wherein a linear array brightness sensor in the system is used for collecting illumination brightness information of each position point of a target area in a set period to obtain a plurality of event sequences; event time triggered neurons in this system include: delay propagation of dendrites, cell bodies and axons; delaying the time sequence corresponding to the event sequence by the delay propagation dendrite, and transmitting the event sequence and the weight to a cell body; the cell body calculates the cell body voltage increment at the current processing moment according to each event sequence and the corresponding weight, so as to determine the cell body voltage, and generates impulse when the cell body voltage is larger than a set threshold value, otherwise, the cell body voltage is attenuated by a set multiple and then is processed for the next time; the axon is configured to output the current processing time as the event occurrence time after receiving the impulse, thereby determining the optical flow velocity. The invention solves the problems of large calculated amount, large power consumption and high cost.
Description
Technical Field
The invention relates to the field of optical flow identification, in particular to an optical flow identification system based on event time triggering neurons.
Background
Currently, optical flow recognition is typically implemented using an artificial neural network of artificial neurons. The existing neural network mainly has the following problems: the traditional neural network is based on multi-layer and multi-neuron operation, the calculated amount is evenly distributed on each neuron, the calculated amount of the order of the power of multiple is increased when the calculated amount of each layer is increased, and the calculated amount of the multi-layer deep neural network is large. The large-scale network is becoming the mainstream in the current development, and has better task performance on various application problems, such as language model, image recognition model, etc., but the large-scale network still has the problems of large calculation amount and large power consumption even though the special hardware is utilized. Furthermore, the traditional neural network needs to perform one operation on the data of all neurons every time, and even if the neurons are not activated, one operation can be completed, so that the operation power consumption is high.
Therefore, the existing optical flow recognition system has the problems of large calculation amount, large power consumption and high cost.
Disclosure of Invention
Based on the above, the embodiment of the invention provides an optical flow identification system based on event time triggering neurons, so as to solve the problems of large calculated amount, large power consumption and high cost.
In order to achieve the above object, the embodiment of the present invention provides the following solutions: an optical flow recognition system based on event time triggered neurons, comprising: a linear array brightness sensor and at least one event time triggered neuron.
The linear array brightness sensor is used for: collecting illumination brightness information of each position point of a target area in a set period of time to obtain a plurality of event sequences; one location point corresponds to one of the event sequences; one of the sequences of events corresponds to one of the time sequences.
The event time triggered neuron comprises: delay spread dendrite, cell body and axon connected in turn; the delay spread dendrite includes: a plurality of parallel propagation channels; one of said propagation channels transmitting one of said sequences of events; the propagation channel comprises: a delay counter and a channel switch; the cell body comprises: a summing counter and a comparator; the axon comprises: a first trigger; the delay counter, the channel switch, the summing counter, the comparator and the first trigger are connected in sequence.
The delay counter is used for: and delaying the time sequence corresponding to the event sequence to obtain a delayed time sequence, and transmitting the event sequence to the cell body according to the delayed time sequence.
The channel switch is used for: and transmitting the weight corresponding to the channel switch to the cell body.
The summing counter is used for: for any event sequence, processing the event sequence at set time intervals; and calculating the cell body voltage increment at the current processing time according to each event sequence and the corresponding weight, and calculating the cell body voltage at the current processing time according to the cell body voltage increment at the current processing time and the cell body voltage at the last processing time.
The comparator is used for: if the cell body voltage at the current processing time is greater than the set threshold value and the difference between the current processing time and the last time impulse generation time is greater than the set duration, impulse is generated at the current processing time, and after impulse is generated, the cell body voltage at the current processing time is set to be zero.
If the cell body voltage at the current processing time is smaller than or equal to the set threshold value, the cell body voltage at the current processing time is attenuated by a set multiple, and then the next processing is carried out after waiting for a set time interval.
The first trigger is used for: if the comparator generates impulse at the current processing time, the current processing time is output as the event occurrence time, so that the event occurrence time of each position point of the target area is obtained; the event occurrence time is the time of giving illumination; the event occurrence time is used to determine an optical flow velocity of the target area.
Optionally, the delay propagates dendrites, further comprising: a weight adjusting module; the input of the weight adjusting module is connected with the output of the first trigger; the output of the weight adjusting module is connected with the input of the channel switch.
The weight adjusting module is used for: when training is carried out by adopting training data, the weight corresponding to the channel switch is adjusted according to a set updating rule when the axon outputs the occurrence time of each event.
Optionally, the event time triggers a neuron, further comprising: an input module; the input module is connected with the delay propagation dendrite; the input module is used for inputting a plurality of event sequences into a plurality of propagation channels in a one-to-one correspondence manner.
Optionally, the delay propagates dendrites, further comprising: a second trigger; the output of the second trigger is connected with the input of the delay counter; the second trigger is configured to input a sequence of events to the delay counter.
Optionally, the axon further comprises: enabling a counter; an input of the enable counter is connected with an output of the first trigger; the output of the enabling counter is connected with the input of the comparator; the enabling counter is used for triggering when the comparator outputs impulse to the first trigger, so that the comparator is disabled within a set time period.
Optionally, when the event time triggering neurons are plural, plural of the event time triggering neurons are connected in series; for any two adjacent event time triggered neurons, the output of the previous event time triggered neuron serves as the enable signal for the next event time triggered neuron.
Optionally, the summing counter is specifically configured to calculate the cell body voltage increment at the current processing time according to each event sequence and the corresponding weight: according to the formulaCalculating the cell body voltage increment at the current processing time t; wherein (1)>The cell body voltage increment at the current processing time is represented; i represents the number of the event sequence; n represents the number of event sequences; j represents the number of the moment in the event sequence i; l (L) i Representing the length of the sequence of events; s is S ij (t) illumination brightness information indicating a time j in the event sequence i; w (w) i Representing the weight corresponding to the event sequence i; t represents a time argument.
Optionally, the set multiple is determined according to the time interval and the decay rate.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects: the embodiment of the invention provides an optical flow identification system based on an event time triggering neuron, which comprises a linear array brightness sensor and the event time triggering neuron; event time triggered neurons include: delay propagation of dendrites, cell bodies and axons; compared with the traditional neural network, the event time triggering neuron of the embodiment of the invention does not need to use special equipment such as a high-computation-power display card and the like for operation, has the characteristics of low power consumption, high speed and reliability, small calculated amount and lower cost, and is easy to use common logic devices for expanding in number, thereby meeting different application requirements and being easier to produce and maintain. Therefore, the optical flow recognition system realized by adopting the event time triggering neuron solves the problems of large calculated amount, large power consumption and high cost of the traditional optical flow recognition system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an event time triggered neuron according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a single event according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of an event sequence provided in an embodiment of the present invention.
Fig. 4 is a schematic diagram of a delay spread dendrite effect according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of a cell discretization processing time axis according to an embodiment of the present invention.
Fig. 6 is a schematic diagram of a hardware structure of an event time triggered neuron according to an embodiment of the present invention.
Fig. 7 is a schematic diagram of connection of a plurality of event time triggered neurons according to an embodiment of the present invention.
Fig. 8 is a schematic diagram of a visual recognition network according to an embodiment of the present invention.
Symbol description: input module-1, delay spread dendrite-2, cell body-3, axon-4.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Artificial neurons are essentially a product abstracted from biological neurons and biological neural networks, and it is feasible to design neuron working models with a priori knowledge of the life sciences midbrain science at the present time of life sciences advancement.
Compared to artificial neurons commonly used in artificial intelligence (Artificial Intelligence, AI), life sciences have found more neuron characteristics, such as membrane potential diffusion characteristics, frequency response characteristics, herb mechanisms, etc., which are functions not possessed by current artificial neurons.
However, this does not negate the achievement of the current artificial neuron, which is a high abstraction of the function of the biological neuron, and utilizes the most fundamental characteristics of the biological neuron to realize a series of functions, and to realize various applications through networking of different modes.
However, in view of the current development trend, the application direction of the artificial neurons is more deep ploughing in a special field, and the artificial neurons are focused on solving a class of problems. The strong artificial intelligence expected for technological development, i.e. the neural network with human logic thinking and memory capability, has not yet been a remarkable network structure for a while.
Based on the above, the embodiment provides an event time triggering neuron, and designs an optical flow recognition system based on the event time triggering neuron.
First, the structure and working steps of the event time triggered neurons are described.
The event time triggering neuron can be used as a data processor which abstracts a method for processing event information by the biological neuron, the event processor can be operated in a computer to simulate the working mode of the neuron and realize the processing of the input event and the event occurrence time thereof, so as to obtain an output event set of the triggering neuron and obtain the identification result of the input event.
The structure of the event time triggered neuron is shown in fig. 1, and the event time triggered neuron includes: input module 1, delay spread dendrite 2, cell body 3 and axon 4.
The event time triggering neuron comprises a plurality of input modules 1, a plurality of delay propagation dendrites 2, a cell body 3 and an axon 4, wherein the input modules 1 are used for receiving events, the delay propagation dendrites 2 add delay to the process of the events from input to the cell body 3, the cell body 3 generates impulse when meeting preset conditions through membrane potential operation and threshold judgment, the impulse is transmitted through the axon 4, and the axon 4 can be connected to dendrites of a plurality of neurons, a plurality of dendrites of a neuron and dendrites of a neuron of the neuron itself.
Wherein the event is a binary signalAs shown in fig. 2.
Wherein S (t) represents a binary signal; t is a time argument representing the time of forward and unidirectional flow, and is used as a reference time during operation to provide a reference for neurons to determine whether an event has occurred.
T represents the moment when the event S occurs on the time axis, S takes a value of 1 or 0, when the time argument increases gradually from 0 to t=t, s=1 represents that the event S occurs at this moment, otherwise other cases s=0 represent that the event does not occur.
An event sequence consisting of a plurality of events is recorded as: s= { S a Wherein 1 is equal to or more than a is equal to or less than L, L is the number of the events of the event sequence (bold letters indicate matrix), S is the event sequence in matrix form, S a Representing the a-th event in the sequence of events.
The sequence of events corresponds to a time sequence: t= { T a }，T a Represents the a-th event S a Corresponding event occurrence time.
S a And T is a One-to-one correspondence, and has:the sequence of events is shown in fig. 3. S is S a (t) TableThe binary signal corresponding to the a-th event is shown.
The working steps of the event time triggered neuron are as follows.
Step 1: the neuron receives a sequence of events.
The concrete steps are as follows: input X to neurons 1 -X n Corresponding to n event sequences S 1 -S n The event sequence corresponds to a time sequence T 1 -T n The method comprises the steps of carrying out a first treatment on the surface of the The length of each event sequence corresponds to L 1 -L n 。
As the time argument T increases from 0, t=t in the time series ij Event S of (2) ij Occurrence of (i is not less than 1 and not more than n, j is not less than 1 and not more than L) n ) I represents the number of the event sequence; n represents the number of event sequences; j represents the number of the moment in the sequence of events i.
The sequence of events is connected to the input X of the neuron, when S ij When=1, at t=t ij The inputs to the neurons are activated at the moment and delay spread is triggered.
Step 2: events are passed to the delay propagation dendrites through event inputs.
The concrete steps are as follows: input X to neurons i (1. Ltoreq.i.ltoreq.n) if the value at the input varies at a certain time from the moment t, the cell body 3 needs to pass τ i The change in value is received after a time.
Input X to neurons 1 -X n Corresponding to n event sequences S 1 -S n The event sequence corresponds to a time sequence T 1 -T n The length of each event sequence corresponds to L 1 -L n The method comprises the steps of carrying out a first treatment on the surface of the Input X to neurons 1 -X n The corresponding event weight is w 1 -w n The method comprises the steps of carrying out a first treatment on the surface of the Neuron input X 1 Sequence of events S 1 Corresponding time series T 1 ={T 1j }（1≤j≤L 1 ) The method comprises the steps of carrying out a first treatment on the surface of the Neuron input X 2 Sequence of events S 2 Corresponding time series T 2 ={T 2j }（1≤j≤L 2 ) The method comprises the steps of carrying out a first treatment on the surface of the … …; neuron input X n Sequence of events S n Corresponding time series T n ={T nj }（1≤j≤L n ）。
The event reaches the cell body 3 after the delay of propagating the dendrite 2, in which process the event sequence S 1 -S n Produce τ 1 -τ n The sequence of events collected at cell 3 is S' i ={S ij }（1≤i≤n，1≤j≤L n ) Time series T 'of the collection at cell body 3' ij ={T ij +τ i }（1≤i≤n，1≤j≤L n ) Wherein S' 1 All event weights included are w 1 The method comprises the steps of carrying out a first treatment on the surface of the Wherein S' 2 All event weights included are w 2 The method comprises the steps of carrying out a first treatment on the surface of the … …; wherein S' n All event weights included are w n . The effect of the delay spread dendrite 2 is shown in fig. 4.
Step 3: and (5) cell body treatment process.
The concrete steps are as follows: the cell body 3 itself has the following intrinsic quantities: cell body voltage U, attenuation rate eta c Setting four parameters of a threshold value theta and a set duration epsilon. The set duration is also called refractory period.
For the cell body 3, the operation is performed once at set time intervals Δt, that is, the operation step time is Δt, and is small enough to be far smaller than the time difference of any two events. The cell body 3 processes the event information once after each delta t time is added to the time independent variable t; the cell discretization time axis is shown in fig. 5.
1 st sub-step: each event increases a certain cell body voltage at the moment of occurrence. Then at a certain time t + (t + >0) At this point, the cell 3 receives all events that produce the voltage increase:the method comprises the steps of carrying out a first treatment on the surface of the The accumulated cell body voltage u=u during this period - +U Increase the number of Wherein U is - For t=t + The voltage result processed at time Δt, i.e. at time t+last Δt. S is S ij (t) a binary signal representing the moment j in the sequence of events i, the binary signal being given different physical meanings in different applications; w (w) i The weight corresponding to the event sequence i is represented.
2 nd sub-step: 1) If the accumulated cell body voltage U is less than the set threshold value theta.
After the cell body voltage passes the delta t time, the voltage of the cell body voltage is attenuated to the original set multiple; setting multiple eta c ^Δt，0＜η c ＜1。
Order U (t=t) + +Δt）=U（η c ^Δt）；U（t=t + +Δt) is t + Cell body voltage at +Δt.
The process is that every time t=1 time passes, the cell body voltage is changed into the original eta c Multiple times.
2) If the accumulated cell body voltage U is more than or equal to the set threshold value theta.
When the cell body voltage U of the cell body 3 is larger than or equal to the set threshold value theta, the cell body 3 generates one impulse and transmits the impulse to the axon 4, so that the axon 4 output Y is 1 in the current moment value, the impulse is continued for delta t time, after the impulse is generated, the cell body voltage U is zeroed, and in the period from the moment when the impulse is generated to the moment when the set duration epsilon is elapsed, even if the cell body voltage is larger than the set threshold value theta, the impulse is not generated.
Step 4: axonal efferent processes.
The concrete steps are as follows: the axon 4 serves as an output terminal of information, and connects the cell body 3 with the delay propagation dendrite 2 of other neurons, or connects the delay propagation dendrites 2 of a plurality of other neurons, or connects the delay propagation dendrites 2 of the neurons themselves.
When the output of an axon 4 is 1 in value at the current moment, this indicates that the cell body 3 is generating impulses, and that other dendrites connected to this axon 4 will receive impulse information.
Step 5: and (5) a neuron weight adjustment process.
The weight can be preset manually or determined by training adjustment using training data. Wherein, training adjustment specifically represents:the weight w will be updated each time a neuron generates impulse n . The update rule is set as follows: weight w n Only at the moment when the neuron generates impulse.
If an event propagates on the delay-propagating dendrite 2, and reaches the cell body 3 within a short period of time t-after the impulse occurs, the weight w of the delay-propagating dendrite 2 n Will be turned down, the larger the t-value, i.e. the greater the difference in time between the impulse and the arrival of the event, the weight w n The smaller the turndown value of (2), the smaller the t-value, i.e. the smaller the difference in time between impulse and arrival of the event, the weight w n The greater the turndown value of (c).
If an event propagates on the delay-propagating dendrite 2, and reaches the cell body 3 within a short period of time t+ before an impulse occurs, the weight w of the delay-propagating dendrite 2 n Will be turned up, the larger the value of t+ is, i.e. the larger the time difference between the arrival of the event and the impulse is, the weight w n The smaller the increment value of (2), the smaller the value of t+ is, i.e. the smaller the time difference between the arrival and impulse of the event is, the weight w is n The greater the turnup value of (c).
The event time triggering neuron has the following advantages: in step 2, a time factor is introduced, signals are introduced from neurons in a traditional neural network model, a time dimension is introduced from a pure digital quantity, information can be continuously processed in a broad range, and association between the information is introduced, and the association information is not required to be introduced by the traditional neurons through a mode of constructing a complex network.
Furthermore, after the time factor is introduced, a large amount of information can be input into the delay propagation neurons in a scattered manner on a time axis, and through freely combining the connection modes of different delay propagation dendrites 2 and axons 4, a plurality of functions can be operated on a single neuron, so that the operation cost is greatly reduced.
In the step 5, a time factor is introduced, so that the Hebb rule of the biological neuron adjustment connection mode can be well realized on the artificial neuron, learning can be realized once per pulse, gradient operation is not needed in the mode, and the operation cost is reduced again.
The application of the event time triggered neurons described above is described below.
Example 1
First, an optical flow recognition system based on the event time triggering neuron is formed by using the event time triggering neuron, so as to realize optical flow recognition.
The system utilizes event time triggering neurons with time resolution learning capability to realize the identification of time sequence signals and further realize the identification of the light flow signals.
The optical flow recognition system based on the event time triggering neuron of the embodiment comprises: a linear array brightness sensor and at least one event time triggered neuron.
The linear array brightness sensor is used for detecting illumination brightness information in a certain linear direction, the linear array brightness sensor transmits the information to the event time triggering neuron, and after the event time triggering neuron array identifies brightness signals in the direction, optical flow speed information in the direction is identified, and corresponding output is made.
In practical application, the linear array brightness sensor is used for: collecting illumination brightness information of each position point of a target area in a set period of time to obtain a plurality of event sequences; one location point corresponds to one of the event sequences; one of the sequences of events corresponds to one of the time sequences.
Referring to fig. 1 and 6, the event time triggered neuron includes: the delayed transmission dendrite 2, cell body 3 and axon 4 are connected in sequence. The delay spread dendrite 2 includes: a plurality of parallel propagation channels; one of said propagation channels transmitting one of said sequences of events; the propagation channel comprises: a delay counter and a channel switch; the cell body 3 includes: a summing counter and a comparator; the axon 4 comprises: a first trigger; the delay counter, the channel switch, the summing counter, the comparator and the first trigger are connected in sequence. It can be seen that each part is composed of digital signal combinational logic, and can be built by using logic circuits or deployed in a CPLD/FPGA programmable logic device.
The delay counter is used for: and delaying the time sequence corresponding to the event sequence to obtain a delayed time sequence, and transmitting the event sequence to the cell body 3 according to the delayed time sequence.
The channel switch is used for: and transmitting the weight corresponding to the channel switch to the cell body 3.
The summing counter is used for: for any event sequence, processing the event sequence at set time intervals; and calculating the cell body voltage increment at the current processing time according to each event sequence and the corresponding weight, and calculating the cell body voltage at the current processing time according to the cell body voltage increment at the current processing time and the cell body voltage at the last processing time.
The comparator is used for: if the cell body voltage at the current processing time is greater than the set threshold value and the difference between the current processing time and the last time impulse generation time is greater than the set duration, impulse is generated at the current processing time, and after impulse is generated, the cell body voltage at the current processing time is set to be zero. If the cell body voltage at the current processing time is smaller than or equal to the set threshold value, the cell body voltage at the current processing time is attenuated by a set multiple, and then the next processing is carried out after waiting for a set time interval. Wherein the set multiple is determined based on the time interval and the decay rate.
The first trigger is used for: if the comparator generates impulse at the current processing time, the current processing time is output as the event occurrence time, so that the event occurrence time of each position point of the target area is obtained; the event occurrence time is the time of giving illumination; the event occurrence time is used to determine an optical flow velocity of the target area.
In one example, the delay propagating dendrite 2 further includes: a weight adjusting module; the input of the weight adjusting module is connected with the output of the first trigger, and the connecting loop is used as an adjusting loop; the output of the weight adjusting module is connected with the input of the channel switch.
The weight adjusting module is used for: when training is performed by using training data, the weight corresponding to the channel switch is adjusted according to a set updating rule when the axon 4 outputs the occurrence time of an event once.
In one example, the event time triggers a neuron, further comprising: an input module 1; the input module 1 is connected with the delay propagation dendrite 2; the input module 1 is configured to input a plurality of event sequences into a plurality of propagation channels in a one-to-one correspondence manner.
In one example, the delay propagating dendrite 2 further includes: a second trigger; the output of the second trigger is connected with the input of the delay counter; the second trigger is configured to input a sequence of events to the delay counter.
In one example, the axon 4 further comprises: enabling a counter; an input of the enable counter is connected with an output of the first trigger; the output of the enable counter is connected to the input of the comparator. The enabling counter is used for triggering when the comparator outputs impulse to the first trigger, so that the comparator is disabled within a set time period.
In one example, when the event time triggered neuron is a plurality of, a plurality of the event time triggered neurons are connected in series; for any two adjacent event time triggered neurons, the output of the previous event time triggered neuron serves as the enable signal for the next event time triggered neuron.
In one example, the summation counter is specifically configured to calculate an increase in cell body voltage at a current processing time according to each of the event sequences and the corresponding weights: according to the formulaCalculating the cell body voltage increment at the current processing time t; wherein (1)>The cell body voltage increment at the current processing time is represented; i represents the number of the event sequence; n represents the number of event sequences; j represents the number of the moment in the event sequence i; l (L) i Representing the length of the sequence of events; the present embodiment uses illumination brightness information as binary valuesSignal, therefore, in the present embodiment, corresponding to S ij (t) illumination brightness information indicating a time j in the event sequence i; w (w) i The weight corresponding to the event sequence i is represented.
The input module 1 of the present embodiment inputs X 1 -X n The second trigger and the delay counter correspond to delay propagation channels in the delay propagation dendrite 2, the channel switch corresponds to weight setting of the neuron, and the adjustment module adjusts the adjustment process of the neuron.
The summation counter sums the input signals and reduces the value of the summation counter after a certain period of time passes, thereby realizing the attenuation rate eta c The purpose of attenuating the cell body voltage U of the neuron.
The sum counter and the set threshold value theta are input into the comparator to form a judging process of the neuron threshold value, if the sum counter and the set threshold value theta are larger than or equal to the set threshold value, the first trigger is output to generate pulse, impulse is generated, and a pulse signal is generated at the output Y.
The impulse is generated and the enabling counter is triggered, so that the comparator is disabled in the counting time, and the refractory period of the neuron model is formed.
The impulse is generated and is transmitted back to the weight adjusting module, according to the step 5: rules in the neuron adjustment procedure "adjust each input signal.
An example of using multiple event time triggered neurons for an optical flow recognition system is described below in connection with FIG. 7.
Referring to fig. 7, this example is made up of several event time triggered neurons (simply referred to as neurons), corresponding to the 1 st neuron, the 2 nd neuron, respectively 1 -X n Each input corresponding to τ 1 -τ n The length increases linearly from no delay to the longest time of the sequence. The latter neuron is enabled by the former neuron control using a significant weight that makes it impossible for the neuron to be activated without an enable input. The final output Y can be activated only if neurons 1 through m are activated in sequence.
The working steps of the neuron method can obtain that each neuron has a learning effect on the regular triggering, so that the same pulse is repeatedly input in a pulse sequence, the 1 st neuron can continuously strengthen the weight of reaching a certain input X at the same time in each repetition, the weight cannot be increased multiple times due to the irregular condition of reaching the input X at other times, and only the pulse reaching the 1 st neuron periodically is finally remained.
When the first 1 st neuron learns the first signal of the pulse sequence, the 2 nd neuron starts to learn the second signal, and the process is repeated until the mth neuron learns the mth signal.
If the pulse sequence length is unknown, it is possible to check where in the neuron cascade system the finally excited neuron is located after learning for a certain period of time, and then determine the neuron as an output neuron.
In connection with the implementation of neurons, a visual recognition network is designed and several examples of applications are given to demonstrate the effectiveness of optical flow recognition systems that trigger neurons based on event time.
The neuron adopts a discrete signal processing mode, and takes a small enough time step delta t as the system stepping time, wherein the time is small enough and is far smaller than the time difference of any two events.
Neuron N has X 0 -X n Each input having w 0 -w n Each input corresponds to a delay spread dendrite 2 having τ 0 -τ n Is a delay of (2); the cell body 3 has a cell body voltage U and an attenuation rate eta c The threshold value θ and the set duration epsilon are set.
Dendrite propagation process: time t s In any dendrite X n A single event S is received on: the time required for the event S to reach the cell body 3 is τ n The method comprises the steps of carrying out a first treatment on the surface of the The amplitude of the signal arriving at the cell body 3 is equal to the weight w n The method comprises the steps of carrying out a first treatment on the surface of the At time t s Will (t) Delay time ，w n ) Record into message queue, where t Delay time =τ n . At t s From time to t s The following events are processed within the time +Δt: will be at t s Time to dendritic event (t Delay time ，w n ) Recorded in a message queue. Every time the neuron system steps for a time deltat, polling the dendrites for the event, and if so, repeating the above steps. Every time the neuronal system steps by a time Δt, poll the message queue, and send t Delay time Subtracting Δt, t Delay time (next time) =t Delay time (current time) - Δt, up to t in the message queue Delay time ≤0，t Delay time The next time after < 0 will remove the signal from the message queue.
Accumulation process of cell body 3: the cell body 3 has a cell body voltage U and an attenuation rate eta c For the cell body voltage U, the neuron system polls the message queue and queries t every time deltat Delay time A signal of=0, and the amplitude corresponding to the signal (equal to the weight w n ) Added to the cell body voltage U to update the cell body voltage after addition to U+w n 。
Detecting cell body voltage after accumulation is completed: when the cell body voltage U after accumulation is lower than the set threshold value theta: the cell body voltage U after accumulation is changed into the current eta c And the process continues to wait for the next round of processing. When the cell body voltage U after accumulation is higher than the set threshold value theta: 1) If the time difference between the last exciting time of the cell body 3 and the current time is larger than the set duration epsilon: the cell body 3 generates impulse, and sends pulse signals through the axon 4 to transmit to other neurons, and at the same time, the cell body voltage U (next time) =0, and the next round of processing is continued. 2) If the time difference between the last exciting time of the cell body 3 and the current time is smaller than the set duration epsilon: and (5) continuing to wait for the next round of processing without any processing.
The following describes a visual recognition network, the structure of which is shown in fig. 8.
Referring to fig. 8, two pixels are arranged in a row and two columns. The 1 st pixel on the left side and the 2 nd pixel on the right side, and the time when the illumination is obtained is defined as the time when the event is generated.
When light is given to the 1 st pixel at time t=1, then an event S is generated at time t=1 1 The method comprises the steps of carrying out a first treatment on the surface of the There is a neuron N connected to the two event occurrence ends, the neuron N having X 1 ，X 2 Two inputs corresponding to the 1 st pixel and the 2 nd pixel; x is X 1 ，X 2 Two inputs correspond to w 1 =6，w 2 Weights of =3; x is X 1 ，X 2 Two inputs to the cell body 3 having τ 1 =1，τ 2 Delay of =2; x is X 1 ，X 2 The weights of the two inputs when reaching the cell body 3 are 6 and 3 respectively; cell 3 has cell voltage u=0, attenuation rate η c =0.5 and a set duration epsilon=5. Setting the threshold value θ depends on the application variation, and a value is given in the application example.
Application example 1: a single pixel is identified, this example identifying the 1 st pixel.
A set threshold θ=5 for cell body 3; the 1 st pixel is illuminated at time t=0 and the 2 nd pixel is not illuminated. Then t=0 time generates a signal S 1 And is passed to input X of neuron N 1 。
From the above conditions, S 1 When the cell voltage u=6 at time t=1 is greater than the set threshold θ=5, the axon 4 of the neuron N generates impulse at time t=1, indicating that the 1 st pixel has illumination. At the same time, after impulse is generated, the cell body voltage u=0.
After a set period of time, the 2 nd pixel is illuminated again at the time t=5, and the 1 st pixel is not illuminated. Then t=5 time generates signal S 2 And is passed to input X of neuron N 2 。
From the above conditions, S 2 When the cell voltage u=3 at time t=7 is smaller than the set threshold θ=5, the neuron cannot generate impulse.
The above application example represents that the neuron has a function of selectively information-identifying different information.
Application example 2: and (5) identifying a time sequence signal.
Setting a set threshold θ=7 of cell body 3; the 2 nd pixel is illuminated at time t=0, and the 2 nd pixel is illuminated at time t=1. Then t=0 time generates a signal S 2 And is passed to input X of neuron N 2 The method comprises the steps of carrying out a first treatment on the surface of the Then t=1 time generates a signal S 1 And is passed to input X of neuron N 1 。
From the above conditions, S 2 Propagates on the 2 nd dendrite at time t=0 and reaches cell body 3 at time t=2, with a weight of 3 when reaching cell body 3.
From the above conditions, S 1 Propagates on the 1 st dendrite at time t=1 and reaches cell body 3 at time t=2, with a weight of 6 when reaching cell body 3.
Then at time t=2 the cell body voltage u=3+6=9, greater than the set threshold θ=7, the neuron N generates an impulse, indicating that the timing signal is identified: the 2 nd pixel is illuminated at time t=0, and the 2 nd pixel is illuminated at time t=1.
At the same time, after impulse is generated, the cell body voltage u=0.
After a set period of time, the 1 st pixel is illuminated at time t=5, and the 2 nd pixel is illuminated at time t=6.
Then t=5 time generates signal S 3 And is passed to input X of neuron N 1 The method comprises the steps of carrying out a first treatment on the surface of the Then t=6 time produces signal S 4 And is passed to input X of neuron N 2 。
From the above conditions, S 3 Propagates on the 1 st dendrite at time t=5 and reaches cell body 3 at time t=6, the voltage at which cell body 3 is reached being 6.
From the above conditions, S 4 Propagates on the 1 st dendrite at time t=6 and reaches cell body 3 at time t=8, the voltage at which cell body 3 is reached being 3.
At t=6, there is only S 3 When the cell body 3 is reached, the cell body voltage u=6 is smaller than the set threshold value θ=6, and no impulse is generated.
At a moment before time t=8, no signal arrives within the time t=6 to t=8, and the cell body voltage is reduced to u=1.5 because the cell body voltage becomes 0.5 times as much as the original cell body voltage every time t=1 time elapses.
time t=8S 4 When the cell body 3 is reached, the cell body voltage u=1.5+3=4.5 is smaller than the set threshold θ=7, and no impulse is generated.
The above application example represents a function in which the neuron has selective information identification for different timing signals.
The hardware configuration of the event time triggered neuron of the present embodiment is composed of a purely digital logic circuit, and this module does not need to use a special device such as a high-power video card to perform the operation, because it is composed of a purely digital logic circuit. The module has the advantages that the module is simple in structure, and is easy to expand in number by using general logic devices, so that different application requirements can be met, simulation and operation of a neuron model can be realized only by using CPLD/FPGA with basic functions, the number of neurons can be easily expanded by using a communication mode between chips, and if a special ASIC digital circuit chip is used, the production cost can be further reduced, and the number of neurons on the chip is improved.
The event time triggering neuron formed by using the pure digital logic circuit has the characteristics of low power consumption, high speed and reliability. Compared with the mode of operation by using special equipment such as a high-power display card, the event time triggering neuron has lower cost and is easier to produce and maintain. In addition, the event time triggered neurons can be applied to various computing and control systems, including the fields of digital signal processing, communication, automatic control, and the like.
Under the development trend of the current chip, the event time triggering neuron formed by adopting the pure digital logic circuit also has the advantage of risk resistance. Because the event time triggered neurons are independent of special devices such as display cards, the neurons can be more flexibly handled when the chip supply shortage is faced.
The neural network composed of the event time triggered neurons of the present embodiment has the following advantages compared with the conventional neural network.
(1) And the running power consumption is reduced.
The operation amount of the neural network is greatly reduced, the huge operation expenditure generated during large-scale network operation is avoided in a mechanism, and the operation power consumption of the neural network is greatly improved.
Furthermore, the conventional neural network needs to perform an operation on the data of all neurons once every operation, and the operation is completed even if the neurons are not activated, but the neural network in this embodiment is a sparse operation, and only a part of the neurons are activated every time an event is activated, so that the non-activated neurons consume little energy (only consume energy when impulse is generated), and the operation power consumption is further reduced.
(2) The real-time training problem is solved.
The training process and the application process are integrated, the neural network can be trained when in use, and the neuron parameters can be manually locked to avoid the change of the neuron parameters, so that the real-time training problem is solved under the structure, and the network can utilize the data during application to continuously improve the expression effect of the neural network on the problem.
(3) The problem that the multilayer network is not easy to train is solved.
The method for self-training by using a single neuron can avoid using a back propagation algorithm, further avoid the problem that a multi-layer network is not suitable for training, and simultaneously has the characteristic of refractory period, so that the network training result is easier to converge.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (8)
1. An optical flow recognition system based on event time triggered neurons, comprising: a linear array brightness sensor and at least one event time triggering neuron;
the linear array brightness sensor is used for:
collecting illumination brightness information of each position point of a target area in a set period of time to obtain a plurality of event sequences; one location point corresponds to one of the event sequences; one of the event sequences corresponds to one of the time sequences;
the event time triggered neuron comprises: delay spread dendrite, cell body and axon connected in turn; the delay spread dendrite includes: a plurality of parallel propagation channels; one of said propagation channels transmitting one of said sequences of events; the propagation channel comprises: a delay counter and a channel switch; the cell body comprises: a summing counter and a comparator; the axon comprises: a first trigger; the delay counter, the channel switch, the summation counter, the comparator and the first trigger are connected in sequence;
the delay counter is used for:
delaying the time sequence corresponding to the event sequence to obtain a delayed time sequence, and transmitting the event sequence to the cell body according to the delayed time sequence;
the channel switch is used for:
transmitting the weight corresponding to the channel switch to the cell body;
the summing counter is used for:
for any event sequence, processing the event sequence at set time intervals;
calculating the cell body voltage increment of the current processing time according to each event sequence and the weight corresponding to the event sequence, and calculating the cell body voltage of the current processing time according to the cell body voltage increment of the current processing time and the cell body voltage of the last processing time;
the comparator is used for:
if the cell body voltage at the current processing time is greater than a set threshold value and the difference between the current processing time and the time of generating impulse last time is greater than a set duration, impulse is generated at the current processing time, and after impulse is generated, the cell body voltage at the current processing time is set to be zero;
if the cell body voltage at the current processing time is smaller than or equal to the set threshold value, attenuating the cell body voltage at the current processing time by a set multiple, waiting for a set time interval, and then performing the next processing;
the first trigger is used for:
if the comparator generates impulse at the current processing time, the current processing time is output as the event occurrence time, so that the event occurrence time of each position point of the target area is obtained; the event occurrence time is the time of giving illumination; the event occurrence time is used to determine an optical flow velocity of the target area.
2. The optical flow recognition system based on event time triggered neurons of claim 1, wherein the delay propagates dendrites further comprising: a weight adjusting module;
the input of the weight adjusting module is connected with the output of the first trigger; the output of the weight adjusting module is connected with the input of the channel switch;
the weight adjusting module is used for:
when training is carried out by training data, the weight corresponding to the channel switch is adjusted according to a set updating rule when the first trigger outputs the event occurrence time every time.
3. The optical flow recognition system based on event time triggered neurons of claim 1, further comprising: an input module;
the input module is connected with the delay propagation dendrite;
the input module is used for inputting a plurality of event sequences into a plurality of propagation channels in a one-to-one correspondence manner.
4. The optical flow recognition system based on event time triggered neurons of claim 1, wherein the delay propagates dendrites further comprising: a second trigger;
the output of the second trigger is connected with the input of the delay counter; the second trigger is configured to input a sequence of events to the delay counter.
5. The optical flow recognition system based on event time triggered neurons of claim 1, wherein the axons further comprise: enabling a counter;
an input of the enable counter is connected with an output of the first trigger; the output of the enabling counter is connected with the input of the comparator;
the enabling counter is used for triggering when the comparator outputs impulse to the first trigger, so that the comparator is disabled for a set time period.
6. The optical flow recognition system based on event time triggered neurons of claim 1, wherein when the event time triggered neurons are a plurality, a plurality of the event time triggered neurons are connected in series;
for any two adjacent event time triggered neurons, the output of the previous event time triggered neuron serves as the enable signal for the next event time triggered neuron.
7. The optical flow recognition system based on event time triggered neurons according to claim 1, wherein the summation counter is specifically configured to calculate an increase of cell body voltage at a current processing time according to each of the event sequences and weights corresponding to the event sequences:
according to the formulaCalculating the cell body voltage increment at the current processing moment; wherein (1)>The cell body voltage increment at the current processing time is represented;ia number representing a sequence of events;nrepresenting the number of event sequences;jrepresenting a sequence of eventsiNumbering of the moment in time; />Representing a sequence of eventsiIs a length of (2); />Representing a sequence of eventsiTime of (a)jIs a luminance information of illumination; />Representing a sequence of eventsiCorresponding weights;trepresenting the time argument.
8. The system of claim 1, wherein the set multiple is determined based on the time interval and the decay rate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310482995.9A CN116205784B (en) | 2023-05-04 | 2023-05-04 | Optical flow recognition system based on event time triggering neuron |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310482995.9A CN116205784B (en) | 2023-05-04 | 2023-05-04 | Optical flow recognition system based on event time triggering neuron |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116205784A CN116205784A (en) | 2023-06-02 |
| CN116205784B true CN116205784B (en) | 2023-08-01 |
Family
ID=86517636
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310482995.9A Active CN116205784B (en) | 2023-05-04 | 2023-05-04 | Optical flow recognition system based on event time triggering neuron |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116205784B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114881070A (en) * | 2022-04-07 | 2022-08-09 | 河北工业大学 | AER object identification method based on bionic hierarchical pulse neural network |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10628699B2 (en) * | 2017-06-13 | 2020-04-21 | Samsung Electronics Co., Ltd. | Event-based image feature extraction |
| CN112101535B (en) * | 2020-08-21 | 2024-04-09 | 深圳微灵医疗科技有限公司 | Signal processing method of impulse neuron and related device |
| CN113255905B (en) * | 2021-07-16 | 2021-11-02 | 成都时识科技有限公司 | Signal processing method of neurons in impulse neural network and network training method |
| CN114548290A (en) * | 2022-02-24 | 2022-05-27 | 西安电子科技大学 | Synaptic convolutional impulse neural network for event stream classification |
| CN114816076A (en) * | 2022-06-24 | 2022-07-29 | 清华大学 | Brain-computer interface computing processing and feedback system and method |
| CN115358261A (en) * | 2022-08-01 | 2022-11-18 | 贵州大学 | Haptic object identification method based on pulse time sequence error back propagation |
-
2023
- 2023-05-04 CN CN202310482995.9A patent/CN116205784B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114881070A (en) * | 2022-04-07 | 2022-08-09 | 河北工业大学 | AER object identification method based on bionic hierarchical pulse neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116205784A (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8655813B2 (en) | Synaptic weight normalized spiking neuronal networks | |
| US20200065658A1 (en) | Neuromorphic event-driven neural computing architecture in a scalable neural network | |
| CN106845633B (en) | Neural network information conversion method and system | |
| US4518866A (en) | Method of and circuit for simulating neurons | |
| CN112699956B (en) | Neuromorphic visual target classification method based on improved impulse neural network | |
| US20140114893A1 (en) | Low-power event-driven neural computing architecture in neural networks | |
| US20100312736A1 (en) | Critical Branching Neural Computation Apparatus and Methods | |
| US20150302294A1 (en) | Multi-scale spatio-temporal neural network system | |
| CN110659666B (en) | Image classification method of multilayer pulse neural network based on interaction | |
| KR102588838B1 (en) | Superconducting neuromorphic core | |
| Bugmann | Biologically plausible neural computation | |
| CN112149815A (en) | Population clustering and population routing method for large-scale brain-like computing network | |
| CN116205784B (en) | Optical flow recognition system based on event time triggering neuron | |
| Hussain et al. | Delay learning architectures for memory and classification | |
| CN111340194A (en) | Pulse convolution neural network neural morphology hardware and image identification method thereof | |
| CN113298231A (en) | Graph representation space-time back propagation algorithm for impulse neural network | |
| US9406015B2 (en) | Transform for a neurosynaptic core circuit | |
| CN116629344A (en) | Spike-BP on-chip learning method, system and processor based on Ca-LIF neuron model | |
| CN115800274B (en) | 5G distribution network feeder automation self-adaptation method, device and storage medium | |
| Sofatzis et al. | The synaptic kernel adaptation network | |
| Barton et al. | The application perspective of izhikevich spiking neural model–the initial experimental study | |
| CN110111234B (en) | Image processing system architecture based on neural network | |
| CN112819072B (en) | Supervision type classification method and system | |
| van den BOOGAARD et al. | The master equation for neural interaction | |
| CN112949833B (en) | Probability calculation neuron calculation unit and construction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |