CN110210563B - Image pulse data space-time information learning and identification method based on Spike cube SNN - Google Patents
Image pulse data space-time information learning and identification method based on Spike cube SNN Download PDFInfo
- Publication number
- CN110210563B CN110210563B CN201910481420.9A CN201910481420A CN110210563B CN 110210563 B CN110210563 B CN 110210563B CN 201910481420 A CN201910481420 A CN 201910481420A CN 110210563 B CN110210563 B CN 110210563B
- Authority
- CN
- China
- Prior art keywords
- pulse
- neuron
- layer
- pulse sequence
- learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for performing joint learning aiming at space-time information of an image pulse sequence and an image identification method, wherein a pulse sequence unit Spike cube and LIF neuron model is established based on a pulse neural network, and an STDP mechanism is adopted to learn synaptic weights and excitation thresholds connected among neurons in each layer of the pulse neural network; and then, carrying out image classification and identification by using the trained model. The invention provides a new technical scheme for structural design and learning of the pulse neural network and learning and identification of the image pulse sequence, and simultaneously provides a new processing method for pulse data output by a bionic vision camera such as DVS (visual dynamic system).
Description
Technical Field
The invention belongs to the technical field of brain-like computation, a pulse neural network (SNN), STDP learning and image recognition, and relates to a learning method of a pulse sequence, in particular to a method for performing joint learning on time and space information contained in an image pulse sequence based on a spike cube pulse neural network.
Background
In recent years, with the rapid development of deep learning, large-scale artificial neural networks are widely applied in many fields. The learning and identification of the sequence information can also be realized by a deep learning technology. For example, a time series processing method represented by a numerical cyclic neural network can learn and process sequence information having time correlation, and is widely applied to two-dimensional scenes including only one-dimensional time information and one-dimensional language information, such as natural language processing, machine translation, and speech recognition. For another example, an image processing method represented by a numerical convolution neural network has a very sophisticated information processing capability for a planar image, is widely applied to the fields of image recognition, target detection, target tracking and the like, and has a learning and recognition capability for a pulse sequence, but the learning and recognition method is to perform temporal accumulation on the pulse sequence and then send the pulse sequence into the neural network. Although this information processing method achieves a relatively high recognition rate, it substantially loses the time series information originally carried by the pulse sequence, and is not a method for learning and recognizing the time-space information of the pulse sequence in a true sense. In addition, the system also has no biological characteristics in the aspects of learning and processing mechanisms, and is higher in the aspect of computing energy consumption.
The impulse neural network (SNN) is known as a "third generation neural network", and compared with a numerical deep learning neural network which is widely popular at present, the impulse neural network has more bionic characteristics, and is specifically embodied in the aspects of an information processing mechanism and a learning mechanism. In terms of information processing mechanisms, the input and processed information of the impulse neural network is a pulse sequence, which is rather similar in form to the corresponding and excitation mechanism of biological neurons to action potential signals in the brain. In terms of learning mechanism, the learning of the spiking neural network is mainly based on the STDP mechanism, which is considered as the learning mode of synapse connection of biological neurons in the brain. The impulse neural network refers to the brain mode in both information processing mechanism and learning mechanism, so the impulse neural network is a research branch of brain-like computation.
Although most of the existing pulse neural networks process the pulse sequences without accumulating the pulse sequences on an input source, because most of synaptic weights of the pulse neural networks are obtained by weight transplantation through a numerical neural network, and the weights of the numerical neural networks are obtained by learning pulse images accumulated by the pulse sequences, the pulse neural networks built through the weight transplantation do not have the capability of exploring pulse sequence time sequence information per se, only depend on space information accumulated by the pulse sequences in time for identification, and do not fully develop the potential of the pulse sequences in space-time information expression. There are also a few types of impulse nerves that use the STDP mechanism to directly learn the pulse sequence that is not accumulated, and although the learning object is the pulse sequence itself, the learning result of such methods is still static values of synaptic weights between neurons and neuron firing thresholds. The processing of a time-characterized pulse sequence with a static synapse without time characteristics naturally fails to make efficient use of the timing information contained in the pulse sequence.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a method for jointly learning time and space information contained in an image pulse sequence based on a spike cube (pulse sequence unit) pulse neural network, provides a new technical scheme for learning and identifying the image pulse sequence, provides a new thought for a structural design and learning method of the pulse neural network, and further provides a new processing method for pulse data output by a bionic vision camera such as DVS (visual system).
The core of the invention is: a method for jointly learning the time and space information of an image pulse sequence based on spike cube and LIF (leakage Integrated and Fire) neuron models and by adopting an STDP (spike time dependent plasticity) mechanism is provided. In the invention, according to the minimum time length of a time sequence mode contained in an image pulse sequence to be recognized, an input image pulse sequence is divided at equal intervals to obtain a pulse sequence segment, and the obtained pulse sequence segment is defined as a pulse sequence unit (Spike cube). Firstly, constructing a pulse neural network according to the framework of the attached figure 2, wherein neurons of the pulse neural network are represented by an LIF model; then preparing an image pulse sequence to be learned, dividing the image pulse sequence into a training set, a verification set and a test set according to a certain ratio, inputting the training set pulse sequence into a pulse neural network for forward propagation, and sequentially passing through a first convolutional layer, a second convolutional layer, a full-link layer and a classification layer; in the layer-by-layer propagation process, learning synaptic weights and excitation thresholds of the connection between neurons by taking spike cubes as learning units and adopting an STDP mechanism; it should be specially noted that the convolutional layer synaptic weights and the excitation threshold have different characteristic forms at different moments, after learning for a period of time, the learning process of synapses is temporarily interrupted, and the verification set image pulse sequence is input to the current impulse neural network for image category identification accuracy test, if the expected standard is reached, the learning process is ended and the synaptic weights and the excitation threshold are stored, if the expected standard is not reached, the learning is continued on the basis until the expected standard is reached or the whole training set image pulse sequence is completely learned; and after learning is finished, carrying out recognition accuracy rate test on the image pulse of the test set by using the pulse neural network. It should be noted that the impulse neural network constructed in fig. 2 provides an existing simpler impulse neural network architecture on which the learning and recognition method proposed in the present invention is based, and the impulse neural network under the architecture can be used for learning and recognition of the handwritten digital image pulse sequence. When the spatio-temporal pattern information contained in the image pulse sequence becomes more complex and diversified, if a better pulse sequence identification effect is to be achieved, the number of layers and the scale of the convolution layer and the full connection layer of the pulse neural network need to be expanded, but the method for performing combined learning on the time and space information contained in the pulse sequence provided by the invention is still suitable for the expanded larger-scale pulse neural network.
The technical scheme provided by the invention is as follows:
a method for carrying out combined learning on time and space information of a pulse sequence based on a Spike cube pulse neural network is characterized in that the pulse sequence is in a two-dimensional image information coding mode, each pixel of a photosensitive chip of a nerve form camera detects light intensity change of the photosensitive chip within the time of the minimum time resolution, when the variation exceeds an excitation threshold corresponding to the photosensitive chip, a signal representing the position and the excitation moment of the pixel is sent out immediately, and the signal is output through a bus, so that the pulse sequence with moment information and space coordinate information is formed. The pulse sequence may be pulse data output from a neuromorphic camera such as a DVS (Dynamic Vision Sensor), an event-type camera, or a retinal-like camera, or may be generated by software simulation, for example, a pulse sequence generated by converting each pixel of a still image into a poisson distribution. The invention is based on spiking neural networks (spike cube and LIF neuron models) and adopts an STDP mechanism to learn the synaptic weights and the excitation threshold values of the connections between the neurons of each layer of the spiking neural network, and the learning process comprises the following steps:
1) constructing a pulse neural network, wherein neurons of the neural network are represented by an LIF model;
2) preparing an image pulse sequence to be learned, and dividing the image pulse sequence into a training set, a verification set and a test set according to a certain ratio (generally, the image pulse sequence can be divided according to the ratio of 5:1: 1);
the Spike cube (pulse sequence unit) is a pulse sequence segment obtained by dividing an input image pulse sequence at equal intervals according to the minimum time length of a time-series pattern included in an image pulse sequence to be learned or recognized. The input image pulse sequence may be pulse data output from a neuromorphic camera such as a DVS (Dynamic Vision Sensor), an event-type camera, or a retinal-like camera, or may be generated by software simulation, for example, a pulse sequence generated by converting each pixel of a still image into a poisson distribution.
3) Inputting the training set image pulse sequence into a pulse neural network for forward propagation, and sequentially passing through a first convolution layer, a second convolution layer, a full-connection layer and a classification layer of the pulse neural network; in the layer-by-layer propagation process, learning the synaptic weights and the excitation thresholds of the connections of all layers of neurons by adopting an STDP mechanism based on spike cube; in particular, the convolutional layer synaptic weights and the excitation thresholds have different characteristic forms at different time instants, and the method for jointly learning the temporal and spatial information of the image pulse sequence comprises the following steps: inputting a calibrated image pulse sequence, forward propagation and layered learning;
31) inputting the image pulse sequence with calibration to a pulse neural network;
inputting a calibrated image pulse sequence in a learning stage, wherein the image pulse sequence is composed of a large number of pulse sequence units (spike cubes); the first one is a pulse sequence which is output by recording some targets to be recognized through a DVS camera, an event-type camera, an imitation retina camera and other nerve morphology cameras, the image category is calibrated by dividing the pulse sequences at equal intervals according to the minimum time length of a pulse time sequence mode required for representing the targets to be recognized, and the calibration value of each divided pulse sequence unit is the recording target of the nerve morphology camera corresponding to the segment. The second method is to convert each pixel of the static image into a pulse sequence unit according to Poisson distribution by simulating the characteristics of the neuromorphic camera through software, wherein the length of the pulse sequence unit is set as the minimum time length of a pulse time sequence mode required for representing the target to be recognized, and the calibration value of the pulse sequence unit is set as the corresponding static image category. 32) Forward propagation, comprising:
321) a sequence of calibration pulses is propagated forward through the first convolutional layer;
the first convolution layer is composed of 1 × N convolution kernels with the step length of 1 and N corresponding characteristic graphs, the characteristic graphs are equivalent to a response area composed of a certain number of neurons, the neurons adopt an LIF neuron model, when synapses have pulse input, the membrane potential of the model increases the strength of the synapses at the arrival moment of the pulses, if the membrane potential of the neurons does not reach an excitation threshold, the membrane potential exponentially decays along with time, and if the membrane potential of the neurons reaches the excitation threshold, the membrane potential suddenly drops and excites one pulse to be transferred to the next connected neuron; the convolution kernels are equivalent to shared synapses of neurons in the connected characteristic graphs, each convolution kernel is connected to the neurons in the corresponding area of each characteristic graph according to the respective positions of all input pulses at the current moment, and the membrane potential of the neurons is correspondingly changed according to the conditions of the input pulses of the convolution synapses. At each time scale, the neuron checks to see if its membrane potential exceeds the firing threshold. If some of the neuron membrane potentials in the profile exceed the firing threshold, firing produces a pulse, and if the firing threshold is not exceeded, the pulse is not fired.
322) The pulse propagates forward through the second convolutional layer;
the pulse excited by each characteristic map neuron of the first convolutional layer is transmitted to the second convolutional layer. The second convolution layer is composed of N × M convolution kernels with the step size of 2 and M characteristic graphs corresponding to the convolution kernels. The neuron adopts a LIF neuron model, the step length is 2, the size of the second convolutional layer feature map can be reduced to half of the size of the first convolutional layer feature map, and the neuron in each feature map of the second convolutional layer is connected with the corresponding neuron in the N feature maps of the first convolutional layer through convolutional core synapses. After the neurons in the first convolutional layer are excited, the signals are transmitted to corresponding neurons of each characteristic diagram of the second convolutional layer through convolutional synapses, after the neurons of the second convolutional layer receive pulses, the membrane potential values of the neurons of the second convolutional layer are correspondingly adjusted according to the condition that the convolutional synapses connected with the neurons input pulses, the membrane potential values of the neurons of the second convolutional layer are detected at each time scale, if the membrane potential values reach an excitation threshold value, one pulse is excited and transmitted to the next layer of neurons connected with the neurons, and if the membrane potential values do not reach the excitation threshold value, the pulses are not excited.
323) The pulse propagates forward through the fully connected layer;
the pulse excited by each characteristic diagram neuron of the second convolution layer is transmitted to the full connection layer. The full-link layer is composed of K LIF neurons, and the value of K can be set to 1.5 times the number of neurons in the second convolutional layer. And if the number of the neurons in the second convolution layer is L, each neuron in the full-connection layer is connected with each neuron in the second convolution layer through L synapses, and L × K synapses are shared between the two layers. When a neuron in the second convolutional layer fires, the pulse is transmitted through synapses to neurons to which the fully-connected layer is connected. And the neurons of the full connection layer adjust the membrane potential of the neurons per se at each time scale according to the condition that the synapses connected with the neurons transmit pulses, detect whether the current membrane potential exceeds an excitation threshold, if so, excite one pulse and transmit the pulse to the neurons connected with the next layer, and if not, not exciting the pulse.
324) The pulse propagates forward to reach the classification layer
The pulse excited by each neuron of the full connection layer is transmitted to the classification layer. The classification layer consists of Y LIF neurons, the value of Y corresponding to the number of targets to be identified in the pulse sequence. Each neuron of the taxonomic layer is connected to each neuron of the fully-connected layer by K synapses, so there are a total of K × Y synapses from the fully-connected layer to the taxonomic layer. When a neuron in the fully connected layer fires, a pulse is transmitted through the synapse to each neuron in the classification layer. And the neurons of the classification layer can adjust the membrane potential of the neurons per se according to the pulse condition transmitted by all synapses connected with the neurons at each time scale, detect whether the membrane potential exceeds an excitation threshold, if so, excite the pulses and record the pulses in a variable corresponding to the neurons per se for storage, and if not, do not excite the pulses and do not record the pulses.
The learning method comprises a spike cube STDP mechanism (a spike cube-based unsupervised STDP learning mechanism) and a spike cube BP-STDP (spike cube STDP based back-propagation algorithm based on a pulse sequence unit and pulse time dependent synaptic plasticity) mechanism. The Spike cube STDP mechanism is used for learning the synaptic weights of the convolutional layer, and the Spike cube BP-STDP mechanism is used for learning the synaptic weights of the full-link layer and the classification layer. The whole learning process is carried out in a layered mode, synaptic weight learning of the spike cube STDP mechanism is carried out firstly, and after learning is finished, synaptic weight learning of the spike cube BP-STDP mechanism is carried out.
4) After learning for a period of time, temporarily interrupting the learning process of synapses, inputting the image pulse sequence of the verification set into the current pulse neural network, and carrying out the identification accuracy rate test of image categories;
5) if the identification accuracy reaches the expected standard, ending the learning process and storing the current synapse weight and the excitation threshold; if the expected standard is not reached, continuing learning on the basis until the expected standard is reached or the whole training set pulse sequence is completely learned;
6) and after learning is finished, the trained pulse neural network is obtained.
The pulse neural network obtained by learning the time and space information of the pulse sequence based on the pulse neural network (spike cube and LIF neuron model) and by adopting the STDP mechanism to carry out the combined learning method is applied to image recognition, and the method comprises the following steps:
1) inputting an image pulse sequence without calibration;
inputting an image pulse sequence without calibration in the identification process, and compared with the calibrated image pulse sequence, the difference is that the image class represented by the image pulse sequence is not required to be calibrated when the spike cube is divided;
2) forward propagation;
the forward propagation in the image sequence identification process is identical to the step "forward propagation" in the learning process. In the identification process, the learned excitation threshold values of each layer are properly adjusted to be low (for example, the learned excitation threshold values of each neuron are adjusted to be 0.9 times in the identification process), so that the final identification rate is weakly improved;
3) classifying and identifying the images; the method comprises the following steps:
after all spike cube image pulse sequence units are input into a pulse neural network and transmitted to the neuron of the classification layer, counting the number of excitation pulses stored in the variable corresponding to each neuron of the classification layer;
and then selecting the classified neuron with the largest number of excitation pulses as the most active neuron, if the number of pulses excited by the two neurons is the same, selecting the neuron with the highest accumulated membrane potential as the most active neuron, and finally, obtaining the image recognition result of the impulse neural network as the image class represented by the most active neuron. For example, a pulse sequence is input into the pulse neural network, the pulses are propagated in the forward direction to finally cause the excitation of the neurons in the classification layer, after all the pulse sequence units are input into the pulse neural network, the number of pulses excited by each neuron in the classification layer is counted, and if the number of pulses excited by the neuron number 2 is the largest, the recognition result of the pulse neural network on the pulse sequence unit is 2.
The classification result obtained by applying the impulse neural network obtained by learning through the joint learning method provided by the invention to image recognition also shows that the impulse neural network obtained by learning through the joint learning method provided by the invention can realize classification recognition on the impulse sequence.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a method for jointly learning time and space information contained in an image pulse sequence based on a spike cube pulse neural network, and the learned pulse neural network can realize the identification of the image pulse sequence. Compared with the existing scheme, such as a method for learning and identifying the sequence through a cyclic neural network, a method for learning a static image through a convolutional neural network, converting the convolutional neural network after learning into a pulse neural network form corresponding to the static image, and then identifying the image pulse sequence, the core innovation point of the invention is to provide a method for jointly learning the time and space information of the pulse sequence by adopting an STDP mechanism based on spike cube and LIF neuron models. The invention creatively realizes the joint learning of the time sequence information and the spatial position information contained in the pulse sequence based on spike cube and STDP mechanisms, the learned pulse neural network has the capability of identifying the pulse sequence, and the trained model is utilized to carry out image classification and identification. The invention provides a new method for learning and classifying and identifying the image pulse sequence.
Drawings
FIG. 1 is a pulse sequence segment generated by Poisson transformation of a static handwritten digit 0 in an embodiment of the present invention;
wherein (a) a static handwritten number of 0; (b) pulse sequence fragments generated by poisson transformation of panel (a).
Fig. 2 is a prior art simple impulse neural network architecture.
FIG. 3 shows the pulse distribution and accumulation effects at simulation times 1,3, and 7, respectively, in an embodiment of the present invention;
wherein, (a) the pulse release effect when the simulation time is 1; (b) a pulse accumulation effect at a simulation time of 3, and (c) a pulse accumulation effect at a simulation time of 7.
FIG. 4 is a diagram of LIF neuron membrane potential characteristics;
t1 (1)、t2 (1)indicates that LIF neuron No. 1 synapses are at t1、t2At the moment a pulse is received, t3 (2)、t4 (2)Indicates that LIF neuron No. 2 synapse is at t3、t4At the moment a pulse is received, tj (f)Denotes the No. f LIF neuron at t j1 pulse is fired at a time.
Detailed Description
The invention will be further described by way of examples, without in any way limiting the scope of the invention, with reference to the accompanying drawings.
The invention provides a method for performing joint learning on time and space information contained in a pulse sequence based on a spike cube pulse neural network, and the method can be applied to the class identification of images. Wherein the pulse sequence comprises an image pulse sequence, voice sequence information, vibration sequence information and the like.
The invention provides a method for performing combined learning on time and space information of a pulse sequence by adopting an STDP mechanism based on spike cube and LIF neuron models. The realization process is as follows: firstly, a set of impulse neural network can be set up by referring to the framework of fig. 2, and neurons of the neural network select a LIF model; then preparing a pulse sequence to be learned, dividing the pulse sequence into a training set, a verification set and a test set (generally, the pulse sequence can be divided according to the ratio of 5:1: 1) according to a certain ratio, inputting the training set pulse sequence into a pulse neural network for forward propagation, and sequentially passing through a first convolutional layer, a second convolutional layer, a full-link layer and a classification layer of the pulse neural network shown in fig. 2; in the layer-by-layer propagation process, learning the synaptic weights and the excitation thresholds of the connections of all layers of neurons by adopting an STDP mechanism based on spike cube; after learning for a period of time, temporarily interrupting the learning process of synapses, inputting a verification set pulse sequence into a current pulse neural network for carrying out a recognition rate test, if the recognition rate reaches an expected standard, ending the learning process and storing the current synapse weight and an excitation threshold, and if the recognition rate does not reach the expected standard, continuing learning on the basis until the expected standard is reached or the whole training set pulse sequence is completely learned; after learning is finished, the pulse neural network can be used for carrying out recognition rate test on the test set pulse sequence.
The Spike cube is a pulse sequence segment obtained by dividing an input pulse sequence at equal intervals according to the minimum time length of a pulse time sequence pattern to be recognized. The input pulse sequence may be pulse data output by a neuromorphic camera such as a DVS (Dynamic Vision Sensor), an event-type camera, or a retinal-like camera, or may be generated by software simulation, for example, a pulse sequence generated by converting each pixel of a still image according to a poisson distribution is shown in fig. 1, which shows a pulse sequence segment generated by poisson converting a static handwritten digit 0 by software simulation.
The method for jointly learning the time and space information of the pulse sequence comprises the following steps: inputting a calibration pulse sequence, forward propagation and layered learning, wherein the pulse neural network after the learning can identify the pulse sequence, and the identification process comprises the following steps: inputting a non-calibration pulse sequence, forward propagation and classification identification. The method specifically comprises the following steps:
the learning process comprises the following steps:
1. inputting a calibrated pulse sequence
The first is a pulse sequence which is output by recording some targets to be recognized through a DVS camera, an event-type camera, a retina-imitating camera and other nerve form cameras, the pulse sequences are divided at equal intervals according to the minimum time length of a pulse time sequence mode required for representing the targets to be recognized, and the calibration value of each divided pulse sequence segment is the recording target of the nerve form camera corresponding to the segment. And secondly, simulating the characteristics of the neuromorphic camera through software, converting each pixel of the static image into a pulse sequence segment according to Poisson distribution, setting the length of the pulse sequence segment as the minimum time length of a pulse time sequence mode required for representing the target to be identified, and setting the calibration value of the pulse sequence segment as the static image corresponding to the segment. (to facilitate understanding of the generation and calibration process of the pulse sequence, i use an example to illustrate that, a pulse output sequence recorded by a neuromorphic camera is simulated by software, the simulation time unit is set to 1 microsecond, as shown in fig. 3, the pulse release and accumulation effects of simulation times 1,3 and 7 are respectively demonstrated, when the simulation time is 1, the output condition of the pulse is demonstrated, at the minimum time resolution, the pulse released at a single time can not represent the converted still image handwritten digit 7, when the simulation time is 3, the accumulated pulse can already preliminarily represent the basic contour of the converted still image handwritten digit 7, but still can not represent the handwritten digit 7 under the influence of surrounding noise pulses, when the simulation time is 7, the accumulated pulse can already represent the recorded still image handwritten digit 7 completely, the length of the pulse sequence segment is thus set to 7. )
After all the calibrated pulse sequence segments are prepared, the pulse sequence segments are input into the pulse neural network one by one.
2. Forward propagation
(1) The pulse propagates forward through the first convolutional layer
The pulse sequence with calibration inputs the pulse of each time into the impulse neural network according to the time scale sequence, and reaches the first convolution layer of the impulse neural network at first. The first convolution layer is composed of 1 × N convolution kernels with step size 1 and N corresponding feature maps, the feature maps correspond to a response region composed of a certain number of neurons, the neurons adopt an LIF neuron model, the membrane potential characteristics of the model are shown in fig. 4 (when a synapse has a pulse input, the membrane potential increases the strength of the synapse at the arrival time of the pulse, if the membrane potential of the neuron does not reach an excitation threshold, the membrane potential exponentially decays with time, if the membrane potential of the neuron reaches the excitation threshold, the membrane potential suddenly drops and excites a pulse to be transferred to the next connected neuron), the convolution kernels correspond to shared synapses of the neurons in the connected feature maps, each convolution kernel is connected to the neurons in the corresponding region of each feature map according to the respective position of all input pulses at the current time, the membrane potential of these neurons is changed accordingly depending on the condition of the convolved synapse input pulse (if the excitatory strength of the convolved synapse is greater than the inhibitory strength, the membrane potential is increased by an amount corresponding thereto, whereas the membrane potential is decreased by an amount corresponding thereto). At each time scale, the neuron checks to see if its membrane potential exceeds the firing threshold. If some of the neuron membrane potentials in the profile exceed the firing threshold, firing produces a pulse, and if the firing threshold is not exceeded, the pulse is not fired.
(2) The pulse propagates forward through the second convolutional layer
The pulse excited by each feature map neuron of the first convolutional layer is transmitted to the second convolutional layer. The second convolution layer is composed of N × M convolution kernels with the step size of 2 and M characteristic graphs corresponding to the convolution kernels. The neuron adopts a LIF neuron model, the step length is 2, the size of the second convolutional layer feature map can be reduced to half of the size of the first convolutional layer feature map, and the neuron in each feature map of the second convolutional layer is connected with the corresponding neuron in the N feature maps of the first convolutional layer through convolutional core synapses. After the neurons in the first convolutional layer are excited, the signals are transmitted to corresponding neurons of each characteristic diagram of the second convolutional layer through convolutional synapses, after the neurons of the second convolutional layer receive pulses, the membrane potential values of the neurons are correspondingly adjusted according to the condition that the convolutional synapses connected with the neurons input the pulses, the membrane potential values of the neurons are detected at each time scale, if the membrane potential values reach an excitation threshold value, one pulse is excited and transmitted to the next layer of neurons connected with the neurons, and if the membrane potential values do not reach the excitation threshold value, the pulses are not excited.
(3) Pulse forward propagation through fully connected layers
The pulse excited by each characteristic diagram neuron of the second convolution layer is transmitted to the full connection layer. The full-link layer is composed of K LIF neurons, and the value of K can be set to 1.5 times the number of neurons in the second convolutional layer. And if the number of the neurons in the second convolution layer is L, each neuron in the full-connection layer is connected with each neuron in the second convolution layer through L synapses, and L × K synapses are shared between the two layers. When a neuron in the second convolutional layer fires, the pulse is transmitted through synapses to neurons to which the fully-connected layer is connected. And the neurons of the full connection layer adjust the membrane potential of the neurons per se at each time scale according to the condition that the synapses connected with the neurons transmit pulses, detect whether the current membrane potential exceeds an excitation threshold, if so, excite one pulse and transmit the pulse to the neurons connected with the next layer, and if not, not exciting the pulse.
(4) The pulse propagates forward to reach the classification layer
The pulse excited by each neuron of the full connection layer is transmitted to the classification layer. The classification layer consists of Y LIF neurons, the value of Y corresponding to the number of targets to be identified in the pulse sequence. Each neuron of the taxonomic layer is connected to each neuron of the fully-connected layer by K synapses, so there are a total of K × Y synapses from the fully-connected layer to the taxonomic layer. When a neuron in the fully connected layer fires, a pulse is transmitted through the synapse to each neuron in the classification layer. And the neurons of the classification layer can adjust the membrane potential of the neurons per se according to the pulse condition transmitted by all synapses connected with the neurons at each time scale, detect whether the membrane potential exceeds an excitation threshold, if so, excite the pulses and record the pulses in a variable corresponding to the neurons per se for storage, and if not, do not excite the pulses and do not record the pulses.
3. Layered learning
The learning algorithm provided by the invention is composed of an unsupervised STDP learning mechanism and a supervised BP-STDP (STDP-based back-propagation algorithm based on synaptic plasticity at pulse time) mechanism. The unsupervised STDP mechanism is used for learning the synaptic weights of the convolutional layer, and the supervised BP-STDP mechanism is used for learning the synaptic weights of the full-link layer and the classification layer. The whole learning process is carried out in a layered mode, synaptic weight learning of an unsupervised learning STDP mechanism is carried out firstly, and after learning is finished, synaptic weight learning of a supervised BP-STDP mechanism is carried out. Next, the hierarchical learning process is specifically described:
(1) unsupervised STDP learning
The unsupervised STDP mechanism is used for learning of convolutional layer synaptic weights. The learning process is performed hierarchically, first learning the synaptic weights of the first convolutional layer. As can be seen from the forward propagation process, when the input pulse passes through the first convolutional layer, part of the neurons in the first convolutional layer will fire. And then only one neuron with the highest membrane potential is selected from the neurons excited by each feature map to carry out synaptic plasticity learning of the STDP mechanism, and simultaneously, learning of other excited neurons at the same position in other feature maps of the layer is inhibited. If excited neurons appear at the same position in multiple signatures, only the neuron with the highest membrane potential is made to learn, and other neurons suppress their learning process. And if the neuron with the highest membrane potential in some feature maps is inhibited to learn, learning synaptic weights of the excitatory neurons with the next highest membrane potential, if the excitatory neurons with the next highest membrane potential are also inhibited, repeating the steps until all the excitatory neurons of the feature maps are traversed. If there are feature maps that do not fire neurons or that fire neurons are all inhibited from learning, then the feature maps do not learn synaptic plasticity at the current time scale. When the cost function of the convolutional synaptic weight of the first layer is converged within a certain range, the learning of the first layer is ended, and the learning of the synaptic weight of the second convolutional layer is started by using the same method. The method for judging the completion of the synaptic weight learning of the second convolutional layer is similar to that of the first convolutional layer.
Learning is also required for the excitation thresholds of the convolutional layer neurons. The convolutional synaptic weights are generally initialized to a normal distribution (mean μ, variance δ), and the firing threshold is generally initialized to the mean μ of the convolutional synaptic weight initialization. Each time the convolutional synapses are learned for a longer time interval, the firing threshold needs to be adjusted according to the learning effect of the convolutional synapse weights. The algorithm provided by the invention is to set the neuron excitation threshold value as the average value of the convolution synapse weights connected with the neuron excitation threshold value and then multiply the average value by a scaling coefficient, wherein the range of the scaling coefficient is generally between 0.5 and 1.0. The learning process of the firing threshold ends with the termination of the convolutional synaptic weight learning process.
In the learning process of the convolution weight and the excitation threshold, spike cube pulse sequence fragments are used as learning units, and the length of the spike cube pulse sequence fragments is assumed to be 5 time scales, so that the convolution weight and the excitation threshold respectively have different characteristic forms under the 5 time scales. And (3) carrying out weight plasticity learning on the current moment on the basis of inheriting the convolution weight information of the previous moment by the convolution weight of each current moment. The convolution synapses after learning are convolved with the input pulses in different characteristic forms at different time scales, and excitation thresholds at different moments are used for judging whether the neurons with the changed membrane potentials are excited or not after convolution. Through the association learning of the convolution synapse and the excitation threshold value, the convolution synapse extracts the time information of the pulse sequence. In addition, the input pulse is convoluted through a plurality of convolutions, the space mode information contained in the input pulse sequence is mapped from a low dimension to a high dimension, and conditions are provided for a classification layer to classify by utilizing high-dimension features. In summary, the learning algorithm provided by the present invention has the capability of fundamentally exploring the timing information and the spatial information contained in the pulse sequence.
(2) Supervised BP-STDP learning
A supervised BP-STDP learning mechanism is used for learning of fully connected and classified layer synaptic weights. The learning process is also carried out hierarchically, but the supervised reverse learning is carried out from the last hierarchical layer. The number of neurons in the classification layer is the same as the number of targets to be identified in the pulse sequence, and thus each neuron in the classification layer corresponds to one target to be identified. In the learning process of the synaptic weights, the input pulse sequence segments are calibrated, and the calibrated values are the categories of targets represented by the pulse sequence segments. After the pulse is propagated forward to reach the classification layer neuron, the excited classification layer neuron is compared with the calibration value of the pulse sequence, and the comparison result is divided into 3 conditions: the first is that the classification layer neurons representing the current calibration value class are not firing, which is labeled 1; second, the excited classification layer neurons do not represent the current calibration value class, which is labeled-1; the third case covers 2 results, one result being that a classified neuron representing the current calibration class fires, another result being that a neuron not representing the current calibration class does not fire, and the third case is labeled 0. According to the learning algorithm provided by the invention, for the first case, the synaptic weight between the firing neuron and the full connection layer is updated according to the STDP mechanism; for the second case, the synaptic weights between the excited neuron and the fully-connected layer are updated according to the anti-STDP mechanism (the anti-STDP mechanism is consistent with the STDP mechanism in principle, but the sign is opposite in the process of updating the weights, that is, the STDP mechanism multiplies the marker value by-1 on the basis of the change of the synaptic weights.
In each spike cube learning unit, every time the learning of the synaptic weight of the classification layer is finished, the learning of the synaptic weight of the full connection layer is immediately carried out according to the principle of back propagation. Unlike the back propagation algorithm in the numerical neural network, back propagation is based on a gradient, and in the impulse neural network, back propagation is based on the impulse condition of neuron excitation. The specific learning process of synaptic weights between the fully-connected layer and the second convolutional layer is as follows: firstly, traversing the excitation condition of each neuron of the full-connection layer one by one, if a pulse is excited at the current time scale, updating the synaptic weight connected between the neuron and the second convolutional layer, wherein the change amount of the weight is related to the excitation condition and the connection weight of the neuron and each neuron of the classification layer, namely, the excitation condition (1, -1,0) of each neuron of the classification layer is multiplied by the synaptic weight between the neuron and the neuron of the full-connection layer, the operation results are summed, and the result is multiplied by a learning rate parameter and the excitation state (excitation is 1, and unexcited is 0) of the corresponding neuron of the previous convolutional layer, so that the obtained value is the change amount of the connection synapse between the neuron and the corresponding neuron of the previous convolutional layer. According to the method, the synaptic weights of all the neurons of the full-connection layer connected with the previous convolutional layer are updated one by one, and one-time learning of the synaptic weights of the full-connection layer is completed. It should be noted that the neuron excitation threshold of the fully-connected layer is not required to be learned, and is preset to a fixed value during initialization.
And stopping the learning process temporarily every time after the synaptic weights of the neurons of the full connection layer and the classification layer are learned for a long period of time, inputting the pulse sequence of the verification set into the current pulse neural network for carrying out recognition rate test, ending the learning process and storing the synaptic weights and the excitation threshold value if the recognition rate reaches an expected standard, continuing learning on the basis if the recognition rate does not reach the expected standard, and ending the BP-STDP learning process of the synaptic weights of the neurons of the full connection layer and the classification layer until the expected standard is reached or the whole training set pulse sequence is completely learned.
The identification process comprises the following steps:
the pulse neural network after learning can identify the pulse sequence, and the identification process comprises the following steps: inputting a non-calibration pulse sequence, forward propagation and classification identification. The first step of inputting the pulse sequence without calibration is similar to the first step of inputting the calibrated pulse sequence in the learning process, and calibration is not needed when the spike cube is divided; the second step of forward propagation is completely the same as the second step of forward propagation in the learning process, but only slightly improves the final recognition rate after properly reducing the learned excitation thresholds of each layer in the recognition process (for example, adjusting the learned excitation thresholds of each neuron to 0.9 times in the recognition process); and a third step of 'classification and identification', wherein after all spike cube pulse sequence segments are input into the pulse neural network and transmitted to the neuron of the classification layer, the number of excitation pulses stored in variables corresponding to each neuron of the classification layer is counted, the neuron of the classification with the largest number of excitation pulses is selected as the most active neuron, if the number of pulses excited by two neurons is the same, the neuron with the highest accumulated membrane potential is selected as the most active neuron, and finally, the classification result output by the pulse neural network is the target class represented by the most active neuron.
It is noted that the disclosed embodiments are intended to aid in further understanding of the invention, but those skilled in the art will appreciate that: various substitutions and modifications are possible without departing from the spirit and scope of the invention and appended claims. Therefore, the invention should not be limited to the embodiments disclosed, but the scope of the invention is defined by the appended claims.
Claims (8)
1. A method for carrying out joint learning aiming at space-time information of an image pulse sequence is characterized in that a pulse sequence unit Spike cube and LIF neuron model is established based on a pulse neural network, and an STDP mechanism is adopted to learn synaptic weights and excitation thresholds connected among neurons of each layer of the pulse neural network; the method comprises the following steps:
1) establishing a pulse neural network, wherein neurons of the neural network are represented by an LIF model;
2) dividing an image pulse sequence to be learned into a training set, a verification set and a test set according to a ratio;
according to the minimum time length of a time sequence mode contained in an image pulse sequence to be learned, dividing the input image pulse sequence at equal intervals to obtain pulse sequence fragments, and calling the obtained pulse sequence fragments as pulse sequence units Spike cube;
3) inputting a training set image pulse sequence into the pulse neural network established in the step 1) for forward propagation, and sequentially passing through a first convolution layer, a second convolution layer, a full connection layer and a classification layer of the pulse neural network;
in the process of layer-by-layer propagation, learning synaptic weights and excitation thresholds which are connected among neurons in each layer in a layering manner, namely performing joint learning on time and space information of an image pulse sequence;
the joint learning method comprises the following steps: inputting a calibrated image pulse sequence, forward propagation and layered learning; firstly, learning the synaptic weight and the excitation threshold of the convolutional layer based on an unsupervised STDP learning mechanism of a pulse sequence unit, and then learning the synaptic weight of the full-link layer and the classification layer based on the pulse sequence unit and a back propagation mechanism depending on synaptic plasticity at the pulse time;
4) after learning for a period of time, temporarily interrupting the learning process of synapses, inputting the image pulse sequence of the verification set into the current pulse neural network, and carrying out the identification accuracy rate test of image categories;
5) if the identification accuracy reaches the expected standard, ending the learning process and storing the current synapse weight and the excitation threshold; if the expected standard is not reached, continuing learning on the basis until the expected standard is reached or the whole training set pulse sequence is completely learned;
6) and after learning is finished, the trained pulse neural network is obtained.
2. The method as claimed in claim 1, wherein the step 3) of the joint learning method comprises the following steps:
31) inputting the image pulse sequence segments with the calibration one by one into a pulse neural network;
32) performing forward propagation and hierarchical learning, including:
321) propagating the scaled pulse sequence segments forward through the first convolutional layer;
the first convolution layer comprises N convolution kernels and N characteristic graphs corresponding to the convolution kernels, the characteristic graphs are response areas formed by neurons, and the neurons adopt LIF neuron models; the convolution kernels are shared synapses of neurons in the characteristic diagrams connected with the convolution kernels, and each convolution kernel is connected to the neurons in the corresponding area of each characteristic diagram according to the respective position of all input pulses at the current moment; the membrane potential of the neuron obtains corresponding change according to the condition of the convolution synapse input pulse;
when a synapse has a pulse input, the membrane potential increases the strength of the synapse at the time of arrival of the pulse; if the membrane potential of the neuron does not reach the excitation threshold, the membrane potential exponentially decays along with time, and if the membrane potential of the neuron reaches the excitation threshold, the membrane potential suddenly drops and excites a pulse to be transferred to the next connected neuron;
at each time scale, the neuron checks whether the membrane potential exceeds a threshold; if certain neuron membrane potentials in the profile exceed a threshold, firing to produce a pulse, and if the threshold is not exceeded, not firing the pulse;
322) the pulse propagates forward through the second convolutional layer;
the second convolution layer comprises N × M convolution kernels and M characteristic graphs corresponding to the convolution kernels; the neuron adopts an LIF neuron model; the number of neurons in the second convolutional layer is L; after a neuron in the first convolutional layer is excited, transmitting the excited neuron to a corresponding neuron of each characteristic diagram of the second convolutional layer through a convolutional synapse, after the neuron of the second convolutional layer receives a pulse, correspondingly adjusting a membrane potential value of the neuron according to the condition that the convolutional synapse connected with the neuron inputs the pulse, detecting the membrane potential value of the neuron at each time scale, if the membrane potential value reaches an excitation threshold value, exciting one pulse and transmitting the pulse to the next layer of neuron connected with the neuron, and if the membrane potential value does not reach the threshold value, not exciting the pulse;
323) the pulse propagates forward through the fully connected layer;
the pulse excited by each characteristic diagram neuron of the second convolution layer is transmitted to the full connection layer; the full-junction layer comprises K LIF neurons; each neuron of the fully-connected layer is connected with each neuron of the second convolutional layer through L synapses, and the two layers have L synapses X K;
when the neurons of the second convolutional layer are excited, the pulse is transmitted to the neurons connected with the fully connected layer through synapses;
the neuron of the full-connection layer adjusts the membrane potential of the neuron according to the pulse transmitted by the connected synapse at each time scale, and detects whether the current membrane potential exceeds a threshold value; if the threshold is exceeded, a pulse is fired and delivered to the next layer of connected neurons; if the threshold is not reached, the pulse is not fired;
324) the pulse is propagated forwards to reach the classification layer;
the pulse excited by each neuron of the full connection layer is transmitted to the classification layer; the classification layer consists of Y LIF neurons, and the value of Y corresponds to the number of targets to be identified in the pulse sequence; each neuron of the classification layer is connected with each neuron of the full connection layer through K synapses, and K × Y synapses are totally obtained from the full connection layer to the classification layer;
when the neuron of the full connection layer is excited, the pulse is transmitted to each neuron of the classification layer through synapses; the neuron of the classification layer adjusts the membrane potential of the neuron according to the pulse transmitted by all connected synapses at each time scale, and detects whether the membrane potential exceeds a threshold value;
if the threshold value is exceeded, the pulse is excited and recorded in a variable corresponding to the pulse to be stored, and if the threshold value is not exceeded, the pulse is not excited.
3. The method of joint learning for spatio-temporal information of an image pulse sequence as set forth in claim 2, wherein the first convolution layer has 1 x N convolution kernels and a convolution kernel step size of 1; the convolution kernels of the second convolution layer are N x M, and the step length is 2; the size of the second convolution layer feature map is reduced to half the size of the first convolution layer feature map.
4. The method of joint learning for spatio-temporal information of image pulse sequences as set forth in claim 1, wherein the obtaining of the scaled image pulse sequences comprises two methods:
the pulse sequence output by recording the target through the neuromorphic camera is specifically characterized in that: dividing the pulse sequence at equal intervals according to the minimum time length of a pulse time sequence mode required by the representation target, wherein the calibration value of each divided pulse sequence segment is the recording target of the nerve morphology camera corresponding to the segment;
for a pulse sequence segment generated by simulating the characteristics of a neuromorphic camera through a software tool, the specific method is as follows: the length of the pulse sequence segment is set as the minimum time length of a pulse time sequence mode required by representing the target to be identified, and the calibration value of the pulse sequence segment is set as a static image corresponding to the segment.
5. A method as claimed in any one of claims 1 to 4 for joint learning of spatio-temporal information of image pulse sequences, characterized in that the pulse sequences may also be speech sequence information or vibration sequence information.
6. An image recognition method for performing joint learning based on image pulse sequence spatio-temporal information, which performs image classification and recognition by using the trained pulse neural network obtained by the method for performing joint learning based on the image pulse sequence spatio-temporal information according to any one of claims 1 to 4, and comprises the following steps:
s1) inputting an image pulse sequence without calibration; the characterized image categories do not need to be calibrated when the Spike cube is divided;
s2) executing the step of forward propagation in the step 3), and performing forward propagation;
s3) classification and identification; the method comprises the following steps:
s31) all the Spike cube image pulse sequence segments are input into the pulse neural network and transmitted to the neuron of the classification layer;
s32), counting the number of excitation pulses stored in the variables corresponding to the neurons of each classification layer;
s33) then taking the classified neuron with the largest number of excitation pulses as the most active neuron;
s34) if the number of pulses excited by the two neurons is the same, taking the neuron with the highest accumulated membrane potential as the most active neuron;
s35) outputs the image class represented by the most active neuron as the image classification result.
7. The image recognition method of claim 6, wherein in step S2), during forward propagation, the values of the excitation thresholds of the learned neurons are adjusted down to improve the recognition rate.
8. The image recognition method for performing joint learning based on spatio-temporal information of an image pulse sequence according to claim 6, wherein the image pulse sequence is pulse data output by a neuromorphic camera or a pulse sequence generated by simulation; the neuromorphic camera includes: dynamic vision sensor cameras, event-type cameras, retinal-like cameras.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910481420.9A CN110210563B (en) | 2019-06-04 | 2019-06-04 | Image pulse data space-time information learning and identification method based on Spike cube SNN |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910481420.9A CN110210563B (en) | 2019-06-04 | 2019-06-04 | Image pulse data space-time information learning and identification method based on Spike cube SNN |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110210563A CN110210563A (en) | 2019-09-06 |
| CN110210563B true CN110210563B (en) | 2021-04-30 |
Family
ID=67790543
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910481420.9A Active CN110210563B (en) | 2019-06-04 | 2019-06-04 | Image pulse data space-time information learning and identification method based on Spike cube SNN |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110210563B (en) |
Families Citing this family (46)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110826602B (en) * | 2019-10-23 | 2022-04-26 | 中国科学院自动化研究所 | Image classification method and system based on membrane potential regulation and control pulse neural network |
| CN110826437A (en) * | 2019-10-23 | 2020-02-21 | 中国科学院自动化研究所 | Intelligent robot control method, system and device based on biological neural network |
| CN110781968B (en) * | 2019-10-29 | 2022-04-19 | 中国人民解放军国防科技大学 | Extensible class image identification method based on plastic convolution neural network |
| CN110991610B (en) * | 2019-11-28 | 2022-08-05 | 华中科技大学 | Probability determination method for nondeterministic problem |
| CN111091815A (en) * | 2019-12-12 | 2020-05-01 | 电子科技大学 | Voice recognition method of aggregation label learning model based on membrane voltage driving |
| CN111046954A (en) * | 2019-12-12 | 2020-04-21 | 电子科技大学 | Image classification method of spiking learning model based on dynamic threshold |
| CN111275742B (en) * | 2020-01-19 | 2022-01-11 | 北京大学 | Target identification method, device and system and computer readable storage medium |
| CN111340194B (en) * | 2020-03-02 | 2022-09-06 | 中国科学技术大学 | Pulse convolution neural network neural morphology hardware and image identification method thereof |
| CN111488908A (en) * | 2020-03-10 | 2020-08-04 | 天津大学 | Brain-imitating image identification method based on enhanced pulse |
| CN113688980A (en) * | 2020-05-19 | 2021-11-23 | 深圳忆海原识科技有限公司 | Brain-like visual neural network with forward learning and meta learning functions |
| CN111612136B (en) * | 2020-05-25 | 2023-04-07 | 之江实验室 | Neural morphology visual target classification method and system |
| CN114022652A (en) * | 2020-07-15 | 2022-02-08 | 中移(苏州)软件技术有限公司 | Data processing method, equipment and device and computer storage medium |
| CN112130118B (en) * | 2020-08-19 | 2023-11-17 | 复旦大学无锡研究院 | Ultra-wideband radar signal processing system and method based on SNN |
| CN112085768B (en) * | 2020-09-02 | 2023-12-26 | 北京灵汐科技有限公司 | Optical flow information prediction method, optical flow information prediction device, electronic equipment and storage medium |
| CN112155549B (en) * | 2020-09-04 | 2023-11-14 | 西北师范大学 | ADHD disease diagnosis auxiliary decision-making system based on deep convolution impulse neural network |
| CN112418296B (en) * | 2020-11-18 | 2024-04-02 | 中国科学院上海微系统与信息技术研究所 | Bionic binocular target identification and tracking method based on human eye visual attention mechanism |
| CN112633497B (en) * | 2020-12-21 | 2023-08-18 | 中山大学 | Convolutional impulse neural network training method based on re-weighted membrane voltage |
| CN112767501B (en) * | 2021-01-06 | 2022-07-12 | 西南大学 | VCSEL-SA image identification system and method based on electric control stimulation |
| CN112699956B (en) * | 2021-01-08 | 2023-09-22 | 西安交通大学 | Neuromorphic visual target classification method based on improved impulse neural network |
| CN112762100B (en) * | 2021-01-14 | 2021-08-10 | 哈尔滨理工大学 | Bearing full-life-cycle monitoring method based on digital twinning |
| CN113111758B (en) * | 2021-04-06 | 2024-01-12 | 中山大学 | SAR image ship target recognition method based on impulse neural network |
| CN112906828A (en) * | 2021-04-08 | 2021-06-04 | 周士博 | Image classification method based on time domain coding and impulse neural network |
| CN113095492B (en) * | 2021-04-14 | 2023-04-18 | 北京大学 | Topological feature detection method and device based on biological neural network |
| CN113397482B (en) * | 2021-05-19 | 2023-01-06 | 中国航天科工集团第二研究院 | Human behavior analysis method and system |
| CN113077017B (en) * | 2021-05-24 | 2022-12-13 | 河南大学 | Synthetic aperture image classification method based on pulse neural network |
| CN113375676B (en) * | 2021-05-26 | 2024-02-20 | 南京航空航天大学 | Detector landing site positioning method based on impulse neural network |
| CN113269113B (en) * | 2021-06-04 | 2024-04-30 | 北京灵汐科技有限公司 | Human behavior recognition method, electronic device, and computer-readable medium |
| CN113269313B (en) * | 2021-06-04 | 2024-05-10 | 北京灵汐科技有限公司 | Synaptic weight training method, electronic device, and computer-readable medium |
| WO2022253229A1 (en) | 2021-06-04 | 2022-12-08 | 北京灵汐科技有限公司 | Synaptic weight training method, target recognition method, electronic device, and medium |
| CN113269264A (en) * | 2021-06-04 | 2021-08-17 | 北京灵汐科技有限公司 | Object recognition method, electronic device, and computer-readable medium |
| CN113408613B (en) * | 2021-06-18 | 2022-07-19 | 电子科技大学 | Single-layer image classification method based on delay mechanism |
| CN113255905B (en) * | 2021-07-16 | 2021-11-02 | 成都时识科技有限公司 | Signal processing method of neurons in impulse neural network and network training method |
| CN114037050B (en) * | 2021-10-21 | 2022-08-16 | 大连理工大学 | Robot degradation environment obstacle avoidance method based on internal plasticity of pulse neural network |
| CN114065806B (en) * | 2021-10-28 | 2022-12-20 | 贵州大学 | Manipulator touch data classification method based on impulse neural network |
| CN114091663A (en) * | 2021-11-28 | 2022-02-25 | 重庆大学 | Lightweight on-chip learning method, system and processor based on impulse neural network |
| CN114220089B (en) * | 2021-11-29 | 2024-06-14 | 北京理工大学 | Method for pattern recognition based on sectional progressive pulse neural network |
| CN113902106B (en) * | 2021-12-06 | 2022-02-22 | 成都时识科技有限公司 | Pulse event decision device, method, chip and electronic equipment |
| CN113962371B (en) * | 2021-12-23 | 2022-05-20 | 中科南京智能技术研究院 | Image identification method and system based on brain-like computing platform |
| CN114155478B (en) * | 2022-02-09 | 2022-05-10 | 苏州浪潮智能科技有限公司 | Emotion recognition method, device and system and computer readable storage medium |
| CN114202068B (en) * | 2022-02-17 | 2022-06-28 | 浙江大学 | Self-learning implementation system for brain-like computing chip |
| CN114659553A (en) * | 2022-02-28 | 2022-06-24 | 联想(北京)有限公司 | Detection method, device, equipment and storage medium |
| CN114429491B (en) * | 2022-04-07 | 2022-07-08 | 之江实验室 | Pulse neural network target tracking method and system based on event camera |
| CN115238857B (en) * | 2022-06-15 | 2023-05-05 | 北京融合未来技术有限公司 | Neural network based on pulse signals and pulse signal processing method |
| CN115246559A (en) * | 2022-07-22 | 2022-10-28 | 鄂尔多斯市国源矿业开发有限责任公司 | Industrial belt longitudinal tearing identification method |
| CN115994563B (en) * | 2022-10-31 | 2023-08-18 | 天津大学 | Brain-like situation learning model construction and training method for intelligent auxiliary driving |
| CN115810138A (en) * | 2022-11-18 | 2023-03-17 | 天津大学 | Image identification method based on multi-electrode array in-vitro culture neuron network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875846A (en) * | 2018-05-08 | 2018-11-23 | 河海大学常州校区 | A kind of Handwritten Digit Recognition method based on improved impulsive neural networks |
| CN109102000A (en) * | 2018-09-05 | 2018-12-28 | 杭州电子科技大学 | A kind of image-recognizing method extracted based on layered characteristic with multilayer impulsive neural networks |
| CN109635938A (en) * | 2018-12-29 | 2019-04-16 | 电子科技大学 | A kind of autonomous learning impulsive neural networks weight quantization method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105404902B (en) * | 2015-10-27 | 2019-02-05 | 清华大学 | Characteristics of image description and accumulating method based on impulsive neural networks |
| US20170236027A1 (en) * | 2016-02-16 | 2017-08-17 | Brainchip Inc. | Intelligent biomorphic system for pattern recognition with autonomous visual feature extraction |
| US10885425B2 (en) * | 2016-12-20 | 2021-01-05 | Intel Corporation | Network traversal using neuromorphic instantiations of spike-time-dependent plasticity |
-
2019
- 2019-06-04 CN CN201910481420.9A patent/CN110210563B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875846A (en) * | 2018-05-08 | 2018-11-23 | 河海大学常州校区 | A kind of Handwritten Digit Recognition method based on improved impulsive neural networks |
| CN109102000A (en) * | 2018-09-05 | 2018-12-28 | 杭州电子科技大学 | A kind of image-recognizing method extracted based on layered characteristic with multilayer impulsive neural networks |
| CN109635938A (en) * | 2018-12-29 | 2019-04-16 | 电子科技大学 | A kind of autonomous learning impulsive neural networks weight quantization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110210563A (en) | 2019-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110210563B (en) | Image pulse data space-time information learning and identification method based on Spike cube SNN | |
| KR102641116B1 (en) | Method and device to recognize image and method and device to train recognition model based on data augmentation | |
| CN109871940B (en) | Multi-layer training algorithm of impulse neural network | |
| CN108510194A (en) | Air control model training method, Risk Identification Method, device, equipment and medium | |
| CN108985252B (en) | Improved image classification method of pulse depth neural network | |
| US5222195A (en) | Dynamically stable associative learning neural system with one fixed weight | |
| CN111417963A (en) | Improved spiking neural network | |
| CN114155478B (en) | Emotion recognition method, device and system and computer readable storage medium | |
| CN106951923B (en) | Robot three-dimensional shape recognition method based on multi-view information fusion | |
| US20170337469A1 (en) | Anomaly detection using spiking neural networks | |
| CN107122798A (en) | Chin-up count detection method and device based on depth convolutional network | |
| Xiao et al. | An event-driven categorization model for AER image sensors using multispike encoding and learning | |
| KR20170031695A (en) | Decomposing convolution operation in neural networks | |
| JP2005267105A (en) | Parallel pulse signal processor, pattern recognition device and picture inputting device | |
| CN108921019A (en) | A kind of gait recognition method based on GEI and TripletLoss-DenseNet | |
| CN113205048B (en) | Gesture recognition method and system | |
| KR20160138042A (en) | Invariant object representation of images using spiking neural networks | |
| WO1990014631A1 (en) | Dynamically stable associative learning neural system | |
| CN112906828A (en) | Image classification method based on time domain coding and impulse neural network | |
| US9361545B2 (en) | Methods and apparatus for estimating angular movement with a single two dimensional device | |
| Pavel et al. | Recurrent convolutional neural networks for object-class segmentation of RGB-D video | |
| CN112101535A (en) | Signal processing method of pulse neuron and related device | |
| CN114186672A (en) | Efficient high-precision training algorithm for impulse neural network | |
| CN116524183A (en) | Camouflage target detection method based on multitask adapter fine adjustment | |
| CN111310816B (en) | Method for recognizing brain-like architecture image based on unsupervised matching tracking coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |