CN115546556A - Training method of pulse neural network for image classification - Google Patents

Training method of pulse neural network for image classification Download PDF

Info

Publication number
CN115546556A
CN115546556A CN202211289610.9A CN202211289610A CN115546556A CN 115546556 A CN115546556 A CN 115546556A CN 202211289610 A CN202211289610 A CN 202211289610A CN 115546556 A CN115546556 A CN 115546556A
Authority
CN
China
Prior art keywords
neural network
branch
pulse
ann
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211289610.9A
Other languages
Chinese (zh)
Inventor
赖剑煌
唐建雄
谢晓华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intelligent Eyes Automation Technology Guangzhou Co ltd
Original Assignee
Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Yat Sen University filed Critical Sun Yat Sen University
Priority to CN202211289610.9A priority Critical patent/CN115546556A/en
Publication of CN115546556A publication Critical patent/CN115546556A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Abstract

Aiming at the limitation of the prior art, the invention provides a training method of a pulse neural network for image classification, provides a rapid and memory-saving pulse neural network training method, designs and applies an artificial neural network-pulse neural network weight sharing frame, and transfers the training of the pulse neural network to the artificial neural network for carrying out, and under the same network structure and time window setting, the training speed and the display memory consumption of the invention are 43-67 percent and 33-55 percent of a pulse back propagation model.

Description

Training method of pulse neural network for image classification
Technical Field
The invention relates to the technical field of brain-like computing methods, in particular to a training method of a pulse neural network for image classification.
Background
Spiking Neural Networks (SNNs) are a Neural Network model based on a mechanism that mimics the discharge of biological neurons. Due to its ability to efficiently handle discrete spatio-temporal events, at low power consumptionAnd has wide application scenes. The basic building blocks of SNNs include Spiking neurons (Spiking Neuron) and synapses (Synapse). The impulse neuron is the basic calculation unit of the SNN, and the synapse is the connection between neurons, and is responsible for converting the impulse signal generated from the pre-synaptic neuron into a stimulus, which is input to the post-synaptic neuron. Integrated-and-Fire (IF) neurons are widely used for modeling SNNs. The impulse neurons are connected by synapses, and the constructed SNN can receive impulse events from the outside world and make corresponding responses, which can be generally expressed by the following formula: z is a radical of formula t ＝g(x t ).
In the formula, x t Representing the pulse event at time t, and the function g (-) represents the SNN model, which combines the pulse events x t As input, and outputs a corresponding value z t Like Artificial Neural Networks (ANNs), SNNs include an input layer, which is responsible for receiving pulse events and passing them to a hidden layer, and an output layer. The hidden layer comprises a plurality of layers of networks, each layer of network comprises a trunk pulse neuron, and each layer of network is connected in a layer-by-layer cascading mode. The output layer does not consider an accumulation-release mechanism, and is responsible for integrating the pulse output of the last hidden layer into the membrane voltage through synapses, and responding to the input pulse event according to the magnitude of the membrane voltage value accumulated in the whole event window. In addition, the connection structure of each layer of neurons can be generally classified into a full connection structure and a convolution structure. The full-connection structure is only that two layers of neurons before and after synapse are connected and can be generally used for connecting the last hidden layer and the output layer; the convolutional linking structure then links several neighboring neurons of the presynaptic network with the postsynaptic neuron by a convolution kernel (typically a tensor of size k × k × c, where k × k denotes the size of the receptive field and c denotes the number of connections to the postsynaptic neuron). SNNs containing convolutional connective structures are often referred to as convolutional impulse Neural networks (CSNN), which mainly apply tasks related to processing vision. SNNs with deep structures are able to respond to incoming impulse events, but to be able to recognize them, algorithms need to be designed to learn the network. So-called learning, also known asFor training, it adjusts synaptic weights between neurons in the network according to samples of the input, so that the network can respond specifically to a particular pattern of input.
At present, methods for learning the SNN can be generalized into two types, one type is to convert a pre-trained Artificial Neural Network (ANN) into a corresponding SNN, the method becomes an ANN-to-SNN (ANN 2 SNN), and the other popular type is to train the SNN directly based on a pulse back propagation algorithm (Spike-based BP). Compared with other methods, the SNN obtained by the artificial neural network-impulse neural network method can obtain better classification accuracy, and the good generalization performance of the pre-trained ANN is mainly benefited. However, the good performance of this kind of method in the classification task is built on a huge inference time step, in other words, the ANN2SNN needs to process the input signal within a larger time window to obtain good classification accuracy, so that the inference efficiency is low. Therefore, the conversion-based SNN has a certain limitation in low power consumption hardware. Pulse backpropagation is a method to train SNNs directly. During training, each training sample is input into the SNN, the SNN outputs a predicted value, then a loss function is constructed, and the error between the predicted value and the sample true label is calculated by using a Stochastic Gradient Descent (SGD) algorithm. The core of the method is to estimate the gradient of pulse output relative to input accumulation potential, so that a space-time Back Propagation (STBP) algorithm can transmit errors to all synaptic weights for parameter updating.
The invention is applied for the Chinese invention with the publication time of 2021.04.09: for example, a convolutional impulse neural network training method based on reweighted membrane voltage has been demonstrated to be able to effectively train the deep SNN based on the pulse back propagation algorithm. Although in most cases it does not perform as well as the ANN2SNN, the pulse backpropagation training updates the parameters by spatio-temporal backpropagation gradients, so that the inference time of the SNN is greatly shortened. However, since the pulse back propagation training needs to perform gradient calculation in the time and space directions, a large amount of memory resources are consumed in the training process, and the training speed is slow.
Disclosure of Invention
Aiming at the limitation of the prior art, the invention provides a training method of a pulse neural network for image classification, and the technical scheme adopted by the invention is as follows:
a method of training a spiking neural network for image classification, comprising the steps of:
s1, acquiring an input image, and performing data preprocessing including image normalization on the input image;
s2, acquiring a pulse neural network to be trained and corresponding task requirements, and performing network initialization on the pulse neural network according to the task requirements;
s3, constructing an artificial neural network-impulse neural network double-branch network structure with shared weight parameters according to the result of the step S2;
s4, training the artificial neural network branches in the artificial neural network-impulse neural network double-branch network structure according to the result of the step S1; and after the training is finished, taking the impulse neural network branches therein as the training result of the impulse neural network which can be used for image classification based on weight sharing.
Compared with the prior art, the invention provides a rapid and memory-saving pulse neural network training method, a weight sharing frame of an artificial neural network-pulse neural network is designed and applied, the training of the pulse neural network is transferred to the artificial neural network for carrying out, and under the same network structure and time window setting, the training speed and the memory consumption of the invention are 43% -67% and 33% -55% of the pulse back propagation model.
As a preferable scheme, the data preprocessing of step S1 includes the following processes:
normalizing the content of each channel of the input image; and according to the preset time window size T, copying the input images into T sheets to construct an image sequence with the length of T.
As a preferred solution, the network initialization of step S2 includes the following procedures:
and setting the number of network layers of the pulse neural network, the structure of each layer of network, the size of a characteristic channel, the size of a convolution kernel, the initial value of each layer of weight and the initial threshold voltage of the IF neuron according to task requirements.
Further, the IF neuron is expressed by the following formula:
and (3) charging process: u. of t+1,n ＝u t,n (1-o t,n )+I t+1,n ；
And (3) discharging:
Figure BDA0003900878040000031
wherein n represents the n-th layer of impulse neural network, the membrane voltage u t,n Representing the internal state of the spiking neuron, the membrane voltage u t,n Accumulating the input stimulus at each moment in the membrane voltage by an accumulation-discharge mechanism; i is t+1,n For the input stimulus at time t +1, I t+1,n Acting on the input pulse according to the network weight to obtain the input pulse; o t.n A pulse signal, V, representing the time t th Representing a voltage threshold.
Further, in step S4, the voltage threshold of the IF neuron is adaptively increased by:
Figure BDA0003900878040000032
Figure BDA0003900878040000033
wherein k represents the kth training iteration in one training algebra, and alpha belongs to [0,1] to represent a momentum coefficient; epsilon [0,1] represents tolerance; τ >0 represents the scale factor and Ω represents the set of indices of the noise pulses.
As a preferable scheme, in the artificial neural network-impulse neural network dual-branch network structure: and taking the rectifying pulse unit or the through pulse unit as an activation function of the ANN branch, and taking pulse output of each layer of the impulse neural network branch as an activation value of each layer of the ANN branch, so that the ANN branch can equivalently represent the impulse neural network branch.
Further, the work flow of the rectification pulse unit is expressed by the following formula:
Figure BDA0003900878040000041
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003900878040000042
the nth layer activation output representing the ANN branch,
Figure BDA0003900878040000043
the output of the nth network of the ANN branch after the convolution and linear rectification unit is represented; the sign function sign (-) indicates that
Figure BDA0003900878040000044
Quantized to 0 and 1, for obtaining the activation position of the output of the linear rectification unit; DEG represents the Hadamard product;
Figure BDA0003900878040000045
representing the cumulative pulse output from the nth layer of the spiking neural network branches.
Further, the work flow of the through pulse unit is expressed by the following formula:
Figure BDA0003900878040000046
wherein the content of the first and second substances,
Figure BDA0003900878040000047
the nth layer activation output representing the ANN branch,
Figure BDA0003900878040000048
convolution and linear rectification unit for n-th network representing ANN branch to pass throughThe latter output;
Figure BDA0003900878040000049
representing the cumulative pulse output from the nth layer of the spiking neural network branches; c is and
Figure BDA00039008780400000410
the tensor of equivalence.
Further, in step S4, training the artificial neural network branches in the artificial neural network-spiking neural network dual-branch network structure includes the following processes:
modeling the ANN branch by using a rectification pulse unit or a through pulse unit, so that forward calculation of each layer of the network of the impulse neural network branch can be represented by the ANN branch; then, a cross entropy loss function is used for acting on the FC layer output of the ANN branch to realize the description of the classification error; and finally, training on the ANN branch by using a back propagation algorithm.
The present invention also provides the following:
a storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the aforementioned training method for an impulse neural network for image classification.
A computer device comprising a storage medium, a processor and a computer program stored in the storage medium and executable by the processor, the computer program, when executed by the processor, implementing the steps of the aforementioned training method for an impulse neural network for image classification.
Drawings
FIG. 1 is a flowchart illustrating a method for training a spiking neural network for image classification according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a structure of an artificial neural network-impulse neural network dual branch network according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a commutation pulse unit used in an embodiment of the present invention;
FIG. 4 is a schematic diagram of a through pulse unit used in an embodiment of the present invention;
fig. 5 is a schematic diagram of a network structure according to a specific example of the embodiment of the present invention;
FIG. 6 is a graphical illustration of the effect of commutating pulse unit (ReSU), through pulse unit (STSU) and Adaptive Threshold Adjustment (ATA) on classification performance in accordance with an embodiment of the present invention;
FIG. 7 is a comparison of training Time (Train Time) and video memory consumption (GPU Mem) of the training method of the impulse neural network for image classification and the spatio-temporal back propagation algorithm (STBP) according to the embodiment of the present invention;
FIG. 8 shows the results of comparing the accuracy of the embodiments of the present invention (SNN 2ANN-ReSU, SNN2 ANN-STSU) with the spatio-temporal back propagation algorithm (STBP-PLIF) and ANN.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the embodiments described are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the embodiments in the present application.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments of the present application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application, as detailed in the claims that follow. In the description of the present application, it is to be understood that the terms "first," "second," "third," and the like are used solely for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order, nor is it to be construed as indicating or implying relative importance. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.
In addition, in the description of the present application, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. The invention is further illustrated below with reference to the figures and examples.
In order to solve the limitation of the prior art, the present embodiment provides a technical solution, and the technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.
Example 1
Referring to fig. 1, a method for training a spiking neural network for image classification includes the following steps:
s1, acquiring an input image, and performing data preprocessing including image normalization on the input image;
s2, acquiring a pulse neural network to be trained and corresponding task requirements, and performing network initialization on the pulse neural network according to the task requirements;
s3, constructing an artificial neural network-impulse neural network double-branch network structure with shared weight parameters according to the result of the step S2;
s4, training the artificial neural network branches in the artificial neural network-impulse neural network double-branch network structure according to the result of the step S1; and after the training is finished, taking the impulse neural network branches therein as the training result of the impulse neural network which can be used for image classification based on weight sharing.
Compared with the prior art, the invention provides a rapid and memory-saving impulse neural network training method, a weight sharing framework of an artificial neural network-impulse neural network (ANN-SNN) is designed and applied, the training of the impulse neural network is migrated to the artificial neural network for carrying out, and under the same network structure and time window setting, the training speed and the memory consumption of the invention are 43% -67% and 33% -55% of a pulse back propagation model.
As a preferred embodiment, the data preprocessing of step S1 includes the following processes:
normalizing the content of each channel of the input image; and according to the preset time window size T, copying the input images into T sheets to construct an image sequence with the length of T.
As a preferred embodiment, the network initialization of step S2 includes the following processes:
and setting the number of network layers of the pulse neural network, the structure of each layer of network, the size of a characteristic channel, the size of a convolution kernel, the initial value of each layer of weight and the initial threshold voltage of the IF neuron according to task requirements.
Further, the IF neuron is expressed by the following formula:
and (3) charging process: u. of t+1,n ＝u t,n (1-o t,n )+I t+1,n ； (1)
And (3) discharging:
Figure BDA0003900878040000071
wherein n represents the n-th layer of the impulse neural network, and the membrane voltage u t,n Representing the internal state of the spiking neuron, the membrane voltage u t Accumulating the input stimulus at each moment in the membrane voltage by an accumulation-discharge mechanism; i is t+1,n For the input stimulus at time t +1, I t+1,n Acting on the input pulse according to the network weight to obtain the input pulse; o t,n Pulse signal, V, representing time t th Representing a voltage threshold.
Specifically, according to equation (2), IF the accumulated membrane voltage exceeds the voltage threshold at a certain time, the IF neuron will issue a pulse signal to the neuron associated therewith. Meanwhile, according to equation (1), the current membrane voltage is reset to 0 until the input stimulus arrives at the next moment. If the current membrane voltage does not reach the threshold, it will accumulate to the next instant.
In the step S2, regarding the number of SNN network layers in the step 2, the network structure of each layer, the size of the characteristic channel, the size of the convolution kernel, the initial distribution state of the weight of each layer, and the threshold membrane voltage, taking a CIFAR10 image data set as an example, which is an RGB image data set including 10 categories in total, 17 layers of residual network structure neural networks may be set to implement an image classification network. The first layer is an input layer, the second to sixteenth layers are hidden convolutional layers, and the seventeenth layer is an output layer of a full-link structure, connecting the final pulse signature to 10 output classes. Since the picture size in the CIFAR10 dataset is 32 × 32 × 3 and the total number of classification categories is 10, the input layer size should be 32 × 32 × 3 neurons, the output layer should be 10 neurons, and assuming that the time length is set to T =10, for each time, the input layer receives the current image, transmits it to the hidden layer for pulse coding to obtain binary pulse features, and finally transmits the features to the output layer to obtain the prediction result. If the user needs to perform more complicated classification tasks, the number of network layers and the number of neurons in each layer can be designed according to own experience.
As a preferred scheme, in the artificial neural network-impulse neural network dual-branch network structure: and taking the rectifying pulse unit or the through pulse unit as an activation function of the ANN branch, and taking pulse output of each layer of the impulse neural network branch as an activation value of each layer of the ANN branch, so that the ANN branch can equivalently represent the impulse neural network branch.
Specifically, please refer to fig. 2 for the artificial neural network-impulse neural network ANN-SNN dual-branch network structure constructed in step S3; the upper part is SNN branch, the lower part is ANN branch, W and FC are weight parameters shared by two branch networks and a classification layer, white arrows represent a network forward calculation process, and gray arrows represent a back propagation training process. In the ANN branch, reSU and STSU represent a commutation pulse unit and a through pulse unit, respectively, which map the pulses accumulated in each layer of the SNN over the entire time window into the ANN branch and forward the pulses as activation function values in the ANN branch. The output of each layer of the ANN branch is equivalent to the accumulated pulse of each layer of the SNN, and the SNN is expressed by using the ANN equivalent. The accumulated pulses of each layer of the SNN are mapped to the activation values of each layer of the branch of the ANN by a commutating pulse unit or a through pulse unit, thereby enabling the SNN to be represented by one ANN.
Further, the work flow of the rectification pulse unit is expressed according to the following formula:
Figure BDA0003900878040000081
wherein the content of the first and second substances,
Figure BDA0003900878040000082
the nth layer activation output representing the ANN branch,
Figure BDA0003900878040000083
the output of the nth network of the ANN branch after the convolution and linear rectification unit is represented; the sign function sign (·) indicates that
Figure BDA0003900878040000084
Quantized to 0 and 1, for obtaining the activation position of the output of the linear rectification unit;
Figure BDA00039008780400000813
representing a Hadamard product;
Figure BDA0003900878040000085
representing the cumulative pulse output from the nth layer of the spiking neural network branches.
In particular, the structure of the rectifier pulse Unit (rectifier pulse Unit, reSU) is shown in FIG. 3; after the process of the Hadamard product,
Figure BDA0003900878040000086
neutralization of
Figure BDA0003900878040000087
The same pulse reservation is activated, as shown by the output pulse in fig. 3, and this part of the reserved pulse information will be passed on to the n +1 th layer network in the ANN branch. While the inconsistent output pulses generated by SNN, such as the red portion of the accumulated pulses in fig. 3 (the red portion is actually present in the original image, but cannot be distinguished from other portions after the color of the image is adjusted to meet the requirements of patent laws), the above contents are only used for explaining the principle of a person who is not skilled in the art, and do not affect the understanding of the person who is skilled in the art on the solution of the present invention, and thus the statement is not involved in the forward calculation of ANN. Each linear rectification unit of the ANN branch is acted upon with the rescu as an activation function, so that the ANN branch, when passing forward, passes cumulative pulses in the SNN branch with the same activation position. According to fig. 2, in the forward calculation, the cross entropy loss uses the pulse information to calculate the training error. Due to the non-derivable property of the sign function sign (), the back propagation algorithm cannot be used directly for training of the ANN branch. To solve this problem, the present invention assumes a forward-delivered pulse on the ANN branch
Figure BDA0003900878040000088
About linear rectified output
Figure BDA0003900878040000089
Is 1, i.e.
Figure BDA00039008780400000810
Enabling the gradient to be propagated back from the deep network of ANN branches to the shallow network by the chain law. At the same time, since training is only performed in the ANN branch, pulses are accumulated
Figure BDA00039008780400000811
The gradient of the weight is 0, so that gradient information is not passed on the SNN branch.
Further, the work flow of the through pulse unit is expressed by the following formula:
Figure BDA00039008780400000812
wherein the content of the first and second substances,
Figure BDA0003900878040000091
the nth layer activation output representing the ANN branch,
Figure BDA0003900878040000092
the output of the nth network of the ANN branch after convolution and linear rectification unit;
Figure BDA0003900878040000093
representing the cumulative pulse output from the nth layer of the spiking neural network branch; c is and
Figure BDA0003900878040000094
the tensor of equivalence.
In particular, the structure of the STSU is shown in FIG. 4. In the forward calculation of the ANN branch, c and
Figure BDA0003900878040000095
cancel each other out so that
Figure BDA0003900878040000096
Is numerically equivalent to all accumulated pulses from the n-th layer SNN
Figure BDA0003900878040000097
After each linear rectifier unit of the ANN branch is acted on by STSU as an activation function, so that each layer in the forward direction of the ANN branch is forwarded with accumulated pulses of the corresponding layer of SNN, and the outputs of the layers of the two branches are completely equivalent. According to fig. 2, in the forward calculation, the cross entropy loss uses the pulse information to calculate the training error. Although c is in contact with
Figure BDA0003900878040000098
Equal value, but because of the constants, c is not a parametric gradient calculation. Back propagation algorithm computing deep network relationships
Figure BDA0003900878040000099
The gradient of (c) allows the ANN branches to be trained directly. Like ReSU, accumulate pulses
Figure BDA00039008780400000910
With a gradient of 0 for the weight parameter, gradient information is not passed on the SNN branch.
Further, in the step S4, training the artificial neural network branches in the artificial neural network-impulse neural network dual-branch network structure includes the following processes:
modeling the ANN branch by using a rectification pulse unit or a through pulse unit, so that forward calculation of each layer of the network of the impulse neural network branch can be represented by the ANN branch; then, a cross entropy loss function is used for acting on the FC layer output of the ANN branch to realize the description of the classification error; and finally, training on the ANN branch by using a back propagation algorithm.
Specifically, the ANN is trained by using a back propagation algorithm, so that the space-time back propagation of the gradient information in the SNN is avoided, and the video memory resource required by training is reduced.
The training diagram is shown in the lower part of fig. 2, an ANN branch, white arrows indicate forward calculation, cross entropy loss is measured according to the last feature calculated by the ANN branch, and a gradient is calculated by a back propagation algorithm for updating parameters of each layer network.
In addition, due to the dynamic characteristics of the IF neuron, in the case of weight sharing, a phenomenon occurs in which the activation position of the pulse output through the IF neuron in the SNN branch and the activation position through the linear rectification unit in the ANN branch do not coincide. In particular, the active position of the IF neuron may be static in the linear rectification unit, as shown in the red portion of the "accumulated pulse" in fig. 3 and 4 (the red portion is actually present in the original image, but cannot be distinguished from other portions after the image color is adjusted to meet the requirement of the patent law, and the above contents are only used for the principle explanation of those skilled in the art, and do not affect the understanding of the solution of the present invention by those skilled in the art, and thus the present invention is stated. False activation of these IF neurons will produce noise pulses that affect the performance of the SNN.
In this regard, in order to suppress the occurrence of the SNN branch and the activation of the output inconsistent with the ANN branch, further, in the step S4, the voltage threshold of the IF neuron is adaptively increased by:
Figure BDA0003900878040000101
Figure BDA0003900878040000102
wherein k represents the kth training iteration in one training algebra, and alpha belongs to [0,1] to represent a momentum coefficient; epsilon [0,1] represents tolerance; τ >0 represents the scale factor and Ω represents the set of indices of the noise pulses.
IF the average of the accumulated noise is greater than ε, the threshold of the IF neuron will be increased. Conversely, the threshold voltage will remain unchanged. The method adaptively increases the threshold value, reduces the chance of noise pulse emission to a certain extent, and reduces the number of active positions in the IF neuron different from the linear rectification unit. The above Adjustment method of the voltage Threshold may be referred to as an Adaptive Threshold Adjustment (ATA).
After training, taking out SNN branches for decision: as shown in step S3, under the ANN-SNN weight sharing framework designed by the present invention, the SNN is mapped to the ANN branch for training and two branches are weight shared. When the ANN branch is trained, the SNN branch responds to the specific pattern similar to the ANN branch. Therefore, after training is completed, the SNN branches can be taken out and directly used for image classification and other decisions.
The specific network structure and image classification tasks will be combined as follows: embodiments of the present invention are specifically described in the context of CIFAR10, CIFAR100, and Tiny-ImageNet image classification tasks based on VGG13 and ResNet-17 network structures.
Firstly, standardizing images of each channel of an input RGB image, wherein after normalization, the mean value of each channel is 0.4914,0.4822 and 0.4465, and the standard deviation is 0.557,0.549 and 0.5534, firstly, standardizing the images of each channel of the input image, and normalizing the images of three channels of the input RGB image. Then, by setting the time window size to T and copying T images to construct an RGB image sequence having a length of T, the RGB images are encoded into an image sequence of T × H × W × 3. In this embodiment, for the VGG-13 network structure, the size of the time window T is set to 5,4,3 on CIFAR10, CIFAR100, and Tiny-ImageNet tasks, respectively; for the ResNet-17 network architecture, the time windows for the three tasks are all set to 5. H = W =32 for CIFAR10 and CIFAR100, and H = W =64 for Tiny-ImageNet.
Network initialization: the VGG-13 and ResNet-17 network structures were chosen as shown in FIG. 5. Wherein [ k3c64s1] represents a convolution layer with a convolution kernel size of 3 × 3, an output channel number of 64, a step size of 1, × 2 represents that the module is repeated 2 times, M represents maximum Pooling (Max), and FC represents a classification layer; [ k3c64-k3c64] denotes a residual network module consisting of two convolution kernels of size 3 × 3 and 64 convolution layers for the number of output channels, with a residual connection between the input and output to set the initial threshold voltage of the IF neuron to a random tensor that is uniformly distributed.
Constructing an ANN-SNN double-branch network structure with shared weight parameters: based on the network layer number and structure set in the foregoing, an ANN-SNN dual-branch network is constructed, specifically as shown in fig. 2, and both share parameters. And outputting each layer of pulses of the SNN as an activation value of each layer of the ANN branch by using a rectifier pulse unit (ReSU) or a through pulse unit (STSU) as an activation function of the ANN branch. Using ReSU, mapping noise pulse in SNN after filtering to ANN branch as activation value of each layer; using STSU, all accumulated pulses in SNN are mapped to activation values for each layer of the ANN for forward calculation.
Training an ANN branch: by modeling the ANN branch using the ReSU or STSU, forward computations for each layer of the SNN can be represented by the ANN branch. And then, a cross entropy loss function is used for acting on the FC layer output of the ANN branch to realize the description of the classification error. And finally, training on the ANN branch by using a back propagation algorithm. Under the action of ReSU and STSU, the gradient information is only passed on the ANN branch and the parameters are updated. In addition, the invention designs a self-adaptive threshold voltage adjusting method for suppressing noise pulses appearing in the SNN branch, so as to further improve the performance of the SNN, and experimental data of the embodiment are shown in fig. 6.
Therefore, after the parameter training on the ANN branch is converged, the SNN branch can be directly used for decision making due to weight sharing. For the VGG-13 network structure, the embodiment trains 400 generations (epoch) on the CIFAR10, the Tiny-ImageNet image classification task and 200 generations on the CIFAR 100; for the ResNet-17 structure, CIFAR10, CIFAR100, tiny-ImageNet were trained for two hundred generations. Because the weights of the ANN branches and the SNN branches are shared, after the ANN branches are trained, the SNN can be taken out for independent classification. This example shows the video memory and training time required for each training generation of VGG-13, resNet-17 on CIFAR10, CIFAR100, and Tiny-ImageNet, as compared to the space-time Back-Propagation (STBP) algorithm, in FIG. 7. The training speed and the video memory consumption of the invention are 43-67% and 33-55% of the pulse back propagation model. Meanwhile, in the embodiment, the variation of the prediction accuracy of the SNN branch with the training algebra is also shown in fig. 8, and is compared with the prediction accuracy obtained by the ANN and STBP methods.
Example 2
A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the training method of the spiking neural network for image classification in embodiment 1.
Example 3
A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, the computer program when executed by the processor implementing the steps of the training method for an impulse neural network for image classification of embodiment 1.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (10)

1. A training method of a pulse neural network for image classification is characterized by comprising the following steps:
s1, acquiring an input image, and performing data preprocessing including image normalization on the input image;
s2, acquiring a pulse neural network to be trained and corresponding task requirements, and initializing the pulse neural network according to the task requirements;
s3, constructing an artificial neural network-impulse neural network double-branch network structure with shared weight parameters according to the result of the step S2;
s4, training the artificial neural network branches in the artificial neural network-impulse neural network double-branch network structure according to the result of the step S1; and after the training is finished, based on weight sharing, taking the impulse neural network branches therein as the training results of the impulse neural network which can be used for image classification.
2. The training method of the impulse neural network for image classification as claimed in claim 1, wherein the data preprocessing of step S1 comprises the following processes:
normalizing the content of each channel of the input image; and according to the preset time window size T, copying the input images into T pieces to construct an image sequence with the length of T.
3. The training method of the impulse neural network for image classification as claimed in claim 1, wherein the network initialization of the step S2 comprises the following processes:
and setting the network layer number of the pulse neural network, the structure of each layer of network, the size of a characteristic channel, the size of a convolution kernel, the initial value of each layer of weight and the initial threshold voltage of the IF neuron according to task requirements.
4. The method of claim 3, wherein the IF neuron is represented by the following equation:
and (3) charging process: u. u t+1,n ＝u t,n (1-o t,n )+I t+1,n ；
And (3) discharging:
Figure FDA0003900878030000011
wherein n represents the n-th layer of the impulse neural network, and the membrane voltage u t,n Representing the internal state of a spiking neuron, the membrane voltage u t,n Accumulating the input stimulus at each moment in the membrane voltage by an accumulation-discharge mechanism; i is t+1,n For the input stimulus at time t +1, I t+1,n Acting on the input pulse according to the network weight to obtain the input pulse; o t,n A pulse signal, V, representing the time t th Representing a voltage threshold.
5. The method for training the spiking neural network for image classification as claimed in claim 4, wherein in the step S4, the voltage threshold of the IF neuron is adaptively increased by:
Figure FDA0003900878030000021
Figure FDA0003900878030000022
wherein k represents the kth training iteration in one training algebra, and alpha belongs to [0,1] to represent a momentum coefficient; epsilon [0,1] represents the tolerance; τ >0 represents the scale factor and Ω represents the set of indices of the noise pulses.
6. The training method of the spiking neural network for image classification according to claim 1, characterized in that in the artificial neural network-spiking neural network dual branch network structure: and taking the rectifying pulse unit or the through pulse unit as an activation function of the ANN branch, and taking pulse output of each layer of the impulse neural network branch as an activation value of each layer of the ANN branch, so that the ANN branch can equivalently represent the impulse neural network branch.
7. The training method of the impulse neural network for image classification as claimed in claim 6, wherein the work flow of the rectifying impulse unit is expressed by the following formula:
Figure FDA0003900878030000023
wherein the content of the first and second substances,
Figure FDA0003900878030000024
the nth layer activation output representing the ANN branch,
Figure FDA0003900878030000025
the output of the nth network of the ANN branch after the convolution and linear rectification unit is represented; the sign function sign (-) indicates that
Figure FDA0003900878030000026
Quantized to 0 and 1, for obtaining active bits of the output of the linear rectifying unitPlacing;
Figure FDA00039008780300000213
representing a Hadamard product;
Figure FDA0003900878030000027
representing the cumulative pulse output from the nth layer of the spiking neural network branches.
8. The method of claim 6, wherein the workflow of the through pulse unit is expressed by the following formula:
Figure FDA0003900878030000028
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003900878030000029
the nth layer activation output representing the ANN branch,
Figure FDA00039008780300000210
the output of the nth network of the ANN branch after convolution and linear rectification unit;
Figure FDA00039008780300000211
representing the cumulative pulse output from the nth layer of the spiking neural network branch; c is and
Figure FDA00039008780300000212
the tensor of equivalence.
9. The method for training the impulse neural network for image classification as claimed in claim 6, wherein in the step S4, the artificial neural network branches in the artificial neural network-impulse neural network dual branch network structure are trained, comprising the following processes:
modeling the ANN branch by using a rectification pulse unit or a through pulse unit, so that forward calculation of each layer of the network of the impulse neural network branch can be represented by the ANN branch; then, a cross entropy loss function is used for acting on the FC layer output of the ANN branch to realize the description of the classification error; and finally, training on the ANN branch by using a back propagation algorithm.
10. A computer device, characterized by: comprising a storage medium, a processor and a computer program stored in the storage medium and executable by the processor, which computer program, when being executed by the processor, carries out the steps of the method of training a spiking neural network for image classification according to any one of claims 1 to 9.
CN202211289610.9A 2022-10-20 2022-10-20 Training method of pulse neural network for image classification Pending CN115546556A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211289610.9A CN115546556A (en) 2022-10-20 2022-10-20 Training method of pulse neural network for image classification

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211289610.9A CN115546556A (en) 2022-10-20 2022-10-20 Training method of pulse neural network for image classification

Publications (1)

Publication Number Publication Date
CN115546556A true CN115546556A (en) 2022-12-30

Family

ID=84735279

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211289610.9A Pending CN115546556A (en) 2022-10-20 2022-10-20 Training method of pulse neural network for image classification

Country Status (1)

Country Link
CN (1) CN115546556A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116051429A (en) * 2023-03-31 2023-05-02 深圳时识科技有限公司 Data enhancement method, impulse neural network training method, storage medium and chip

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116051429A (en) * 2023-03-31 2023-05-02 深圳时识科技有限公司 Data enhancement method, impulse neural network training method, storage medium and chip

Similar Documents

Publication Publication Date Title
CN112633497B (en) Convolutional impulse neural network training method based on re-weighted membrane voltage
CN110309732B (en) Behavior identification method based on skeleton video
Alaloul et al. Data processing using artificial neural networks
CN109102000B (en) Image identification method based on hierarchical feature extraction and multilayer pulse neural network
CN112699956B (en) Neuromorphic visual target classification method based on improved impulse neural network
CN110288555B (en) Low-illumination enhancement method based on improved capsule network
CN113449864B (en) Feedback type impulse neural network model training method for image data classification
US10776691B1 (en) System and method for optimizing indirect encodings in the learning of mappings
CN113705769A (en) Neural network training method and device
KR102152374B1 (en) Method and system for bit quantization of artificial neural network
CN112906828A (en) Image classification method based on time domain coding and impulse neural network
CN108304912B (en) System and method for realizing pulse neural network supervised learning by using inhibition signal
CN113094357A (en) Traffic missing data completion method based on space-time attention mechanism
CN112183742A (en) Neural network hybrid quantization method based on progressive quantization and Hessian information
CN113537365B (en) Information entropy dynamic weighting-based multi-task learning self-adaptive balancing method
CN112712170B (en) Neuromorphic visual target classification system based on input weighted impulse neural network
CN111310816B (en) Method for recognizing brain-like architecture image based on unsupervised matching tracking coding
CN111130909B (en) Network flow prediction method based on self-adaptive reserve pool ESN
CN115546556A (en) Training method of pulse neural network for image classification
CN114266351A (en) Pulse neural network training method and system based on unsupervised learning time coding
CN113239897A (en) Human body action evaluation method based on space-time feature combination regression
CN117056785A (en) Federal learning classification model training method based on self-adaptive model disturbance
US20220358346A1 (en) Systems, methods, and media for generating and using spiking neural networks with improved efficiency
CN113239949A (en) Data reconstruction method based on 1D packet convolutional neural network
WO2021086861A1 (en) Quantized architecture search for machine learning models

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20230605

Address after: Room A019, Building 2, No. 69, Huangge Section, Shinan Road, Nansha District, Guangzhou City, Guangdong Province, 511458

Applicant after: INTELLIGENT EYES AUTOMATION TECHNOLOGY (GUANGZHOU) CO.,LTD.

Address before: 510275 No. 135 West Xingang Road, Guangzhou, Guangdong, Haizhuqu District

Applicant before: SUN YAT-SEN University