CN112288080A - Pulse neural network-oriented adaptive model conversion method and system - Google Patents

Pulse neural network-oriented adaptive model conversion method and system Download PDF

Info

Publication number
CN112288080A
CN112288080A CN202011294317.2A CN202011294317A CN112288080A CN 112288080 A CN112288080 A CN 112288080A CN 202011294317 A CN202011294317 A CN 202011294317A CN 112288080 A CN112288080 A CN 112288080A
Authority
CN
China
Prior art keywords
neural network
pulse
layer
neuron
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011294317.2A
Other languages
Chinese (zh)
Inventor
张毅
刘晓东
彭龙
马俊
黄辰林
余杰
谭郁松
吴庆波
廖湘科
汪昌棋
李文杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202011294317.2A priority Critical patent/CN112288080A/en
Publication of CN112288080A publication Critical patent/CN112288080A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Feedback Control In General (AREA)

Abstract

The invention discloses a pulse neural network-oriented self-adaptive model conversion method and a pulse neural network-oriented self-adaptive model conversion system, wherein the method comprises the steps of obtaining a deep artificial neural network model to be converted, wherein the deep artificial neural network model to be converted is a deep neural network model under any one frame of Keras, Pythrch and TensorFlow; if the deep artificial neural network model to be converted is a deep artificial neural network model under a Pythrch or TensorFlow framework, converting the deep artificial neural network model into a deep artificial neural network model under a Keras framework; and converting the deep artificial neural network model under the Keras framework into a pulse neural network. The invention can realize a deep artificial neural network model under various intelligent frames, and completes the self-adaptive conversion to the novel impulse neural network on the premise of ensuring a small amount of model precision loss.

Description

Pulse neural network-oriented adaptive model conversion method and system
Technical Field
The invention relates to a neural network model, in particular to a pulse neural network-oriented adaptive model conversion method and system.
Background
As a new generation of neural network model, the pulse neural network can capture the key dynamics of the input and output characteristics of biological neurons, and compared with the traditional artificial neural network, the pulse neural network is easier to establish a larger-scale network and has low operation power consumption. Therefore, the spiking neural network model is a more promising brain-like computational model.
Because the dynamics principle of the impulse neural network is more complex than that of the deep neural network, and the output state of the neuron is a step impulse signal, the impulse neural network has no mathematic conductibility and is difficult to establish a gradient descent learning theory similar to that of the deep neural network, the challenge of directly learning and training the impulse neural network is very large. Currently, many artificial intelligence researchers often convert the trained artificial neural network model into the impulse neural network model by using a model conversion method, thereby effectively avoiding the training of the impulse neural network model.
Almost artificial neural network development is currently performed based on intelligent frameworks, such as TensorFlow, Keras, Pythrch, Caffe and the like, however, interface differences among different frameworks can cause different trained model format definitions, so that intercommunication among existing models cannot be realized. The method further enables model transformation methods under multiple intelligent frames to be considered when the artificial neural network models under different frames are transformed in the model oriented to the impulse neural network, and brings a larger problem of repeated development.
The great success of the traditional deep artificial neural network model is mainly attributed to the development of high-performance computing hardware and the large amount of data available for training the model. However, solving the more complex application problem in reality requires more advanced models and complex data, which means a huge increase in the amount of computation and power consumption. To overcome this challenge, many researchers have attempted to make deep neural network models more efficient by compressing them. However, the problem of huge computation and power consumption still remains with the increasing complexity of the model.
The method is more direct and effective, the trained artificial neural network model is converted into the impulse neural network model, and the problems of huge energy consumption and calculation delay can be solved fundamentally. Early studies converted the convolutional neural elements in the model into impulsive elements with biological excitations of leakage and refractory periods, with the aim of processing the input from event-based sensors. Meanwhile, a method for customizing a deep artificial neural network model is researched and proposed to adapt to the working principle of the impulse neural network, and the customization content comprises the following steps: 1) eliminating negative value input between the input layer and each layer; 2) using ReLU as an activation function of neurons; 3) the bias parameters are no longer used. Meanwhile, a weight parameter regularization technology is added, so that the performance loss after conversion is further reduced, and nearly lossless conversion is realized on MNIST data sets through testing. It can be seen that after the deep artificial neural network model is converted into the impulse neural network model technology, the converted impulse neural network model can have performance comparable to that of the original model by tuning, and meanwhile, the model obtained by conversion not only has real-time response capability but also has power consumption far lower than that of the original model when running on a special device. However, deep artificial neural network models under different intelligent frameworks have great differences, and the transformation of multiple models to the impulse neural network cannot be simultaneously realized by using the same method, so that the development and application of the impulse neural network are greatly restricted, and great inconvenience is brought to developers.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: aiming at the problems in the prior art, the invention provides a pulse neural network-oriented self-adaptive model conversion method and system, which can realize a deep artificial neural network model under various intelligent frameworks and complete self-adaptive conversion to a novel pulse neural network on the premise of ensuring a small amount of model precision loss.
In order to solve the technical problems, the invention adopts the technical scheme that:
an adaptive model conversion method facing to a pulse neural network comprises the following steps:
1) obtaining a deep artificial neural network model to be converted, wherein the deep artificial neural network model to be converted is a deep neural network model under any one frame of Keras, Pythrch and TensorFlow;
2) if the deep artificial neural network model to be converted is a deep artificial neural network model under a Pythrch or TensorFlow framework, converting the deep artificial neural network model into a deep artificial neural network model under a Keras framework;
3) and converting the deep artificial neural network model under the Keras framework into a pulse neural network.
Optionally, the detailed steps of step 2) include:
2.1) judging the type of the deep artificial neural network model to be converted, and if the type is the deep artificial neural network model under the Keras framework, skipping to execute the step 3); otherwise, skipping to execute the step 2.2);
2.2) reasoning the deep artificial neural network model to be converted under the original frame, so as to analyze the deep artificial neural network model to be converted, and obtain a weight file of the deep artificial neural network model to be converted, an information file of a model structure and a logical reasoning graph of the model;
and 2.3) constructing and obtaining the converted deep artificial neural network model under the Keras framework based on the weight file of the deep artificial neural network model to be converted, the information file of the model structure and the logic inference diagram of the model.
Optionally, when the deep artificial neural network model under the Keras framework is converted into the impulse neural network in step 3), the method includes five operations of bias conversion, weight parameter regularization, BN layer conversion, softmax layer conversion, and pooling conversion; and after the five operations are completed, the method also comprises the step of adjusting the super parameters, and the adjusted super parameters comprise a neuron membrane potential threshold value, a membrane potential resetting mode, setting of a refractory period, pulse delay emission and membrane potential leakage.
Optionally, the step of bias switching comprises: and converting the bias in the deep artificial neural network model under the Keras framework into constant input current with the same sign and inputting the constant input current to the corresponding pulse neuron, wherein the bias value is that the membrane potential is increased when the bias value is positive and plays a role in exciting the pulse neuron, and the membrane potential is reduced when the bias value is negative.
Optionally, the step of regularizing the weight parameter includes: setting the maximum activation value in the l layer of the impulse neural network to be lambda for the l layer in the convolutional neural networkl＝max[al]Wherein a islIs the activation value in the l-th layer of the spiking neural network, max.]Representing the maximum value, weighted by WlOffset is bl，WlIs a matrix representing the weight parameters of the l layers, blIs a vector representing the bias of l-layer neurons or convolution kernels, the weights after regularization are:
Figure BDA0002784761360000031
in the above formula, Wl' weight after regularization, WlAs a weight before regularization, λl-1Is the maximum activation value, lambda, in layer l-1lIs the maximum activation value in the l-th layer;
the bias is as follows:
Figure BDA0002784761360000032
in the above formula, bl' bias after regularization, blFor bias before regularization, λlIs the maximum activation value in the l-th layer.
Optionally, the BN layer switching step includes: firstly, regularizing input according to a mean value and a variance of the input of a BN layer:
Figure BDA0002784761360000033
in the above formula, the first and second carbon atoms are,
Figure BDA0002784761360000034
for the regularized input, xiIs the original input, mu is the mean, sigma2Is a variance, and ε is a variance σ2A very small positive value of error at 0;
the regularized input is then transformed as follows:
Figure BDA0002784761360000035
in the above formula, yiIs the original input xiOutputting after passing through a BN layer; gamma and beta are parameters which need to be learned in the training stage and are used for restoring activated information representation;
Figure BDA0002784761360000036
is a regularized input;
then, the calculation parameters of the BN layer are integrated into the weight of the immediately previous layer, and the calculation function of fusing the parameters of the BN layer into the weight parameters of the previous network layer is expressed as follows:
Figure BDA0002784761360000037
in the above formula, the first and second carbon atoms are,
Figure BDA0002784761360000038
respectively, the weight and bias parameters of the layer after the BN layer parameters are fused, W and b are respectively the weight and bias parameters of the layer immediately before the BN layer, gamma is a scaling coefficient, mu is a mean value, and sigma is2Is a variance, and ε is a variance σ2A very small positive value of error at 0.
Optionally, the step of softmax layer conversion comprises: firstly, performing softmax calculation shown in the following formula on the membrane potential of a pulse neuron;
Figure BDA0002784761360000041
in the above formula, xiRepresents the value of the membrane potential, λ, of each neuroniIs the ratio of membrane potential of each pulse neuron, the ratio lambdaiHas a value range of [0,1 ]]；
Then the ratio lambda of the membrane potential of each pulse neuroniAs an arrival rate parameter of the poisson process of the corresponding pulse neuron, wherein the function expression of the poisson process is as follows:
Figure BDA0002784761360000042
in the above formula, N (t)i+ τ) represents the number of times that the neuron i emits a pulse during the period t + τ, and naturally N (0) has a value of 0, kiRepresenting the number of pulses emitted by a pulse neuron i during a time period t, τ -1 represents the time interval during which the pulse neuron emits pulses, where the known arrival rate parameter λ isiAnd the number k of the pulse neurons i transmitting pulses in the time period t, so as to judge which pulse neuron should transmit pulses at the next time of t and determine the function value r of the poisson processti+τThe one neuron with the largest value fires at the next instant of t.
Optionally, the detailed step of pooling conversion comprises: recording pulse emission sequence information of each neuron aiming at a convolutional layer which needs to be maximally pooled, recording the pulse emission sequence information of each neuron as 1 if the membrane potential of the pulse neuron is not lower than a threshold value at each emission moment, and recording the pulse emission sequence information of the pulse neuron as a string S consisting of 0 and 1 if the membrane potential of the pulse neuron is lower than the threshold value; the pulse of the pulse neuron with the largest emissivity in the neurons with the largest pooling core number can only pass through each pulse time, and on the premise that the membrane potential of the pulse neuron is not lower than a threshold value to enable the pulse neuron to generate, T is the current time point, N is the total number of pulses generated by the neurons with the pulse transmitting time, and the neurons with the non-largest activation emissivity are filtered by a threshold function shown by the following formula aiming at a sequence S, so that the largest pooling operation is realized in a pulse neural network model;
Figure BDA0002784761360000043
in the above formula, N is the total number of pulses generated by the neurons at the pulse transmission time, T is the current time point, S (T) is the value of the corresponding time in the sequence S, the threshold function is used to pass the pulse of the neuron with the maximum N value among the pulse neurons generating the pulses at the current time, and if the N values are the same, a pulse is passed randomly.
In addition, the invention also provides a pulse neural network-oriented adaptive model conversion system, which comprises a computer device, wherein the computer device is programmed or configured to execute the steps of the pulse neural network-oriented adaptive model conversion method, or a computer program which is programmed or configured to execute the pulse neural network-oriented adaptive model conversion method is stored in a memory of the computer device.
Furthermore, the present invention also provides a computer-readable storage medium having stored therein a computer program programmed or configured to execute the method for adaptive model transformation for a spiking neural network.
Compared with the prior art, the invention has the following advantages: the method comprises the steps of obtaining a deep artificial neural network model to be converted, wherein the deep artificial neural network model to be converted is a deep neural network model under any one of Keras, Pythrch and TensorFlow; if the deep artificial neural network model to be converted is a deep artificial neural network model under a Pythrch or TensorFlow framework, converting the deep artificial neural network model into a deep artificial neural network model under a Keras framework; and converting the deep artificial neural network model under the Keras framework into a pulse neural network. The invention can realize a deep artificial neural network model under various intelligent frames, and completes the self-adaptive conversion to the novel impulse neural network on the premise of ensuring a small amount of model precision loss.
Drawings
FIG. 1 is a basic flow diagram of a method according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of a convolutional neural network model adaptive transformation framework for implementing the method of the embodiment of the present invention.
FIG. 3 is a diagram illustrating the cumulative connection of weights (potentials) and biases at synapses of a spiking neuron.
Figure 4 loss of precision test data before and after model conversion.
Detailed Description
As shown in fig. 1, the adaptive model transformation method for the spiking neural network of the present embodiment includes:
1) obtaining a deep artificial neural network model to be converted, wherein the deep artificial neural network model to be converted is a deep neural network model under any one frame of Keras, Pythrch and TensorFlow;
2) if the deep artificial neural network model to be converted is a deep artificial neural network model under a Pythrch or TensorFlow framework, converting the deep artificial neural network model into a deep artificial neural network model under a Keras framework;
3) and converting the deep artificial neural network model under the Keras framework into a pulse neural network.
Convolutional neural networks are a typical model of deep artificial neural networks. Here, a detailed description of the technology will be given by way of example. Fig. 2 shows the process of adaptively converting the convolutional neural network model under an arbitrary framework into the impulse neural network model, which can be mutually converted among tensoflow, Pytorch and Keras framework, and finally converted into the impulse neural network model under the Keras framework. And developing a trained high-precision CNN model by using any one of Tensorflow, Pythrch and Keras frameworks, then acquiring a logic inference diagram of the CNN model by using the development framework, and realizing random lossless conversion of the CNN model among the three frameworks on the basis of the logic inference diagram of the model.
The general design idea of the method of the embodiment is as follows: the deep artificial neural network models under different frames are firstly converted into models under a Keras frame, and then the deep artificial neural network models under the Keras frame are converted into pulse neural network models. The technology of inter-conversion of the deep neural network model can avoid the problem that the model needs to be retrained on different deployment platforms, and improves the working efficiency of developers. To accomplish this conversion, the detailed steps of step 2) in this embodiment include:
2.1) judging the type of the deep artificial neural network model to be converted, and if the type is the deep artificial neural network model under the Keras framework, skipping to execute the step 3); otherwise, skipping to execute the step 2.2);
2.2) reasoning the deep artificial neural network model to be converted under the original frame, so as to analyze the deep artificial neural network model to be converted, and obtain a weight file of the deep artificial neural network model to be converted, an information file of a model structure and a logical reasoning graph of the model;
and 2.3) constructing and obtaining the converted deep artificial neural network model under the Keras framework based on the weight file of the deep artificial neural network model to be converted, the information file of the model structure and the logic inference diagram of the model.
When acquiring a logical inference graph of a pytore framework development model, the pytore framework provides a way for TorchScript to conveniently deploy the model to other platforms without relying on a Python environment. This embodiment acquires a logical inference graph of a pytorech model based on a trace mode of the torch. Because the trace mode is to obtain the computation graph in the computation process, data information with the dimension consistent with that of the model input information in actual application needs to be input at the same time for the model to carry out reasoning computation. Finally, 3 files are obtained, one json file provides information of later model visualization, one is a weight file of the model, and the other is a file which stores information of the logical inference graph of the model.
When the logical inference graph of the Tensorflow framework development model is obtained, the model developed by the Tensorflow framework is calculated in a mode of strictly following a calculation graph, a network structure file and a weight file are separately stored, a sub-graph of an original network structure is extracted through the operation of a known network output node, and the sub-graph is the logical inference graph of the model. Finally, 3 files are obtained, one json file provides information of later model visualization, one is a weight file of the model, and the other is a file which stores information of the logical inference graph of the model.
The principle that the 3 frames generate the model under the self frame by using the logical inference graph and the model weight is consistent, the node operation in the logical inference graph is mapped to the corresponding network layer under the self frame to generate the network structure code file under the self frame, and then the weight parameter of the model is loaded to finally obtain the complete inference model under the self frame.
Due to the simplicity of operation of the Keras framework, the example chooses to convert the deep artificial neural network model into the impulse neural network model under the Keras framework. The transformations specifically achieved under the Keras framework mainly comprise the following 5 major parts: (1) biasing: realizing bias operation in the impulse neural network by using a constant input current with the same sign as that of a bias parameter in the deep neural network; (2) regularization of weight parameters: regularizing the weight parameters of the fully-connected layer on the basis of layers, and regularizing the convolutional layer on the basis of characteristic diagram channels; (3) BN layer: the parameters of the layer are merged into the weight parameters and the bias parameters of the previous layer; (4) softmax layer: the operation is generally applied to the final output of the classification network, and the classification result is judged by calculating the membrane potential of the neuron in the last layer; (5) pooling: the calculation of the average pooling is supported by the impulse neural network, and no special processing is needed; while maximal pooling allows only the pulse with the most emittance neuron to pass by adding a threshold function to the neuron. Therefore, in this embodiment, when the deep artificial neural network model under the Keras framework is converted into the impulse neural network in step 3), five operations including bias conversion, weight parameter regularization, BN layer conversion, softmax layer conversion, and pooling conversion are included.
The bias plays a role in enhancing the capability of the network to express information in the convolutional neural network, so the bias is a standard configuration in the convolutional neural network, but in the prior art conversion work, the bias is limited by the imperfection of a conversion method, and the bias does not support the conversion. In a pulsed neural network, biasing can be achieved simply with a constant input current of the same sign. In this embodiment, the step of bias conversion includes: and converting the bias in the deep artificial neural network model under the Keras framework into constant input current with the same sign and inputting the constant input current to the corresponding pulse neuron, wherein the bias value is that the membrane potential is increased when the bias value is positive and plays a role in exciting the pulse neuron, and the membrane potential is reduced when the bias value is negative. FIG. 3 is a diagram of a pulsed neuron accumulating weights (potentials) and biases at synapses whose magnitude is accumulated at the membrane potential of a post-synaptic neuron only if a pre-synaptic neuron emits a pulse. However, the bias does not have such a requirement, and the bias is continuously input to the neuron, and the bias value is regular, so that the membrane potential is increased for stimulating the neuron, and is negative, so that the membrane potential is reduced for inhibiting the neuron.
In this embodiment, the regularizing of the weight parameter includes: setting the maximum activation value in the l layer of the impulse neural network to be lambda for the l layer in the convolutional neural networkl＝max[al]Wherein a islIs the activation value in the l-th layer of the spiking neural network, max.]Representing the maximum value, weighted by WlOffset is bl，WlIs a matrix representing the weight parameters of the l layers, blIs a vector representing the bias of l-layer neurons or convolution kernels, the weights after regularization are:
Figure BDA0002784761360000071
in the above formula, Wl' weight after regularization, WlAs a weight before regularization, λl-1Is the maximum activation value, lambda, in layer l-1lIs the maximum activation value in the l-th layer;
the bias is as follows:
Figure BDA0002784761360000072
in the above formula, bl' bias after regularization, blFor bias before regularization, λlIs the maximum activation value in the l-th layer. Finally, W obtainedl',bl' are the weight and bias, respectively, of the l layers after regularization. softmax is commonly used in the output layer of deep artificial neural networks because its result is a normalized and strictly positive class probability with an output of 0,1]The above regularization operation is therefore not performed with the network layer with softmax operation.
In addition, a more robust regularization strategy can be used simultaneously, and an activation value between output activation percentiles [99.0 and 99.9] is selected to replace the maximum activation value for regularization instead of regularization by using the maximum activation value for regularization. If the selection is proper, the problems of high emissivity and low emissivity can be well balanced.
The BN layer can reduce internal covariate transitions in convolutional neural networks to speed up model convergence, typically just after the convolutional or fully-connected layers, regularizing the activation of the convolutional or fully-connected layers. The BN layer has four types of parameters which are respectively: mean μ, variance σ2Scaling factor γ, transition factor β. If the output of the immediately preceding convolutional layer is activated as X ═ X1...mAnd then, there are:
Figure BDA0002784761360000081
Figure BDA0002784761360000082
it can be seen that the mean and variance are the mean and variance of the calculated input. With the mean and the variance, the input is regularized using these two parameters. In this embodiment, the BN layer switching includes: firstly, regularizing input according to a mean value and a variance of the input of a BN layer:
Figure BDA0002784761360000083
in the above formula, the first and second carbon atoms are,
Figure BDA0002784761360000084
for the regularized input, xiIs the original input, mu is the mean, sigma2For variance, ε is an infinitesimally small positive value, avoiding σ2An error of 0 has little effect on the regularization result, so that regularization directly on activation changes the characterization of the information.
The regularized input is then transformed as follows:
Figure BDA0002784761360000085
in the above formula, yiIs the original input xiOutputting after passing through a BN layer; gamma and beta are parameters which need to be learned in the training stage and are used for restoring activated information representation;
Figure BDA0002784761360000086
is a regularized input; for the model that has been trained, the mean μ, the variance σ2In the embodiment, before the artificial neural network is converted to the impulse neural network, the calculated parameters of the BN layer are firstly integrated into the weights of the immediately previous layer, so that the effect of BN is reserved for the whole model, and meanwhile, no processing is needed in the conversion process.
Then, the calculation parameters of the BN layer are integrated into the weight of the immediately previous layer, and the calculation function of fusing the parameters of the BN layer into the weight parameters of the previous network layer is expressed as follows:
Figure BDA0002784761360000087
in the above formula, the first and second carbon atoms are,
Figure BDA0002784761360000088
respectively, the weight and bias parameters of the layer after the BN layer parameters are fused, W and b are respectively the weight and bias parameters of the layer immediately before the BN layer, gamma is a scaling coefficient, mu is a mean value, and sigma is2Is a variance, and ε is a variance σ2A very small positive value of error at 0.
softmax is typically used in the output layer of deep artificial neural networks because its result is a normalized and strictly positive-class probability. In this embodiment, the step of converting the softmax layer includes: firstly, performing softmax calculation shown in the following formula on the membrane potential of a pulse neuron;
Figure BDA0002784761360000091
in the above formula, xiRepresents the value of the membrane potential, λ, of each neuroniIs the ratio of membrane potential of each pulse neuron, the ratio lambdaiHas a value range of [0,1 ]]；
Then the ratio lambda of the membrane potential of each pulse neuroniAs an arrival rate parameter of the poisson process of the corresponding pulse neuron, wherein the function expression of the poisson process is as follows:
Figure BDA0002784761360000092
in the above formula, N (t)i+ τ) represents the number of times that the neuron i emits a pulse during the period t + τ, and naturally N (0) has a value of 0, kiTo representThe number of pulses emitted by the pulse neuron i in the time period t, τ ═ 1 represents the time interval during which the pulse neuron emits pulses, and the known arrival rate parameter λ in the above equationiAnd the number k of the pulse neurons i transmitting pulses in the time period t, so as to judge which pulse neuron should transmit pulses at the next time of t and the function value of the poisson process
Figure BDA0002784761360000094
The one neuron with the largest value fires at the next instant of t.
The way of pooling in convolutional neural networks mainly includes both maximal pooling and average pooling. Many successful convolutional neural networks use maximal pooling to downsample spatial features, which is empirically slightly better than average pooling. In this embodiment, a threshold function is set to filter out non-maximal activation emission rate neurons, so that maximal pooling operation is realized in the impulse neural network model. In this embodiment, the detailed steps of the pooling conversion include: recording pulse emission sequence information of each neuron aiming at a convolutional layer which needs to be maximally pooled, recording the pulse emission sequence information of each neuron as 1 if the membrane potential of the pulse neuron is not lower than a threshold value at each emission moment, and recording the pulse emission sequence information of the pulse neuron as a string S consisting of 0 and 1 if the membrane potential of the pulse neuron is lower than the threshold value; allowing only the pulse of the pulse neuron with the highest emissivity in the neurons with the largest pooled nucleus number to pass through at each pulse time, and if the membrane potential of the pulse neuron is not lower than the threshold value, allowing the pulse neuron to generate a pulse (if the membrane potential of the pooled nucleus with the largest probability is 2x2, namely allowing only the pulse of the neurons with the largest emissivity in the 4 neurons [ if the membrane potential is not lower than the threshold value, namely the pulse is generated ] to pass through at each pulse time), allowing T to be the current time point, allowing N to be the total number of the pulses generated by the neurons containing the pulse transmission time, and filtering the neurons with the highest non-activation emissivity by a threshold function shown in the following formula aiming at the sequence S, thereby realizing the operation of the largest pooling in the pulse neural network model;
Figure BDA0002784761360000093
in the above formula, N is the total number of pulses generated by the neurons at the pulse transmission time, T is the current time point, S (T) is the value of the corresponding time in the sequence S, the threshold function is used to pass the pulse of the neuron with the maximum N value among the pulse neurons generating the pulses at the current time, and if the N values are the same, one pulse is randomly passed.
In addition, since the pulse transmission mechanism is a special case of the ReLU and the multiply-accumulate operation performed by the neurons of the artificial neural network is different from the accumulate input potential operation of the pulse neurons, the transformed pulse model generally needs to be adjusted by the hyper-parameter to obtain the performance comparable to that of the original model. And after the five operations are completed, the method also comprises the step of adjusting the super parameters, and the adjusted super parameters comprise a neuron membrane potential threshold value, a membrane potential resetting mode, setting of a refractory period, pulse delay emission and membrane potential leakage. Through the adjustment of the hyper-parameters, the performance of the converted pulse model can be comparable to that of the original model, and the accuracy of the converted model is ensured not to be greatly reduced. And storing the pulse neural network model after adjustment, so that all conversion work can be completed.
In summary, the present embodiment can be uniformly converted into the impulse neural network model for the deep neural network models under the intelligent frameworks such as tensrflow, Keras, and pittch. In this embodiment, for all special operation layers of the deep neural network model, calculation operations are retained in the transformed impulse neural network, which mainly include a bias, a regularization layer, a BN layer, a Softmax layer, a pooling layer, and the like. In order to verify the effect of the method of the embodiment, a test hardware environment is set up to be AMD R72700X, a video card adopts NVIDIA RTX2070Super, and an operating system is ubuntu 16.04. Meanwhile, two pulse neural network simulation tools are selected: the INISim (Keras framework-based impulse neural network simulator), nest (dedicated neural network simulator) was tested as a comparison to the method of this example. The pulse neural network models obtained by converting the binary and non-binary deep artificial neural network models are tested on MNIST and CIFAR-10 data respectively, and the result is shown in figure 4. As can be seen from fig. 3, for the MNIST dataset, the non-binary network can achieve lossless conversion, and the precision before and after the conversion of the binary network falls by 1.23%; aiming at the CIFAR-10 data set, the precision before and after the non-binary network conversion is reduced by 0.73 percent, and the precision before and after the binary network conversion is reduced by 4.38 percent. The method can realize the self-adaptive conversion of various typical deep artificial neural networks to the impulse neural network under the condition of no loss or little loss of precision.
In addition, the present embodiment also provides an adaptive model conversion system for a spiking neural network, which includes a computer device programmed or configured to execute the steps of the aforementioned adaptive model conversion method for a spiking neural network, or a computer program programmed or configured to execute the aforementioned adaptive model conversion method for a spiking neural network is stored in a memory of the computer device.
Furthermore, the present embodiment also provides a computer-readable storage medium having stored therein a computer program programmed or configured to execute the aforementioned pulse neural network-oriented adaptive model conversion method.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is directed to methods, apparatus (systems), and computer program products according to embodiments of the application wherein instructions, which execute via a flowchart and/or a processor of the computer program product, create means for implementing functions specified in the flowchart and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.

Claims (10)

1. An adaptive model conversion method for a pulse neural network is characterized by comprising the following steps:
1) obtaining a deep artificial neural network model to be converted, wherein the deep artificial neural network model to be converted is a deep neural network model under any one frame of Keras, Pythrch and TensorFlow;
2) if the deep artificial neural network model to be converted is a deep artificial neural network model under a Pythrch or TensorFlow framework, converting the deep artificial neural network model into a deep artificial neural network model under a Keras framework;
3) and converting the deep artificial neural network model under the Keras framework into a pulse neural network.
2. The method for transforming the pulse neural network-oriented adaptive model according to claim 1, wherein the detailed steps of step 2) comprise:
2.1) judging the type of the deep artificial neural network model to be converted, and if the type is the deep artificial neural network model under the Keras framework, skipping to execute the step 3); otherwise, skipping to execute the step 2.2);
2.2) reasoning the deep artificial neural network model to be converted under the original frame, so as to analyze the deep artificial neural network model to be converted, and obtain a weight file of the deep artificial neural network model to be converted, an information file of a model structure and a logical reasoning graph of the model;
and 2.3) constructing and obtaining the converted deep artificial neural network model under the Keras framework based on the weight file of the deep artificial neural network model to be converted, the information file of the model structure and the logic inference diagram of the model.
3. The method for transforming the impulse neural network-oriented adaptive model according to any one of claims 1-2, wherein the step 3) of transforming the deep artificial neural network model under the Keras framework into the impulse neural network comprises five operations of bias transformation, weight parameter regularization, BN layer transformation, softmax layer transformation and pooling transformation; and after the five operations are completed, the method also comprises the step of adjusting the super parameters, and the adjusted super parameters comprise a neuron membrane potential threshold value, a membrane potential resetting mode, setting of a refractory period, pulse delay emission and membrane potential leakage.
4. The method for the adaptive model conversion for the spiking neural network according to claim 3, wherein the step of bias conversion comprises: and converting the bias in the deep artificial neural network model under the Keras framework into constant input current with the same sign and inputting the constant input current to the corresponding pulse neuron, wherein the bias value is that the membrane potential is increased when the bias value is positive and plays a role in exciting the pulse neuron, and the membrane potential is reduced when the bias value is negative.
5. The impulse-oriented neural network of claim 3The adaptive model conversion method according to (1), wherein the step of regularizing the weight parameter includes: setting the maximum activation value in the l layer of the impulse neural network to be lambda for the l layer in the convolutional neural networkl＝max[al]Wherein a islIs the activation value in the l-th layer of the spiking neural network, max.]Representing the maximum value, weighted by WlOffset is bl，WlIs a matrix representing the weight parameters of the l layers, blIs a vector representing the bias of l-layer neurons or convolution kernels, the weights after regularization are:
Figure FDA0002784761350000011
in the above formula, Wl'For the normalized weight, WlAs a weight before regularization, λl-1Is the maximum activation value, lambda, in layer l-1lIs the maximum activation value in the l-th layer;
the bias is as follows:
Figure FDA0002784761350000021
in the above formula, bl'For normalized bias, blFor bias before regularization, λlIs the maximum activation value in the l-th layer.
6. The method for the adaptive model conversion for the spiking neural network according to claim 3, wherein the BN layer conversion step comprises: firstly, regularizing input according to a mean value and a variance of the input of a BN layer:
Figure FDA0002784761350000022
in the above formula, the first and second carbon atoms are,
Figure FDA0002784761350000023
for the regularized input, xiIs the original input, mu is the mean, sigma2Is a variance, and ε is a variance σ2A very small positive value of error at 0;
the regularized input is then transformed as follows:
Figure FDA0002784761350000024
in the above formula, yiIs the original input xiOutputting after passing through a BN layer; gamma and beta are parameters which need to be learned in the training stage and are used for restoring activated information representation;
Figure FDA0002784761350000025
is a regularized input;
then, the calculation parameters of the BN layer are integrated into the weight of the immediately previous layer, and the calculation function of fusing the parameters of the BN layer into the weight parameters of the previous network layer is expressed as follows:
Figure FDA0002784761350000026
in the above formula, the first and second carbon atoms are,
Figure FDA0002784761350000027
respectively, the weight and bias parameters of the layer after the BN layer parameters are fused, W and b are respectively the weight and bias parameters of the layer immediately before the BN layer, gamma is a scaling coefficient, mu is a mean value, and sigma is2Is a variance, and ε is a variance σ2A very small positive value of error at 0.
7. The impulse neural network-oriented adaptive model transformation method according to claim 3, wherein the step of softmax layer transformation comprises: firstly, performing softmax calculation shown in the following formula on the membrane potential of a pulse neuron;
Figure FDA0002784761350000028
in the above formula, xiRepresents the value of the membrane potential, λ, of each neuroniIs the ratio of membrane potential of each pulse neuron, the ratio lambdaiHas a value range of [0,1 ]]；
Then the ratio lambda of the membrane potential of each pulse neuroniAs an arrival rate parameter of the poisson process of the corresponding pulse neuron, wherein the function expression of the poisson process is as follows:
Figure FDA0002784761350000031
in the above formula, N (t)i+ τ) represents the number of times that the neuron i emits a pulse during the period t + τ, and naturally N (0) has a value of 0, kiRepresenting the number of pulses emitted by a pulse neuron i during a time period t, τ -1 represents the time interval during which the pulse neuron emits pulses, where the known arrival rate parameter λ isiAnd the number k of the pulse neurons i transmitting pulses in the time period t, so as to judge which pulse neuron should transmit pulses at the next time of t and the function value of the poisson process
Figure FDA0002784761350000032
The one neuron with the largest value fires at the next instant of t.
8. The method for adaptive model transformation for spiking neural networks according to claim 4, wherein the detailed steps of pooling transformation include: recording pulse emission sequence information of each neuron aiming at a convolutional layer which needs to be maximally pooled, recording the pulse emission sequence information of each neuron as 1 if the membrane potential of the pulse neuron is not lower than a threshold value at each emission moment, and recording the pulse emission sequence information of the pulse neuron as a string S consisting of 0 and 1 if the membrane potential of the pulse neuron is lower than the threshold value; the pulse of the pulse neuron with the largest emissivity in the neurons with the largest pooling core number can only pass through each pulse time, and on the premise that the membrane potential of the pulse neuron is not lower than a threshold value to enable the pulse neuron to generate, T is the current time point, N is the total number of pulses generated by the neurons with the pulse transmitting time, and the neurons with the non-largest activation emissivity are filtered by a threshold function shown by the following formula aiming at a sequence S, so that the largest pooling operation is realized in a pulse neural network model;
Figure FDA0002784761350000033
in the above formula, N is the total number of pulses generated by the neurons at the pulse transmission time, T is the current time point, S (T) is the value of the corresponding time in the sequence S, the threshold function is used to pass the pulse of the neuron with the maximum N value among the pulse neurons generating the pulses at the current time, and if the N values are the same, a pulse is passed randomly.
9. An impulse neural network-oriented adaptive model transformation system, comprising a computer device, wherein the computer device is programmed or configured to execute the steps of the impulse neural network-oriented adaptive model transformation method according to any one of claims 1 to 8, or a computer program programmed or configured to execute the impulse neural network-oriented adaptive model transformation method according to any one of claims 1 to 8 is stored in a memory of the computer device.
10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, the computer program being programmed or configured to perform the method for adaptive model transformation for a spiking neural network according to any one of claims 1 to 8.
CN202011294317.2A 2020-11-18 2020-11-18 Pulse neural network-oriented adaptive model conversion method and system Pending CN112288080A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011294317.2A CN112288080A (en) 2020-11-18 2020-11-18 Pulse neural network-oriented adaptive model conversion method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011294317.2A CN112288080A (en) 2020-11-18 2020-11-18 Pulse neural network-oriented adaptive model conversion method and system

Publications (1)

Publication Number Publication Date
CN112288080A true CN112288080A (en) 2021-01-29

Family

ID=74397917

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011294317.2A Pending CN112288080A (en) 2020-11-18 2020-11-18 Pulse neural network-oriented adaptive model conversion method and system

Country Status (1)

Country Link
CN (1) CN112288080A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112966815A (en) * 2021-03-31 2021-06-15 中国科学院自动化研究所 Target detection method, system and equipment based on impulse neural network
CN113158869A (en) * 2021-04-15 2021-07-23 深圳市优必选科技股份有限公司 Image recognition method and device, terminal equipment and computer readable storage medium
CN113159276A (en) * 2021-03-09 2021-07-23 北京大学 Model optimization deployment method, system, equipment and storage medium
CN114580622A (en) * 2022-05-07 2022-06-03 中科南京智能技术研究院 Method and system for converting neuron membrane potential into pulse sequence

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113159276A (en) * 2021-03-09 2021-07-23 北京大学 Model optimization deployment method, system, equipment and storage medium
CN113159276B (en) * 2021-03-09 2024-04-16 北京大学 Model optimization deployment method, system, equipment and storage medium
CN112966815A (en) * 2021-03-31 2021-06-15 中国科学院自动化研究所 Target detection method, system and equipment based on impulse neural network
CN113158869A (en) * 2021-04-15 2021-07-23 深圳市优必选科技股份有限公司 Image recognition method and device, terminal equipment and computer readable storage medium
CN114580622A (en) * 2022-05-07 2022-06-03 中科南京智能技术研究院 Method and system for converting neuron membrane potential into pulse sequence
CN114580622B (en) * 2022-05-07 2022-07-05 中科南京智能技术研究院 Method and system for converting neuron membrane potential into pulse sequence

Similar Documents

Publication Publication Date Title
CN107092959B (en) Pulse neural network model construction method based on STDP unsupervised learning algorithm
CN112288080A (en) Pulse neural network-oriented adaptive model conversion method and system
WO2023284142A1 (en) Signal processing method for neuron in spiking neural network and method for training said network
Alaloul et al. Data processing using artificial neural networks
CN108491874B (en) Image list classification method based on generation type countermeasure network
CN111858989B (en) Pulse convolution neural network image classification method based on attention mechanism
WO2022134391A1 (en) Fusion neuron model, neural network structure and training and inference methods therefor, storage medium, and device
CN109829541A (en) Deep neural network incremental training method and system based on learning automaton
JP2019032808A (en) Mechanical learning method and device
EP3121767A1 (en) Methods and systems for implementing deep spiking neural networks
KR20170031695A (en) Decomposing convolution operation in neural networks
CN111639754A (en) Neural network construction, training and recognition method and system, and storage medium
CN110223785A (en) A kind of infectious disease transmission network reconstruction method based on deep learning
CN111260124A (en) Chaos time sequence prediction method based on attention mechanism deep learning
CN110717627A (en) Full traffic prediction method based on dual graph framework
CN112906828A (en) Image classification method based on time domain coding and impulse neural network
JP2020123345A (en) Learning method and learning device for generating training data acquired from virtual data on virtual world by using generative adversarial network (gan), to thereby reduce annotation cost required in learning processes of neural network for autonomous driving, and testing method and testing device using the same
CN111382840B (en) HTM design method based on cyclic learning unit and oriented to natural language processing
CN114118375A (en) Continuous dynamic network characterization learning method based on time sequence diagram Transformer
Yu et al. Efficient and reliable training of neural networks
Harikrishnan et al. Handwritten digit recognition with feed-forward multi-layer perceptron and convolutional neural network architectures
KR102191346B1 (en) Method for generating spiking neural network based on burst spikes and inference apparatus based on spiking neural network
CN111260054A (en) Learning method for improving accuracy of associative memory impulse neural network
JP2907486B2 (en) Neural network device
WO2020220926A1 (en) Multimedia data identification method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210129