CN113837380A - Neural network training method and device based on biological self-organization back propagation - Google Patents
Neural network training method and device based on biological self-organization back propagation Download PDFInfo
- Publication number
- CN113837380A CN113837380A CN202111095440.6A CN202111095440A CN113837380A CN 113837380 A CN113837380 A CN 113837380A CN 202111095440 A CN202111095440 A CN 202111095440A CN 113837380 A CN113837380 A CN 113837380A
- Authority
- CN
- China
- Prior art keywords
- neural network
- training
- layer
- back propagation
- plasticity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 201
- 238000000034 method Methods 0.000 title claims abstract description 151
- 238000012549 training Methods 0.000 title claims abstract description 148
- 210000002569 neuron Anatomy 0.000 claims abstract description 103
- 238000005457 optimization Methods 0.000 claims abstract description 57
- 238000012545 processing Methods 0.000 claims abstract description 20
- 230000003956 synaptic plasticity Effects 0.000 claims abstract description 20
- 230000001419 dependent effect Effects 0.000 claims abstract description 10
- 230000002441 reversible effect Effects 0.000 claims abstract description 5
- 239000012528 membrane Substances 0.000 claims description 49
- 230000006870 function Effects 0.000 claims description 42
- 210000000225 synapse Anatomy 0.000 claims description 34
- 230000008569 process Effects 0.000 claims description 30
- 238000013139 quantization Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000008859 change Effects 0.000 claims description 10
- 208000028659 discharge Diseases 0.000 claims description 10
- 230000001105 regulatory effect Effects 0.000 claims description 10
- 238000011084 recovery Methods 0.000 claims description 9
- 230000000946 synaptic effect Effects 0.000 claims description 9
- 230000003518 presynaptic effect Effects 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000001537 neural effect Effects 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 230000002964 excitative effect Effects 0.000 claims description 6
- 230000002401 inhibitory effect Effects 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 6
- 230000000644 propagated effect Effects 0.000 claims description 5
- 238000004088 simulation Methods 0.000 claims description 4
- 238000013459 approach Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 230000005764 inhibitory process Effects 0.000 claims description 3
- 230000007996 neuronal plasticity Effects 0.000 claims description 3
- 239000002858 neurotransmitter agent Substances 0.000 claims description 3
- 230000036390 resting membrane potential Effects 0.000 claims description 3
- 239000004576 sand Substances 0.000 claims description 3
- 238000000844 transformation Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000005265 energy consumption Methods 0.000 abstract description 3
- 230000002829 reductive effect Effects 0.000 abstract description 3
- 238000012421 spiking Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 210000005056 cell body Anatomy 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000033228 biological regulation Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000027928 long-term synaptic potentiation Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001242 postsynaptic effect Effects 0.000 description 2
- 240000002627 Cordeauxia edulis Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000003284 homeostatic effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 210000004205 output neuron Anatomy 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a neural network training method and a device based on biological self-organization back propagation, wherein the training method comprises the following steps: obtaining a training sample; inputting the data into a neural network, and outputting the data after data processing; obtaining a first optimized weight parameter of an output layer neuron by an error back propagation method or a time sequence dependent plasticity method; sequentially generating biological self-organization synaptic plasticity from the hidden layer to the neuron of the input layer in the reverse direction, thereby obtaining second optimized weight parameters of the hidden layer and the input layer of the neural network; and optimizing the neural network once according to the first optimized weight parameter and the second optimized weight parameter, and repeating the optimization for multiple times until the loss function of the neural network converges to a preset value. The second optimized weight parameter is obtained in a biological self-organizing mode and is combined with the first optimized weight parameter obtained by a traditional error back propagation method or a time sequence dependence plasticity method, so that the computer operation amount is less, and the training energy consumption of the computer on the whole network is obviously reduced.
Description
Technical Field
The invention relates to the technical field of brain-like intelligence and artificial intelligence, in particular to a neural network training method and device based on biological self-organization back propagation.
Background
Biological intelligence has been through thousands of years of genetic evolution, and many special structures and functions such as neuronal diversity, network functional loop specificity, and neural synapse pruning with on-line learning and memory maintenance capabilities have been consolidated. Among them, biological synaptic plasticity plays a great role in the aspects of network function maintenance, learning and the like, and is also the core of brain-like intelligence research. Conventionally, a learning algorithm mainly based on an artificial neural network generalizes various types of parameters in the network to a global and unified optimization framework by means of a Back Propagation (BP) method, and by designing an error function, parameters in the network are adjusted in a direction of maximizing network task performance and minimizing errors, so that excellent performance is finally obtained. However, the BP method does not obtain evidence of biological discovery, and the method belongs to a supervised learning neural network training method, and has high power consumption and high requirement on the number of training samples in the training process.
Disclosure of Invention
The invention provides a neural network training method and device based on biological self-organizing back propagation, which are used for solving the defects that supervised learning needs to be carried out on more training samples and the power consumption is high when an error back propagation method is applied to neural network training in the prior art, and the purpose that unsupervised and self-organizing training is carried out on a neural network by utilizing biological synapse plasticity is achieved, the number of required training samples is small, and the power consumption in the training process is low.
The invention provides a neural network training method based on biological self-organizing back propagation, which is applied to a pulse neural network or an artificial neural network and comprises the following steps:
obtaining a training sample;
inputting the training sample into a neural network, and outputting the training sample after data processing is carried out on the training sample through an input layer, at least one hidden layer and an output layer of the neural network;
obtaining a first optimization weight parameter of a neuron of the neural network output layer by an error back propagation method or a time sequence dependent plasticity method according to the data output by the neural network output layer;
sequentially generating biological self-organizing synaptic plasticity from the hidden layer to the input layer through neurons, and performing back propagation to obtain second optimized weight parameters of the hidden layer and the input layer of the neural network;
performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter;
and performing the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
According to the neural network training method based on the biological self-organizing back propagation provided by the invention, when the training method is applied to the impulse neural network, the training sample is obtained, and then the method further comprises the following steps: carrying out Spike quantization coding on the training samples to generate time sequence input information; the quantization coding formula is:
wherein, Ispikes(t) is quantization coding, Iraw(t) is the training sample, IrdAnd (t) is a random number between 0 and 1 which is randomly generated.
According to the neural network training method based on the biological self-organizing back propagation provided by the invention, when the training method is applied to the artificial neural network, the training sample is obtained, and then the method further comprises the following steps: and carrying out normalization processing on the training samples to obtain input information.
According to the neural network training method based on the biological self-organizing back propagation provided by the invention, when the training method is applied to the impulse neural network, the training sample is input into the neural network, and the training sample is output after being subjected to data processing through an input layer, at least one hidden layer and an output layer of the neural network, and the method specifically comprises the following steps:
Short-Term synaptic Plasticity (Short-Term Plasticity) occurs at the synapses of neurons in the input and hidden layers of the neural network, as follows:
wherein u and x are the parameters regulating the short-term synaptic potentiation or inhibition, respectively, and represent the amount of neurotransmitter used in the synapse, taufAnd τdThe integral recovery quantities of U and x, respectively, U is a hyperparameter, and is the increment of a regulating parameter U generated by Spike quantization coding, and A is a synapse weight Wi,jUpper learnable amount, τsAs synaptic currents IsynThe integral recovery amount of (1);
the neuronal cell bodies were subjected to leaky-integrate-and-discharge treatment as follows:
wherein, taumIs the integral recovery of V (t), where V (t) is the neuronal membrane potential, VLIs the membrane potential leakage current, gLIs the conductance of the leakage current of the membrane potential, gE|IConductance being of excitatory or inhibitory neurons, VE|IMembrane potential reset voltage, V, for excitatory or inhibitory activityTrTo discharge threshold, VresetIs the resting membrane potential.
According to the neural network training method based on the biological self-organizing back propagation provided by the invention, when the training method is applied to an artificial neural network, the training sample is input into the neural network, and the training sample is output after being subjected to data processing through an input layer, at least one hidden layer and an output layer of the neural network, and the method specifically comprises the following steps: after information passes through the input layer, the information is transmitted to the neurons of all the hidden layers through full-connection weight, and after the information is integrated, the information is transmitted to the output layer through the nonlinear neurons.
According to the neural network training method based on the biological self-organization back propagation provided by the invention, when the training method is applied to a pulse neural network, biological self-organization synapse plasticity occurs from a hidden layer to an input layer through neurons in sequence, and back propagation is carried out, so that second optimized weight parameters of the hidden layer and the input layer of the neural network are obtained, and the method specifically comprises the following steps:
the information transferred between neurons keeps the input-output information stable, making the neuron membrane potential plastic, as follows:
wherein, Vj(t) is the membrane potential at the current instant,
for the membrane potential at a future moment, ηeIn order to obtain a learning rate,
fusing the membrane potential plasticity and the membrane potential in the forward propagation process of the information in the neural network to obtain a comprehensive membrane potential:
wherein the content of the first and second substances,
for membrane potential changes during information forward propagation,
is a change in the membrane potential balance after synaptic plasticity of neurons, Δ Vj(t) is
And
weighted sum of (1), TeTotal learning period, teThe current time instant, as the simulation time increases,
gradually approaches 1 and thus will eventually remain only
And (3) performing plasticity learning on part of synapses in neuron output synapses by adopting a pulse time sequence dependent plasticity rule, and learning neuron weights between a hidden layer and an output layer, wherein the following steps are as follows:
wherein A is+And A-To scale the parameters, tj,sAnd tk,sTo the discharge time, tau+And τ_Is an attenuation parameter;
and
postsynaptic adjustment of positive and negative values, respectively
j is the hidden layer neuron serial number, k is the output layer neuron serial number;
plasticity change
Plasticity propagated backwards to presynaptic by corresponding nonlinear transformations
In (1), as follows:
wherein the content of the first and second substances,
respectively the presynaptic regulating quantity of the second optimized weight parameter
Positive and negative values of, σn(. is) a non-linear transformation function for normalization, λpIs a range factor, λfIs an amplitude factor, y ═ Ediag,i(x) Representing the information transformation, transforming a vector into a diagonal matrix, yj,j＝xjI is the input layer neuron sequence number and j is the hidden layer neuron sequence number.
According to the neural network training method based on the biological self-organization back propagation provided by the invention, when the training method is applied to an artificial neural network, biological self-organization synapse plasticity occurs from a hidden layer to an input layer through neurons in sequence, and back propagation is carried out, so that second optimized weight parameters of the hidden layer and the input layer of the neural network are obtained, and the method specifically comprises the following steps:
construction of an energy function ERBMConstraining plasticity between neurons of adjacent layers, and setting a hyper-parameter thetasbpRepresents a range of constraints, as follows:
energy function ERBMThe Loss function of the error back propagation method is combined into a unified constraint function, which is as follows:
Closs＝βCRBM+ERBM
wherein, CRBMThe method is a loss function of an artificial neural network, and is a restricted Boltzmann machine;
neuronal plasticity between adjacent layers is modulated as follows:
wherein eta isbpFor learning rate based on error back propagation method, etaSBPFor the learning rate based on the bio-self-organized back propagation method,
i is a matrix with diagonal 1, λfIs an amplitude factor, λpIs a range factor.
The invention also provides a neural network training device based on biological self-organizing back propagation, which is applied to a pulse neural network or an artificial neural network, and comprises:
the sample acquisition unit is used for acquiring a training sample;
the forward propagation unit is used for inputting the training sample into a neural network, and the training sample is output after being subjected to data processing through an input layer, at least one hidden layer and an output layer of the neural network;
the output layer back propagation unit is used for obtaining a first optimization weight parameter of the neural network output layer neuron through an error back propagation method or a time sequence dependence plasticity method according to the data output by the neural network output layer;
the self-organization reverse propagation unit is used for sequentially generating biological self-organization synapse plasticity from the hidden layer to the input layer through neurons and performing reverse propagation so as to obtain second optimized weight parameters of the hidden layer and the input layer of the neural network;
the network single optimization unit is used for performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter;
and the network cycle optimization unit is used for carrying out the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the method for training the neural network based on the self-organizing back propagation.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for training a neural network based on back propagation of biological self-organization as defined in any one of the above.
According to the neural network training method and device based on biological self-organizing back propagation, a traditional error back propagation method is adopted in the training process of an artificial neural network and a pulse neural network to obtain a first optimized weight parameter, back propagation is carried out by utilizing the plasticity of the biological self-organizing synapses of neurons to change the plasticity of front and back synapses of the neurons so as to obtain a second optimized weight parameter through self-organization, and the weight parameter in the training process of the neural network is comprehensively adjusted through the first optimized weight parameter and the second optimized weight parameter so as to carry out network optimization; compared with other supervision type methods, the method has the characteristics of less computer operation amount, realization of synchronous operation and parallel operation of the computer, acceleration of network convergence speed and obvious reduction of training energy consumption of the computer on the whole network.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart of a neural network training method based on biological self-organization back propagation provided by the invention;
FIG. 2 is a flowchart illustrating the training method 120 applied to the spiking neural network according to the present invention;
FIG. 3 is a flowchart illustrating the step 140 of the training method applied to the spiking neural network;
FIG. 4 is a schematic diagram of simulation experiment results of the training method applied to the impulse neural network;
FIG. 5 is a schematic structural diagram of a neural network training device based on biological self-organization back propagation provided by the invention;
fig. 6 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the continuous development of the fields of brain-like intelligence and artificial intelligence, a plurality of unsupervised and self-organized biological plasticity methods are gradually proposed, and the methods are obviously different from a Back Propagation (BP) method. The Self-organizing back propagation (SBP) provided by the embodiment of the invention is a plasticity method for biological discovery, emphasizes the site where synapse plasticity occurs, can reversely transmit the plasticity to other sites, has biological discovery evidences in the transmission direction, the transmission range and the transmission type, and can simultaneously show advantages on a Spiking Neural Network (SNN) and an Artificial Neural Network (ANN) through modeling.
The embodiment of the invention shown in fig. 1 provides a neural network training method based on biological self-organization back propagation, wherein the training method is applied to an impulse neural network or an artificial neural network, and the training method comprises the following steps:
step 110: obtaining a training sample;
when the training method provided by the embodiment of the invention is applied to a pulse neural network (SNN), the training sample is obtained, and then Spike quantization coding is carried out on the training sample to generate time sequence input information, wherein the quantization coding formula is as follows:
wherein, Ispikes(t) is quantization coding, Iraw(t) is the training sample, IrdAnd (t) is a random number between 0 and 1 which is randomly generated. Inputting the generated input information into the neural network of step 120.
When the training method provided by the embodiment of the invention is applied to an Artificial Neural Network (ANN), the training sample is obtained, and then the training sample is subjected to normalization (including normalization of mean value, variance and the like)) to obtain input information.
Step 120: inputting the training sample into a neural network, and outputting the training sample after data processing is carried out on the training sample through an input layer, at least one hidden layer and an output layer of the neural network;
after the training samples of step 110 are encoded, the input information is input into the Spiking Neural Network (SNN), and then the information is forward propagated in the spiking neural network, as shown in fig. 2, the specific process is as follows:
step 210: the input sequence of the neurons of the input layer and the hidden layer of the neural network is different according to the input, and Short-Term synaptic Plasticity (Short-Term Plasticity) of different degrees is generated on synapses, as follows:
wherein u and x are the parameters regulating the short-term synaptic potentiation or inhibition, respectively, and represent the amount of neurotransmitter used in the synapse, taufAnd τdThe integral recovery quantities of U and x, respectively, U is a hyperparameter, and is the increment of a regulating parameter U generated by Spike quantization coding, and A is a synapse weight Wi,jUpper learnable amount, τsAs synaptic currents IsynThe integral recovery amount of (1);
step 220: the neuronal cell bodies were subjected to leaky-integrate-and-discharge treatment as follows:
wherein, taumIs the integral recovery of V (t), where V (t) is the neuronal membrane potential, VLIs the membrane potential leakage current, gLIs the conductance of the leakage current of the membrane potential, gE|IConductance being of excitatory or inhibitory neurons, VE|IMembrane potential reset voltage, V, for excitatory or inhibitory activityTrTo discharge threshold, VresetIs the resting membrane potential.
After the training samples in step 110 are encoded, and the input information is input into an Artificial Neural Network (ANN), the information is forward propagated in the artificial neural network, and the specific process is as follows: after information passes through the input layer, the information is transmitted to the neurons of all the hidden layers through full-connection weight, and after the information is integrated, the information is transmitted to the output layer through the nonlinear neurons.
Step 130: obtaining a first optimization weight parameter of a neuron of the neural network output layer by an error back propagation method or a time sequence dependent plasticity method according to the data output by the neural network output layer;
step 140: sequentially generating biological self-organizing synaptic plasticity from the hidden layer to the input layer through neurons, and performing back propagation to obtain second optimized weight parameters of the hidden layer and the input layer of the neural network;
in the embodiment of the invention, in the process of information transmission through the neurons of the input layer and the hidden layer, synapses output by the neurons generate Plasticity, and the Plasticity can be derived from time-sequence-Dependent Plasticity (STDP) of a pulse neural network or weight modification (such as BP-based) of partial Readout in an artificial neural network. Embodiments of the invention allow for the adoption of multi-source, multi-type post-synaptic plasticity occurrences of neurons, such as Long-Term Potentiation (LTP), Long-Term suppression (LTD), Hebb's learning rule, or other gradient-based weight changes. In the embodiments of the present invention, a post-synaptic or post-neuronal synaptic refers to a synapse on the output side of a neuron, and a pre-synaptic or pre-neuronal synaptic refers to a synapse on the input side of a neuron.
In this step, when the neural network model is a Spiking Neural Network (SNN), as shown in fig. 3, this step specifically includes:
step 310: the information transferred between neurons keeps the input-output information stable, making the neuron membrane potential plastic, as follows:
wherein, Vj(t) is the membrane potential at the current instant,
for the membrane potential at a future moment, ηeIs the learning rate;
specifically, information maintains steady state information representation in neurons, forms self-organizing synchronous Membrane Potential regulation (Homeo-Static Membrane Potential) in cells, and the stable plasticity of the Membrane Potential at the neuron level describes corresponding target neurons, and the update of the Membrane Potential of the target neurons is kept stable at the previous moment and the next moment.
Step 320: fusing the membrane potential plasticity and the membrane potential in the forward propagation process of the information in the neural network to obtain a comprehensive membrane potential:
wherein the content of the first and second substances,
for membrane potential changes during information forward propagation,
is a change in the membrane potential balance after synaptic plasticity of neurons, Δ Vj(t) is
And
weighted sum of (1), TeTotal learning period, teThe current time instant, as the simulation time increases,
gradually approaches 1 and thus will eventually remain only
Of formula (5)
Is the same as that of the formula (3), where
Has a specific physical meaning and represents the change amount of membrane potential in the forward propagation process, and the formula in the formula (3) is the basic formula of membrane potential change.
Step 330: and (3) performing plasticity learning on part of synapses in neuron output synapses by adopting a pulse time sequence dependent plasticity rule, and learning neuron weights between a hidden layer and an output layer, wherein the following steps are as follows:
wherein A is+And A-To scale the parameters, tj,sAnd tk,sTo the discharge time, tau+And τ_Is an attenuation parameter;
and
postsynaptic adjustment of positive and negative values, respectively
j is the hidden layer neuron serial number, k is the output layer neuron serial number;
step 330 describes the process of learning the synapses of neurons as a function of the time of the pre-and post-firing.
Step 340: plasticity change
Plasticity propagated backwards to presynaptic by corresponding nonlinear transformations
Its plasticity is transferred from the plasticity between cryptolayer neuron j and exporter neuron k as follows:
wherein the content of the first and second substances,
respectively the presynaptic regulating quantity of the second optimized weight parameter
Positive and negative values of, σn(. is) a non-linear transformation function for normalization, λpIs a range factor, λfIs an amplitude factor, y ═ Ediag,i(x) Representing the information transformation, transforming a vector into a diagonal matrix, yj,j＝xjI is the input layer neuron sequence number and j is the hidden layer neuron sequence number.
In this step, when the neural network model is an artificial neural network (SNN), this step specifically includes:
construction of an energy function ERBMConstraining plasticity between neurons of adjacent layers, and setting a hyper-parameter thetasbpRepresents a range of constraints, as follows:
energy function ERBMThe Loss function of the error back propagation method is combined into a unified constraint function, which is as follows:
Closs＝βCRBM+ERBM (10)
wherein, CRBMIs a loss function of artificial neural network, limited Boltzmann machine, ClossIs the loss required for the first optimization parameter optimization.
Neuronal plasticity between adjacent layers is modulated as follows:
wherein eta isbpFor learning rate based on error back propagation method, etaSBPFor the learning rate based on the bio-self-organized back propagation method,
i is a matrix with diagonal 1, λfIs an amplitude factor, λpIs a range factor.
Equation (9) represents an energy function, equation (10) represents that the loss function of the artificial neural network in the step includes both an energy function and an error propagation loss function, and equation (11) represents a mode of updating the loss function of the artificial neural network error back propagation.
In step 140, the process of backward propagation is self-organized, unsupervised, and uniform, well-defined in direction, from the synapse location where synaptic plasticity occurs, back-propagates along the connected soma into the input synapses input to the soma. This process differs between spiking neural networks and artificial neural networks. In the impulse neural network, information is spread by means of a simple range factor and an amplitude factor, and the stability of the network is kept by means of local steady state regulation (Homeo-mean Potential) of a neuron; in the artificial neural network, a limit similar to energy optimal constraint is added by means of setting a Loss function, and the plasticity of front and rear layers of neurons is connected together, so that the purposes of self-organization and unsupervised learning are achieved.
Step 150: performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter;
step 160: and performing the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
The training method provided by the embodiment of the invention can improve the classification performance of the original network, for a fixed data set such as MNIST, NETtalk, DVS-Gesture and the like, SBP is added into different neural networks SNN and ANN and fused into STDP corresponding to the SNN or BP corresponding to the ANN to form a fused training method, and the network trained by the fusion method improves the classification accuracy of the network.
Compared with other supervision type methods, the training method provided by the embodiment of the invention has the characteristics of less calculation amount, synchronous calculation and parallel calculation, so that the original part of plasticity learning is changed into SBP, the calculation amount can be obviously reduced greatly, and the convergence speed of the network is not changed greatly in the learning process, so that the energy consumption of computer resources in the training process of the whole network is greatly reduced compared with that of the original network.
The following table 1 shows the design of training by a self-organizing back propagation method (SBP) on a Spiking Neural Network (SNN) and testing on three data sets of MNIST, NETtalk and DVS-Gesture, and the results are shown in the table 1, and the results in the table 1 show that the method provides an effective and biologically reasonable self-organizing back propagation optimization method and achieves the performance exceeding that of other methods.
TABLE 1
Wherein, FF1 and FF2 are two processes of information forward propagation reaching the hidden layer and the output layer, m, n, k are the number of input, hidden and output neurons of the network, and FB1 and FB2 are two processes of feedback reaching the hidden layer and the input layer.
The design of training by a self-organizing back propagation method (SBP) on an Artificial Neural Network (ANN) is shown in FIG. 4, and the training is carried out on three large data sets, namely MNIST, NETtalk and DVS-Gesture, the result is shown in the figure, wherein the 'without SBP' in the artificial neural network training represents the performance of the training result on the data set when the SBP method is not added, the 'with SBP' represents the performance of the training result on the data set when the SBP method is added in the artificial neural network training, the 'Accuracy' represents the Accuracy of the training result, the 'cost' represents the computational consumption of the training, and the result in FIG. 4 shows that the self-organizing back propagation method (SBP) provides an effective and biologically reasonable self-organizing back propagation optimization method and achieves the performance exceeding the standard ANN-RBM.
In the following, a neural network training device based on self-organizing backward propagation according to an embodiment of the present invention is described, and the neural network training device based on self-organizing backward propagation described below and the neural network training method based on self-organizing backward propagation described above may be referred to correspondingly, as shown in fig. 5, and the neural network training device based on self-organizing backward propagation according to an embodiment of the present invention includes:
a sample obtaining unit 510, configured to obtain a training sample;
a forward propagation unit 520, configured to input the training sample into a neural network, where the training sample is output after being subjected to data processing through an input layer, at least one hidden layer, and an output layer of the neural network;
in the embodiment of the present invention, the forward propagation unit 520 needs to perform encoding processing on the training samples before performing forward propagation on the training samples. For the application of the training impulse neural network, the forward propagation unit 520 first performs Spike quantization coding on the training samples acquired by the sample acquisition unit 510 to generate time sequence input information; for the application of training the artificial neural network, the forward propagation unit 520 firstly normalizes the training samples acquired by the sample acquisition unit 510 to obtain input information.
In this embodiment of the present invention, for the application of training the impulse neural network, the forward propagation unit 520 includes:
a Short-time synaptic Plasticity generating subunit, configured to generate Short-time synaptic Plasticity (Short-Term Plasticity) on synapses of neurons of the input layer and the hidden layer of the neural network;
and the neuron cell body processing subunit is used for performing electric leakage, integration and discharge processing on the neuron cell body.
In the embodiment of the present invention, in the application of training the artificial neural network, when the forward propagation unit 520 performs processing, information is transmitted to the neurons of the hidden layers through full-link weights after passing through the input layer, and is transmitted to the output layer through the nonlinear neurons after being integrated.
An output layer back propagation unit 530, configured to obtain a first optimized weight parameter of a neuron in the output layer of the neural network by an error back propagation method or a time-dependent plasticity method according to the data output by the output layer of the neural network;
the self-organizing backward propagation unit 540 is used for sequentially generating biological self-organizing synaptic plasticity from the hidden layer to the input layer through neurons and performing backward propagation so as to obtain second optimized weight parameters of the hidden layer and the input layer of the neural network;
in an embodiment of the present invention, for the application of training the spiking neural network, the self-organizing back propagation unit 540 includes:
a neuron membrane potential plasticity generating subunit, for keeping the input-output information stable by the information transmitted between neurons, so that the neuron membrane potential generates plasticity;
the comprehensive membrane potential acquisition subunit is used for fusing the membrane potential plasticity and the membrane potential in the forward propagation process of the information in the neural network to obtain a comprehensive membrane potential;
the neuron weight learning subunit is used for performing plasticity learning on part of synapses in neuron output synapses by adopting a pulse time sequence dependence plasticity rule and learning neuron weights from the hidden layer to the output layer;
and the neuron weight learning subunit between the input layer and the hidden layer is used for carrying out nonlinear transformation on the plasticity change and reversely transmitting the plasticity change to presynaptic plasticity.
In an embodiment of the present invention, for an application of training an artificial neural network, the self-organizing back propagation unit 540 includes:
the neuron output synapse updating subunit is used for updating the numerical value of the neuron output synapse by an error back propagation method;
the plasticity constraint subunit between the adjacent layer neurons is used for constructing an energy function, constraining the plasticity between the adjacent layer neurons and setting a hyper-parameter representing constraint range;
the uniform constraint function acquisition subunit is used for combining the energy function and the loss function of the error back propagation method into a uniform constraint function;
and the neuron plasticity regulation subunit between adjacent layers is used for regulating the neuron plasticity between adjacent layers.
A network single optimization unit 550, configured to perform single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter;
and a network cycle optimization unit 560, configured to perform the single optimization process on the neural network for multiple times until a loss function of the neural network converges to a preset value.
An entity structure schematic diagram of an electronic device provided in an embodiment of the present invention is described below with reference to fig. 6, and as shown in fig. 6, the electronic device may include: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a method of training a neural network based on back propagation of a biological ad hoc network, the method comprising: obtaining a training sample; inputting the training sample into a neural network, and outputting the training sample after data processing is carried out on the training sample through an input layer, at least one hidden layer and an output layer of the neural network; obtaining a first optimization weight parameter of a neuron of the neural network output layer by an error back propagation method or a time sequence dependent plasticity method according to the data output by the neural network output layer; the neuron generates biological self-organizing synaptic plasticity from the hidden layer to the input layer in sequence and carries out back propagation, so that second optimized weight parameters of the hidden layer and the input layer of the neural network are obtained; performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter; and performing the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or make a contribution to the prior art, or may be implemented in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by a computer, the computer can execute the method for training a neural network based on biological self-organization back propagation provided by the above methods, the method includes: obtaining a training sample; inputting the training sample into a neural network, and outputting the training sample after data processing is carried out on the training sample through an input layer, at least one hidden layer and an output layer of the neural network; obtaining a first optimization weight parameter of a neuron of the neural network output layer by an error back propagation method or a time sequence dependent plasticity method according to the data output by the neural network output layer; the neuron generates biological self-organizing synaptic plasticity from the hidden layer to the input layer in sequence and carries out back propagation, so that second optimized weight parameters of the hidden layer and the input layer of the neural network are obtained; performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter; and performing the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
In yet another aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the method for training a neural network based on self-organizing back propagation of living beings provided in the foregoing: obtaining a training sample; inputting the training sample into a neural network, and outputting the training sample after data processing is carried out on the training sample through an input layer, at least one hidden layer and an output layer of the neural network; obtaining a first optimization weight parameter of a neuron of the neural network output layer by an error back propagation method or a time sequence dependent plasticity method according to the data output by the neural network output layer; the neuron generates biological self-organizing synaptic plasticity from the hidden layer to the input layer in sequence and carries out back propagation, so that second optimized weight parameters of the hidden layer and the input layer of the neural network are obtained; performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter; and performing the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. The neural network training method based on biological self-organization back propagation is characterized in that the training method is applied to an impulse neural network or an artificial neural network, and comprises the following steps:
obtaining a training sample;
inputting the training sample into a neural network, and outputting the training sample after data processing is carried out on the training sample through an input layer, at least one hidden layer and an output layer of the neural network;
obtaining a first optimization weight parameter of a neuron of the neural network output layer by an error back propagation method or a time sequence dependent plasticity method according to the data output by the neural network output layer;
sequentially generating biological self-organizing synaptic plasticity from the hidden layer to the input layer through neurons, and performing back propagation to obtain second optimized weight parameters of the hidden layer and the input layer of the neural network;
performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter;
and performing the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
2. The method for training the neural network based on the back propagation of the biological self-organization as claimed in claim 1, wherein when the training method is applied to the impulse neural network, the obtaining of the training samples further comprises: carrying out Spike quantization coding on the training samples to generate time sequence input information; the quantization coding formula is:
wherein, Ispikes(t) is quantization coding, Iraw(t) is the training sample, IrdAnd (t) is a random number between 0 and 1 which is randomly generated.
3. The method for training the neural network based on the back propagation of the biological self-organization as claimed in claim 1, wherein when the training method is applied to the artificial neural network, the obtaining of the training samples further comprises: and carrying out normalization processing on the training samples to obtain input information.
4. The method according to claim 2, wherein when the training method is applied to a pulse neural network, the training samples are input to the neural network, and the training samples are output after being subjected to data processing by an input layer, at least one hidden layer and an output layer of the neural network, and specifically comprises:
Short-Term synaptic Plasticity (Short-Term Plasticity) occurs at the synapses of neurons in the input and hidden layers of the neural network, as follows:
wherein u and x are the parameters regulating the short-term synaptic potentiation or inhibition, respectively, and represent the amount of neurotransmitter used in the synapse, taufAnd τdThe integral recovery quantities of U and x are respectively, U is a hyper-parameter and is the increment of a regulating parameter U generated by Spike quantization coding, A is the synaptic weight W between the hidden layer neuron sequence number i and the input layer neuron sequence number ji,jUpper learnable amount, τsAs synaptic currents IsynThe integral recovery amount of (1);
the neuronal cell bodies were subjected to leaky-integrate-and-discharge treatment as follows:
wherein, taumIs the integral recovery of V (t), where V (t) is the neuronal membrane potential, VLIs the membrane potential leakage current, gLIs the conductance of the leakage current of the membrane potential, gE|IConductance being of excitatory or inhibitory neurons, VE|IMembrane potential reset voltage, V, for excitatory or inhibitory activityTrTo discharge threshold, VresetIs the resting membrane potential.
5. The method according to claim 3, wherein when applied to an artificial neural network, the training samples are input to the neural network, and the training samples are output after being subjected to data processing by an input layer, at least one hidden layer and an output layer of the neural network, and specifically comprises: after information passes through the input layer, the information is transmitted to the neurons of all the hidden layers through full-connection weight, and after the information is integrated, the information is transmitted to the output layer through the nonlinear neurons.
6. The method for training the neural network based on the bio-self-organized back propagation as claimed in claim 4, wherein when the training method is applied to the impulse neural network, the sequential occurrence of the bio-self-organized synaptic plasticity from the hidden layer to the input layer through the neurons and the back propagation are performed, so as to obtain the second optimized weight parameters of the hidden layer and the input layer of the neural network, specifically comprising:
the information transferred between neurons keeps the input-output information stable, making the neuron membrane potential plastic, as follows:
wherein the content of the first and second substances,
is the amount of change in membrane potential between neuron input and output, Vj(t) is the membrane potential at the current instant,
for the membrane potential at a future moment, ηeIs the learning rate;
fusing the membrane potential plasticity and the membrane potential in the forward propagation process of the information in the neural network to obtain a comprehensive membrane potential:
wherein, is Δ Vj F(t) Membrane potential Change, Δ V, during Forward propagation of informationj E(t) Membrane potential equilibrium change, Δ V, after synaptic plasticity of neuronsj(t) is Δ Vj F(t) and Δ Vj E(T) weighted summation, TeTotal learning period, teThe current time instant, as the simulation time increases,
gradually approaches 1, so eventually only Δ V will remainj F(t)；
And (3) performing plasticity learning on part of synapses in neuron output synapses by adopting a pulse time sequence dependent plasticity rule, and learning neuron weights between a hidden layer and an output layer, wherein the following steps are as follows:
wherein A is+And A_To scale the parameters, tj,sAnd tk,sTo the discharge time, tau+And τ-Is an attenuation parameter;
and
postsynaptic adjustment of positive and negative values, respectively
j is the hidden layer neuron serial number, k is the output layer neuron serial number;
plasticity change
Plasticity propagated backwards to presynaptic by corresponding nonlinear transformations
In (1), as follows:
wherein the content of the first and second substances,
respectively the presynaptic regulating quantity of the second optimized weight parameter
I is a matrix with diagonal 1, σn(. is) a non-linear transformation function for normalization, λpIs a range factor, λfIs an amplitude factor, y ═ Ediag,i(x) Representing the information transformation, transforming a vector into a diagonal matrix, yj,j＝xjI is the input layer neuron sequence number and j is the hidden layer neuron sequence number.
7. The method for training the neural network based on the bio-self-organized back propagation as claimed in claim 5, wherein when the training method is applied to the artificial neural network, the bio-self-organized synaptic plasticity sequentially occurs from the hidden layer to the input layer through the neurons, and the back propagation is performed, so as to obtain the second optimized weight parameters of the hidden layer and the input layer of the neural network, specifically comprising:
construction of an energy function ERBMConstraining plasticity between neurons of adjacent layers, and setting a hyper-parameter thetasbpRepresents a range of constraints, as follows:
energy function ERBMThe Loss function of the error back propagation method is combined into a unified constraint function, which is as follows:
Closs＝βCRBM+ERBM
wherein, CRBMThe method is a loss function of an artificial neural network, and is a restricted Boltzmann machine;
neuronal plasticity between adjacent layers is modulated as follows:
wherein eta isbpFor learning rate based on error back propagation method, etaSBPFor the learning rate based on the bio-self-organized back propagation method,
i is a matrix with diagonal 1, λfIs an amplitude factor, λpIs a range factor.
8. A neural network training device based on biological self-organizing back propagation, which is applied to an impulse neural network or an artificial neural network, and comprises:
the sample acquisition unit is used for acquiring a training sample;
the forward propagation unit is used for inputting the training sample into a neural network, and the training sample is output after being subjected to data processing through an input layer, at least one hidden layer and an output layer of the neural network;
the output layer back propagation unit is used for obtaining a first optimization weight parameter of the neural network output layer neuron through an error back propagation method or a time sequence dependence plasticity method according to the data output by the neural network output layer;
the self-organization reverse propagation unit is used for sequentially generating biological self-organization synapse plasticity from the hidden layer to the input layer through neurons and performing reverse propagation so as to obtain second optimized weight parameters of the hidden layer and the input layer of the neural network;
the network single optimization unit is used for performing single optimization on the neural network according to the first optimization weight parameter and the second optimization weight parameter;
and the network cycle optimization unit is used for carrying out the single optimization process on the neural network for multiple times until the loss function of the neural network converges to a preset value.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method for training a neural network based on bio-ad hoc back propagation according to any one of claims 1 to 7.
10. A non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the steps of the method for training a neural network based on back propagation of biological self-organization according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111095440.6A CN113837380A (en) | 2021-09-17 | 2021-09-17 | Neural network training method and device based on biological self-organization back propagation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111095440.6A CN113837380A (en) | 2021-09-17 | 2021-09-17 | Neural network training method and device based on biological self-organization back propagation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113837380A true CN113837380A (en) | 2021-12-24 |
Family
ID=78959947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111095440.6A Pending CN113837380A (en) | 2021-09-17 | 2021-09-17 | Neural network training method and device based on biological self-organization back propagation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113837380A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114998659A (en) * | 2022-06-17 | 2022-09-02 | 北京大学 | Image data classification method for training impulse neural network model on line along with time |
-
2021
- 2021-09-17 CN CN202111095440.6A patent/CN113837380A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114998659A (en) * | 2022-06-17 | 2022-09-02 | 北京大学 | Image data classification method for training impulse neural network model on line along with time |
| CN114998659B (en) * | 2022-06-17 | 2024-05-24 | 北京大学 | Image data classification method for training impulse neural network model on line along with time |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107092959B (en) | Pulse neural network model construction method based on STDP unsupervised learning algorithm | |
| Saadat et al. | Training echo state neural network using harmony search algorithm | |
| Alaloul et al. | Data processing using artificial neural networks | |
| Jia et al. | A novel optimized GA–Elman neural network algorithm | |
| KR20170031695A (en) | Decomposing convolution operation in neural networks | |
| KR101700145B1 (en) | Automated method for modifying neural dynamics | |
| TW201531966A (en) | Configuring sparse neuronal networks | |
| Jadav et al. | Optimizing weights of artificial neural networks using genetic algorithms | |
| Qi et al. | Jointly learning network connections and link weights in spiking neural networks. | |
| CN111401547B (en) | HTM design method based on circulation learning unit for passenger flow analysis | |
| Chandra et al. | Encoding subcomponents in cooperative co-evolutionary recurrent neural networks | |
| Shi et al. | A curiosity-based learning method for spiking neural networks | |
| CN111382840B (en) | HTM design method based on cyclic learning unit and oriented to natural language processing | |
| Valdez et al. | Parallel genetic algorithms for optimization of Modular Neural Networks in pattern recognition | |
| Chen et al. | A Survey of Learning Spiking Neural P Systems and A Novel Instance. | |
| Wang et al. | Online versus offline learning for spiking neural networks: A review and new strategies | |
| Wang et al. | Optimizing deep belief echo state network with a sensitivity analysis input scaling auto-encoder algorithm | |
| CN113837380A (en) | Neural network training method and device based on biological self-organization back propagation | |
| De Campos et al. | Automatic design of neural networks with l-systems and genetic algorithms-a biologically inspired methodology | |
| Wang et al. | A self-organizing deep belief network based on information relevance strategy | |
| CN116080688B (en) | Brain-inspiring-like intelligent driving vision assisting method, device and storage medium | |
| Belatreche et al. | An evolutionary strategy for supervised training of biologically plausible neural networks | |
| Belatreche et al. | Evolutionary design of spiking neural networks | |
| Zhao et al. | A genetic algorithm for reinforcement learning | |
| Vaila et al. | Spiking CNNs with PYNN and NEURON |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |