CN115222026A

CN115222026A - Hardware circuit of impulse neural network

Info

Publication number: CN115222026A
Application number: CN202110421047.5A
Authority: CN
Inventors: 梁翔
Original assignee: Individual
Current assignee: Individual
Priority date: 2021-04-19
Filing date: 2021-04-19
Publication date: 2022-10-21

Abstract

A hardware circuit of a pulse neural network comprises an electronic synapse module, a front neuron module, a rear neuron module, a power supply module and a microprocessor. Each pre-neuron in the pre-neuron module sequentially generates pre-pulses entering the electronic synapse module in accordance with a one-dimensional time sequence input by the microprocessor. The front pulse entering the rear neuron module through the electronic synapse module is integrated by the rear neuron module and continuously promotes the reduction of the resistance value of the low-doped memristor until the rear neuron generates a spike and a rear pulse. The back pulse can be quickly propagated to the electronic synapse module to be integrated with the front pulse, and the resistance value of the specific electronic synapse is adjusted (namely, the synapse weight value is modified) based on the synapse plasticity learning rule which is time-dependent; the microprocessor records the time of the earliest spike and the bit sequence of the post-neuron and marks the learning information accordingly. After learning all the information, the system can classify different information according to the bit sequence and time in the training result.

Description

Hardware circuit of impulse neural network

Technical Field

The invention belongs to the field of SNN (Spike Neural Network for short) hardware, and particularly relates to an electronic synapse module based on a memristor, a pre-neuron module capable of generating a pre-pulse, a post-neuron module with a cell membrane mode voltage integration function and a post-pulse generation function, a power supply allocation module and a microprocessor.

Background

The memristor is a fourth basic electronic component except for a resistor, an inductor and a capacitor, which is predicted from a theoretical perspective in 1971 by professor of begonia of university of California in Berkeley and division and is confirmed to exist by researchers of Hewlett packard company in 2008. The characteristics of the memristor are mainly represented in that the resistance state of the memristor can have a memory function on charges flowing through, and the memristor can be used for preparing electronic synapses and further used for building a simulation computer and an artificial neural network of a human brain-like information processing mode.

The resistance change mechanism of the memristor is not unified at present, but the Conductive filament theory (Conductive film) based on Oxygen vacancy (Ov for short) or metal ion migration such as copper (Cu), silver (Ag) and the like is the most widely accepted and practical mechanism at present. The metal electrode loses electrons under the action of voltage and is oxidized into metal ions, the metal ions enter the material under the action of an electric field to form conductive filaments, so that the resistance of the material is reduced, and conversely, the metal ions reversely migrate under the action of a negative electric field to promote the rupture of the conductive filaments so as to return the material from a Low Resistance State (LRS) to a High Resistance State (HRS). By adjusting the doping ratio in the material and the manner of applied voltage, the resistance state of the memristor may vary non-linearly between HRS and LRS, and may exhibit synaptic plasticity such as time-dependent and frequency-dependent, which indicates that the memristor has great potential for making electronic synapses for synaptic functions in biological-like nervous systems.

The data Retention characteristic (Retention characteristic) is one of important performances of the memristor, and is used for representing the data Retention capability of the memristor, and the performance is related to the ion (or defect) doping concentration when the memristor is prepared; the appropriate doping concentration allows the memristor to exhibit a higher retentivity characteristic, while the memristor with a lower doping concentration can rapidly return to the HRS resistance after being stimulated by the same electric pulse, thereby exhibiting a poorer retentivity characteristic.

The impulse neural network is the next generation neural network. Compared with the most common neural network at present, the SNN is characterized by being closer to a biological system and is considered to be a neural network closer to the information processing mode of a biological brain.

The large-scale neural network hardware circuit built based on the resistors, the capacitors and the inductors has the following problems: the number of elements required for a single synapse and neuron is large; a large number of components increase power consumption and reduce integration; building an electronic synapse with memristors may reduce this limitation, reducing power consumption to a greater extent.

There is a spike-Timing Dependent synaptic Plasticity (STDP) learning rule in the cranial nervous system, abbreviated as STDP learning rule: synaptic weight is increased (decreased) when the pre-synaptic pulse stimulus reaches the synapse earlier (later) than the post-synaptic pulse stimulus.

The first pulse triggering time coding (TTFS coding) is an information coding mode in SNN; the coding mode expresses information by the latency of the first pulse of the neuron, namely the time of the first pulse release of the neuron after the stimulation starts; TTFS coding is more suitable for coding edge features where the stimulus is well defined (the main difference is significant).

The LeakIntegrate-and-Fire (LeakIntegrate-and-Fire) neuron model has long been widely used in the field of neurocomputing. LIF neurons are characterized by: when the external stimulation is applied, the membrane voltage of the cell membrane rises, and when the stimulation is removed, the membrane voltage quickly drops to the original level; only when intensive electrical stimulation stimulates LIF neurons, the membrane pressure will rise continuously until the threshold voltage is reached, spikes (Spike) are issued, and the refractory period is rapidly entered.

Disclosure of Invention

The main purposes of the invention are as follows: a pulsed neural network hardware circuit of electronic synapses prepared based on memristors and a learning mechanism thereof are provided.

The technical scheme of the invention is as follows: the information to be learned (e.g. pictures) is encoded into a one-dimensional time series by the TTFS and passed to the microprocessor; under the control of the microprocessor, each pre-neuron in the pre-neuron module generates a pre-pulse entering the electronic synapse module by taking time as an axis according to an afferent one-dimensional time sequence; the pre-pulse enters the post-neuron module after passing through the electronic synapse module; the front pulse passing through the electronic synapse module is integrated into an electrical stimulation sequence by the back neuron module and continuously promotes the reduction of the resistance value of the low-doped memristor until the back neuron generates a Spike (Spike) and a back pulse. The back pulse quickly propagates to the electronic synapse module and adjusts the resistance value of a specific memristor in the electronic synapse module (namely, the operation of modifying the synaptic weight) based on the STDP learning rule with the front pulse; the microprocessor records the generation time of the earliest Spike and the bit sequence of the post-neuron, and marks the learning information according to the training result (sequence and time), thus completing one learning training. After learning all the information, the system can classify different information according to the bit sequence and time in the training result.

As shown in fig. 1, the present invention mainly includes a microprocessor, a pre-neuron module, an electronic synapse module, a post-neuron module, and a power distribution module.

The microprocessor inputs a square wave signal with the pulse width d into the integrator of the corresponding anterior neuron according to a one-dimensional time sequence, and the square wave can synchronously enter a control port of the single-pole double-throw analog switch so as to control the switching of the ground channel of the single-pole double-throw analog switch; an integrator consisting of a resistor and a capacitor inputs a forward quasi-triangular wave (as shown in figure 2) with the pulse width slightly larger than 2 times d at a common port of the single-pole double-throw analog switch, wherein the pulse width of a rising edge (generated by charging the capacitor) of the quasi-triangular wave is strictly equal to d, and the pulse width of a falling edge (generated by discharging the capacitor) on the right side is slightly larger than d; when the input signal of the microprocessing is the High level of the square wave (within the range of the pulse width d), the two single-pole double-throw analog switches can switch the line into the High path (High) state, at the moment, the reverse amplifier enables the rising edge of the similar triangular wave to be input into the voltage follower after the phase inversion, when the input signal of the microprocessing is the Low level of the square wave (within the range of the pulse width d-2 d), the two single-pole double-throw analog switches can switch the line into the Low path (Low) state, and the falling edge of the similar triangular wave can 'skip' the reverse amplifier and directly enter the voltage follower; the output signal of the voltage follower is the output signal of the pre-neuron module.

As shown in fig. 3, the control voltage converting circuit is composed of two Nmos transistors and 2 pull-up resistors. When the square wave signal (such as the high level of the STM32 is 3.3V) sent by the microprocessor cannot directly drive the single-pole double-throw analog switch 2 (5V is needed) or the single-pole four-throw analog switch 8 (5V is needed), the circuit can synchronously replace the square wave signal sent by the microprocessor with the square wave signal (5V) which can drive the single-pole double-throw analog switch or the single-pole four-throw analog switch. If the square wave signal sent by the microprocessor can directly drive the analog switch, the part of the circuit can be omitted.

As shown in fig. 4, the memristor adopted by the invention has a sandwich structure of a copper electrode/manganese ion doped tin dioxide polycrystalline film/FTO conductive glass substrate; wherein the array copper electrode as the top electrode resembles a presynaptic membrane of biological synapses, the manganese ion doped tin dioxide polycrystalline thin film resembles a synaptic cleft of biological synapses, and the fluorine doped tin dioxide conductive thin film glass substrate (FTO conductive glass) as the bottom electrode resembles a postsynaptic membrane of biological synapses; the front pulse sent by each front neuron independently flows through the tin dioxide polycrystalline film due to the one-to-one correspondence of the copper electrode and the front neuron, and enters the rear neuron module after being converged at the FTO conductive film.

As shown in fig. 5, the post-neuron circuit module mainly includes a single-pole four-throw analog switch (e.g., CD4052 BE), an inverse adder, a cell membrane voltage integrating circuit, a voltage comparator, a winner take all' competition circuit (WTA), and a post-pulse generator; the single-pole four-throw analog switch is always in a 0-number passage state under the control of the microprocessor (the microprocessor can refer to the circuit shown in the figure 3 for the control circuit of the single-pole four-throw analog switch); each pre-pulse flowing through the electronic synapse module is input into a cell membrane pressure integrating circuit after being logically integrated through the addition of an inverse adder formed by an operational amplifier and a resistor; the cell membrane pressure integration circuit simulates a leakage integration-and-fire (LIF) neuron in a pulse neural network; the cell membrane voltage of the neuron continuously rises under the stimulation of a pre-pulse, and a membrane voltage integrating circuit in the invention is a series voltage dividing circuit which is composed of a stannic oxide polycrystalline thin-film memristor (low-doped memristor for short) with manganese ion doping concentration lower than that of an electronic synapse module and a constant-value resistor with resistance value smaller than high-resistance-state resistance of the low-doped memristor; under the stimulation of an electric pulse, the phenomenon that the resistance of the low-doped memristor quickly or slowly returns to HRS after the stimulation action is finished occurs due to poor Retention characteristics (referred to as retentivity characteristics for short), and then the voltage on the constant-value resistance also changes along with the change of the resistance value of the low-doped memristor, and the phenomenon is shown that the lower (higher) the resistance value of the doped memristor is, the larger (smaller) the voltage on the constant-value resistance is, and the leakage integral phenomenon is almost consistent with the leakage integral phenomenon of the cell membrane voltage of the LIF neuron; the voltage on the fixed resistor is compared with the reference voltage in the voltage comparator in real time, when the voltage of the fixed resistor is higher (lower) than the reference voltage, the voltage comparator outputs 5V (0V) under the condition of 5V power supply, and the signal output by the voltage comparator is detected by an external fracture of the microprocessor through a Pmos tube (normally closed analog switch) corresponding to a winner 'eating all' competition circuit; the comparator generates a high-level (5V) signal, and the rear neuron corresponding to the comparator generates Spike response; the competition circuit for winning people to eat is mainly composed of a plurality of normally closed Pmos tubes, each Pmos tube is connected with the output end of a comparator of a posterior neuron, and the grid electrode of the Pmos tube is connected with the source electrodes of the Pmos tubes of other lines; the response speed of the mos tube is usually several nanoseconds, which is much faster than the response speed of the microprocessor, and the connection of the competition circuit is eaten by the winner, so that an external interrupt port of the microprocessor can definitely receive the first generated Spike, the bit sequence of the later neuron is recorded, and the time of the Spike generation is further recorded according to a timer program inside the microprocessor; after detecting the bit sequence and time information of the first Spike, the microprocessor controls a rear pulse generator in a rear neuron of the bit sequence to generate rear pulse, and simultaneously switches a single-pole four-throw analog switch of the rear neuron which generates the first Spike to a 1-channel state, while the neuron which does not generate the first Spike or does not generate the first Spike is switched to a 2-channel state (open circuit); the single-pole four-throw analog switch has four paths (0-channel, 1-channel, 2-channel, 3-channel), and 3 channels not shown in fig. 4 can be used for weight monitoring of the electronic synapses in other scientific researches, but since the weights of the electronic synapses are not known in the actual operation of the spiking neural network (each weight of the electronic synapse may affect the integration effect of the adder on the pre-pulse, and further affect the results of the bit sequence and time), 3 channels are shown in fig. 4.

FIG. 6: schematic diagram of power supply module. The module mainly provides positive voltage, negative voltage, reference voltage and the like for other circuits.

Drawings

Fig. 1 is a basic framework schematic diagram of a hardware circuit of a spiking neural network.

FIG. 2 is a schematic diagram of the structure and function of an anterior neuron module:

1-an integrator; 2-single pole double throw analog switch (e.g., CD4053 BE); a 3-inverting amplifier; 4-a voltage follower; the schematic diagram of the waveform change in the dashed line frame is the schematic diagram of the processing and output of the input signal by the anterior neuron module.

Fig. 3 is a control voltage conversion circuit.

FIG. 4 is a schematic diagram of an electronic synapse module structure:

5-array top electrode (top electrode for short); 6-manganese ion doped tin dioxide polycrystalline film; 7-array bottom electrode (bottom electrode for short, namely FTO conductive glass plated with tin dioxide film doped with fluorine ions).

FIG. 5 is a schematic diagram of a posterior neuron module:

8-single pole four throw analog switches (e.g., CD4052 BE); 9-inverse adder; 10-feedback resistance. 11-cell membrane voltage integrating circuit (membrane voltage integrating circuit for short); 12-a voltage comparator; 13- "winner takes all" competition circuit (WTA); 14-post pulse generator.

Fig. 6 is a schematic diagram of a power supply module.

FIG. 7 is a schematic diagram of the synthesis of the front pulse and the back pulse based on the STDP learning rule:

the maximum amplitude of the back pulse is larger (even much larger) than that of the front pulse, but the pulse width is smaller than half of that of the front pulse; after the front pulse and the rear pulse are integrated, a region which exceeds the memristor point forming voltage and can adjust the resistance value of the memristor exists; when in use

When positive (i.e. the back pulse arrives at the electronic synapse later than the front pulse), this region follows two pulse widths

Become larger and smaller, even if

The larger the resultant pulse, the weaker the ability of the resultant pulse to positively modulate an electronic synapse; when in use

When the value is negative (i.e. the back pulse reaches the electronic synapse before the front pulse), the region follows two pulse widths

Become larger and larger, even if

The larger the resultant pulse, the stronger the negative regulation of the electronic synapse.

Fig. 8 is a schematic diagram of the Spike excitation (Spike excitation) principle of the posterior neuron module:

FIG. 8 replaces the actual pulse with a square wave with positive and negative spikes; a series of discrete pre-pulses are generated by the pre-pulse, and after the series of discrete pre-pulses are processed by the post-neuron adder, the discrete pre-pulses are integrated into a pulse sequence, and the voltage amplitude is related to the resistance value of an electronic synapse through which a single pre-pulse flows; the pulses which are very close to each other and have larger voltage amplitude form an area with the maximum energy density, and the time range of the area is synchronous with the pulse dense area in the front pulse; the integrated pulse stimulates a low-doped memristor in a cell membrane voltage integration circuit, and the voltage of a constant-value resistor in the circuit also changes according to the characteristics of the LIF neuron, because the stimulation energy given to the low-doped memristor by the region with the maximum energy density is the largest, so that the Spike excitation is most easily generated in the region.

FIG. 9 is a schematic diagram of a mechanism for a hardware circuit of a spiking neural network to perform pattern classification:

each small open circle represents an electronic synapse, the resistance of each electronic synapseValue is

。

Fig. 10 is data Retention characteristic (retentivity characteristic) data for a low-doped memristor.

FIG. 11 is a dynamic graph of the convergence of training for pattern recognition of the present invention:

the Y mode and the Z mode are encoded into one-dimensional time sequence by adopting TTFS (similar to the flattening processing in CNN); by adopting the invention to learn the two modes, the training convergence is found and the training results can be clearly distinguished.

Detailed Description

A spiking neural network hardware circuit comprising: the device comprises a front neuron module, a rear neuron module, an electronic synapse module, a microprocessor and a power supply distribution module.

The specific implementation of the hardware circuit is as follows:

the function of the pre-neuron circuit is to generate pre-pulses according to the time sequence of the input. The specific operation of the anterior neuron circuit is: when a front pulse needs to be generated, the microprocessor inputs a square wave at the input port of the front neuron module, and the square wave passes through the integrator 1 to generate a triangular wave-like waveform with the positive direction at the front end of the capacitor. The pulse width of the rising edge of the triangle-like wave is equal to that of the square wave, and the pulse width of the falling edge of the triangle-like wave is slightly larger than that of the square wave, so that the pulse width of the triangle-like wave can be considered to be 2 times of that of the input square wave. In addition to entering the integrator 1, the square wave emitted by the microprocessor also controls the channel switching of the single pole double throw analog switch 2. When the amplitude of the square wave is at a High level (3.3V), the single-pole double-throw analog switch 2 is switched to a High channel (High channel), so that the rising edge of the quasi-triangular wave enters the voltage follower 4 after passing through the reverse amplifier 3; when the amplitude of the square wave is at a Low level (0V), the single-pole double-throw analog switch 2 is switched to a Low channel (Low channel), so that the falling edge of the similar triangular wave directly enters the voltage follower 4 without passing through the reverser 3; therefore, as shown in the dashed box of fig. 2, a square wave with a pulse width d entering the pre-neuron module will generate a pulse with a pulse width 2 times d at the output of the pre-neuron circuit module, which is a pre-pulse generated by a single pre-neuron in the present invention, and has a positive spike and a negative spike.

The invention can expand the number of the front neuron units according to the actual requirement to build larger-scale neural network hardware; meanwhile, the parameters of resistance or capacitance in the integrator 1, the single-pole double-throw analog switch 2 and the inverting amplifier 3 can be adjusted, so that different pre-pulses are generated to adapt to memristors with different characteristics, namely, the parameters of electric pulses are properly adjusted according to the characteristics of the memristors, and the purpose of improving the circuit applicability is achieved.

The control voltage conversion circuit shown in fig. 3 is composed of two Nmos transistors and 2 pull-up resistors; when the square wave signal (for example, the high level of the STM32 is 3.3V) sent by the microprocessor cannot directly drive the single-pole double-throw analog switch 2 (5V is needed) or the single-pole four-throw analog switch 8 (5V is needed), the circuit can synchronously replace the square wave signal sent by the microprocessor with the square wave signal (5V) which can drive the single-pole double-throw analog switch 2 or the single-pole four-throw analog switch 8 to generate the square wave signal; if the square wave signal sent by the microprocessor can directly drive the analog switch, the part of the circuit can be omitted.

The electronic synapse module has the main function of transmitting a prepulse to the back neuron module and storing the information learned by the system; the electronic synapse in the invention may be composed of a manganese ion doped tin oxide thin film memristor array; the basic frame of a single memristor is a top electrode (Cu)/manganese ion doped tin oxide thin film/bottom electrode (FTO conductive glass); an array of memristors may constitute a synaptic connection as shown in FIG. 1. Each array top electrode 5 is connected with an output port of a fixed front neuron module, and each array top electrode 7 is connected with an input port of a back neuron module; all the front pulses pass through the manganese ion-doped tin dioxide polycrystalline film 6 and then enter the rear neuron module to be further integrated.

The main functions of the posterior neuron module are to integrate the anterior impulses sent by each anterior neuron, generate Spike signals (Spike) and release the posterior impulses; the back neuron module mainly comprises a single-pole four-throw analog switch 8, a reverse amplifier 9, a cell membrane voltage integrating circuit 11 (membrane voltage integrating circuit for short), a voltage comparator 12, a winner eating competition circuit (WTA) 13 and a back pulse generator 14; the single-pole four-throw analog switch 8 is normally in the channel No. 0; after passing through the electronic synapse module and the channel 0 of the single-pole four-throw analog switch 8, the front pulses are integrated by the inverse adder 9, and the integrated pulse sequence acts on the cell membrane voltage integrating circuit 11; the cell membrane voltage integrating circuit 11 is essentially a voltage dividing circuit composed of a manganese ion low-doped memristor (low-doped memristor for short), a diode and a constant-value resistor; the integrated pulse with positive and negative spikes from the reverse adder 9 will be filtered by the diode to remove the negative spikes and only retain the positive spikes. According to the characteristics of the low-doped memristor, the integrated pulse of the forward spike is retained, so that the resistance of the low-doped memristor can be continuously adjusted towards the LRS direction but cannot be completely adjusted to the LRS (because the maximum amplitude of the pre-pulse is usually not greater than one half of the electrical Forming voltage (Forming voltage) of the memristor, and in actual operation, the maximum amplitude of the pre-pulse is usually far less than the value of the Forming voltage so as to avoid that the integrated pulse with too high amplitude breaks through the low-doped memristor due to too many pre-pulses sent at the same time); the gradual decrease in the resistance value of the low-doped memristor results in an increasing voltage across the fixed-value resistor. When the voltage across the voltage dividing resistor is greater than the reference voltage of the voltage comparator 12 (i.e., the Spike-excited threshold voltage), the voltage comparator 12 outputs a high level (typically 5V); the rear neuron module is provided with a plurality of rear neuron units, the rear neuron which generates Spike at the earliest controls the Pmos tube in other rear neurons which do not generate Spike at the earliest in a winner eating competition circuit 13 (WTA) to be cut off, and then the winner eating competition circuit realizes that only the generated Spike at the earliest is detected by an interrupt port of the microprocessor in one training; meanwhile, the rear pulse generator 14 of the rear neuron which generates Spike at the earliest generates rear pulse under the control of the microprocessor; the posterior pulse generator 14 is substantially identical to the structure in the anterior neuron module, except that: the output of the post-pulse generator 14 is output by a voltage inverting amplifier and a voltage forward amplifier in a matching way, but not output by a voltage follower in a unified way; this is because the generation of the back pulse means that the weight of the electronic synapse changes, the amplitude value of the front pulse is small, and the back pulse with a larger amplitude can be matched with the front pulse to ensure that the electronic synapse weight is adjusted based on the STDP learning rule, and the adjustment of each resistance value in the pulse generator 14 can satisfy the requirement, and further improve the adaptability of the present invention to memristors with different characteristics.

When the earliest generated Spike is detected, the microprocessor immediately records the bit sequence and time of the rear neuron generated by the Spike, and simultaneously immediately controls the single-pole four-throw analog switch 8 to be switched to the No. 1 line, so that the output end of the rear pulse generator 14 of the rear neuron generating the Spike earliest is communicated with the array bottom electrode 7 of the electronic synapse module, and simultaneously, the single-pole four-throw analog switches 8 of other rear neurons not generating the Spike earliest are switched to the No. 2 line, and further the input ends of other rear neurons not generating the Spike earliest are disconnected with the bottom electrode 7 of the electronic synapse module. This is done to ensure that only one post-neuron (the output of the post-pulse generator 14) is connected to the electronic synapse module in the event of a post-pulse generation, while the other post-neurons are disconnected from the electronic synapse module; the generated back pulse will quickly propagate into the electronic synapse module to adjust the resistance value of a specific memristor in the electronic synapse module in accordance with the STDP rule with the front pulse.

As shown in fig. 5, the power distribution module mainly includes a single power supply to dual power supply module (generally, DC-DC module) and a specific voltage output module; the single power supply-to-double power supply module provides positive and negative power supplies for each operational amplifier device needing double power supply in the whole circuit, and ensures that pulses with positive and negative information in the circuit can pass through each analog switch without distortion; the specific voltage output module mainly comprises two resistor series voltage-dividing circuits and a voltage follower, and is mainly used for providing reference voltage (0V-5V) for a comparator in the posterior neural module; when the reference voltage is a specific voltage (e.g., 5V, 3.3V, 1.5V), the specific voltage output module may be replaced with some voltage regulator devices (e.g., AMS1117 may generate a stable voltage of 5V, 3.3V, 1.5V, etc. when 5V is input).

The learning mechanism of the circuit of the present invention will now be explained.

The learning mechanism of a single learning object is as follows:

the external equipment encodes the information to be learned into a one-dimensional time sequence in a TTFS coding mode and inputs the sequence into the microprocessor. For example, as shown in fig. 8, if the information to be learned is a gray-scale image, the gray-scale value of each pixel of the image may be encoded by using a TTFS coding method as follows:

\8230; (formula 1)

Wherein,

is an anterior neuron

The moment at which the pulse is delivered,

is the maximum issue time in the TTFS,

is a gray-scale value that is,

is the maximum gray value 255. And coding the gray value of a gray value picture into a one-dimensional time series by adopting a TTFS coding mode according to the mode that each pixel point corresponds to one front neuron.

The microprocessor inputs the sequence to the corresponding anterior neuron; each anterior neuron according to its own

The front pulse electrical stimulation is delivered, and the inverse adder 9 in the back neuron integrates the discrete front pulse electrical stimulation into the same timeA pulse stimulation sequence on the secondary axis; the integrated stimulation pulse acts on the low-doped memristor of the cell membrane voltage integrating circuit 11; the formula for the integrated pulse is as follows:

\8230; (formula 2)

Wherein,

is an anterior neuron

The pulse stimulation is sent out, and the pulse stimulation,

is the resistance value of the feedback resistor 10 of the inverse adder 9 (the resistance values of the feedback resistors of all the rear neurons in the present invention are the same),

is an anterior neuron

And posterior neuron

The resistance value of the electronic synapse connected between,

is the first

Integrated impulses of individual posterior neurons.

Because the TTFS coding mode is adopted, the voltage amplitude of the pre-pulse voltage

Is related to the time of release of the pulse by the pre-neuron, thus

The formula of (1) is as follows:

8230; formula 3

Wherein,

is the actual time that the circuitry is operating,

is an anterior neuron

The time instants at which the pulses are delivered according to a one-dimensional time sequence,

is the electrical pulse from the pulse generator, in the present invention, although the pre-pulse and post-pulse are non-square wave pulses with positive and negative spikes, the pulse generator parameters of all pre-neuron modules are identical, so here will be the pulse generator parameters

It is understood that the pulses fired by the individual anterior neurons are identical in pulse shape, except that the pulse firing times are different.

As can be seen from equations 1 and 2: to pair

Resistance values affecting a larger electronic synapse

(ii) a Because memristors used as electronic synapses typically have a good on-off ratio (HRS/LRS, typically about 100-1000), while the resistance values of the memristors are in the denominator position in equation 2, equation 2 may be further simplified to equation 4 using a scaled mathematical approach:

\8230; type 4)

Wherein,

is the bit sequence of the posterior neuron that first generated Spike (i.e.

），

Is to

The resistance value of the electronic synapse that most affects.

The Spike excitation principle is shown in fig. 8, combined with the specific circuit of the back neuron module (i.e. shown in fig. 5),

will also act on the cell membrane voltage integrating circuit 11; the cell membrane voltage integrating circuit 11 is actually composed of a low-doped memristor and a constant-value resistor, and simultaneously

Each electric pulse is a positive and negative peak pair, and the diode only filters out a negative voltage peak and keeps a forward voltage peak, so that the diode has no influence on the voltage dividing effect of the low-doped memristor and the constant-value resistor, and the voltage on the constant-value voltage dividing resistor

(i.e., the voltage comparator 12 needs to compare the voltage in real time to determine whether the post neuron is activated) as shown in equation 5:

\8230; type 5)

Wherein,

is the resistance value of the constant-value resistor,

is the resistance value of the low-doped memristor; the poor data retention characteristic of the low-doped memristor enables the resistance value of the low-doped memristor to quickly decay from a low resistance state to a high resistance state after being subjected to the electric stimulation, and further enables the low-doped memristor to be in the high resistance state

Are also constantly changing; under the action of continuous electric stimulation, the region with the maximum energy density can finally let

At the moment when the threshold voltage is exceeded, i.e. the post-neuron is activated and releases Spike immediately, the post-neuron, which produced Spike first, will also produce a post-pulse at the same time as Spike.

As can be seen from FIG. 5, the WTA circuit is present such that for a single learning of a learning object, only Spike from one posterior neuron is detected and a posterior pulse is generated, and the microprocessor generates the posterior neural sequence of Spike at the earliest

And the time of Spike generation

As a set of labeled amounts: (

，

) And marking the single learning object together, namely completing one learning.

After the earliest Spike is generated, the post-pulse is controlled by the microprocessorThe post-pulse generated by the generator 14 will quickly propagate to the electronic synapse module and synthesize a new pulse with the post-and post-pulses to the electronic synapse module according to the STDP learning rule

Is regulated (because

For is to

The influence is large, and the flow is caused to flow through

The front pulse is always in

Energy density maximum region) that allows the features of the learning object to remain in the electronic synapse. This process is illustrated in FIG. 7, where

Is the interval between the pre-pulse and the post-pulse; only when the amplitude of the stimulation exceeds the electrical formation Voltage (Forming Voltage) of the memristor, the resistance of the memristor can be effectively adjusted and kept for a long time, namely, the learning result of the machine on the information can be effectively memorized; as can be seen from FIG. 7, when

The larger the positive value, the smaller the maximum amplitude of the synthesized new pulse, the synthesized pulse pair

The positive regulation ability is also weakened; when the temperature is higher than the set temperature

The smaller the negative value, the smaller the maximum amplitude of the new pulse synthesized, and the pulse pair synthesized

The negative regulatory capacity is also diminished, which follows the STDP learning rule.

The above is the process of completing one training; when the same object is retrained, the result of the previous training will result in

Reduction of (2), new

Then substituted into equation 4 to obtain the new

The maximum voltage value of the maximum energy density region of (2) is increased, which allows

The threshold voltage is reached more quickly and the last neuron that produced Spike first triggers Spike and

until the Spike generation time is reduced

No re-enhancement (i.e., reaching or very close to the LRS); this convergence mechanism is similar to a "trap",

is the driving force for convergence, and allows the training of a single subject (

，

) In

Gradually decrease to a certain fixed value

. The final result of this convergence (

，

) As the final result of the training of a single subject.

Mechanism of pattern classification: as can be seen from equation 4, the effect

The area of maximum energy density is not limited to

And also

；

Is the convergence driving force (let) of single-subject training

The amplitude of (c) is larger and larger as the number of training increases to ensure training convergence),

determining the position of the region with the maximum energy density on the time axis; as can be seen from the TTFS coding scheme and equation 1, the characteristics of the information to be learned (the dense interval of the pulse emission time in the one-dimensional time series) first determine

The position of the medium energy density maximum interval on a time axis; in other words,

is not associated with a posterior neuron

The resistance value of the electronic synapse with the smallest resistance value among all the connected electronic synapses is connected with the post-neuron

The resistance value of the memristor which is connected and passed by the front pulse flow in the region with the maximum energy density and has the minimum resistance value.

A pulse dense region exists in a one-dimensional time sequence of the learning information coded according to TTFS, the dense region has a screening effect on all electronic synapses in an electronic synapse module actually, and a coverage region exists; the final result is that the voltage amplitude of the pre-pulse flowing through the electronic synapse with the smallest resistance is the largest after it is integrated, and further, the probability that the post-neuron connected to the electronic synapse will generate Spike at the earliest is the largest

，

) In

Principle of generation, and can be further deduced in a single training

Will be substantially unchanged once generated; the dense areas and coverage areas in the one-dimensional time series after different learning information is coded are often different, so that the electronic synapses covered by the dense areas and the coverage areas are often different, further leading to specific learning information

Also different, ultimately leading to the result of training: (

，

) Also different, the present invention can perform pattern classification; for example, as shown in fig. 9, the coverage areas of the information 1 and the information 2 with large difference after encoding are different, so that the coverage area of the information 1 is different

Will only exist in

And

medium (assuming the resistance of the third column is minimal in the footprint of information 1), and information 2

Will only exist in

And

in (assume in the coverage area of information 2, the second

The resistance of the column is minimal), apparently two pieces of information

The information can not be overlapped, so that the invention can classify different information.

In the training of mode classification, the resistance parameters of a pulse generator in the pre-neuron module need to be adjusted, so that the maximum voltage amplitude of the pre-pulse is smaller, and the effect of the method is to avoid the damage to the low-doped memristor due to the overlarge integrated pulse. In addition, the resistance parameters of the rear pulse generator 14 in the rear neuron module need to be adjusted to make the maximum voltage amplitude of the rear pulse larger, so as to ensure that the adjustment of the STDP learning rule is effective.

And (3) experimental data verification: FIG. 10 is a test result of poor data retention characteristics of low-doped memristors; the effect of a single stimulus on the enhancement of the synapse weight is delayed to a certain extent, but the enhancement effect disappears quickly; when a group of intensive stimuli is applied to the low-doped memristor, the weight enhancement effect has an accumulation effect, and further the voltage of the constant-value resistor in the 11-cell membrane voltage integration circuit is gradually increased. This is similar to the effect of LIF in neuronal cells.

Fig. 11 is a learning and recognition process of the Y pattern and the Z pattern. The training results of the two groups of pictures can be clearly distinguished, so that the convergence of the network can be proved, and the classification effect of the network on the modes with obvious differences is also proved.

Claims

1. A hardware circuit of a pulse neural network comprises an electronic synapse module, a front neuron module, a rear neuron module, a power supply module and a microprocessor: the pre-neuron module sequentially generates pre-pulses entering the electronic synapse module according to a one-dimensional time sequence input by the microprocessor; the pre-pulse enters the post-neuron module through the electronic synapse module; the discrete front pulses are integrated into an electrical stimulation sequence by the rear neuron module and continuously promote the reduction of the resistance value of the low-doped memristor until the rear neurons activate (generate spikes) and the rear pulses; the back pulse can be quickly propagated to the electronic synapse module to be integrated with the front pulse, and the resistance value of a specific electronic synapse is adjusted (namely the synapse weight value is modified) based on a time-dependent synapse plasticity learning rule; the microprocessor records the earliest time of producing spike and the bit sequence of the neuron, and marks the learned information with the recorded time, and after all the learned information is obtained, the system can classify different information based on the bit sequence and time in the training result.

2. The pre-neuron module of a spiking neural network hardware circuit according to claim 1, wherein: the front neuron module mainly comprises an integrator, an inverting amplifier, a voltage follower and a single-pole double-throw analog switch; the integrator converts a square wave with the pulse width d into a forward similar triangular wave with the pulse width slightly larger than 2 times d; the reverse amplifier is matched with the channel switching of 2 single-pole double-throw analog switches, so that the rising edge of the quasi-triangular wave can be converted into a negative peak, and the falling edge of the quasi-triangular wave is input into the voltage follower unchanged; the output pulse of the voltage follower is the pre-pulse output by the pre-neuron module; a pre-pulse is an electrical stimulation pulse with a positive spike (pulse width d), a negative spike (pulse width slightly greater than d).

3. The post-neuron module of a spiking neural network hardware circuit according to claim 1, wherein: the back neuron module mainly comprises a single-pole four-throw analog switch, a reverse adder, a cell membrane pressure integrating circuit, a voltage comparator, a winner eating competition circuit and a back pulse generator; the inverse adder integrates the discrete pre-pulses into an electrical stimulation pulse sequence; a diode in the cell membrane voltage integrating circuit filters the negative spike of the integrated pulse and only keeps the positive spike; the positive Spike sequence continuously stimulates a low-doped memristor in the cell membrane voltage integration circuit, the low-doped memristor is continuously adjusted to a low resistance state, the voltage on a constant-value resistor in the cell membrane voltage integration circuit is continuously increased until the voltage is greater than the reference voltage of a voltage comparator, and a back neuron generates a Spike (Spike) and a back pulse; the rear neuron module is provided with a plurality of rear neuron units, the rear neuron which generates Spike at the earliest controls the cut-off of Pmos tubes in other rear neurons which do not generate Spike at the earliest in a winner eating-all competition circuit, and then the 'winner eating-all' is realized so as to ensure that only the Spike generated at the earliest in a single training is detected by an interrupt port of the microprocessor; the posterior neuron that produced Spike at the earliest also produced a posterior pulse under the control of the microprocessor; the circuit structure for the generation of the posterior pulse differs from that in the anterior neuron module in that: the inverter and the follower are replaced by an amplifier in the post-pulse generating circuit, and the amplifier is used for generating post-pulse with larger amplitude to ensure that the electronic synapse weight is effectively adjusted.

4. The electronic synapse module of a spiking neural network hardware circuit, as claimed in claim 1, wherein: the memristor array can adopt but is not limited to a copper electrode/a manganese ion doped tin dioxide polycrystalline film/a fluorine residue tin dioxide polycrystalline film conductive glass substrate with a sandwich structure; any memristor array or Crossbar array structure (Crossbar structure) with the resistance value adjustable by voltage can be adopted.

5. The learning mechanism of the hardware circuit of the spiking neural network as claimed in claim 1, wherein: the learning information has a pulse dense region in a one-dimensional time sequence coded in a first pulse triggering time coding mode, and in the process of integrating a pre-pulse by a post-neuron, the smaller the resistance value of an electronic synapse covered by the pulse dense region is, the larger the voltage amplitude value after the pre-pulse is integrated by the electronic synapse is, the maximum probability that the post-neuron connected with the electronic synapse generates Spike at the earliest is also, which is the principle of generating a bit sequence parameter in a training result (bit sequence and time) of single learning information; continuously learning a single piece of learning information, the electronic synapse adjusted in the previous training will reduce the generation time of the next earliest Spike until the time can not be further reduced, i.e. the electronic synapse is already in a low resistance state, which is the convergence mechanism of learning and the principle of generation of time parameters in the training result (bit sequence, time); the pulse dense areas corresponding to the learning objects with large differences are different, which causes different electronic synapses most affecting single training, and further causes different training results (bit sequence and time), so that after all information is learned, the system can classify different information according to the bit sequence and time in the training results.

6. The pre-neuron module of a spiking neural network hardware circuit of claim 2 and the post-neuron module of the spiking neural network hardware circuit of claim 3: through the resistance value parameters of the resistors in the front neuron module and the rear neuron module, pulse stimulation with different amplitudes and slopes can be generated, so that the method is better suitable for any memristor array or horizontal-vertical array structure (Crossbar structure) with the resistance value adjustable by voltage, and the applicability of the method is improved.