CN109359734A - A kind of neural network synaptic structure and its adjusting method based on memristor unit - Google Patents

A kind of neural network synaptic structure and its adjusting method based on memristor unit Download PDF

Info

Publication number
CN109359734A
CN109359734A CN201811245921.9A CN201811245921A CN109359734A CN 109359734 A CN109359734 A CN 109359734A CN 201811245921 A CN201811245921 A CN 201811245921A CN 109359734 A CN109359734 A CN 109359734A
Authority
CN
China
Prior art keywords
memristor
round
unit
cascade
weight
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811245921.9A
Other languages
Chinese (zh)
Other versions
CN109359734B (en
Inventor
帅垚
乔石珺
吴传贵
罗文博
王韬
张万里
彭赟
潘忻强
梁翔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201811245921.9A priority Critical patent/CN109359734B/en
Publication of CN109359734A publication Critical patent/CN109359734A/en
Application granted granted Critical
Publication of CN109359734B publication Critical patent/CN109359734B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/06Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Neurology (AREA)
  • Feedback Control In General (AREA)

Abstract

The present invention relates to computers and electronic information technical field, and in particular to a kind of neural network synaptic structure and its adjusting method based on memristor unit.Multiple memristor cell levels are tied together as an electronic synapse (i.e. cascade memristor unit group) by the present invention, the synaptic weight being mapped to its resistance value using a kind of mapping ruler of weighting in neural network, and using it is complete adjust or adjusted stepwise by the way of come avoid neural network in learning process because memristor can not accurately adjust brought by influence, to make each neural network that can operate normally in the system based on memristor.The present invention solves resistance value to the mapping problems of synaptic weight and the uncertain problem of memristor unit change in resistance.

Description

A kind of neural network synaptic structure and its adjusting method based on memristor unit
Technical field
The present invention relates to computers and electronic information technical field, and in particular to a kind of nerve net based on memristor unit Network synaptic structure and its adjusting method.
Background technique
Cynapse is that the intermediate structure of different neurons is connected in neural network, and weight passes through corresponding neural network algorithm Continuous renewal is the basis of neural network information processing.Memristor as a kind of new component with resistance tunable characteristic, It is considered as 4 basic circuit elements one of arranged side by side with resistance, inductance and capacitor.
Researcher find synaptic connection strengths changing rule and memristor electrology characteristic (its electric conductivity value by apply electricity The control characteristic of signal) there is similar changing rule, therefore single memristor can simulate the function of a cynapse.With use Electronic computer (transistor circuit of i.e. traditional CMOS complementary metal-oxide-semiconductor base) runs neural network procedure phase Than energy consumption can be greatly reduced in memristor unit, and improve read or write speed reduces the complexity of integrated circuit simultaneously.However, memristor Also there is its birth defect, as analog device, its resistance is nonlinear analog quantity variation under the regulation of voltage first, and The weight that expectation reaches cannot be precisely adjusted as digital device.Major part researcher uses the resistance of memristor at present It is worth the one-to-one weight for being mapped to neural network, the neural network based on such mapping method can not accomplish accurately to adjust power How each memristor resistance value in memristor circuit is effectively mapped to corresponding cynapse in neural network matrix and weighed by value Value, is very crucial problem to reach the optimum of operational precision, power consumption.
Currently, the synaptic structure based on memory resistor mainly has in the prior art: Prezioso, M.1Et al. recalled with two The difference of unit is hindered to define the weight in neural network used, and only to previous memristor list during adjusting Member applies driving voltage, and keeps the resistance value of the latter memristor unit constant, can guarantee the weight of all cynapses all in this way There is identical a reference value.It updates using manhattan right value update rule, that is,To The amplitude of the driving voltage of the application to required for the memristor unit for needing to adjust is positive and negative.Ligang Gao2Et al. way be Using the driving voltage (including pulsewidth and amplitude) under process control, and in conjunction with additional write verification step in a program come by The resistance of step section memristor, until its resistance is close or equal to target resistance values, so that the right value update of each step be made to reach To desired value so that neural network can operate normally.
The weight of a cynapse is indicated using the difference of two memristor units, and is advised using Manhattan right value update Then, due to causing to realize simple image recognition without fundamentally solving the inexactness that synaptic weight is adjusted, because This is unable to complete more complicated neural network function.And the technology by the way that write verification step is added, which increase program controls The complexity of system, and the weight that can be rapidly completed originally is adjusted into advantage, it is changed into time-consuming verification process repeatedly.
Summary of the invention
For above-mentioned there are problem or deficiency, to solve resistance value to the mapping problems of synaptic weight and solution is motivating The uncertain problem of memristor unit change in resistance under the action of voltage, the present invention provides a kind of minds based on memristor unit Through network synaptic structure and its adjusting method, realize memristor resistance value to synaptic weight by memristor cascade.
A kind of neural network synaptic structure based on memristor unit is recalled by the unit cascaded cascade constituted of n memristor Hinder device unit group.
The accuracy of synaptic weight required for the quantity n of the memristor unit is operated normally by neural network determines, Carry out each memristor unit of tissue in the connection type of physical layer, by the resistance value of each memristor unit using cascade connection type It is pre-processed according to following formula:
Wherein, i refers to i-th of cascade memristor unit in cascade memristor unit group, RiTo cascade memristor unit group In i-th of unit real-time resistance value, RmidFor the intermediate value of the change in resistance range of each unit in cascade memristor unit group, Rmin For the minimum value of the change in resistance range of each unit in cascade memristor unit group, wiIt is in cascade memristor unit group i-th The numerical value that the resistance value of a cascade memristor unit maps.Each memristor unit is identical, RmidAnd RminFor definite value.
The numerical value that above-mentioned pretreatment is obtained carries out the tissue of algorithm level according to following formula:
W=round (wX, Y) and * 10X-1+round(wX-1, Y) and * 10X-2+…+round(wi, Y) and * 10i-1 +…+round (w1, Y) and * 100+round(w-1, Y-1) and * 10-1+round(w-2, Y-2) and * 10-2 +…+round(w-i, Y-i) and * 10-i+…+ round(w-Y, 0) and * 10-Y
Or
W=round (wX, 0) and * 10X-1+round(wX-1, 0) and * 10X-2+…+round(wi, 0) and * 10i-1 +…+round (w1, 0) and * 100+round(w-1, 0) and * 10-1+round(w-2, 0) and * 10-2 +…+round(w-i, 0) and * 10-i+…+round(w-Y, 0)*10-Y
Wherein, W is the weight for cascading memristor unit group (i.e. a neural network electronic synapse), and X and Y are respectively to cascade The corresponding quantity for indicating weight integer and fractional part unit in memristor unit group, Y are also the accurate of represented weight simultaneously Degree; round(wi, Y-i) and it is to indicate to wiIt rounds up and retains Y-i decimals.
The emulation cynapse that tissue is completed in the manner described above carries out the update of synaptic weight using two kinds of regulative modes.
The first is adjusted to be complete, i.e., cascade all memristor units are involved in adjusting during right value update, also It is to change the resistance value of cascade each memristor unit by applying the operation of electric pulse.
Second is adjusted stepwise, i.e., according to the situation of change of error during the update of weight, to judge to need The memristor unit of adjusting.Specially in the early period of the changing greatly due to weight of program operation, start only to aforementioned expression The higher position of weight in method is adjusted;In the later period of program operation, the update of each weight is smaller, that is, learns Journey is nearly completed, and starts to carry out fine adjusting to all unit application electric pulses at this time again, to reach desired weight as early as possible. Error is considered as early period when being greater than 5~20 times of err_goal, is otherwise the later period, err_goal is the EP (end of program) set in program Error trigger condition, error current be less than the value when learning process terminate.
Multiple memristor cell levels are tied together as an electronic synapse (i.e. cascade memristor unit group) by the present invention, Its resistance value is mapped to the synaptic weight in neural network using a kind of mapping ruler of weighting, and using full adjusting or substep The mode of adjusting come avoid neural network in learning process because memristor can not accurately adjust brought by influence, to make each Neural network can operate normally in the system based on memristor.
In conclusion the present invention solve resistance value to synaptic weight mapping problems and memristor unit change in resistance not Certain problem.
Detailed description of the invention
Fig. 1 is the neural network synaptic knob composition based on memory resistor in the prior art;
Fig. 2 is the test and control system that embodiment uses;
Fig. 3 is neural network synaptic knob composition of the embodiment based on five memristor units;
Fig. 4 is the modulation principle flow chart of synaptic structure of the present invention;
Fig. 5 is that the neural network of embodiment surveys the tolerance of error in adjustment process in three kinds of different modes of learning Examination.
Specific embodiment
It is further to the present invention with reference to the accompanying drawings and examples to be described in detail
Fig. 2 is the test and control system that embodiment uses, and top is 3706A system control switch in figure, it passes through The connection and disconnection of internal circuit are come the memristor unit of choosing needs to operate.2400 interfaces are used to connect Keithley2400Source Meter digital multimeter is also connected with generation excitation to read the resistance value of memristor unit thereon The function generator interface of voltage realizes the update of memristor unit resistance value by the pulse voltage that function generator generates.In figure DUT be memristor unit to be controlled, a unit is illustrated only in figure to illustrate.
Fig. 3 is the memristor cell array of embodiment preparation, and each intersection is a memristor unit, and both ends are connected to control The circuit and access system control switch of the unit are made, wherein five electronic synapses simulated as one of a row, pass through program Control applies write-in voltage VwriteWith reading voltage Vread.Five memristor units in dotted line frame are the nerve net of the present embodiment Network synaptic structure.
Specific weight indicates and updates that details is shown in Fig. 4, runs the BP nerve net based on Iris data set by computer Network determines that weight indicates that required accuracy is comprising totally five decimals including integer-bit, it is therefore desirable to five memristor units Cascade is to indicate a synaptic weight.
After initialization operation after the control unit of each outside is connect with memristor array Jing Guo each unit Can start program operation, by each memristor unit resistance value that digital multimeter is successively read return to after computer according to Secondary label is1, R2... R5, then according to formula wi=(Ri-Rmid)/(Rmax-Rmin) pre-processed, all resistance values It is modulated in [- 1,1] range.According still further to weighting scheme above-mentioned:
W=w1·1+w2·0.1+w3·0.01+w4·0.001+w5·0.0001
Obtain the practical weight of entire cynapse.
During subsequent right value update, this example is carried out using adjusted stepwise method.During program operation, The weight for needing to adjust some cynapse in backpropagation by the neural network error that computer calculates, then passes through process control Instrument applies the square-wave voltage of specified quantity and pulse width to corresponding cynapse unit, in the excitation of these adjustment pulses Under the migration of carrier occurs inside each memristor unit, so that the conductance of the unit is increased or is reduced.
(err_goal is in program when we judge that current error is greater than 10 times of err_goal by program in this example The error trigger condition of the EP (end of program) of setting, learning process terminates when error current is less than the value), at this time this weight with The target weight for needing to learn falls far short, and then only influences maximum first place to weight and is adjusted, by multiple study Current error value adjusts each memristor unit less than 10 times of err_goal and then simultaneously after circulation, fine-tuned with Meet the requirement of err_goal.
Specific mode of operation carries out detailed so that one group of five memristor unit is a cascade memristor unit group as an example Illustrate: one integer, four decimals being selected according to neural network precise requirements to w in this example, therefore use one group five Memristor unit is a cascade memristor unit group.
The resistance value of all units is all reset as the intermediate state of its change in resistance range in the initial state, it is assumed that some The weight group of cynapse unit becomes
W=0.47281-0.2310.1+0.800.01+0.20.001-00.0001=0.4579
It can be seen that such method indicates the weight of cynapse, the value of first memristor unit plays main function, subsequent Memristor unit can accurately adjust the weight of cynapse, to meet the requirement of accuracy.It is followed assuming that having run first It is needed after ring by weighed value adjusting to w=1.0482, particular number of pulse is applied to the first memristor unit by process control Voltage, and subsequent unit remains unchanged, and gradually slows down in the variation of the weight after the adjusting of multiple learning process, It is exactly that learning process is nearly completed, pulse control is all applied to all units by process control at this time, error amount gradually subtracts In the error range as low as allowed, learning process terminates, and right value update is completed.
Fig. 5 is the actual Computer Simulation to above-mentioned processing mode of our embodiments as a result, be to adjust entirely respectively, point Step section and when without tandem states under the influence of different errors accuracy rate situation of change.Abscissa is the cynapse of computer simulation The error amount of weight variation, that is, the random error added.The accuracy rate when longitudinal axis is the end of the program.It can be seen that opening Begin just tolerate the error of maximum number magnitude using full regulative mode, and adjusted stepwise is not much different to the tolerance of error but It is that the accuracy rate that adjusted stepwise can reach is higher.

Claims (2)

1. a kind of neural network synaptic structure based on memristor unit, it is characterised in that:
By the unit cascaded cascade memristor unit group constituted of n memristor, cynapse required for n is operated normally by neural network is weighed The accuracy of value determines;The resistance value of each memristor unit is pre-processed according to following formula:
Wherein, i refers to i-th of cascade memristor unit in cascade memristor unit group, RiFor i-th of cascade memristor unit Real-time resistance value, RmidFor the change in resistance range intermediate value of each unit in cascade memristor unit group, RminTo cascade memristor The change in resistance stated range minimum of each unit in unit group, wi are that the resistance value of i-th of cascade memristor unit maps to obtain Numerical value;Each memristor unit is identical, RmidAnd RminFor definite value;
The numerical value that above-mentioned pretreatment is obtained carries out the tissue of algorithm level according to following formula:
W=round (wX, Y) and * 10X-1+round(wX-1, Y) and * 10x-2+…+round(wi, Y) and * 10i-1
+…+round(w1, Y) and * 100+round(w-1, Y-1) and * 10-1+round(w-2, Y-2) and * 10-2
+…+round(w-i, Y-i) and * 10-i+…+round(w-Y, 0) and * 10-Y
Or
W=round (wX, 0) and * 10X-1+round(wX-1, 0) and * 10X-2+…+round(wi, 0) and * 10i-1
+…+round(w1, 0) and * 100+round(w-1, 0) and * 10-1+round(w-2, 0) and * 10-2
+…+round(w-i, 0) and * 10-i+…+round(w-Y, 0) and * 10-Y
Wherein, W is the weight for cascading memristor unit group, and king and Y are respectively to cascade corresponding expression weight in memristor unit group The quantity of integer and fractional part unit, Y are also the accuracy of represented weight simultaneously;round(wi, Y-i) and it is to indicate to wiFour House five enters to retain Y-i decimals.
2. the update mode of synaptic weight is such as described in claim 1 based on the neural network synaptic structure of memristor unit Under:
The first is adjusted to be complete, and cascade all memristor units are involved in adjusting during right value update, by applying electricity Pulse operates to change the resistance value of cascade each memristor unit;
Second is adjusted stepwise, i.e., according to the situation of change of error during the update of weight, to judge to need to adjust Memristor unit;
Specially only the higher position of weight is adjusted in the early period of program operation, is just started in the later period of program operation to institute Some units apply electric pulse and carry out fine adjusting, to reach desired weight as early as possible;When error is greater than 5~20 times of err_goal It is considered as early period, is otherwise the later period, err_goal is the error trigger condition of the EP (end of program) set in program, and error current is less than Learning process terminates when the value.
CN201811245921.9A 2018-10-24 2018-10-24 Neural network synaptic structure based on memristor unit and adjusting method thereof Active CN109359734B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811245921.9A CN109359734B (en) 2018-10-24 2018-10-24 Neural network synaptic structure based on memristor unit and adjusting method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811245921.9A CN109359734B (en) 2018-10-24 2018-10-24 Neural network synaptic structure based on memristor unit and adjusting method thereof

Publications (2)

Publication Number Publication Date
CN109359734A true CN109359734A (en) 2019-02-19
CN109359734B CN109359734B (en) 2021-10-26

Family

ID=65346390

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811245921.9A Active CN109359734B (en) 2018-10-24 2018-10-24 Neural network synaptic structure based on memristor unit and adjusting method thereof

Country Status (1)

Country Link
CN (1) CN109359734B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110364232A (en) * 2019-07-08 2019-10-22 河海大学 It is a kind of based on memristor-gradient descent method neural network Strength of High Performance Concrete prediction technique
WO2023059265A1 (en) * 2021-10-08 2023-04-13 Agency For Science, Technology And Research Neural processing core, method of programming a synaptic memory array thereof and method of performing neural network inference thereon
CN118428429A (en) * 2024-07-05 2024-08-02 中国人民解放军国防科技大学 Memristive synapse, memristive crossover array circuit and conductance updating method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106650922A (en) * 2016-09-29 2017-05-10 清华大学 Hardware neural network conversion method, computing device, compiling method and neural network software and hardware collaboration system
CN107578014A (en) * 2017-09-06 2018-01-12 上海寒武纪信息科技有限公司 Information processor and method
US20180018358A1 (en) * 2013-10-16 2018-01-18 University Of Tennessee Research Foundation Method and apparatus for constructing a neuroscience-inspired artificial neural network with visualization of neural pathways
CN108009640A (en) * 2017-12-25 2018-05-08 清华大学 The training device and its training method of neutral net based on memristor

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180018358A1 (en) * 2013-10-16 2018-01-18 University Of Tennessee Research Foundation Method and apparatus for constructing a neuroscience-inspired artificial neural network with visualization of neural pathways
CN106650922A (en) * 2016-09-29 2017-05-10 清华大学 Hardware neural network conversion method, computing device, compiling method and neural network software and hardware collaboration system
CN107578014A (en) * 2017-09-06 2018-01-12 上海寒武纪信息科技有限公司 Information processor and method
CN108009640A (en) * 2017-12-25 2018-05-08 清华大学 The training device and its training method of neutral net based on memristor

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
TIANGUI YOU等: "Exploiting Memristive BiFeO3 Bilayer Structures for Compact Sequential Logics", 《ADVANCED FUNCTIONAL MATERIALS》 *
王颜: "基于忆阻器的耦合行为与突触电路研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 *
贾真真: "基于记忆元件的细胞神经网络研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110364232A (en) * 2019-07-08 2019-10-22 河海大学 It is a kind of based on memristor-gradient descent method neural network Strength of High Performance Concrete prediction technique
WO2023059265A1 (en) * 2021-10-08 2023-04-13 Agency For Science, Technology And Research Neural processing core, method of programming a synaptic memory array thereof and method of performing neural network inference thereon
CN118428429A (en) * 2024-07-05 2024-08-02 中国人民解放军国防科技大学 Memristive synapse, memristive crossover array circuit and conductance updating method
CN118428429B (en) * 2024-07-05 2024-09-13 中国人民解放军国防科技大学 Memristive synapse, memristive crossover array circuit and conductance updating method

Also Published As

Publication number Publication date
CN109359734B (en) 2021-10-26

Similar Documents

Publication Publication Date Title
CN109359734A (en) A kind of neural network synaptic structure and its adjusting method based on memristor unit
Hong et al. Memristive circuit implementation of biological nonassociative learning mechanism and its applications
CN107341539B (en) neural network processing system
Arthur et al. Silicon-neuron design: A dynamical systems approach
CN109657787A (en) A kind of neural network chip of two-value memristor
JP2017538994A (en) Neuromorphic synapse, its array, and its system
CN109816096B (en) Memristor-based perceptron neural network circuit and adjusting method thereof
Yakopcic et al. Energy efficient perceptron pattern recognition using segmented memristor crossbar arrays
CN110852429B (en) 1T 1R-based convolutional neural network circuit and operation method thereof
WO2023000586A1 (en) Storage and computation integrated apparatus and calibration method therefor
CN109165730A (en) State quantifies network implementation approach in crossed array neuromorphic hardware
CN110991629A (en) Memristor-based neuron circuit
Li et al. ReSiPE: ReRAM-based single-spiking processing-in-memory engine
US20190228295A1 (en) Synapse memory cell driver
Wei et al. Emerging Memory-Based Chip Development for Neuromorphic Computing: Status, Challenges, and Perspectives
DE102021123580A1 (en) METHOD OF DEVELOPING AN INITIALIZATION FUNCTION FOR PROGRAMMING A MEMORY ELEMENT
CN115605026A (en) Passive memristor cross array device capable of directly realizing weight difference
CN116863296A (en) Self-adaptive optimization sensing and storage integrated target detection and recognition system and method
Lin et al. A High-Speed and High-Efficiency Diverse Error Margin Write-Verify Scheme for an RRAM-Based Neuromorphic Hardware Accelerator
CN113971981A (en) Operation method and device of memory unit and memory array
Schrauwen et al. Reservoir computing toolbox manual
Schwartz Reproducing biologically realistic regimes on a highly-accelerated neuromorphic hardware system
KR102433941B1 (en) New device performance test system and the new device performance test method
CN114550808A (en) Nerve morphology calculation and evaluation device
CN116129961A (en) Method for setting resistance state of memristor cross array device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant