CN109359734A - A kind of neural network synaptic structure and its adjusting method based on memristor unit - Google Patents
A kind of neural network synaptic structure and its adjusting method based on memristor unit Download PDFInfo
- Publication number
- CN109359734A CN109359734A CN201811245921.9A CN201811245921A CN109359734A CN 109359734 A CN109359734 A CN 109359734A CN 201811245921 A CN201811245921 A CN 201811245921A CN 109359734 A CN109359734 A CN 109359734A
- Authority
- CN
- China
- Prior art keywords
- memristor
- round
- unit
- cascade
- weight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Neurology (AREA)
- Feedback Control In General (AREA)
Abstract
The present invention relates to computers and electronic information technical field, and in particular to a kind of neural network synaptic structure and its adjusting method based on memristor unit.Multiple memristor cell levels are tied together as an electronic synapse (i.e. cascade memristor unit group) by the present invention, the synaptic weight being mapped to its resistance value using a kind of mapping ruler of weighting in neural network, and using it is complete adjust or adjusted stepwise by the way of come avoid neural network in learning process because memristor can not accurately adjust brought by influence, to make each neural network that can operate normally in the system based on memristor.The present invention solves resistance value to the mapping problems of synaptic weight and the uncertain problem of memristor unit change in resistance.
Description
Technical field
The present invention relates to computers and electronic information technical field, and in particular to a kind of nerve net based on memristor unit
Network synaptic structure and its adjusting method.
Background technique
Cynapse is that the intermediate structure of different neurons is connected in neural network, and weight passes through corresponding neural network algorithm
Continuous renewal is the basis of neural network information processing.Memristor as a kind of new component with resistance tunable characteristic,
It is considered as 4 basic circuit elements one of arranged side by side with resistance, inductance and capacitor.
Researcher find synaptic connection strengths changing rule and memristor electrology characteristic (its electric conductivity value by apply electricity
The control characteristic of signal) there is similar changing rule, therefore single memristor can simulate the function of a cynapse.With use
Electronic computer (transistor circuit of i.e. traditional CMOS complementary metal-oxide-semiconductor base) runs neural network procedure phase
Than energy consumption can be greatly reduced in memristor unit, and improve read or write speed reduces the complexity of integrated circuit simultaneously.However, memristor
Also there is its birth defect, as analog device, its resistance is nonlinear analog quantity variation under the regulation of voltage first, and
The weight that expectation reaches cannot be precisely adjusted as digital device.Major part researcher uses the resistance of memristor at present
It is worth the one-to-one weight for being mapped to neural network, the neural network based on such mapping method can not accomplish accurately to adjust power
How each memristor resistance value in memristor circuit is effectively mapped to corresponding cynapse in neural network matrix and weighed by value
Value, is very crucial problem to reach the optimum of operational precision, power consumption.
Currently, the synaptic structure based on memory resistor mainly has in the prior art: Prezioso, M.1Et al. recalled with two
The difference of unit is hindered to define the weight in neural network used, and only to previous memristor list during adjusting
Member applies driving voltage, and keeps the resistance value of the latter memristor unit constant, can guarantee the weight of all cynapses all in this way
There is identical a reference value.It updates using manhattan right value update rule, that is,To
The amplitude of the driving voltage of the application to required for the memristor unit for needing to adjust is positive and negative.Ligang Gao2Et al. way be
Using the driving voltage (including pulsewidth and amplitude) under process control, and in conjunction with additional write verification step in a program come by
The resistance of step section memristor, until its resistance is close or equal to target resistance values, so that the right value update of each step be made to reach
To desired value so that neural network can operate normally.
The weight of a cynapse is indicated using the difference of two memristor units, and is advised using Manhattan right value update
Then, due to causing to realize simple image recognition without fundamentally solving the inexactness that synaptic weight is adjusted, because
This is unable to complete more complicated neural network function.And the technology by the way that write verification step is added, which increase program controls
The complexity of system, and the weight that can be rapidly completed originally is adjusted into advantage, it is changed into time-consuming verification process repeatedly.
Summary of the invention
For above-mentioned there are problem or deficiency, to solve resistance value to the mapping problems of synaptic weight and solution is motivating
The uncertain problem of memristor unit change in resistance under the action of voltage, the present invention provides a kind of minds based on memristor unit
Through network synaptic structure and its adjusting method, realize memristor resistance value to synaptic weight by memristor cascade.
A kind of neural network synaptic structure based on memristor unit is recalled by the unit cascaded cascade constituted of n memristor
Hinder device unit group.
The accuracy of synaptic weight required for the quantity n of the memristor unit is operated normally by neural network determines,
Carry out each memristor unit of tissue in the connection type of physical layer, by the resistance value of each memristor unit using cascade connection type
It is pre-processed according to following formula:
Wherein, i refers to i-th of cascade memristor unit in cascade memristor unit group, RiTo cascade memristor unit group
In i-th of unit real-time resistance value, RmidFor the intermediate value of the change in resistance range of each unit in cascade memristor unit group, Rmin
For the minimum value of the change in resistance range of each unit in cascade memristor unit group, wiIt is in cascade memristor unit group i-th
The numerical value that the resistance value of a cascade memristor unit maps.Each memristor unit is identical, RmidAnd RminFor definite value.
The numerical value that above-mentioned pretreatment is obtained carries out the tissue of algorithm level according to following formula:
W=round (wX, Y) and * 10X-1+round(wX-1, Y) and * 10X-2+…+round(wi, Y) and * 10i-1 +…+round
(w1, Y) and * 100+round(w-1, Y-1) and * 10-1+round(w-2, Y-2) and * 10-2 +…+round(w-i, Y-i) and * 10-i+…+
round(w-Y, 0) and * 10-Y
Or
W=round (wX, 0) and * 10X-1+round(wX-1, 0) and * 10X-2+…+round(wi, 0) and * 10i-1 +…+round
(w1, 0) and * 100+round(w-1, 0) and * 10-1+round(w-2, 0) and * 10-2 +…+round(w-i, 0) and * 10-i+…+round(w-Y,
0)*10-Y
Wherein, W is the weight for cascading memristor unit group (i.e. a neural network electronic synapse), and X and Y are respectively to cascade
The corresponding quantity for indicating weight integer and fractional part unit in memristor unit group, Y are also the accurate of represented weight simultaneously
Degree； round(wi, Y-i) and it is to indicate to wiIt rounds up and retains Y-i decimals.
The emulation cynapse that tissue is completed in the manner described above carries out the update of synaptic weight using two kinds of regulative modes.
The first is adjusted to be complete, i.e., cascade all memristor units are involved in adjusting during right value update, also
It is to change the resistance value of cascade each memristor unit by applying the operation of electric pulse.
Second is adjusted stepwise, i.e., according to the situation of change of error during the update of weight, to judge to need
The memristor unit of adjusting.Specially in the early period of the changing greatly due to weight of program operation, start only to aforementioned expression
The higher position of weight in method is adjusted；In the later period of program operation, the update of each weight is smaller, that is, learns
Journey is nearly completed, and starts to carry out fine adjusting to all unit application electric pulses at this time again, to reach desired weight as early as possible.
Error is considered as early period when being greater than 5~20 times of err_goal, is otherwise the later period, err_goal is the EP (end of program) set in program
Error trigger condition, error current be less than the value when learning process terminate.
Multiple memristor cell levels are tied together as an electronic synapse (i.e. cascade memristor unit group) by the present invention,
Its resistance value is mapped to the synaptic weight in neural network using a kind of mapping ruler of weighting, and using full adjusting or substep
The mode of adjusting come avoid neural network in learning process because memristor can not accurately adjust brought by influence, to make each
Neural network can operate normally in the system based on memristor.
In conclusion the present invention solve resistance value to synaptic weight mapping problems and memristor unit change in resistance not
Certain problem.
Detailed description of the invention
Fig. 1 is the neural network synaptic knob composition based on memory resistor in the prior art；
Fig. 2 is the test and control system that embodiment uses；
Fig. 3 is neural network synaptic knob composition of the embodiment based on five memristor units；
Fig. 4 is the modulation principle flow chart of synaptic structure of the present invention；
Fig. 5 is that the neural network of embodiment surveys the tolerance of error in adjustment process in three kinds of different modes of learning
Examination.
Specific embodiment
It is further to the present invention with reference to the accompanying drawings and examples to be described in detail
Fig. 2 is the test and control system that embodiment uses, and top is 3706A system control switch in figure, it passes through
The connection and disconnection of internal circuit are come the memristor unit of choosing needs to operate.2400 interfaces are used to connect
Keithley2400Source Meter digital multimeter is also connected with generation excitation to read the resistance value of memristor unit thereon
The function generator interface of voltage realizes the update of memristor unit resistance value by the pulse voltage that function generator generates.In figure
DUT be memristor unit to be controlled, a unit is illustrated only in figure to illustrate.
Fig. 3 is the memristor cell array of embodiment preparation, and each intersection is a memristor unit, and both ends are connected to control
The circuit and access system control switch of the unit are made, wherein five electronic synapses simulated as one of a row, pass through program
Control applies write-in voltage VwriteWith reading voltage Vread.Five memristor units in dotted line frame are the nerve net of the present embodiment
Network synaptic structure.
Specific weight indicates and updates that details is shown in Fig. 4, runs the BP nerve net based on Iris data set by computer
Network determines that weight indicates that required accuracy is comprising totally five decimals including integer-bit, it is therefore desirable to five memristor units
Cascade is to indicate a synaptic weight.
After initialization operation after the control unit of each outside is connect with memristor array Jing Guo each unit
Can start program operation, by each memristor unit resistance value that digital multimeter is successively read return to after computer according to
Secondary label is1, R2... R5, then according to formula wi=(Ri-Rmid)/(Rmax-Rmin) pre-processed, all resistance values
It is modulated in [- 1,1] range.According still further to weighting scheme above-mentioned:
W=w1·1+w2·0.1+w3·0.01+w4·0.001+w5·0.0001
Obtain the practical weight of entire cynapse.
During subsequent right value update, this example is carried out using adjusted stepwise method.During program operation,
The weight for needing to adjust some cynapse in backpropagation by the neural network error that computer calculates, then passes through process control
Instrument applies the square-wave voltage of specified quantity and pulse width to corresponding cynapse unit, in the excitation of these adjustment pulses
Under the migration of carrier occurs inside each memristor unit, so that the conductance of the unit is increased or is reduced.
(err_goal is in program when we judge that current error is greater than 10 times of err_goal by program in this example
The error trigger condition of the EP (end of program) of setting, learning process terminates when error current is less than the value), at this time this weight with
The target weight for needing to learn falls far short, and then only influences maximum first place to weight and is adjusted, by multiple study
Current error value adjusts each memristor unit less than 10 times of err_goal and then simultaneously after circulation, fine-tuned with
Meet the requirement of err_goal.
Specific mode of operation carries out detailed so that one group of five memristor unit is a cascade memristor unit group as an example
Illustrate: one integer, four decimals being selected according to neural network precise requirements to w in this example, therefore use one group five
Memristor unit is a cascade memristor unit group.
The resistance value of all units is all reset as the intermediate state of its change in resistance range in the initial state, it is assumed that some
The weight group of cynapse unit becomes
W=0.47281-0.2310.1+0.800.01+0.20.001-00.0001=0.4579
It can be seen that such method indicates the weight of cynapse, the value of first memristor unit plays main function, subsequent
Memristor unit can accurately adjust the weight of cynapse, to meet the requirement of accuracy.It is followed assuming that having run first
It is needed after ring by weighed value adjusting to w=1.0482, particular number of pulse is applied to the first memristor unit by process control
Voltage, and subsequent unit remains unchanged, and gradually slows down in the variation of the weight after the adjusting of multiple learning process,
It is exactly that learning process is nearly completed, pulse control is all applied to all units by process control at this time, error amount gradually subtracts
In the error range as low as allowed, learning process terminates, and right value update is completed.
Fig. 5 is the actual Computer Simulation to above-mentioned processing mode of our embodiments as a result, be to adjust entirely respectively, point
Step section and when without tandem states under the influence of different errors accuracy rate situation of change.Abscissa is the cynapse of computer simulation
The error amount of weight variation, that is, the random error added.The accuracy rate when longitudinal axis is the end of the program.It can be seen that opening
Begin just tolerate the error of maximum number magnitude using full regulative mode, and adjusted stepwise is not much different to the tolerance of error but
It is that the accuracy rate that adjusted stepwise can reach is higher.
Claims (2)
1. a kind of neural network synaptic structure based on memristor unit, it is characterised in that:
By the unit cascaded cascade memristor unit group constituted of n memristor, cynapse required for n is operated normally by neural network is weighed
The accuracy of value determines；The resistance value of each memristor unit is pre-processed according to following formula:
Wherein, i refers to i-th of cascade memristor unit in cascade memristor unit group, RiFor i-th of cascade memristor unit
Real-time resistance value, RmidFor the change in resistance range intermediate value of each unit in cascade memristor unit group, RminTo cascade memristor
The change in resistance stated range minimum of each unit in unit group, wi are that the resistance value of i-th of cascade memristor unit maps to obtain
Numerical value；Each memristor unit is identical, RmidAnd RminFor definite value；
The numerical value that above-mentioned pretreatment is obtained carries out the tissue of algorithm level according to following formula:
W=round (wX, Y) and * 10X-1+round(wX-1, Y) and * 10x-2+…+round(wi, Y) and * 10i-1
+…+round(w1, Y) and * 100+round(w-1, Y-1) and * 10-1+round(w-2, Y-2) and * 10-2
+…+round(w-i, Y-i) and * 10-i+…+round(w-Y, 0) and * 10-Y
Or
W=round (wX, 0) and * 10X-1+round(wX-1, 0) and * 10X-2+…+round(wi, 0) and * 10i-1
+…+round(w1, 0) and * 100+round(w-1, 0) and * 10-1+round(w-2, 0) and * 10-2
+…+round(w-i, 0) and * 10-i+…+round(w-Y, 0) and * 10-Y
Wherein, W is the weight for cascading memristor unit group, and king and Y are respectively to cascade corresponding expression weight in memristor unit group
The quantity of integer and fractional part unit, Y are also the accuracy of represented weight simultaneously；round(wi, Y-i) and it is to indicate to wiFour
House five enters to retain Y-i decimals.
2. the update mode of synaptic weight is such as described in claim 1 based on the neural network synaptic structure of memristor unit
Under:
The first is adjusted to be complete, and cascade all memristor units are involved in adjusting during right value update, by applying electricity
Pulse operates to change the resistance value of cascade each memristor unit；
Second is adjusted stepwise, i.e., according to the situation of change of error during the update of weight, to judge to need to adjust
Memristor unit；
Specially only the higher position of weight is adjusted in the early period of program operation, is just started in the later period of program operation to institute
Some units apply electric pulse and carry out fine adjusting, to reach desired weight as early as possible；When error is greater than 5~20 times of err_goal
It is considered as early period, is otherwise the later period, err_goal is the error trigger condition of the EP (end of program) set in program, and error current is less than
Learning process terminates when the value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811245921.9A CN109359734B (en) | 2018-10-24 | 2018-10-24 | Neural network synaptic structure based on memristor unit and adjusting method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811245921.9A CN109359734B (en) | 2018-10-24 | 2018-10-24 | Neural network synaptic structure based on memristor unit and adjusting method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109359734A true CN109359734A (en) | 2019-02-19 |
| CN109359734B CN109359734B (en) | 2021-10-26 |
Family
ID=65346390
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811245921.9A Active CN109359734B (en) | 2018-10-24 | 2018-10-24 | Neural network synaptic structure based on memristor unit and adjusting method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109359734B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110364232A (en) * | 2019-07-08 | 2019-10-22 | 河海大学 | It is a kind of based on memristor-gradient descent method neural network Strength of High Performance Concrete prediction technique |
| WO2023059265A1 (en) * | 2021-10-08 | 2023-04-13 | Agency For Science, Technology And Research | Neural processing core, method of programming a synaptic memory array thereof and method of performing neural network inference thereon |
| CN118428429A (en) * | 2024-07-05 | 2024-08-02 | 中国人民解放军国防科技大学 | Memristive synapse, memristive crossover array circuit and conductance updating method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650922A (en) * | 2016-09-29 | 2017-05-10 | 清华大学 | Hardware neural network conversion method, computing device, compiling method and neural network software and hardware collaboration system |
| CN107578014A (en) * | 2017-09-06 | 2018-01-12 | 上海寒武纪信息科技有限公司 | Information processor and method |
| US20180018358A1 (en) * | 2013-10-16 | 2018-01-18 | University Of Tennessee Research Foundation | Method and apparatus for constructing a neuroscience-inspired artificial neural network with visualization of neural pathways |
| CN108009640A (en) * | 2017-12-25 | 2018-05-08 | 清华大学 | The training device and its training method of neutral net based on memristor |
-
2018
- 2018-10-24 CN CN201811245921.9A patent/CN109359734B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180018358A1 (en) * | 2013-10-16 | 2018-01-18 | University Of Tennessee Research Foundation | Method and apparatus for constructing a neuroscience-inspired artificial neural network with visualization of neural pathways |
| CN106650922A (en) * | 2016-09-29 | 2017-05-10 | 清华大学 | Hardware neural network conversion method, computing device, compiling method and neural network software and hardware collaboration system |
| CN107578014A (en) * | 2017-09-06 | 2018-01-12 | 上海寒武纪信息科技有限公司 | Information processor and method |
| CN108009640A (en) * | 2017-12-25 | 2018-05-08 | 清华大学 | The training device and its training method of neutral net based on memristor |
Non-Patent Citations (3)
| Title |
|---|
| TIANGUI YOU等: "Exploiting Memristive BiFeO3 Bilayer Structures for Compact Sequential Logics", 《ADVANCED FUNCTIONAL MATERIALS》 * |
| 王颜: "基于忆阻器的耦合行为与突触电路研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 * |
| 贾真真: "基于记忆元件的细胞神经网络研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110364232A (en) * | 2019-07-08 | 2019-10-22 | 河海大学 | It is a kind of based on memristor-gradient descent method neural network Strength of High Performance Concrete prediction technique |
| WO2023059265A1 (en) * | 2021-10-08 | 2023-04-13 | Agency For Science, Technology And Research | Neural processing core, method of programming a synaptic memory array thereof and method of performing neural network inference thereon |
| CN118428429A (en) * | 2024-07-05 | 2024-08-02 | 中国人民解放军国防科技大学 | Memristive synapse, memristive crossover array circuit and conductance updating method |
| CN118428429B (en) * | 2024-07-05 | 2024-09-13 | 中国人民解放军国防科技大学 | Memristive synapse, memristive crossover array circuit and conductance updating method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109359734B (en) | 2021-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359734A (en) | A kind of neural network synaptic structure and its adjusting method based on memristor unit | |
| Hong et al. | Memristive circuit implementation of biological nonassociative learning mechanism and its applications | |
| CN107341539B (en) | neural network processing system | |
| Arthur et al. | Silicon-neuron design: A dynamical systems approach | |
| CN109657787A (en) | A kind of neural network chip of two-value memristor | |
| JP2017538994A (en) | Neuromorphic synapse, its array, and its system | |
| CN109816096B (en) | Memristor-based perceptron neural network circuit and adjusting method thereof | |
| Yakopcic et al. | Energy efficient perceptron pattern recognition using segmented memristor crossbar arrays | |
| CN110852429B (en) | 1T 1R-based convolutional neural network circuit and operation method thereof | |
| WO2023000586A1 (en) | Storage and computation integrated apparatus and calibration method therefor | |
| CN109165730A (en) | State quantifies network implementation approach in crossed array neuromorphic hardware | |
| CN110991629A (en) | Memristor-based neuron circuit | |
| Li et al. | ReSiPE: ReRAM-based single-spiking processing-in-memory engine | |
| US20190228295A1 (en) | Synapse memory cell driver | |
| Wei et al. | Emerging Memory-Based Chip Development for Neuromorphic Computing: Status, Challenges, and Perspectives | |
| DE102021123580A1 (en) | METHOD OF DEVELOPING AN INITIALIZATION FUNCTION FOR PROGRAMMING A MEMORY ELEMENT | |
| CN115605026A (en) | Passive memristor cross array device capable of directly realizing weight difference | |
| CN116863296A (en) | Self-adaptive optimization sensing and storage integrated target detection and recognition system and method | |
| Lin et al. | A High-Speed and High-Efficiency Diverse Error Margin Write-Verify Scheme for an RRAM-Based Neuromorphic Hardware Accelerator | |
| CN113971981A (en) | Operation method and device of memory unit and memory array | |
| Schrauwen et al. | Reservoir computing toolbox manual | |
| Schwartz | Reproducing biologically realistic regimes on a highly-accelerated neuromorphic hardware system | |
| KR102433941B1 (en) | New device performance test system and the new device performance test method | |
| CN114550808A (en) | Nerve morphology calculation and evaluation device | |
| CN116129961A (en) | Method for setting resistance state of memristor cross array device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |