CN111461312B - Random neuron discarding based on memristor - Google Patents
Random neuron discarding based on memristor Download PDFInfo
- Publication number
- CN111461312B CN111461312B CN202010225486.4A CN202010225486A CN111461312B CN 111461312 B CN111461312 B CN 111461312B CN 202010225486 A CN202010225486 A CN 202010225486A CN 111461312 B CN111461312 B CN 111461312B
- Authority
- CN
- China
- Prior art keywords
- signal
- memristor
- random
- neuron
- discarding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Neurology (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
The invention belongs to the technical field of semiconductor information correlation, and discloses a random neuron discarding method based on a memristor. The random discarding neuron is used for accepting or rejecting received signals, a discarding control unit and an MOS switch are arranged in the random discarding neuron, the discarding control unit is used for converting the received signals into control instructions to control the switch of the MOS switch, the random discarding neuron comprises a memristor, a divider resistor, a comparator and a register, the memristor is used for receiving excitation signals and converting the excitation signals into random current signals, the divider resistor is used for converting the random current signals into random voltage signals, the comparator is used for comparing the random voltage signals with a preset threshold value, and the register is used for receiving the signals from the comparator and transmitting the signals as the control instructions to the MOS switch to control the switch of the MOS. According to the invention, the recognition precision of the neural network is improved, the requirement on hardware artificial synapse is reduced, and the problems of overfitting and nonlinear weight updating in the neural network training process are solved.
Description
Technical Field
The invention belongs to the technical field of semiconductor information correlation, and particularly relates to a random neuron discarding method based on a memristor.
Background
With the great success of software-based artificial neural networks, in order to further improve the use efficiency of the neural networks, research on hardware accelerators for the artificial neural networks has been started so as to achieve the purposes of increasing the calculation efficiency and reducing the operation power consumption.
At this stage, some researches have been made by those skilled in the art, such as using a memristor with a simple sandwich structure to simulate most of the functions of synapses, showing continuous increase or decrease of memristor conductance under the action of artificial neurons, and realizing simulation of important learning rules of some nervous systems. However, as the algorithm is continuously advanced, more requirements are put on the function of the artificial neurons (such as randomly discarding the neurons) for improving the accuracy of the neural network under a special state, and the corresponding hardware implementation needs to be continuously improved. Accordingly, there is a need in the art to develop an artificial neuron capable of performing a discarding function.
Disclosure of Invention
In view of the above drawbacks or needs for improvement in the prior art, the present invention provides a memristor-based random discarding neuron, wherein the structure and function of the discarding control unit are designed for the key components thereof, so that when the discarding neuron in the present invention is used, the random discarding of part of neurons is realized, the recognition accuracy of a neural network is improved, the requirement on hardware artificial synapses is reduced, and the problems of overfitting in the neural network training process and nonlinear weight updating of hardware artificial synapses are further solved.
To achieve the above object, the present invention provides a memristor-based random discarding neuron for rejecting a received signal, wherein,
the random discarding neuron is provided with a discarding control unit and an MOS switch, and the discarding control unit is used for converting a received signal into a control instruction so as to control the switch of the MOS switch;
the discard control unit comprises a memristor, a voltage division resistor, a comparator and a register, wherein the memristor is used for receiving an excitation signal and converting the excitation signal into a random current signal, the voltage division resistor is connected with the memristor and used for converting the random current signal from the memristor into a random voltage signal, the comparator is connected with the voltage division resistor and used for comparing the random voltage signal from the voltage division resistor with a preset threshold value, when the random voltage signal is larger than the preset threshold value, the comparator outputs a low level signal, and when the random voltage signal is smaller than the preset threshold value, the comparator outputs a high level signal;
the register is connected with the comparator and used for receiving the signal from the comparator and transmitting the currently received signal to the MOS switch as a control instruction, when the register receives a low-level signal, the MOS switch is switched on, the signal is transmitted to the next neuron through the random discarding neuron, and when the register receives a high-level signal, the MOS switch is switched off, and the signal cannot be transmitted to the next neuron through the random discarding neuron and is discarded.
Further preferably, one end of the voltage dividing resistor is connected with the memristor, and the other end of the voltage dividing resistor is grounded.
Further preferably, the randomly discarded neuron further comprises an artificial synapse for preprocessing a signal received by the randomly discarded neuron.
Further preferably, the random discarding neuron further comprises a signal receiving and issuing unit, one end of which is connected to the artificial synapse, and the other end of which is connected to the discarding control unit, and which receives a signal from the artificial synapse after being preprocessed and transmits the signal to the discarding control unit.
Further preferably, the artificial synapse is comprised of a non-volatile memristor whose resistance state is continuously tunable.
Further preferably, the preset threshold is obtained by a threshold voltage connected to the comparator.
Further preferably, the register is bistable in nature and outputs a continuous signal after receiving a signal until the current signal is replaced by the next received signal.
Generally, compared with the prior art, the technical scheme of the invention has the following beneficial effects:
1. the randomly discarded neurons provided by the invention can be randomly discarded through the design of the discarding control unit, so that different neurons are activated during each training, different neural networks are mutually combined to obtain an average result of a plurality of different neural networks, and the average process is favorable for weakening invalid characteristics, thereby solving the over-fitting problem existing in the use of a hardware neural network; on the other hand, the weight of the hardware artificial synapse based on the memristor has the characteristic of nonlinear updating, the accuracy of the neural network is reduced due to frequent nonlinear updating in the training process, and the random discarding of neurons does not participate in each training, so that the updating times of the artificial synapses connected with the random discarding of neurons are reduced, and the influence caused by the nonlinear updating of the artificial synapses is reduced due to the random discarding of the neurons;
2. the randomly discarded neurons provided by the invention have certain probability to be randomly inhibited in the training process and are excited in the using process. In the process of random inhibition, a neuron with a random resistance change effect can be used for sending a control signal to determine whether the neuron is randomly discarded to be excited or not, and the memristor can spontaneously return to an initial state after resistance change without additional zero setting operation.
Drawings
FIG. 1 is a schematic diagram of a structure of a memristor-based random-discard neuron constructed in accordance with a preferred embodiment of the present invention;
FIG. 2 is a graph of electrical performance of memristors constructed in accordance with a preferred embodiment of the present invention, wherein (a) is the memristor current-voltage response curve, (b) is the memristor current response curve for electrical impulses, (c) is the memristor current response curve for electrical impulses of multiple magnitudes, and (d) is the memristor response distribution for electrical impulses of different magnitudes;
FIG. 3 is a neural network constructed from randomly discarded neurons constructed in accordance with a preferred embodiment of the present invention;
FIG. 4 is a recognition result of handwritten numbers by randomly discarded neurons constructed in accordance with a preferred embodiment of the present invention, where (a) is the recognition accuracy of artificial synapses using actual memristions, and (b) is the recognition accuracy of artificial synapses using ideal ones.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
As shown in fig. 1, a memristor-based random discarding neuron is provided with an artificial synapse, a signal receiving and issuing unit, a discarding control unit and an MOS switch, an output end of the artificial synapse is connected to an input end of the signal receiving and issuing unit, an output end of the signal receiving and issuing unit is connected to one signal end of the MOS switch, the discarding control unit issues a control signal of the MOS switch, and the other end of the MOS switch is connected to a synapse of a next-stage neuron; in this embodiment, the artificial synapse is M2, the signal receiving and issuing unit, the memristor is M1, the comparator is OP1, the register is S2, and the MOS switch is S1.
The discarding control unit comprises a memristor, a voltage division resistor, a comparator and a register, one end of the memristor is connected with a signal source, the other end of the memristor is divided into two paths, one path of the memristor is grounded through the voltage division resistor, the other path of the memristor is used as the negative input of the comparator, the positive input of the comparator is used for comparing a threshold value, the output of the comparator is connected with the control end of the register, and the output end of the register is used as the output of the discarding control unit; the memristor has a random resistance change effect. The abandon control unit is responsible for sending out the control signal of MOS switch, and the memristor is for having the complete volatility memristor of random resistance change effect.
Furthermore, the artificial synapse is used for connection among all the randomly discarded neurons, receives an electric signal transmitted by a previous-stage neuron, and transmits the signal to a later-stage neuron according to the self electric conductance in proportion; the artificial synapse consists of nonvolatile memristors with continuously adjustable resistance states, one end of each memristor is connected to the upper-level neuron through a lead, and the other end of each memristor is connected to the input end of the signal receiving and issuing unit; the artificial synapse is used for preprocessing the received signal, and the preprocessing may be an operation, a filtering, or other processing method for the received signal.
Furthermore, the signal receiving and sending unit is used for receiving the neural signal processed by the artificial synapse, and the output end of the signal receiving and sending unit is connected with the MOS switch and is responsible for sending a detection signal or a synapse weight adjusting signal.
Furthermore, the output end of the discarding control unit is connected with the control end of the MOS switch and is used for controlling the state of the MOS switch, and a signal for making the MOS switch in a conducting state is randomly sent out in the training process; and in the detection process, the MOS switch is always kept in a conducting state.
Furthermore, the memristor is a memristor with a random resistance change effect, one end of the memristor is connected with an excitation signal, the other end of the memristor is connected with one end of a voltage dividing resistor, and the other end of the voltage dividing resistor is grounded.
Furthermore, a negative input end of the comparator is connected with one end, connected with the memristor and the voltage dividing resistor, of the memristor, a positive input end of the comparator is connected with a threshold voltage, and an output end of the comparator is connected with a control end of the register.
Furthermore, the zero setting end of the register is connected with a zero setting signal generator, and the output end of the register is connected to the control end of the MOS switch. When the memristor receives an excitation signal, the memristor changes resistance to different low-resistance states according to a certain probability to cause the divided voltage on the voltage dividing resistor to change, when the divided voltage on the voltage dividing resistor is larger than the threshold voltage, the output end of the comparator outputs a low level signal, the switch of the register is turned on, and then the MOS switch is in a conducting state, and the signal of the neuron can be transmitted to the next stage of neuron and is in an activated state; when the voltage of the voltage dividing resistor is lower than the threshold voltage, the related signal cannot be output backwards, so that the MOS switch is in a cut-off state, the signal of the neuron cannot be transmitted backwards, and the signal is in a discarded state.
The register is characterized by bistability, and when receiving a low level signal, the register outputs a continuous high level signal to enable the MOS switch to be in a conducting state, namely the register stores the currently accepted comparison result until the current comparison result is replaced by the next comparison result.
The random discarding of the neurons can realize the disconnection of the connection between the neurons and the later neurons according to a certain probability in the training process so as to realize the functions of inhibiting the over-fitting phenomenon of the artificial neural network in the training process and reducing the use requirement of the artificial synapses.
In this embodiment, the artificial synapse array employs two different memristor arrays, both of which are continuously adjustable devices with non-volatility, one is an actual device whose resistance state changes non-linearly in the weight updating process, and the other is an ideal device whose resistance state changes fully linearly, and is used for comparing optimization effects of randomly discarding neurons on the non-ideal devices.
The memristor M2 with the random resistance change effect is used for embodying the probability characteristics of the neuron. As shown in fig. 2, (a) to (d) of fig. 2 show that the change in resistance of the memristor M2 is completely volatile in a short time, and the resistance value of the low resistance state has a random effect, and the memristor with the random resistance change effect spontaneously decays to return to the high resistance state in a short time after receiving a pulse, and waits for the next selection. In the embodiment, the memristor with the random resistance change effect adopts Ag/Ta2O5: Ag/Pt (silver/silver doped tantalum pentoxide/platinum) fully volatile memristors; when the applied forward scan voltage exceeds the threshold, the current of the memristor suddenly increases to the current limit, and the memristor shows obvious random characteristics on the threshold distribution. The performance of the pulse test shows that the low-resistance states of the memristor also have obvious random distribution and can be used for realizing random signals. The manufacturing process of the memristor comprises the following steps: in magnetron sputtering equipment, a monocrystalline silicon wafer coated with an oxide layer with a certain thickness is used as a substrate, a Ti layer is plated firstly as an adhesion layer, and then Pt is plated as a lower electrode; then simultaneously depositing Ag and Ta by a magnetron sputtering co-sputtering method2O5To obtain Ag-doped Ta2O5(ii) a Finally, depositing an upper electrode Ag through magnetron sputtering to prepare and obtain the completely volatile Ag/Ta with random resistance change effect2O5: Ag/Pt memristor.
The negative input end of the comparator is connected with the voltage division of the voltage division resistor R1, and the positive input end is connected with the threshold voltage VthThe output end of the register is connected with the control end of the register S2, and the output end of the register is connected with the MOS switch S1. The comparator is used for comparing the voltage-to-ground voltage of the voltage dividing resistor R1 with the threshold voltage VthWhen dividing the pressureThe voltage of the resistor R1 to the ground is larger than the threshold voltage VthAt this time, the comparator OP1 outputs a low level for driving the register S2 to output a continuous high level for driving the MOS switch S1, thereby activating the association between the corresponding neuron and synapse and the neuron at the subsequent stage.
The randomly discarded neurons are connected with each other according to the algorithm of the neural network, as shown in fig. 3, the recognition function of the neural network can be realized by using hardware, and in the present embodiment, the recognition function of the handwritten numbers is realized by using a software simulation method. As shown in (a) and (b) of FIG. 4, the recognition rate of the neural network is significantly improved and the gap between the non-ideal synapse and the ideal synapse is significantly narrowed after the pattern of randomly discarding neurons is adopted.
The random discarding neuron based on the memristor provided by the invention has the advantages that the memristor of the synapse array is a nonvolatile memristor, and resistors, comparators, MOS switches and the like are mature commercial devices or modules. By simulating the selected devices and the constructed artificial neural network, and has a synapse basic unit. When the training process of the neural network is completed by utilizing the randomly discarded neurons, the overfitting phenomenon generated by the neural network when the training set is small can be well inhibited, and meanwhile, the influence on the identification precision due to the non-ideal characteristic of the hardware artificial synapse is reduced.
It will be understood by those skilled in the art that the foregoing is merely a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included within the scope of the present invention.
Claims (7)
1. A memristor-based random discard neuron for rejecting a received signal, wherein,
the random discarding neuron is provided with a discarding control unit and an MOS switch, and the discarding control unit is used for converting a received signal into a control instruction so as to control the switch of the MOS switch;
the discard control unit comprises a memristor, a voltage division resistor, a comparator and a register, wherein the memristor is used for receiving an excitation signal and converting the excitation signal into a random current signal, the voltage division resistor is connected with the memristor and used for converting the random current signal from the memristor into a random voltage signal, the comparator is connected with the voltage division resistor and used for comparing the random voltage signal from the voltage division resistor with a preset threshold value, when the random voltage signal is larger than the preset threshold value, the comparator outputs a low level signal, and when the random voltage signal is smaller than the preset threshold value, the comparator outputs a high level signal;
the register is connected with the comparator and used for receiving the signal from the comparator and transmitting the currently received signal to the MOS switch as a control instruction, when the register receives a low-level signal, the MOS switch is switched on, the signal is transmitted to the next neuron through the random discarding neuron, and when the register receives a high-level signal, the MOS switch is switched off, and the signal cannot be transmitted to the next neuron through the random discarding neuron and is discarded.
2. The memristor-based random discard neuron as defined in claim 1, wherein the voltage dividing resistor is connected to the memristor at one end and to ground at the other end.
3. The memristor-based stochastic discard neuron of claim 1, further comprising an artificial synapse for pre-processing signals accepted by the stochastic discard neuron.
4. The memristor-based random discard neuron as claimed in claim 3, wherein the random discard neuron further comprises a signal acceptance issuing unit connected with the artificial synapse at one end and with the discard control unit at the other end, and the signal acceptance issuing unit accepts a signal from the artificial synapse preprocessing and transmits the signal to the discard control unit.
5. The memristor-based random-discard neuron of claim 3, wherein the artificial synapse is composed of a non-volatile memristor whose resistance state is continuously adjustable.
6. The memristor-based random discard neuron as defined in claim 1, wherein the preset threshold is obtained by a threshold voltage connected to the comparator.
7. The memristor-based random discard neuron as claimed in claim 1, wherein the register is bistable in nature, and upon receiving a signal, outputs a continuous signal until the current signal is replaced by the next received signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010225486.4A CN111461312B (en) | 2020-03-26 | 2020-03-26 | Random neuron discarding based on memristor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010225486.4A CN111461312B (en) | 2020-03-26 | 2020-03-26 | Random neuron discarding based on memristor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111461312A CN111461312A (en) | 2020-07-28 |
| CN111461312B true CN111461312B (en) | 2022-04-22 |
Family
ID=71682473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010225486.4A Active CN111461312B (en) | 2020-03-26 | 2020-03-26 | Random neuron discarding based on memristor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461312B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906880B (en) * | 2021-04-08 | 2022-04-26 | 华中科技大学 | Adaptive neuron circuit based on memristor |
| CN114461178A (en) * | 2022-02-15 | 2022-05-10 | 清华大学 | Random number generator, electronic device and operation method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105229675A (en) * | 2013-05-21 | 2016-01-06 | 高通股份有限公司 | The hardware-efficient of shunt peaking realizes |
| CN109447250A (en) * | 2018-09-14 | 2019-03-08 | 华中科技大学 | A kind of artificial neuron based on battery effect in memristor |
| WO2019100036A1 (en) * | 2017-11-20 | 2019-05-23 | The Regents Of The University Of California | Memristive neural network computing engine using cmos-compatible charge-trap-transistor (ctt) |
-
2020
- 2020-03-26 CN CN202010225486.4A patent/CN111461312B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105229675A (en) * | 2013-05-21 | 2016-01-06 | 高通股份有限公司 | The hardware-efficient of shunt peaking realizes |
| WO2019100036A1 (en) * | 2017-11-20 | 2019-05-23 | The Regents Of The University Of California | Memristive neural network computing engine using cmos-compatible charge-trap-transistor (ctt) |
| CN109447250A (en) * | 2018-09-14 | 2019-03-08 | 华中科技大学 | A kind of artificial neuron based on battery effect in memristor |
Non-Patent Citations (2)
| Title |
|---|
| Quasi-Hodgkin–Huxley Neurons with Leaky Integrate-and-Fire Functions Physically Realized with Memristive Devices;He-Ming Huang等;《pubmed.gov》;20181120;第1-8页 * |
| 杨蕊等.忆阻类脑器件及其神经形态运算.《 TFC’19第十五届全国薄膜技术学术研讨会》.2019,第1页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111461312A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109447250B (en) | Artificial neuron based on battery effect in memristor | |
| Wu et al. | Full imitation of synaptic metaplasticity based on memristor devices | |
| Prezioso et al. | Modeling and implementation of firing-rate neuromorphic-network classifiers with bilayer Pt/Al2O3/TiO2− x/Pt memristors | |
| US11586884B2 (en) | Artificial neurons using diffusive memristor | |
| US10740672B2 (en) | Capacitative artificial neural networks | |
| US8447714B2 (en) | System for electronic learning synapse with spike-timing dependent plasticity using phase change memory | |
| US9208434B2 (en) | Neuromorphic system exploiting the intrinsic characteristics of memory cells | |
| US20180330236A1 (en) | Neural network processing system | |
| CN111461312B (en) | Random neuron discarding based on memristor | |
| Dong et al. | Convolutional neural networks based on RRAM devices for image recognition and online learning tasks | |
| Ambrogio et al. | Novel RRAM-enabled 1T1R synapse capable of low-power STDP via burst-mode communication and real-time unsupervised machine learning | |
| CN111882064B (en) | Method and system for realizing pulse neural network competition learning mechanism based on memristor | |
| CN111967589A (en) | Neuron analog circuit, driving method thereof and neural network device | |
| Weiss et al. | Stdp based online learning for a current-controlled memristive synapse | |
| Khanal et al. | Synaptic behaviour in ZnO–rGO composites thin film memristor | |
| Chen et al. | Competitive neural network circuit based on winner-take-all mechanism and online hebbian learning rule | |
| Jiang et al. | Circuit design of RRAM-based neuromorphic hardware systems for classification and modified Hebbian learning | |
| CN117151176A (en) | Synaptic array, operation circuit and operation method for neural network learning | |
| US11551073B2 (en) | Modulation device and method, artificial synapse comprising said modulation device, short term plasticity method in an artificial neural network comprising said artificial synapse | |
| KR102322131B1 (en) | Probabilistic memristive artificial synapse device and manufacturing method thereof | |
| CN111244270B (en) | Electronic device and method for simulating function of biological neuron | |
| Wu et al. | Enabling bio-plausible multi-level stdp using cmos neurons with dendrites and bistable rrams | |
| CN109978019B (en) | Image mode recognition analog and digital mixed memristor equipment and preparation thereof, and STDP learning rule and image mode recognition method are realized | |
| EP4002219A1 (en) | Circuit and method for spike time dependent plasticity | |
| Wei et al. | Artificial neuron with spike frequency adaptation based on mott memristor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |