CN111461312B - Random neuron discarding based on memristor - Google Patents
Random neuron discarding based on memristor Download PDFInfo
- Publication number
- CN111461312B CN111461312B CN202010225486.4A CN202010225486A CN111461312B CN 111461312 B CN111461312 B CN 111461312B CN 202010225486 A CN202010225486 A CN 202010225486A CN 111461312 B CN111461312 B CN 111461312B
- Authority
- CN
- China
- Prior art keywords
- signal
- memristor
- random
- discarding
- neuron
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000002569 neuron Anatomy 0.000 title claims abstract description 67
- 210000000225 synapse Anatomy 0.000 claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 19
- 230000005284 excitation Effects 0.000 claims abstract description 7
- 238000013528 artificial neural network Methods 0.000 abstract description 21
- 230000008569 process Effects 0.000 abstract description 13
- 238000012549 training Methods 0.000 abstract description 11
- 239000004065 semiconductor Substances 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 8
- BASFCYQUMIYNBI-UHFFFAOYSA-N platinum Substances [Pt] BASFCYQUMIYNBI-UHFFFAOYSA-N 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 238000001755 magnetron sputter deposition Methods 0.000 description 3
- 229910052709 silver Inorganic materials 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 239000004332 silver Substances 0.000 description 2
- 230000000946 synaptic effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 229910021421 monocrystalline silicon Inorganic materials 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- BPUBBGLMJRNUCC-UHFFFAOYSA-N oxygen(2-);tantalum(5+) Chemical compound [O-2].[O-2].[O-2].[O-2].[O-2].[Ta+5].[Ta+5] BPUBBGLMJRNUCC-UHFFFAOYSA-N 0.000 description 1
- 229910052697 platinum Inorganic materials 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004544 sputter deposition Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 229910052715 tantalum Inorganic materials 0.000 description 1
- PBCFLUZVCVVTBY-UHFFFAOYSA-N tantalum pentoxide Inorganic materials O=[Ta](=O)O[Ta](=O)=O PBCFLUZVCVVTBY-UHFFFAOYSA-N 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Neurology (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
本发明属于半导体信息相关技术领域，并公开了一种基于忆阻器随机丢弃神经元。该随机丢弃神经元用于对接受到的信号进行取舍，其中设置有丢弃控制单元和MOS开关，丢弃控制单元用于将接受到的信号转化控制指令，以此控制MOS开关的开关，其包括忆阻器、分压电阻、比较器和寄存器，忆阻器用于接受激发信号并转化为随机电流信号，分压电阻用于将随机电流信号转化为随机电压信号，比较器用于将随机电压信号与预设阈值进行比较，寄存器用于接受来自比较器的信号并将其作为控制指令传递给MOS开关，以此控制MOS的开关。通过本发明，提高神经网络的识别精度，降低对硬件人工突触的要求，解决神经网络训练过程中的过拟合与非线性权重更新的问题。
The invention belongs to the technical field of semiconductor information, and discloses a method of randomly discarding neurons based on a memristor. The random discarding neuron is used to select the received signal, wherein a discarding control unit and a MOS switch are provided, and the discarding control unit is used to convert the received signal into a control instruction, so as to control the switch of the MOS switch, which includes a memristor resistor, voltage divider, comparator and register, the memristor is used to receive the excitation signal and convert it into a random current signal, the voltage divider is used to convert the random current signal into a random voltage signal, and the comparator is used to convert the random voltage signal with the preset The threshold value is compared, and the register is used to receive the signal from the comparator and pass it to the MOS switch as a control command, so as to control the switch of the MOS. Through the invention, the recognition accuracy of the neural network is improved, the requirement for the hardware artificial synapse is reduced, and the problems of overfitting and nonlinear weight update in the training process of the neural network are solved.
Description
技术领域technical field
本发明属于半导体信息相关技术领域，更具体地，涉及一种基于忆阻器随机丢弃神经元。The invention belongs to the technical field related to semiconductor information, and more particularly, relates to a method of randomly discarding neurons based on memristors.
背景技术Background technique
随着基于软件的人工神经网络取得了巨大成功，为了进一步提升神经网络的使用效率，目前已经开始研究用于人工神经网络的硬件加速器，以达到增加计算效率、降低运算功耗的目的。With the great success of software-based artificial neural networks, in order to further improve the use efficiency of neural networks, hardware accelerators for artificial neural networks have been studied to achieve the purpose of increasing computing efficiency and reducing computing power consumption.
现阶段，本领域相关技术人员已经做了一些研究，如用一个具有简单三明治结构的忆阻器就可以模拟突触的大部分功能，在人工神经元的作用下表现出忆阻器电导的连续增加或者降低，并实现了对一些神经系统重要学习规则的模拟。然而，由于算法的不断进步，对人工神经元的功能提出了更多的要求(例如随机丢弃神经元)，用于提高神经网络在特殊状态下的准确性，对应的硬件实现也需要不断改进。相应地，本领域存在着发展一种能够实现丢弃功能的人工神经元的技术需求。At this stage, those skilled in the art have done some research. For example, a memristor with a simple sandwich structure can simulate most of the functions of the synapse, and show the continuity of the memristor conductance under the action of artificial neurons. Increase or decrease, and simulate some important learning rules of the nervous system. However, due to the continuous progress of algorithms, more requirements are placed on the functions of artificial neurons (such as randomly discarding neurons) to improve the accuracy of neural networks in special states, and the corresponding hardware implementation also needs to be continuously improved. Accordingly, there is a technical need in the art to develop an artificial neuron capable of discarding functions.
发明内容SUMMARY OF THE INVENTION
针对现有技术的以上缺陷或改进需求，本发明提供了一种基于忆阻器随机丢弃神经元，其中对其关键组件丢弃控制单元的结构和功能的设计，使得在使用本发明中的丢弃神经元时，实现随机丢弃部分的神经元，提高神经网络的识别精度，降低对硬件人工突触的要求，进而解决神经网络训练过程中的过拟合问题和硬件人工突触的非线性权重更新的问题。In view of the above defects or improvement needs of the prior art, the present invention provides a memristor-based random discarding of neurons, wherein the structure and function of the key components of the discarding control unit are designed, so that the discarded neurons in the present invention are used in the design of the structure and function. It realizes the random discarding of some neurons, improves the recognition accuracy of the neural network, reduces the requirements for hardware artificial synapses, and solves the problem of overfitting in the process of neural network training and the nonlinear weight update of hardware artificial synapses. question.
为实现上述目的，本发明提供了一种基于忆阻器随机丢弃神经元，该随机丢弃神经元用于对接受到的信号进行取舍，其中，In order to achieve the above object, the present invention provides a memristor-based random discarding of neurons, which is used to select received signals, wherein,
所述随机丢弃神经元中设置有丢弃控制单元和MOS开关，所述丢弃控制单元用于将接受到的信号转化控制指令，以此控制所述MOS开关的开关；The random discarding neuron is provided with a discarding control unit and a MOS switch, and the discarding control unit is used to convert the received signal into a control instruction, thereby controlling the switching of the MOS switch;
所述丢弃控制单元包括忆阻器、分压电阻、比较器和寄存器，所述忆阻器用于将接受激发信号并将该激发信号转化为随机电流信号，所述分压电阻与所述忆阻器连接，用于将来自所述忆阻器的随机电流信号转化为随机电压信号，所述比较器与所述分压电阻连接，用于将来自所述分压电阻的随机电压信号与预设阈值进行比较，当所述随机电压信号大于预设阈值时，所述比较器输出一个低电平信号，当所述随机电压信号小于预设阈值时，所述比较器输出一个高电平信号；The discarding control unit includes a memristor, a voltage dividing resistor, a comparator and a register. The memristor is used to receive an excitation signal and convert the excitation signal into a random current signal. The voltage dividing resistor is connected to the memristor. The comparator is connected to convert the random current signal from the memristor into a random voltage signal, the comparator is connected to the voltage dividing resistor, and is used to convert the random voltage signal from the voltage dividing resistor with a preset thresholds are compared, when the random voltage signal is greater than the preset threshold, the comparator outputs a low-level signal, and when the random voltage signal is less than the preset threshold, the comparator outputs a high-level signal;
所述寄存器与所述比较器连接，用于接受来自所述比较器的信号，并将当前接受的信号作为控制指令传递给所述MOS开关，当所述寄存器接受到低电平信号时，所述MOS开关导通，信号通过所述随机丢弃神经元传输到下一个神经元，当所述寄存器接受到高电平信号时，所述MOS开关断开，信号不能通过所述随机丢弃神经元传输到下一个神经元，被丢弃。The register is connected to the comparator for accepting the signal from the comparator, and transmits the currently accepted signal as a control command to the MOS switch. When the register receives a low level signal, the The MOS switch is turned on, and the signal is transmitted to the next neuron through the randomly discarded neuron. When the register receives a high-level signal, the MOS switch is turned off, and the signal cannot be transmitted through the randomly discarded neuron. to the next neuron, which is discarded.
进一步优选地，所述分压电阻一端与所述忆阻器连接，另一端接地。Further preferably, one end of the voltage dividing resistor is connected to the memristor, and the other end is grounded.
进一步优选地，所述随机丢弃神经元中还包括人工突触，所述人工突触用于将该随机丢弃神经元接受的信号进行预处理。Further preferably, the randomly discarded neurons further include artificial synapses, and the artificial synapses are used to preprocess the signals received by the randomly discarded neurons.
进一步优选地，所述随机丢弃神经元还包括信号接受发放单元，该信号接受发放单元一端与所述人工突触连接，另一端与所述丢弃控制单元连接，其接受来自人工突触预处理后的信号并传递给所述丢弃控制单元。Further preferably, the randomly discarded neuron also includes a signal receiving and issuing unit, one end of the signal receiving and issuing unit is connected with the artificial synapse, and the other end is connected with the discarding control unit, which receives the pre-processing from the artificial synapse. signal and pass it to the discard control unit.
进一步优选地，所述人工突触是由电阻态连续可调的非易失性忆阻器组成。Further preferably, the artificial synapse is composed of a non-volatile memristor whose resistance state is continuously adjustable.
进一步优选地，所述预设阈值通过与所述比较器连接的阈值电压获得。Further preferably, the preset threshold is obtained by a threshold voltage connected to the comparator.
进一步优选地，所述寄存器呈双稳态的特性，其接受到一个信号后，输出一个持续的信号，直至当前信号被下一次接收信号所取代。Further preferably, the register has a bistable characteristic, after receiving a signal, it outputs a continuous signal until the current signal is replaced by the next received signal.
总体而言，通过本发明所构思的以上技术方案与现有技术相比，具备下列有益效果：In general, compared with the prior art, the above technical solutions conceived by the present invention have the following beneficial effects:
1.本发明提供的随机丢弃神经元，通过对丢弃控制单元的设计，可实现神经元的随机丢弃，从而导致每次训练时激活不同的神经元，相互组合成不同的神经网络，最终得到的神经网络是多个不同的神经网络的平均结果，这种平均过程有利于弱化无效的特征，从而解决硬件神经网络使用中存在的过拟合间题；另一方面，基于忆阻器的硬件人工突触的权重具有非线性更新的特征，训练过程中频繁地非线性更新会降低神经网络的准确率，由于随机丢弃神经元并非参与每一次训练，所以与之连接的人工突触的更新次数降低，因此随机丢弃神经元降低了人工突触的非线性更新所产生的影响；1. The random discarding of neurons provided by the present invention can realize the random discarding of neurons through the design of the discarding control unit, so that different neurons are activated during each training, and they are combined into different neural networks. The neural network is the average result of multiple different neural networks. This averaging process is beneficial to weaken invalid features, thereby solving the problem of overfitting in the use of hardware neural networks. On the other hand, the hardware artificial The weight of the synapse has the characteristics of nonlinear update. Frequent nonlinear update during the training process will reduce the accuracy of the neural network. Since the randomly discarded neurons do not participate in every training, the number of updates of the artificial synapses connected to it is reduced. , so randomly dropping neurons reduces the impact of nonlinear updates of artificial synapses;
2.本发明提供的随机丢弃神经元在训练过程中有一定概率被随机抑制，在使用过程中被激发。其中随机抑制的过程可以用一个具有随机阻变效应的神经元来发出控制信号，决定随机丢弃神经元是否被激发，且该忆阻器在阻变后可以自发回到初始状态，无需额外的置零操作。2. The randomly discarded neurons provided by the present invention have a certain probability to be randomly suppressed during the training process, and are excited during the use process. In the process of random inhibition, a neuron with random resistive effect can be used to send a control signal to decide whether the randomly discarded neuron is excited or not, and the memristor can spontaneously return to the initial state after resistive switching without additional setting. Zero operations.
附图说明Description of drawings
图1是按照本发明的优选实施例所构建的基于忆阻器的随机丢弃神经元的结构示意图；FIG. 1 is a schematic structural diagram of a randomly discarded neuron based on a memristor constructed according to a preferred embodiment of the present invention;
图2中按照本发明的优选实施例所构建的忆阻器的电学性能图，其中，(a)是忆阻器电流-电压响应曲线，(b)是忆阻器对电脉冲的电流响应曲线，(c)是忆阻器对多个电脉冲的电流响应曲线，(d)是忆阻器对不同幅值的电脉冲的响应分布；Figure 2 shows the electrical performance diagram of the memristor constructed according to the preferred embodiment of the present invention, wherein (a) is the current-voltage response curve of the memristor, and (b) is the current response curve of the memristor to electrical pulses , (c) is the current response curve of the memristor to multiple electrical pulses, (d) is the response distribution of the memristor to electrical pulses of different amplitudes;
图3是按照本发明的优选实施例所构建的随机丢弃神经元构建的神经网络；Fig. 3 is a neural network constructed by randomly discarding neurons constructed according to a preferred embodiment of the present invention;
图4是按照本发明的优选实施例所构建的随机丢弃神经元对手写体数字的识别结果，其中，(a)是使用实际忆阻人工突触的识别准确率，(b)是使用理想人工突触的识别准确率。4 is the recognition result of handwritten digits by randomly discarding neurons constructed according to the preferred embodiment of the present invention, wherein (a) is the recognition accuracy using actual memristive artificial synapses, (b) is the recognition accuracy using ideal artificial synapses touch recognition accuracy.
具体实施方式Detailed ways
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。此外，下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
如图1所示，一种基于忆阻器的随机丢弃神经元，随机丢弃神经元中设置有人工突触、信号接收发放单元、丢弃控制单元和MOS开关，人工突触的输出端连接于信号接收发放单元的输入端，信号接收发放单元的输出端连接于MOS开关的信号一端，丢弃控制单元发出MOS开关的控制信号，MOS开关的另一端端与下一级的神经元的突触相连接；本实施例中，人工突触是M2、信号接收发放单元、忆阻器是M1、比较器是OP1、寄存器是S2，MOS开关是S1。As shown in Figure 1, a randomly discarded neuron based on memristor is provided with an artificial synapse, a signal receiving and issuing unit, a discarding control unit and a MOS switch, and the output end of the artificial synapse is connected to the signal The input end of the receiving and distributing unit, the output end of the signal receiving and distributing unit is connected to one end of the signal of the MOS switch, the control signal of the MOS switch is sent by the discarding control unit, and the other end of the MOS switch is connected to the synapse of the neuron at the next level. In this embodiment, the artificial synapse is M2, the signal receiving and distributing unit, the memristor is M1, the comparator is OP1, the register is S2, and the MOS switch is S1.
丢弃控制单元包括忆阻器、分压电阻、比较器和寄存器，该忆阻器的一端连接信号源，另一端分两路，一路经分压电阻接地，另一路作为比较器的负向输入，比较器的正向输入比较阈值，比较器的输出连接寄存器的控制端，寄存器的输出端作为丢弃控制单元的输出；该忆阻器具有随机阻变效应。丢弃控制单元负责发出MOS开关的控制信号，忆阻器为具有随机阻变效应的完全易失性忆阻器。The discarding control unit includes a memristor, a voltage dividing resistor, a comparator and a register. One end of the memristor is connected to the signal source, the other end is divided into two paths, one is grounded through the voltage dividing resistor, and the other is used as the negative input of the comparator. The positive input of the comparator compares the threshold, the output of the comparator is connected to the control terminal of the register, and the output terminal of the register is used as the output of the discarding control unit; the memristor has random resistance effect. The discard control unit is responsible for sending out the control signal of the MOS switch, and the memristor is a completely volatile memristor with random resistance switching effect.
进一步地，人工突触用于各个随机丢弃神经元之间的连接，其接收上一级神经元传来的电信号，并根据自身的电导按照比例将信号向后级神经元传播；人工突触由电阻态连续可调的非易失性忆阻器组成，各个忆阻器的一端通过导线连接于上一级神经元，另一端连接于信号接收发放单元的输入端；人工突触用于对接受的信号进行预处理，预处理可以为对接受的信号进行运算，过滤或者其他处理方式。Further, the artificial synapse is used for the connection between each randomly discarded neuron, which receives the electrical signal from the neuron at the upper level, and propagates the signal to the neuron at the later level according to its own conductance in proportion; artificial synapse. It consists of non-volatile memristors whose resistance state is continuously adjustable. One end of each memristor is connected to the upper-level neuron through a wire, and the other end is connected to the input end of the signal receiving and distributing unit; artificial synapses are used to The received signal is preprocessed, and the preprocessing may be operation, filtering or other processing methods on the received signal.
进一步地，信号接收发放单元用于接收经人工突触处理过的神经信号，输出端与MOS开关相连接，负责发出检测信号或突触权重调节信号。Further, the signal receiving and distributing unit is used for receiving the nerve signal processed by the artificial synapse, and the output end is connected with the MOS switch, and is responsible for sending out the detection signal or the synaptic weight adjustment signal.
进一步地，丢弃控制单元的输出端与MOS开关的控制端相连接，用于控制MOS开关的状态，在训练过程中，随机发出令MOS开关处于导通状态的信号；在检测过程，始终保持MOS开关处于导通状态。Further, the output terminal of the discarding control unit is connected to the control terminal of the MOS switch, and is used to control the state of the MOS switch. During the training process, a signal to make the MOS switch in the conducting state is randomly issued; during the detection process, the MOS switch is always kept on. The switch is in the ON state.
进一步地，忆阻器为具有随机阻变效应的忆阻器，一端与激发信号相连，另一端与分压电阻的一端相连接，分压电阻另一端接地。Further, the memristor is a memristor with random resistance effect, one end is connected to the excitation signal, the other end is connected to one end of a voltage dividing resistor, and the other end of the voltage dividing resistor is grounded.
进一步地，比较器的负向输入端与忆阻器和分压电阻器相连接的一端相连接，正向输入端与一阈值电压相连接，输出端连接至寄存器的控制端。Further, the negative input end of the comparator is connected to the end connected with the memristor and the voltage dividing resistor, the positive input end is connected to a threshold voltage, and the output end is connected to the control end of the register.
进一步地，寄存器的置零端与一置零信号发生器相连，输出端连接至MOS开关的控制端。当忆阻器收到激发信号时，会根据一定概率阻变到不同的低阻态，导致分压电阻上的分压变化，当分压电阻上的分压大于阈值电压时，比较器的输出端输出低电平信号，打开寄存器的开关，进而使MOS开关处于导通状态，该神经元的信号可以传输到下一级神经元上，处于激活状态；当分压电阻的分压不到阈值电压时，相关信号无法向后输出，导致MOS开关处于截止状态，神经元的信号无法向后传输，处于被丢弃状态。Further, the zero-setting terminal of the register is connected to a zero-setting signal generator, and the output terminal is connected to the control terminal of the MOS switch. When the memristor receives the excitation signal, it will change to different low-resistance states according to a certain probability, resulting in a change in the voltage divider on the voltage divider resistor. When the voltage divider on the voltage divider resistor is greater than the threshold voltage, the output terminal of the comparator Output a low-level signal, turn on the switch of the register, and then make the MOS switch in an on state, the neuron's signal can be transmitted to the next-level neuron and is in an active state; when the voltage division of the voltage divider resistor is less than the threshold voltage , the related signal cannot be output backward, resulting in the cut-off state of the MOS switch, the signal of the neuron cannot be transmitted backward, and is in the state of being discarded.
寄存器体现出双稳态的特性，其接收到一个低电平信号时会输出一个持续的高电平信号，使所述MOS开关处于导通状态，即寄存器将当前接受的比较结果存储，直至当前的比较结果被下一次的比较结果替代。The register reflects the characteristics of bistable. When it receives a low-level signal, it will output a continuous high-level signal, so that the MOS switch is in a conducting state, that is, the register stores the currently accepted comparison result until the current The result of the comparison is replaced by the result of the next comparison.
随机丢弃神经元能够实现在训练过程中按一定概率断开神经元与后级神经元之间的联系，以实现抑制人工神经网络在训练过程中出现的过拟合现象以及降低对人工突触使用要求的功能。Randomly discarding neurons can disconnect the connection between neurons and subsequent neurons with a certain probability during the training process, so as to suppress the overfitting phenomenon of artificial neural networks during the training process and reduce the use of artificial synapses. requested function.
本实施方式中，人工突触阵列采用两种不同的忆阻器阵列，二者都是具有非易失性的连续可调器件，一种是在权重更新过程中阻态非线性变化的实际器件，另一种是完全线性更新的理想器件，用于比较随机丢弃神经元对非理想器件的优化效果。In this embodiment, the artificial synapse array adopts two different memristor arrays, both of which are non-volatile and continuously adjustable devices, and the other is an actual device whose resistance state changes nonlinearly during the weight update process. , the other is an ideal device with fully linear updates, which is used to compare the optimization effect of randomly dropping neurons on a non-ideal device.
具有随机阻变效应的忆阻器M2用于体现神经元的概率特性。如图2所示，图2中(a)～(d)所示，忆阻器M2电阻变化在短时间内完全易失且具其低阻态的阻值具有随机效应，具有随机阻变效应的忆阻器在接收脉冲后的很短时间内，自发地衰减使其恢复到高阻态，等待下一次地选择。本实施方式中，具有随机阻变效应地忆阻器采用Ag/Ta2O5：Ag/Pt(银/银掺杂的五氧化二钽/铂)完全易失性忆阻器；当施加正向扫描电压超过阈值时，忆阻器的电流突然增加至限流，忆阻器在阈值分布上表现出了明显的随机特性。脉冲测试的性能表明，忆阻器的低阻态也存在着明显的随机分布，可以用于随机信号的实现。该忆阻器的制作流程为：在磁控溅射设备中，以覆有一定厚度的氧化层的单晶硅片作为基体，首先镀一层Ti作为为黏附层，随后镀Pt作为下电极；再通过磁控溅射共溅射的方法同时沉积Ag和Ta2O5，得到Ag掺杂的Ta2O5；最后通过磁控溅射沉积上电极Ag，以制备得到具有随机阻变效应的完全易失性的Ag/Ta2O5：Ag/Pt忆阻器。The memristor M2 with random resistive effect is used to reflect the probabilistic properties of neurons. As shown in Figure 2, as shown in (a) to (d) of Figure 2, the resistance change of the memristor M2 is completely volatile in a short period of time, and the resistance value with its low resistance state has a random effect, which has a random resistance change effect. The memristor attenuates spontaneously within a short time after receiving the pulse, returning it to a high-impedance state, waiting for the next ground selection. In this embodiment, the memristor with random resistance switching effect adopts Ag/Ta 2 O 5 : Ag/Pt (silver/silver doped tantalum pentoxide/platinum) fully volatile memristor; when positive When the scanning voltage exceeds the threshold, the current of the memristor suddenly increases to the current limit, and the memristor exhibits obvious random characteristics in the threshold distribution. The performance of the pulse test shows that the low resistance state of the memristor also has an obvious random distribution, which can be used for the realization of random signals. The manufacturing process of the memristor is as follows: in the magnetron sputtering equipment, a single crystal silicon wafer covered with an oxide layer of a certain thickness is used as the substrate, firstly, a layer of Ti is plated as an adhesion layer, and then Pt is plated as a lower electrode; Then, Ag and Ta 2 O 5 are simultaneously deposited by the method of magnetron sputtering and co-sputtering to obtain Ag-doped Ta 2 O 5 ; finally, the upper electrode Ag is deposited by magnetron sputtering to prepare a stochastic resistive effect. Fully volatile Ag/Ta 2 O 5 : Ag/Pt memristor.
比较器的负向输入端接分压电阻R1的分压，正向输入端接阈值电压Vth，其输出端与寄存器S2的控制端相连接，寄存器的输出端与MOS开关S1相连接。比较器用于比较分压电阻R1对地电压与阈值电压Vth的大小，当分压电阻R1对地的电压大于阈值电压Vth时，比较器OP1输出低电平以用于激励寄存器S2输出持续的高电平，用于激励MOS开关S1，进而激活相应的神经元与突触、后级神经元之间的联系。The negative input terminal of the comparator is connected to the voltage division of the voltage dividing resistor R1, the positive input terminal is connected to the threshold voltage V th , the output terminal of the comparator is connected to the control terminal of the register S2, and the output terminal of the register is connected to the MOS switch S1. The comparator is used to compare the voltage of the voltage dividing resistor R1 to ground and the threshold voltage V th . When the voltage of the voltage dividing resistor R1 to the ground is greater than the threshold voltage V th , the comparator OP1 outputs a low level to stimulate the register S2 to output a continuous output voltage. A high level is used to excite the MOS switch S1, thereby activating the connection between the corresponding neuron and the synapse and subsequent neurons.
将随机丢弃神经元按照神经网络的算法相互连接，如图3所示，可以利用硬件实现神经网络的识别功能，在本实施方式中，采用软件仿真的方式实现对手写体数字的识别功能。结果如图4中(a)和(b)所示，采用随机丢弃神经元的模式后，神经网络的识别率得到了明显提升，而且明显拉近了非理想突触和理想突触之间的差距。The randomly discarded neurons are connected to each other according to the algorithm of the neural network, as shown in FIG. 3 , the recognition function of the neural network can be realized by hardware. In this embodiment, the recognition function of handwritten digits is realized by software simulation. The results are shown in (a) and (b) in Figure 4. After adopting the mode of randomly discarding neurons, the recognition rate of the neural network has been significantly improved, and the gap between non-ideal synapses and ideal synapses has been significantly narrowed. gap.
本发明提供的基于忆阻器的随机丢弃神经元，突触阵列的忆阻器为非易失性忆阻器，电阻、比较器、MOS开关等等均为成熟商业器件或模块。通过对所选器件和构建的人工神经网络进行仿真，并具有突触基本单元。利用随机丢弃神经元完成神经网络的训练过程时，可以较好得抑制神经网络在训练集较小时产生得过拟合现象，同时减小因硬件人工突触的非理想特性对识别精度产生的影响。The memristor-based random discarding of neurons provided by the present invention, the memristor of the synapse array is a non-volatile memristor, and the resistors, comparators, MOS switches and the like are all mature commercial devices or modules. By simulating selected devices and constructed artificial neural networks with synaptic basic units. When using randomly discarded neurons to complete the training process of the neural network, it can better suppress the overfitting phenomenon of the neural network when the training set is small, and at the same time reduce the impact of the non-ideal characteristics of hardware artificial synapses on the recognition accuracy. .
本领域的技术人员容易理解，以上仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be Included in the protection scope of the present invention.
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010225486.4A CN111461312B (en) | 2020-03-26 | 2020-03-26 | Random neuron discarding based on memristor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010225486.4A CN111461312B (en) | 2020-03-26 | 2020-03-26 | Random neuron discarding based on memristor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111461312A CN111461312A (en) | 2020-07-28 |
| CN111461312B true CN111461312B (en) | 2022-04-22 |
Family
ID=71682473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010225486.4A Active CN111461312B (en) | 2020-03-26 | 2020-03-26 | Random neuron discarding based on memristor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461312B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906880B (en) * | 2021-04-08 | 2022-04-26 | 华中科技大学 | Adaptive neuron circuit based on memristor |
| CN114461178A (en) * | 2022-02-15 | 2022-05-10 | 清华大学 | Random number generator, electronic device and operation method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105229675A (en) * | 2013-05-21 | 2016-01-06 | 高通股份有限公司 | The hardware-efficient of shunt peaking realizes |
| CN109447250A (en) * | 2018-09-14 | 2019-03-08 | 华中科技大学 | A kind of artificial neuron based on battery effect in memristor |
| WO2019100036A1 (en) * | 2017-11-20 | 2019-05-23 | The Regents Of The University Of California | Memristive neural network computing engine using cmos-compatible charge-trap-transistor (ctt) |
-
2020
- 2020-03-26 CN CN202010225486.4A patent/CN111461312B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105229675A (en) * | 2013-05-21 | 2016-01-06 | 高通股份有限公司 | The hardware-efficient of shunt peaking realizes |
| WO2019100036A1 (en) * | 2017-11-20 | 2019-05-23 | The Regents Of The University Of California | Memristive neural network computing engine using cmos-compatible charge-trap-transistor (ctt) |
| CN109447250A (en) * | 2018-09-14 | 2019-03-08 | 华中科技大学 | A kind of artificial neuron based on battery effect in memristor |
Non-Patent Citations (2)
| Title |
|---|
| Quasi-Hodgkin–Huxley Neurons with Leaky Integrate-and-Fire Functions Physically Realized with Memristive Devices;He-Ming Huang等;《pubmed.gov》;20181120;第1-8页 * |
| 杨蕊等.忆阻类脑器件及其神经形态运算.《 TFC’19第十五届全国薄膜技术学术研讨会》.2019,第1页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111461312A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106845634B (en) | A kind of neuron circuit based on memory resistor | |
| US8447714B2 (en) | System for electronic learning synapse with spike-timing dependent plasticity using phase change memory | |
| US11586884B2 (en) | Artificial neurons using diffusive memristor | |
| US10275706B2 (en) | Neuristor-based reservoir computing devices | |
| Zhang et al. | Synaptic characteristics of Ag/AgInSbTe/Ta-based memristor for pattern recognition applications | |
| US9208434B2 (en) | Neuromorphic system exploiting the intrinsic characteristics of memory cells | |
| US10740672B2 (en) | Capacitative artificial neural networks | |
| Prezioso et al. | Modeling and implementation of firing-rate neuromorphic-network classifiers with bilayer Pt/Al2O3/TiO2− x/Pt memristors | |
| CN109447250B (en) | An artificial neuron based on the battery effect in memristors | |
| CN111461312B (en) | Random neuron discarding based on memristor | |
| CN110289317B (en) | Ferroelectric graphene transistor, complementary type synapse device based on ferroelectric graphene transistor and regulation and control method | |
| CN107909146B (en) | Neuron circuit based on volatile threshold transition device | |
| CN112906880B (en) | Adaptive neuron circuit based on memristor | |
| CN111967589A (en) | Neuron analog circuit, driving method thereof and neural network device | |
| US20130173515A1 (en) | Electronic synapses from stochastic binary memory devices | |
| Zarudnyi et al. | Spike-timing dependent plasticity in unipolar silicon oxide RRAM devices | |
| Khanal et al. | Synaptic behaviour in ZnO–rGO composites thin film memristor | |
| Chen et al. | Optimal tuning of memristor conductance variation in spiking neural networks for online unsupervised learning | |
| Singha et al. | Analog memristive time dependent learning using discrete nanoscale RRAM devices | |
| Kiani et al. | Electroforming-free BiFeO 3 switches for neuromorphic computing: Spike-timing dependent plasticity (STDP) and cycle-number dependent plasticity (CNDP) | |
| KR102322131B1 (en) | Probabilistic memristive artificial synapse device and manufacturing method thereof | |
| CN115222029A (en) | Stable conductance obtaining method, regulation and control method and application of memristor | |
| CN111244270B (en) | An electronic device and method for simulating the function of biological neurons | |
| Wu et al. | Enabling bio-plausible multi-level stdp using cmos neurons with dendrites and bistable rrams | |
| Chang et al. | Modeling and implementation of oxide memristors for neuromorphic applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |