TW202422557A - Verification method and system in artificial neural network array - Google Patents

Abstract

Numerous examples are disclosed of verification circuitry and associated methods in an artificial neural network. In one example, a system comprises a vector-by-matrix multiplication array comprising a plurality of non-volatile memory cells arranged in rows and columns, the non-volatile memory cells respectively capable of storing one of N possible levels corresponding to one of N possible currents, and a plurality of output blocks to receive current from respective columns of the vector-by-matrix multiplication array and generate voltages during a verify operation of the vector-by-matrix multiplication and generate digital outputs during a read operation of the vector-by-matrix multiplication.

Description

Verification method and system in artificial neural network array

This application claims priority to U.S. Patent Application No. 18/080,545, filed on December 13, 2022, entitled “VERIFICATION METHODS AND SYSTEMS IN ARTIFICIAL NEURAL NETWORK ARRAYS,” and U.S. Provisional Patent Application No. 63/409,142, filed on September 22, 2022, entitled “VERIFICATION METHODS AND SYSTEMS IN ARTIFICIAL NEURAL NETWORK ARRAYS.”

Many examples of verification circuits and related methods in artificial neural networks are revealed.

Artificial neural networks simulate biological neural networks (the central nervous system of animals, especially the brain) and are used to estimate or approximate functions that may depend on a large number of inputs and are usually unknown. Artificial neural networks typically consist of layers of interconnected "neurons" that can exchange information with each other.

Figure 1 illustrates an artificial neural network, where circles represent neuron inputs or layers. Connections (called synapses) are represented by arrows and have numerical weights that can be adjusted based on experience. This allows the neural network to adapt to the inputs and be able to learn. Typically, a neural network includes a layer of multiple inputs. There is usually one or more intermediate layers of neurons and a neuron output layer that provides the output of the neural network. The neurons at each level make decisions based on the data received from the synapses, either individually or collectively.

One of the main challenges in developing artificial neural networks for high-performance information processing is the lack of adequate hardware technology. In practice, practical neural networks rely on very large numbers of synapses to enable high connectivity between neurons (i.e., very high computational parallelism). In principle, such complexity could be achieved using digital supercomputers or clusters of dedicated graphics processing units. However, in addition to high cost, these approaches suffer from mediocre energy efficiency compared to biological networks, which consume very little energy, primarily because they perform low-precision analog computations. CMOS analog circuits have been used for artificial neural networks, but the synapses of most CMOS implementations are too large given the large number of neurons and synapses.

The applicant previously disclosed an artificial (analog) neural network in U.S. Patent Application Publication No. 2017/0337466A1, which is hereby incorporated by reference. The non-volatile memory array operates as an analog neural memory and includes non-volatile memory cells arranged in rows and columns. The neural network includes a plurality of first synapses configured to receive a plurality of first inputs and thereby generate a plurality of first outputs; and a plurality of first neurons configured to receive a plurality of first outputs. The plurality of first contacts include a plurality of memory cells, wherein each memory cell includes spaced-apart source and drain regions formed in a semiconductor substrate and having a channel region extending therebetween; a floating gate disposed over and insulated from a first portion of the channel region; and a non-floating gate disposed over and insulated from a second portion of the channel region. Each of the plurality of memory cells stores a weight value corresponding to a number of electrons on the floating gate. The plurality of memory cells multiply the plurality of first inputs by the stored weight values to generate a plurality of first outputs. Non-volatile memory cells

Non-volatile memory is well known. For example, U.S. Patent No. 5,029,130 (the "'130 patent") discloses an array of split-gate non-volatile memory cells (flash memory cells) and is incorporated herein by reference. Such a memory cell 210 is shown in FIG. 2 . Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12 and having a channel region 18 therebetween. A floating gate 20 is formed over and insulated from a first portion of the channel region 18 (and controls its conductivity), and is formed over a portion of the source region 14 . The word line terminal 22 (usually coupled to the word line) has a first portion and a second portion, wherein the first portion is disposed above and insulated from the second portion of the channel region 18 (and controls its conductivity), and the second portion extends upward and above the floating gate 20. The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. The bit line 24 is coupled to the drain region 16.

The memory cell 210 is erased (where electrons are removed from the floating gate) by applying a high positive voltage on the word line terminal 22 , which causes the electrons on the floating gate 20 to tunnel from the floating gate 20 through the intermediate insulator to the word line terminal 22 by Fowler-Nordheim tunneling.

The memory cell 210 is programmed by source side injection (SSI) with hot electrons by applying a positive voltage on the word line terminal 22 and a positive voltage on the source 14 (where electrons are placed on the floating gate). Electron flow will flow from the drain region 16 to the source region 14. When the electrons reach the gap between the word line terminal 22 and the floating gate 20, the electrons will accelerate and become hot. Due to the electrostatic attraction from the floating gate 20, some of the heated electrons will be injected onto the floating gate 20 through the gate oxide.

The memory cell 210 is read by applying a positive read voltage to the drain region 16 and the word line terminal 22 (which turns on the portion of the channel region 18 below the word line terminal). If the floating gate 20 is positively charged (i.e., erased electrons), the portion of the channel region 18 below the floating gate 20 is also turned on, and current will flow through the channel region 18, which is sensed as an erased state or state "1". If the floating gate 20 is negatively charged (i.e., programmed with electrons), the portion of the channel region below the floating gate 20 is mostly or completely turned off, and current will not flow (or almost not flow) through the channel region 18, which is sensed as a programmed state or state "0".

Table 1 describes typical voltage and current ranges that may be applied to the terminals of the memory cell 210 to perform read, erase, and program operations: Table 1: Operation of the flash memory cell 210 of FIG. 2 WL BL SL Read 2-3V 0.6-2V 0V Erase ~11-13V 0V 0V Programming 1-2V 10.5-3µA 9-10V

Other split gate memory cell configurations are other types of flash memory cells and are known. For example, FIG. 3 depicts a 4-gate memory cell 310 that includes a source region 14, a drain region 16, a floating gate 20 over a first portion of a channel region 18, a select gate 22 (typically coupled to a word line WL) over a second portion of the channel region 18, a control gate 28 over the floating gate 20, and an erase gate 30 over the source region 14. Such a configuration is described in U.S. Patent No. 6,747,310, which is incorporated herein by reference for all purposes. Here, except for the floating gate 20, all other gates are non-floating gates, which means they are electrically connected or connectable to a voltage source. Programming is performed by injecting heated electrons from the channel region 18 onto the floating gate 20. Erasing is performed by tunneling electrons from the floating gate 20 to the erase gate 30.

Table 2 depicts typical voltage and current ranges that may be applied to the terminals of the memory cell 310 to perform read, erase, and program operations: Table 2: Operation of the flash memory cell 310 of FIG. 3 WL/SG BL CG EG SL Read 1.0-2V 0.6-2V 0-2.6V 0-2.6V 0V Erase -0.5V/0V 0V 0V/-8V 8-12V 0V Programming 1V 0.1-1µA 8-11V 4.5-9V 4.5-5V

FIG4 depicts a 3-gate memory cell 410, which is another type of flash memory cell. Memory cell 410 is the same as memory cell 310 of FIG3, except that memory cell 410 does not have a separate control gate. Erase operations (erasing by use of the erase gate) and read operations are similar to those of FIG3, except that no control gate bias is applied. Programming operations are also performed without a control gate bias, and as a result, a higher voltage must be applied to the source line during programming operations to compensate for the lack of a control gate bias.

Table 3 describes typical voltage and current ranges that may be applied to the terminals of the memory cell 410 to perform read, erase, and program operations: Table 3: Operation of the flash memory cell 410 of FIG. 4 WL/SG BL EG SL Read 0.7-2.2V 0.6-2V 0-2.6V 0V Erase -0.5V/0V 0V 11.5V 0V Programming 1V 0.2-3µA 4.5V 7-9V

5 depicts a stacked gate memory cell 510, which is another type of flash memory cell. Memory cell 510 is similar to memory cell 210 of FIG. 2 except that floating gate 20 extends over the entire channel region 18 and control gate 22 (which will be coupled to the word line here) extends over floating gate 20 and is separated by an insulating layer (not shown). Erasing is accomplished by FN tunneling of electrons from FG to the substrate, programming is accomplished by channel hot electron (CHE) injection in the region between channel 18 and drain region 16 using electrons flowing from source region 14 to drain region 16, and the read operation is similar to the read operation of memory cell 210 with a higher control gate voltage.

Table 4 describes typical voltage ranges that may be applied to the terminals of the memory cell 510 and the substrate 12 to perform read, erase, and program operations: Table 4: Operation of the flash memory cell 510 of FIG. 5 CG BL SL Substrate Read 2-5V 0.6–2V 0V 0V Erase -8 to -10V/0V FLT FLT 8-10V/15-20V Programming 8-12V 3-5V 0V 0V

The methods and means described herein may be applied to other non-volatile memory technologies, such as, but not limited to, FINFET split gate flash memory or stacked gate flash memory, NAND flash memory, SONOS (silicon-oxide-nitride-oxide-silicon, charge trapping in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trapping in nitride), ReRAM (resistive RAM), PCM (phase change memory), MRAM (magnetic RAM), FeRAM (ferroelectric RAM), CT (charge trapping) memory, CN (carbon tube) memory, OTP (two-level or multi-level one-time programmable) and CeRAM (correlated electronic RAM).

In order to utilize a memory array including one of the above-described types of non-volatile memory cells in an artificial neural network, two modifications are implemented. First, as further described below, the circuitry is configured so that each memory cell can be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array. Second, continuous (analog) programming of memory cells is provided.

Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be changed from a completely erased state to a fully programmed state and vice versa independently and continuously with minimal disturbance to other memory cells. This means that the cell storage is actually analog, or at least can store one of many discrete values (e.g., 16 or 64 different values), which allows very precise and individual adjustments to all memory cells in the memory array, and this makes memory arrays very suitable for storing and fine-tuning the synaptic weights of neural networks. Neural Network Using Arrays of Non-Volatile Memory Cells

FIG6 conceptually illustrates a non-limiting example of a neural network utilizing the non-volatile memory array of the present example. This example uses the non-volatile memory array neural network for a face recognition application, but any other suitable application may be implemented using a non-volatile memory array-based neural network.

S0 is the input layer, which for this example is a 32×32 pixel RGB image with 5 bits of precision (i.e., three 32×32 pixel arrays, one for each color R, G, and B, with 5 bits of precision per pixel). Synapse CB1 from input layer S0 to layer C1 applies different sets of weights in some cases and shared weights in other cases, and scans the input image with a 3×3 pixel overlapping filter (kernel), shifting the filter by 1 pixel (or more than 1 pixel as dictated by the model). Specifically, the values for 9 pixels in a 3×3 portion of the image (i.e., called a filter or kernel) are provided to synapse CB1, where the 9 input values are multiplied by appropriate weights and, after calculating the sum of the multiplication outputs, a single output value is determined and provided by the first synapse of CB1 to produce a pixel in one of the feature maps of layer C1. The 3×3 filter is then shifted one pixel to the right within input layer S0 (i.e., the row of three pixels on the right is added and the row of three pixels on the left is discarded), whereby the 9 pixel values in this newly positioned filter are provided to synapse CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process continues until the 3×3 filter has scanned the entire 32×32 pixel image of the input layer S0 for all three colors and all bits (precision values). This process is then repeated using a different set of weights to produce a different feature map for layer C1 until all feature maps for layer C1 have been calculated.

At layer C1, in this example, there are 16 feature maps, each of which is 30×30 pixels. Each pixel is a new feature pixel obtained by multiplying the input by the kernel, so each feature map is a two-dimensional array, so in this example, layer C1 constitutes 16 layers of two-dimensional arrays (remember that the layers and arrays referenced here are logical relationships, not necessarily physical relationships - that is, arrays are not necessarily oriented in terms of physical two-dimensional arrays). Each of the 16 feature maps in layer C1 is generated by one of 16 different sets of synaptic weights applied to the filter sweep. The C1 feature maps can all be related to different aspects of the same image feature, such as boundary recognition. For example, a first image (generated using a first set of weights that are shared by all scans used to generate the first image) may identify circular edges, a second image (generated using a second set of weights that is different from the first set of weights) may identify rectangular edges or the aspect ratio of certain features, and so on.

Before going from layer C1 to layer S1, an activation function P1 (pooling) is applied, which pools the values from consecutive non-overlapping 2×2 regions in each feature map. The purpose of the pooling function P1 is to calculate the average of nearby locations (or a maximum function can also be used) to reduce the dependency of edge locations, for example, and to reduce the data size before entering the next stage. At layer S1, there are 16 15×15 feature maps (i.e., 16 different arrays, each with 15×15 pixels). The synapse CB2 from layer S1 to layer C2 scans the map in S1 with a 4×4 filter and a filter shift of one pixel. At layer C2, there are 22 12×12 feature maps. An activation function P2 (pooling) is applied before layer C2 to layer S2, which pools the values from consecutive non-overlapping 2×2 regions in each feature map. At layer S2, there are 22 6×6 feature maps. An activation function (pooling) is applied at synapse CB3 from layer S2 to layer C3, where each neuron in layer C3 is connected to each map in layer S2 via an individual synapse of CB3. At layer C3, there are 64 neurons. Synapse CB4 from layer C3 to output layer S3 fully connects C3 to S3, that is, every neuron in layer C3 is connected to every neuron in layer S3. The output at S3 includes 10 neurons, where the highest output neuron determines the class. This output may, for example, represent recognition or classification of the content of the original image.

Each layer of synapses is implemented using an array of nonvolatile memory cells or a portion of an array of nonvolatile memory cells.

FIG7 is a block diagram of an array that can be used for that purpose. The vector matrix multiplication (VMM) array 32 includes non-volatile memory cells and is used as a synapse between one layer and the next layer (e.g., CB1, CB2, CB3, and CB4 in FIG6). Specifically, the VMM array 32 includes a non-volatile memory cell array 33, an erase gate and word line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode individual inputs of the non-volatile memory cell array 33. The input to the VMM array 32 may come from the erase gate and word line gate decoder 34 or from the control gate decoder 35. The source line decoder 37 in this example also decodes the output of the non-volatile memory cell array 33. Alternatively, the bit line decoder 36 may decode the output of the non-volatile memory cell array 33.

The non-volatile memory cell array 33 serves two purposes. First, it stores weights to be used by the VMM array 32. Second, the non-volatile memory cell array 33 effectively multiplies the input by the weights stored in the non-volatile memory cell array 33 and adds them together according to the output line (source line or bit line) to produce an output, which will be the input of the next layer or the input of the last layer. By performing multiplication and addition functions, the non-volatile memory cell array 33 does not require separate multiplication and addition logic circuits and is also power efficient due to in-situ memory calculations.

The outputs of the non-volatile memory cell array 33 are supplied to a differential adder (e.g., a summing operational amplifier or a summing current mirror) 38, which calculates the sum of the outputs of the non-volatile memory cell array 33 to produce a single value for convolution. The differential adder 38 is configured to perform the sum of positive weights and negative weights.

Then, the summed output value of the difference adder 38 is supplied to the excitation function block 39, which rectifies the output. The excitation function block 39 may provide a sigmoid, tanh, or ReLU function. The rectified output value of the excitation function block 39 becomes an element of the feature map of the next layer (e.g., C1 in FIG. 6 ), and is then applied to the next synapse to generate the next feature map layer or the last layer. Thus, in this example, the non-volatile memory array 33 constitutes a plurality of synapses (which receive their inputs from a previous neuron layer or from an input layer such as an image database), and the summing operational amplifier 38 and the excitation function block 39 constitute a plurality of neurons.

The inputs to the VMM array 32 in FIG. 7 (WLx, EGx, CGx, and optionally BLx and SLx) may be analog levels, binary levels, or digital bits (in which case a DAC is provided to convert the digital bits to the appropriate input analog level), and the outputs may be analog levels, binary levels, or digital bits (in which case an output ADC is provided to convert the output analog level to digital bits).

FIG8 is a block diagram depicting the use of many layers of VMM arrays 32, here labeled VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in FIG8, the input (denoted as Inputx) is converted from digital to analog by a digital to analog converter 31 and provided to the input VMM array 32a. The converted analog input can be a voltage or a current. The input D/A conversion for the first layer can be accomplished by using a function or LUT (lookup table) that maps the input Inputx to the appropriate analog level for the matrix multiplier of the input VMM array 32a. Input conversion may also be accomplished by an analog-to-analog (A/A) converter to convert external analog input to a mapped analog input to the VMM array 32a.

The output generated by the input VMM 32a is provided as input to the next VMM array (hidden level 1) 32b, which in turn generates outputs that are provided as input to the next VMM array (hidden level 2) 32c, and so on. The various layers of VMM arrays 32 act as synapses and neurons at different layers of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be an independent physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. The example shown in FIG8 includes five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those skilled in the art will understand that this is merely exemplary and that the system may instead include more than two hidden layers and more than two fully connected layers. Vector Matrix Multiplication (VMM) Arrays

FIG9 depicts a neuron VMM array 900 that is particularly applicable to the memory unit 310 shown in FIG3 and is used as a synapse between the input layer and the next layer and part of the neuron. The VMM array 900 includes a memory array 901 of non-volatile memory units and a reference array 902 of non-volatile reference memory units (above the array). Alternatively, another reference array can be placed below.

In the VMM array 900, control gate lines such as control gate line 903 extend in the vertical direction (thus, the reference array 902 in the column direction is orthogonal to the control gate line 903), and erase gate lines such as erase gate line 904 extend in the horizontal direction. Here, inputs to the VMM array 900 are provided on the control gate lines (CG0, CG1, CG2, CG3), and outputs of the VMM array 900 appear on the source lines (SL0, SL1). In one embodiment, only even columns are used, and in another embodiment, only odd columns are used. The current on each source line (SL0, SL1) performs a summation function of all currents from the memory cells connected to that particular source line.

As described herein with respect to neural networks, the non-volatile memory cells of the VMM array 900 (ie, the memory cells 310 of the VMM array 900) are preferably configured to operate in a subcritical region.

Applying bias in weak inversion (subcritical region) to the nonvolatile reference memory cell and nonvolatile memory cell described herein: Ids = Io*e ^(Vg-Vth)/nVt = w*Io*e ^(Vg)/nVt where w = e ^(-Vth)/nVt where Ids is the drain-source current; Vg is the gate voltage on the memory cell; Vth is the critical voltage of the memory cell; Vt is the thermal voltage = k*T/q, where k is the Boltzmann constant, T is the temperature in Kelvin, and q is the electron charge; n is the slope factor = 1+(Cdep/Cox), where Cdep is the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer; Io is the memory cell current when the gate voltage is equal to the critical voltage, and Io is related to (Wt/L)*u*Cox*(n-1)*Vt ² , where u is the carrier mobility, Wt and L are the width and length of the memory cell respectively.

For an I to V logarithmic converter that uses a memory cell (e.g., a reference memory cell or a peripheral memory cell) or a transistor to convert input current to input voltage: Vg=n*Vt*log[Ids/wp*Io] Here, wp is the w of the reference or peripheral memory cell.

For a memory array used as a vector matrix multiplication VMM array with current input, the output current is: Iout=wa*Io*e ^{(Vg)/ n Vt} , that is, Iout=(wa/wp)*Iin=W*Iin W=e ^{(Vthp-Vtha)/ n VtHere} , wa=w for each memory cell in the memory array. Vthp is the effective critical voltage of the peripheral memory cell and Vtha is the effective critical voltage of the main (data) memory cell. Note that the critical voltage of the transistor is a function of the substrate bulk bias, and the substrate bulk bias (denoted as Vsb) can be adjusted to compensate for various conditions at such temperatures. The critical voltage Vth can be expressed as: Vth=Vth0+gamma(SQRT|Vsb–2*φF)-SQRT|2*φF|) where Vth0 is the critical voltage with zero substrate bias, φF is the surface potential, and gamma is the matrix effect parameter.

The word line or control gate may be used as the input of the memory cell for input voltage.

Alternatively, the flash memory cells of the VMM array described in this article can be configured to operate in the linear region: Ids = beta*(Vgs-Vth)*Vds; beta = u*Cox*Wt/L W = α(Vgs-Vth) Meaning that the weight W in the linear region is proportional to (Vgs-Vth).

The word line or control gate or bit line or source line can be used as the input of the memory cell operating in the linear region. The bit line or source line can be used as the output of the memory cell.

For an I to V linear converter, a memory cell (e.g., a reference memory cell or a peripheral memory cell) or a transistor operating in a linear region may be used to linearly convert an input/output current into an input/output voltage.

Alternatively, the memory cells of the VMM array described herein can be configured to operate in the saturation region: Ids=½*beta*(Vgs-Vth) ² ; beta=u*Cox*Wt/L Wα(Vgs-Vth) ² , which means that the weight W is proportional to (Vgs-Vth) ² .

The word line, control gate, or erase gate can be used as the input of a memory cell operating in the saturation region. The bit line or source line can be used as the output of an output neuron.

Alternatively, memory cells of the VMM array described herein may be used for all regions or combinations thereof (subcritical, linear, or saturated) of each layer or multiple layers of a neural network.

Other specific embodiments of the VMM array 32 of FIG. 7 are described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in the above application, source lines or bit lines can be used as neuron outputs (current sum outputs).

FIG10 depicts a neuron VMM array 1000, which is particularly applicable to the memory cell 210 shown in FIG2 and is used as a synapse between an input layer and a next layer. The VMM array 1000 includes a memory array 1003 of a non-volatile memory cell, a reference array 1001 of a first non-volatile reference memory cell, and a reference array 1002 of a second non-volatile reference memory cell. The reference arrays 1001 and 1002 arranged in the row direction of the array are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In practice, the first and second non-volatile reference memory cells are connected in diode form via a multiplexer 1014 (only partially depicted) with current flowing into their inputs. The reference cells are adjusted (e.g., programmed) to a target reference level. The target reference level is provided by a reference microarray matrix (not shown).

Memory array 1003 serves two purposes. First, it stores weights on its individual memory cells to be used by VMM array 1000. Second, memory array 1003 effectively multiplies the inputs (i.e., current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3; reference arrays 1001 and 1002 convert these current inputs into input voltages to be supplied to word lines WL0, WL1, WL2, and WL3) by the weights stored in memory array 1003, and then adds all the results (memory cell currents) to produce outputs on individual bit lines (BL0-BLN), which will be the inputs to the next layer or the inputs to the last layer. By performing multiplication and addition functions, memory array 1003 does not require separate multiplication and addition logic circuits and is also power efficient. Here, voltage inputs are provided on word lines WL0, WL1, WL2, and WL3, and the outputs appear on bit lines BL0-BLN during a read (inference) operation. The current on each of the bit lines BL0-BLN performs a summing function of the currents from all non-volatile memory cells connected to that particular bit line.

Table 5 depicts the operating voltages and currents of the VMM array 1000. The rows in the table represent the voltages on the word line of the selected cell, the word line of the unselected cell, the bit line of the selected cell, the bit line of the unselected cell, the source line of the selected cell, and the source line of the unselected cell. The columns represent the operations of read, erase, and program. Table 5: Operations of the VMM array 1000 of FIG. 10 WL WL-unsel BL BL-unsel SL SL-unsel Read 1-3.5V -0.5V/0V 0.6-2V(I neuron) 0.6V-2V/0V 0V 0V Erase ~5-13V 0V 0V 0V 0V 0V Programming 1-2V -0.5V/0V 0.1-3µA Vinh~2.5V 4-10V 0-1V/FLT

FIG11 depicts a neuron VMM array 1100, which is particularly applicable to the memory unit 210 shown in FIG2 and is used as a synapse between an input layer and a next layer and a portion of a neuron. The VMM array 1100 includes a memory array 1103 of a non-volatile memory unit, a reference array 1101 of a first non-volatile reference memory unit, and a reference array 1102 of a second non-volatile reference memory unit. The reference arrays 1101 and 1102 extend in the column direction of the VMM array 1100. The VMM array is similar to the VMM 1000, except that in the VMM array 1100, the word lines extend in the vertical direction. Here, the inputs are provided on word lines (WLA0, WLB0, WLA1, WLB1, WLA2, WLB2, WLA3, WLB3) and the outputs appear on source lines (SL0, SL1) during a read operation. The current on each source line performs a summing function of all the currents from the memory cells connected to that particular source line.

Table 6 depicts the operating voltages and currents of the VMM array 1100. The rows in the table represent the voltages on the word line of the selected cell, the word line of the unselected cell, the bit line of the selected cell, the bit line of the unselected cell, the source line of the selected cell, and the source line of the unselected cell. The columns represent the operations of read, erase, and program. Table 6: Operations of the VMM array 1100 of FIG. 11 WL WL-unsel BL BL-unsel SL SL-unsel Read 1-3.5V -0.5V/0V 0.6-2V 0.6V-2V/0V ~0.3-1V (I neuron) 0V Erase ~5-13V 0V 0V 0V 0V Disable SL (~4-8V) Programming 1-2V -0.5V/0V 0.1-3µA Vinh~2.5V 4-10V 0-1V/FLT

FIG12 depicts a neuron VMM array 1200, which is particularly applicable to the memory cell 310 shown in FIG3 and is used as a synapse between an input layer and the next layer and a portion of a neuron. The VMM array 1200 includes a memory array 1203 of a non-volatile memory cell, a reference array 1201 of a first non-volatile reference memory cell, and a reference array 1202 of a second non-volatile reference memory cell. The reference arrays 1201 and 1202 are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In practice, the first and second non-volatile reference memory cells are connected in diode form via multiplexers 1212 (only partially shown) with current inputs flowing into them via BLR0, BLR1, BLR2 and BLR3. Multiplexers 1212 each include a respective multiplexer 1205 and a stacked transistor 1204 to ensure a fixed voltage on the bit line (e.g. BLR0) of each of the first and second non-volatile reference memory cells during a read operation. The reference cells are adjusted to a target reference level.

Memory array 1203 serves two purposes. First, it stores weights to be used by VMM array 1200. Second, memory array 1203 effectively multiplies the inputs (current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3; reference arrays 1201 and 1202 convert these current inputs into input voltages to be supplied to control gates (CG0, CG1, CG2, and CG3)) by the weights stored in the memory array, and then adds all the results (cell currents) to produce an output, which appears at BL0-BLN and will be the input to the next layer or the input to the last layer. By performing both multiplication and addition functions, the memory array does not require separate multiplication and addition logic circuits and is also power efficient. Here, the inputs are provided on the control gate lines (CG0, CG1, CG2, and CG3) and the outputs appear on the bit lines (BL0-BLN) during a read operation. The current on each bit line performs a summing function of all the currents from the memory cells connected to that particular bit line.

VMM array 1200 implements a one-way adjustment for non-volatile memory cells in memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is achieved. If too much charge is placed on the floating gate (causing an erroneous value to be stored in the cell), the cell is erased and the sequence of partial programming operations is restarted. As shown, two rows that share the same erase gate (e.g., EG0 or EG1) are erased together (called a page erase), and then each cell is partially programmed until the desired charge on the floating gate is achieved.

Table 7 depicts the operating voltages and currents of the VMM array 1200. The rows in the table represent the voltages on the word line of the selected cell, the word line of the unselected cell, the bit line of the selected cell, the bit line of the unselected cell, the control gate of the selected cell, the control gate of the unselected cell in the same segment as the selected cell, the control gate of the unselected cell in a different segment from the selected cell, the erase gate of the selected cell, the erase gate of the unselected cell, the source line of the selected cell, and the source line of the unselected cell. The columns represent the operations of read, erase, and program. Table 7: Operations of the VMM array 1200 of FIG. 12 WL WL-unsel BL BL-unsel CG CG-unsel in the same section CG-unsel EG EG-unsel SL SL-unsel Read 1.0-2V -0.5V/0V 0.6-2V (I neuron) 0V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0V 0V Erase 0V 0V 0V 0V 0V 0-2.6V 0-2.6V 5-12V 0-2.6V 0V 0V Programming 0.7-1V -0.5V/ 0V 0.1-1µA Vinh(1-2V) 4-11V 0-2.6V 0-2.6V 4.5-5V 0-2.6V 4.5-5V 0-1V

FIG13 depicts a neuron VMM array 1300, which is particularly applicable to the memory cell 310 shown in FIG3 and is used as a synapse between the input layer and the next layer and part of the neuron. The VMM array 1300 includes a memory array 1303 of a non-volatile memory cell, a reference array 1301 of a first non-volatile reference memory cell, and a reference array 1302 of a second non-volatile reference memory cell. The EG lines EGR0, EG0, EG1, and EGR1 extend vertically, while the CG lines CG0, CG1, CG2, and CG3 and the SL lines WL0, WL1, WL2, and WL3 extend horizontally. VMM array 1300 is similar to VMM array 1200 except that VMM array 1300 implements bidirectional scaling, wherein each individual cell can be fully erased, partially programmed, and partially erased as needed to achieve the desired amount of charge on the floating gate due to the use of individual EG lines. As shown, reference arrays 1301 and 1302 convert input currents at terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 to be applied to memory cells in the row direction (through the action of reference cells connected in diode form via multiplexer 1314). The current outputs (neurons) are in bit lines BL0-BLN, where each bit line calculates the sum of all currents from the non-volatile memory cells connected to that particular bit line.

Table 8 depicts the operating voltages and currents of the VMM array 1300. The rows in the table represent the voltages on the word line of the selected cell, the word line of the unselected cell, the bit line of the selected cell, the bit line of the unselected cell, the control gate of the selected cell, the control gate of the unselected cell in the same segment as the selected cell, the control gate of the unselected cell in a different segment from the selected cell, the erase gate of the selected cell, the erase gate of the unselected cell, the source line of the selected cell, and the source line of the unselected cell. The columns represent the operations of read, erase, and program. Table 8: Operations of the VMM array 1300 of FIG. 13 WL WL-unsel BL BL-unsel CG CG -unsel in the same section CG-unsel EG EG-unsel SL SL-unsel Read 1.0-2V -0.5V/0V 0.6-2V (I neuron) 0V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0V 0V Erase 0V 0V 0V 0V 0V 4-9V 0-2.6V 5-12V 0-2.6V 0V 0V Programming 0.7-1V -0.5V/0V 0.1-1µA Vinh (1-2V) 4-11V 0-2.6V 0-2.6V 4.5-5V 0-2.6V 4.5-5V 0-1V

FIG22 depicts a neuron VMM array 2200 that is particularly suitable for the memory cell 210 shown in FIG2 and is used as part of the synapses and neurons between the input layer and the next layer. In the VMM array 2200, inputs INPUT ₀ , ..., INPUT _N are received on bit lines BL ₀ , ..., BL _N , respectively, and outputs OUTPUT ₁ , OUTPUT ₂ , OUTPUT ₃ , and OUTPUT ₄ are generated on source lines SL ₀ , SL ₁ , SL ₂ , and SL ₃ , respectively.

FIG23 depicts a neuron VMM array 2300 that is particularly suitable for the memory cell 210 shown in FIG2 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , INPUT ₁ , INPUT ₂ , and INPUT ₃ are received on source lines SL ₀ , SL ₁ , SL ₂ , and SL ₃ , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N.

FIG24 depicts a neuron VMM array 2400 that is particularly suitable for the memory cell 210 shown in FIG2 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., _WLM , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., _BLN .

FIG25 depicts a neuron VMM array 2500 that is particularly suitable for the memory cell 310 shown in FIG3 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., _WLM , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., _BLN .

FIG26 depicts a neuron VMM array 2600 that is particularly suitable for the memory cell 410 shown in FIG4 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _N are received on vertical control gate lines CG ₀ , ..., CG _N , respectively, and outputs OUTPUT ₁ and OUTPUT ₂ are generated on source lines SL ₀ and SL ₁ .

FIG27 depicts a neuron VMM array 2700 that is particularly suitable for the memory cell 410 shown in FIG4 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _N are received at the gates of bit line control gates 2701-1, 2701-2, ..., 2701-(N-1), and 2701-N, which are respectively coupled to bit lines BL ₀ , ..., BL _N. Exemplary outputs OUTPUT ₁ and OUTPUT ₂ are generated at source lines SL ₀ and SL ₁ .

FIG28 depicts a neuron VMM array 2800 that is particularly suitable for the memory cell 310 shown in FIG3, the memory cell 510 shown in FIG5, and the memory cell 710 shown in FIG7, and is used as part of the synapse and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., _WLM , and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N , respectively.

FIG29 depicts a neuron VMM array 2900 that is particularly suitable for the memory cell 310 shown in FIG3, the memory cell 510 shown in FIG5, and the memory cell 710 shown in FIG7, and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG ₀ , ..., CG _M , and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical source lines SL ₀ , ..., SL _N , respectively, where each source line SLi is coupled to the source lines of all memory cells in the i-th row.

FIG30 depicts a neuron VMM array 3000 that is particularly suitable for memory cell 310 shown in FIG3 , memory cell 510 shown in FIG5 , and memory cell 710 shown in FIG7 , and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG ₀ , ..., CG _M , and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical bit lines BL ₀ , ..., BL _N , respectively, where each bit line BLi is coupled to the bit lines of all memory cells in the i-th row. Long-Term Memory

Learning techniques include a concept called long short-term memory (LSTM). LSTM cells are commonly used in neural networks. LSTM allows neural networks to remember information for a predetermined arbitrary time interval and use that information in subsequent operations. A traditional LSTM cell consists of a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information in and out of the cell and the time interval for which information is remembered in the LSTM. VMMs are particularly useful in LSTM cells.

FIG14 depicts an exemplary LSTM 1400. LSTM 1400 in this example includes units 1401, 1402, 1403, and 1404. Unit 1401 receives an input vector _x0 , and produces an output vector _h0 and a unit state vector _c0 . Unit 1402 receives an input vector _x1 , an output vector (hidden state) _h0 from unit 1401, and a unit state _c0 , and produces an output vector _h1 and a unit state vector _c1 . Unit 1403 receives an input vector _x2 , an output vector (hidden state) _h1 from unit 1402, and a unit state _c1 , and produces an output vector _h2 and a unit state vector _c2 . Unit 1404 receives input vector x ₃ , output vector (hidden state) h ₂ from unit 1403 , and unit state c ₂ , and produces output vector h ₃ . Additional units may be used, and an LSTM with four units is just one example.

FIG15 depicts an exemplary implementation of an LSTM unit 1500, which may be used for units 1401, 1402, 1403, and 1404 in FIG14. LSTM unit 1500 receives an input vector x(t), a unit state vector c(t-1) from a previous unit, and an output vector h(t-1) from a previous unit, and generates a unit state vector c(t) and an output vector h(t).

LSTM unit 1500 includes sigmoid function devices 1501, 1502, and 1503, each of which applies a number between 0 and 1 to control how much of each component in the input vector is allowed to reach the output vector. LSTM unit 1500 also includes tanh devices 1504 and 1505 for applying a hyperbolic tangent function to the input vector, multiplication devices 1506, 1507, and 1508 for multiplying two vectors, and addition device 1509 for adding two vectors. The output vector h(t) can be provided to the next LSTM unit in the system, or it can be accessed for other purposes.

FIG. 16 depicts an LSTM unit 1600, which is an example of an implementation of the LSTM unit 1500. For the convenience of the reader, the same numbering as that of the LSTM unit 1500 is used in the LSTM unit 1600. The sigmoid function devices 1501, 1502 and 1503 and the tanh device 1504 each include a plurality of VMM arrays 1601 and an excitation function block 1602. Therefore, it can be seen that the VMM array is particularly useful in the LSTM unit used in certain neural network systems. The multiplication devices 1506, 1507 and 1508 and the addition device 1509 are implemented digitally or in an analog manner. The excitation function block 1602 can be implemented digitally or in an analog manner.

FIG17 shows an alternative to LSTM cell 1600 (and another example of LSTM cell 1500 implementation). In FIG17 , sigmoid function devices 1501, 1502, and 1503 and tanh device 1504 share the same physical hardware (VMM array 1701 and incentive function block 1702) in a time multiplexed manner. LSTM unit 1700 also includes: a multiplication device 1703 for multiplying two vectors; an addition device 1708 for adding two vectors; a tanh device 1505 (which includes an activation function block 1702); a register 1707 for storing the value i(t) when i(t) is output from the sigmoid function block 1702; and a register 1704 for storing the value f(t)*c(t-1) when the value f(t)*c(t-1) is output from the multiplication device 1703. A register 1705 stores the value f(t)*c(t-1) when it is outputted via the multiplexer 1710; a register 1705 stores the value i(t)*u(t) when it is outputted from the multiplication device 1703 via the multiplexer 1710; and a register 1706 stores the value o(t)*c~(t) when it is outputted from the multiplication device 1703 via the multiplexer 1710; and a multiplexer 1709.

LSTM cell 1600 includes multiple sets of VMM arrays 1601 and individual excitation function blocks 1602, while LSTM cell 1700 includes only one set of VMM arrays 1701 and excitation function blocks 1702, which are used to represent multiple layers in the specific example of LSTM cell 1700. LSTM cell 1700 will require less space than LSTM cell 1600 because LSTM cell 1700 only requires 1/4 of the space for VMM and excitation function blocks compared to LSTM cell 1600.

It will be further appreciated that an LSTM cell will typically include multiple VMM arrays, each of which requires functionality provided by certain circuit blocks external to the VMM array (e.g., adder and excitation function blocks and high voltage generation blocks). Providing a separate circuit block for each VMM array would require a large amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the example described below reduces the circuitry required external to the VMM array itself. Gated Recursive Cell

The analog VMM implementation can be applied to GRU (Gated Recurrent Unit) systems. GRU is a gating mechanism in recurrent neural networks. GRU is similar to LSTM, except that GRU cells usually contain fewer components than LSTM cells.

FIG18 depicts an exemplary GRU 1800. The GRU 1800 in this example includes units 1801, 1802, 1803, and 1804. Unit 1801 receives an input vector _x0 and produces an output vector _h0 . Unit 1802 receives an input vector _x1 and an output vector _h0 from unit 1801 and produces an output vector _h1 . Unit 1803 receives an input vector _x2 and an output vector (hidden state) _h1 from unit 1802 and produces an output vector _h2 . Unit 1804 receives an input vector _x3 and an output vector (hidden state) _h2 from unit 1803 and produces an output vector _h3 . Additional units may be used, and a GRU with four units is just one example.

FIG. 19 depicts an exemplary implementation of a GRU unit 1900, which may be used for units 1801, 1802, 1803, and 1804 of FIG. 18. GRU unit 1900 receives an input vector x(t) and an output vector h(t-1) from a previous GRU unit, and generates an output vector h(t). GRU unit 1900 includes sigmoid function devices 1901 and 1902, each of which applies a number between 0 and 1 to components from the output vector h(t-1) and the input vector x(t). The GRU unit 1900 also includes a tanh device 1903 for applying a hyperbolic tangent function to an input vector, a plurality of multiplication devices 1904, 1905 and 1906 for multiplying two vectors, an addition device 1907 for adding two vectors, and a complementary device 1908 for subtracting the input from 1 to produce an output.

FIG. 20 depicts a GRU unit 2000, which is an example of an implementation of the GRU unit 1900. For the convenience of the reader, the same numbering as that of the GRU unit 1900 is used in the GRU unit 2000. As can be seen from FIG. 20, the sigmoid function devices 1901 and 1902 and the tanh device 1903 each include a plurality of VMM arrays 2001 and an excitation function block 2002. Therefore, it can be seen that the VMM array is particularly used in the GRU unit used in certain neural network systems. The multiplication devices 1904, 1905 and 1906, the addition device 1907 and the complementary device 1908 are implemented digitally or in an analog manner. The excitation function block 2002 can be implemented digitally or in an analog manner.

FIG21 shows an alternative to GRU cell 2000 (and another example of an implementation of GRU cell 1900). In FIG21, GRU cell 2100 utilizes VMM array 2101 and excitation function block 2102, which applies a number between 0 and 1 when configured as a sigmoid function to control how much of each component in the input vector is allowed to reach the output vector. In FIG21, sigmoid function devices 1901 and 1902 and tanh device 1903 share the same physical hardware (VMM array 2101 and excitation function block 2102) in a time multiplexed manner. The GRU unit 2100 also includes: a multiplication device 2103 for multiplying two vectors; an addition device 2105 for adding two vectors; a complement device 2109 for subtracting an input from 1 to generate an output; a multiplexer 2104; and a register 2106 for holding the value h(t-1)*r(t) when the value h(t-1)*r(t) is output from the multiplication device 2103 via the multiplexer 2104. )*r(t); a register 2107 for holding the value h(t-1)*z(t) when the value h(t-1)*z(t) is output from the multiplication device 2103 via the multiplexer 2104; and a register 2108 for holding the value h^(t)*(1-z(t)) when the value h^(t)*(1-z(t)) is output from the multiplication device 2103 via the multiplexer 2104.

The GRU unit 2000 includes multiple sets of VMM arrays 2001 and excitation function blocks 2002, while the GRU unit 2100 includes only one set of VMM arrays 2101 and excitation function blocks 2102, which are used to represent multiple layers in the specific example of the GRU unit 2100. The GRU unit 2100 will require less space than the GRU unit 2000 because the GRU unit 2100 only needs 1/3 of the space for the VMM and excitation function blocks compared to the GRU unit 2000.

It will be further appreciated that a GRU system will typically include multiple VMM arrays, each of which requires functionality provided by certain circuit blocks external to the VMM array (e.g., adder and excitation function blocks and high voltage generation blocks). Providing individual circuit blocks for each VMM array would require a significant amount of space within a semiconductor device and would be somewhat inefficient. Therefore, the embodiments described below reduce the circuitry required external to the VMM array itself.

The inputs to the VMM array can be analog levels, binary levels, pulses, time modulated pulses, or digital bits (in which case a DAC is required to convert the digital bits to the appropriate input analog level), and the outputs can be analog levels, binary levels, timing pulses, pulses, or digital bits (in which case an output ADC is required to convert the output analog level to digital bits).

Typically, for each memory unit in a VMM array, each weight W can be implemented by a single memory unit, or by a differential unit, or by two hybrid memory units (average of 2 units). In the case of a differential unit, two memory units are required to implement the weight W as a differential weight (W=W+-W-). In the case of two hybrid memory units, two memory units are required to implement the weight W as the average of the two units.

FIG. 31 depicts a VMM system 3100. In some embodiments, weights W stored in a VMM array are stored as a differential pair of W+ (positive weight) and W- (negative weight), where W=(W+) -(W-). In the VMM system 3100, half of the bit lines are referred to as W+ lines, i.e., bit lines connected to memory cells that will store positive weights W+, and the other half of the bit lines are referred to as W- lines, i.e., bit lines connected to memory cells that implement negative weights W-. The W- lines are interspersed between the W+ lines in an alternating manner. Subtraction operations are performed by summing circuits (e.g., summing circuits 3101 and 3102) that receive current from the W+ lines and the W- lines. The output of the W+ line is combined with the output of the W- line to effectively provide W=W+-W- for each pair of (W+, W-) cells of all (W+, W-) line pairs. Although the above description has been made for the W- lines to be interspersed between the W+ lines in an alternating manner, in other examples the W+ lines and the W- lines can be arbitrarily placed at any position in the array.

Another example is depicted in Figure 32. In a VMM system 3210, positive weights W+ are implemented in a first array 3211, negative weights W- are implemented in a second array 3212, the second array 3212 is separate from the first array, and the resulting weights are appropriately combined by a summing circuit 3213.

FIG. 33 depicts a VMM system 3300. The weights W stored in the VMM arrays are stored as a differential pair of W+ (positive weight) and W- (negative weight), where W=(W+)-(W-). The VMM system 3300 includes arrays 3301 and 3302. Half of the bit lines of each of arrays 3301 and 3302 are called W+ lines, i.e., bit lines connected to memory cells that will store positive weights W+, and the other half of the bit lines of each of arrays 3301 and 3302 are called W- lines, i.e., bit lines connected to memory cells that implement negative weights W-. The W- lines are interspersed between the W+ lines in an alternating manner. The subtraction operation is performed by summing circuits (e.g., summing circuits 3303, 3304, 3305, and 3306) that receive current from the W+ line and the W- line. The output of the W+ line from each array 3301, 3302 is combined with the output of the W- line to effectively provide W=W+-W- for each pair of (W+, W-) cells of all (W+, W-) line pairs. In addition, the W values from each array 3301 and 3302 can be further combined by summing circuits 3307 and 3308 so that each W value is the result of subtracting the W value from array 3302 from the W value from array 3301, which means that the final result of summing circuits 3307 and 3308 is the difference value of two difference values.

Each nonvolatile memory cell used in an analog neural memory system will be erased and programmed to hold a very specific and precise amount of charge, i.e., number of electrons, in the floating gate. For example, each floating gate should hold one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, 64, 128, and 256.

It is very important to be able to accurately verify the programmed operation after executing it.

Many examples of verification circuits and related methods in artificial neural networks are revealed.

VMM system architecture

34 depicts a block diagram of a VMM system 3400. The VMM system 3400 includes a VMM array 3401, a row decoder 3402, a high voltage decoder 3403, a row decoder 3404, a bit line driver 3405, an input circuit 3406, an output circuit 3407, a control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high voltage generation block 3410, which includes a charge pump 3411, a charge pump regulator 3412, and a high voltage level generator 3413. The VMM system 3400 further includes a (programming/erasing, or weight adjustment) algorithm controller 3414, an analog circuit 3415, a control engine 3416 (which may include, but is not limited to, specific functions such as arithmetic functions, excitation functions, and embedded microcontroller logic), a test control logic 3417, and a static random access memory (SRAM) block 3418 for storing intermediate data such as for input circuits (e.g., excitation data) or output circuits (neuron output data) or for programmed input data (e.g., for an entire row or multiple rows of input data).

Input circuit 3406 may include circuits such as a DAC (digital to analog converter), a DPC (digital to pulse converter, digital to time modulated pulse converter), an AAC (analog to analog converter, e.g., current to voltage converter, logarithmic converter), a PAC (pulse to analog level converter), or any other type of converter. Input circuit 3406 may implement one or more of normalization, linear or nonlinear up/down scaling functions, and arithmetic functions. Input circuit 3406 may implement a temperature compensation function on the input level. Input circuit 3406 may implement an excitation function such as ReLU or sigmoid. Input circuit 3406 may store digital stimulus data to be used as an input signal or combined with an input signal during a programming or read operation. The digital stimulus data may be stored in a register. Input circuit 3406 may include circuitry for driving array terminals (e.g., CG, WL, EG, and SL lines), which may include a sample-and-hold circuit and a buffer. A DAC may be used to convert the digital stimulus data into an analog input voltage for application to the array.

Output circuit 3407 may include circuits such as ITV (current to voltage circuit), ADC (analog to digital converter to convert the analog output of the neuron into digital bits), AAC (analog to analog converter, e.g., current to voltage converter, logarithmic converter), APC (analog to pulse converter, analog to time modulated pulse converter), or any other type of converter. Output circuit 3407 may convert the array output into excitation data. Output circuit 3407 may implement excitation functions such as rectified linear excitation function (ReLU) or sigmoid. Output circuit 3407 may implement one or more of statistical normalization, regularization, up/down scaling/gain functions, statistical rounding, and arithmetic functions (e.g., addition, subtraction, division, multiplication, shift, logarithm) on the neuron output. Output circuit 3407 may implement a temperature compensation function on the neuron output or array output (e.g., bit line output) to, for example, keep the power consumption of the array approximately constant with temperature or improve the accuracy of the array (neuron) output by keeping the IV slope approximately the same over the entire temperature range. Output circuit 3407 may include a register for storing output data.

FIG. 35A depicts a programming method 3500. First, the method begins (step 3501), which typically occurs in response to receiving a program command. Next, a mass programming operation programs all cells to state "0" (step 3502). Then, a soft erase operation erases all cells to a weak erase level such that each cell draws, for example, about 1-5 µA during a read operation (step 3503). This is in contrast to a deep erase level where each cell draws, for example, about 20-30 µA during a read operation. Next, a hard programming operation is performed on all unselected cells to a very deeply programmed state to add electrons to the floating gates of the cells (step 3504), thereby ensuring that those cells are truly "off," meaning that those cells will draw negligible current during a read operation.

Then, a coarse programming operation is performed to program the selected cells to a level closer to the target, for example, 2X to 100X of the target. A coarse programming operation is performed on the selected cells (step 3505), and then a fine programming operation is performed on the selected cells (step 3506) to program each selected cell to the desired fine value.

The coarse programming operation (3505) may consist of multiple coarse verification/programming cycles. In each coarse verification/programming cycle, a verification operation is performed to verify whether the unit output meets the coarse target; if not, the programming operation is performed again for that unit. The verification/programming cycle is repeated until the unit outputs of all target units meet the coarse target.

The precise programming operation (3506) may consist of multiple precise verification/programming cycles. In each precise verification/programming cycle, a verification operation is performed to verify whether the unit output meets the precise target; if not, the programming operation is performed again for that unit. The verification/programming cycle is repeated until the unit output of all target units meets the precise target.

FIG35B depicts another programming method 3510 that is similar to the programming method 3500. However, instead of a programming operation that programs all cells to state "0" as in step 3502 of FIG35A, after the method is initiated (step 3501), an erase operation is used to erase all cells to state "1" (step 3512). Then, a soft programming operation (step 3513) is used to program all cells to a weakly programmed state (level) such that each cell draws, for example, about 3-5 µA during a read operation. Thereafter, a hard programming operation is performed on all unselected cells to a very deeply programmed state (step 3504), followed by coarse and fine programming (3505-3506) as in Figure 35A. A variation of the example of Figure 35B removes the soft programming operation (step 3513).

FIG. 36 depicts a first example of a coarse programming operation 3505, which is a search and execute method 3600. First, a lookup table search is performed to determine a coarse target current value (I _CT ) for a selected cell based on the value intended to be stored in that cell (step 3601). For example, this table is built from silicon characterization or calibration of wafer testing. It is assumed that the selected cell can be programmed to store one of N possible values (e.g., 128, 64, 32, but not limited thereto). Each of the N values will correspond to a different desired current value (I _D ) to be drawn by the selected cell during a read operation. In one example, the lookup table may include M possible current values to be used as a rough target current value I _CT for the selected cell during the search and execution method 3600, where M is an integer less than N. For example, if N is 8, then M may be 4, which means that the selected cell may store 8 possible values, and one of the 4 rough target current values will be selected as the rough target for the search and execution method 3600. That is, the search and execution method 3600 (which, as described above, is an example of the coarse programming operation 3505) is intended to quickly program the selected cell to a value (I _CT ) that is somewhat close to the desired value (I _D ), and then the fine programming operation 3506 is intended to more accurately program the selected cell to the desired value (I _D ).

For the simple example of N=8 and M=4, examples of cell values, desired current values, and rough target current values are described in Tables 9 and 10: Table 9: Example of N desired current values for N=8 The value stored in the selected cell Expected current value (I _D ) 000 100pA 001 200pA 010 300pA 011 400pA 100 500pA 101 600pA 110 700pA 111 800pA Table 10: Example of M target current values for M=4 Rough target current value (I _CT ) Related unit value 800pA+I _CTOFFSET1 000,001 1600pA+I _CTOFFSET2 010,011 2400pA+I _CTOFFSET3 100,101 3200pA+I _CTOFFSET4 110,111 The offset value I _CTOFFSETx is used to prevent the desired current value from being exceeded during coarse adjustment.

Once the coarse target current value I _CT is selected, the selected cell is formatted by applying a voltage v ₀ to the appropriate terminal of the selected cell based on the cell architecture type of the selected cell (e.g., memory cell 210, 310, 410, or 510) (step 3602). If the selected cell is of the type of memory cell 310 in FIG. 3, a voltage v ₀ is applied to the control gate terminal 28, and depending on the coarse target current value I _CT , v ₀ may be 5-7 V. The value of v ₀ may optionally be determined by a voltage lookup table that stores the value of v ₀ against the coarse target current value I _CT .

Next, the selected cell is programmed by applying a voltage of v _i =v _i-1 +v _increment , where i starts at 1 and increments each time this step is repeated, where v _increment is a small, fine voltage that will result in a degree of programming appropriate for the granularity of the desired change (step 3603). Thus, the first time step 3603 is performed, i=1, and v ₁ will be v ₀ +v _increment . A verification operation is then performed (step 3604), where a read operation is performed on the selected cell and the current drawn through the selected cell (I _cell ) is compared to a coarse target threshold value I _CT . If I _cell is less than or equal to I _CT (which is the first critical value here), the search and execution method 3600 is complete and the precise programming operation 3506 can begin. If I _cell is not less than or equal to I _CT , the value of i is increased and step 3603 is repeated.

Thus, at the end of the coarse programming method 3505 and the start of the fine programming method 3506, the voltage v _i will be the last voltage used to program the selected cell, and the selected cell will store a value associated with the coarse target current value I _CT , where I _cell >= I _CT . The fine programming method 3506 programs the selected cell to the point in time at which it draws a current _ID during a read operation (plus or minus an acceptable amount of deviation, e.g., +/- 30% or less, e.g., +/- 50 pA), the current _ID being the desired current value associated with the value to be stored in the selected cell.

37 depicts an example of a sequence of different voltages that may be applied to the control gates of selected memory cells during a coarse programming operation 3505 and/or a fine programming operation 3506. It consists of multiple verification/programming cycles.

According to the first method, increasing voltages are gradually applied to the control gates to program the selected memory cell. The starting point is v _i , which during the fine programming operation 3506 will be the last voltage applied during the coarse programming method 3505. The increment of v _p1 is added to v ₁ , and the voltage v ₁ +v _p1 is then used to program the selected cell (represented by the second pulse from the left in sequence 3701). v _p1 is an increment less than v _increment (the voltage increment used during the coarse programming operation 3505). After each programming voltage is applied, a verification operation (similar to step 3404) is performed, in which it is determined whether I _cell is less than or equal to I _PT1 (which is a first precise target current value, here a second critical value), where I _PT1 = I _D + I _PT1OFFSET , where I _PT1OFFSET is an offset value added to prevent programming overshoot. If not, another increment v _p1 is added to the previously applied programming voltage, and the process is repeated. When I _cell is less than or equal to I _PT1 , this portion of the programming sequence stops. Alternatively, if I _PT1 is equal to I _D , or is nearly equal to I _D with sufficient accuracy (i.e., plus or minus an acceptable amount of deviation), the selected memory cell has been successfully programmed.

If _IPTI is not close enough to _ID , i.e., is not nearly equal to _ID with sufficient accuracy, then further programming with a smaller granularity is performed. Here, sequence 3702 is now used. The starting point for sequence 3702 is the last voltage used for programming under sequence 3701. An increment of _Vp2 (which is less than _vp1) is added to that voltage, and the combined voltage is applied to program the selected memory cell. After each programming voltage is applied, a verification operation (similar to step 3404) is performed, in which it is determined whether I _cell is less than or equal to I _PT2 (which is a second precise target current value, here a third critical value), where I _PT2 = I _D + I _PT2OFFSET , where I _PT2OFFSET is an offset value added to prevent programming overshoot. If not, another increment V _p2 is added to the previously applied programming voltage, and the process is repeated. When I _cell is less than or equal to I _PT2 plus or minus an acceptable amount of deviation, this portion of the programming sequence stops. Here, I _PT2 is considered equal to I _D or close enough to I _D that programming can stop because the target value has been reached with sufficient accuracy. Those skilled in the art will appreciate that additional sequences may be applied with smaller and smaller programmable increments. For example, in Figure 38, three sequences (3801, 3802, and 3803) are applied instead of just two sequences.

The second approach is shown in sequence 3703 in FIG. 37 and sequence 3803 in FIG. 38 . Rather than increasing the voltage applied during programming of the selected memory cell, the same voltage is applied for increasing periods of time. That is, additional time increments _tp1 are added to the programming pulses so that each applied pulse is _tp1 longer than the previously applied pulse. After each programming pulse is applied, the same verification operations are performed as previously described for sequence 3701. Alternatively, additional sequences may be applied in which the additional time increments added to the programming pulses have a smaller duration than the previously used sequence. Although only one time series is shown, one of ordinary skill in the art will appreciate that any number of different time series may be used.

Additional details will now be provided for three examples of coarsely programmed operation 3505.

FIG. 39 depicts another example of a coarse programming operation 3505, which is an adjustable calibration method 3900. The adjustable calibration method begins implementation (step 3901). The cell is programmed with a preset starting value _v0 (step 3902). Unlike the search and execute method 3600, here _v0 is not obtained from a lookup table, but can be a relatively small initial value. The control gate voltage of the cell is measured at a first current value IR1 (e.g., 100 nA) and a second current value IR2 (e.g., 10 nA), and a subcritical slope (e.g., 360 mV/dec) is determined and stored based on those measurements (step 3903).

Determine a new programming voltage, V _i . The first time this step is performed, i=1, and V ₁ is determined based on the stored subcritical slope value and the current target and offset values using, for example, a subcritical equation (e.g., the following subcritical equation): V _i =V _i-1 +V _increment , where V _increment is proportional to the slope of Vg Vg= n*Vt*log[Ids/wa*Io]. Here, wa is the w of the memory cell, and Ids is the current target plus the offset value.

If the stored slope value is relatively steep, a relatively small current offset value can be used. If the stored slope value is relatively flat, a relatively large current offset value can be used. Therefore, determining the slope information allows the selection of a current offset value that is customized for the particular cell in question. This will ultimately make the programming process shorter. As this step is repeated, i is incremented and _vi = _{vi- 1} + _vincrement . The cell is then programmed using _vi . _vincrement can be determined by a lookup table that stores _vincrement values relative to target current values.

Next, a verification operation is performed, in which a read operation is performed on the selected cell, and the current drawn through the selected cell (I _cell ) is compared to the coarse target threshold value I _CT (step 3905). If I _cell is less than or equal to I _CT , it sets I _CT = I _D + I _CTOFFSET , where I _CTOFFSET is an offset value added to prevent programming overshoot, and the adjustable calibration method 3900 is completed and the fine programming operation 3506 can be started. If I _cell is not less than or equal to I _CT , steps 3904-3905 are repeated, and i is incremented. Then, the fine programming method 3506 is started when the voltage _vi is the last voltage used to program the selected cell.

FIG40 depicts an example of an adjustable calibration operation 3900. During step 3903, current source 4001 applies current values IR1 and IR2 to the selected cell (here, memory cell 4002), and then the voltage at the control gate of memory cell 4002 is measured (CGR1 for IR1 and CGR2 for IR2). The slope is determined to be (CGR2-GR1)/dec of current, which is the slope of VCG with respect to LOG(I).

FIG41 depicts another example of a coarse programming operation 3505, which is an adjustable calibration method 4100. The adjustable calibration method begins (step 4101). The cell is programmed with a preset starting value _v0 (step 4102). _v0 is obtained from a lookup table, such as created by silicon characterization, and the table value is offset, such as not to exceed the target program value.

In the next step 4103, an IV slope parameter is established for predicting the next programmed voltage, a first control gate read voltage V _CGR1 is applied to the selected cell, and the resulting cell current IR ₁ is measured. Then, a second control gate read voltage V _CGR2 is applied to the selected cell, and the resulting cell current IR ₂ is measured. The slope is determined and stored based on those measurements, for example, according to the equation in the subcritical region (the cell is operating in the subcritical region): Slope = (V _CGR1 – V _CGR2 )/(LOG(IR1) – LOG(IR2)) (step 4103). Examples of values for V _CGR1 and V _CGR2 are, for example, 1.5V and 1.3V, respectively.

Determining the slope information allows the selection of a V _increment value that is tailored to the particular cell in question. This will ultimately make the programming process shorter.

When step 4104 is repeated, i is incremented and a new desired programming voltage V _i is determined based on the stored slope value and the current target and offset values using the following equation: V _i =V _i-1 +V _increment , where for i-1, V _increment =alpha*slope*(LOG (IR ₁ )–LOG(I _CT )), where I _CT is the target current and alpha is a predetermined constant <1 (programming offset value) to prevent overshoot, e.g., 0.9. For example, V _i is VSLP or VCGP, a source line or control gate programming voltage.

Then, the _cell is formatted using Vi (step 4105).

Next, a verification operation is performed, in which a read operation is performed on the selected cell, and the current drawn by the selected cell (I _cell ) is compared with I _CT (step 4106). If I _cell is less than or equal to I _CT (here, a rough target threshold), I _CT = I _D + I _CTOFFSET is set, where I _CTOFFSET is an offset value added to prevent programmed overshoot, and the process proceeds to step 4107. If not, the process returns to step 4104 and i is incremented.

In step 4107, I _cell is compared to a critical value I _CT2 that is less than I _CT . The purpose of this is to see if an overshoot occurs. That is, although the goal of I _cell is to be lower than I _CT , if it is too much lower than I _CT , an overshoot occurs and the stored value may actually correspond to an erroneous value. If I _cell is not less than or equal to I _CT2 , no overshoot occurs and the adaptive calibration method 4100 is complete, at which point the process proceeds to the precise programming operation 3506. If I _cell is less than or equal to I _CT2 , an overshoot occurs. Next, the selected cell is erased (step 4108), and with i reset to 0, the programming process restarts at step 4102. Optionally, if step 4108 is executed more than a predetermined number of times, the selected unit may be considered a bad unit that should not be used.

The fine programming operation 3506 consists of multiple verification and programming (V/P) cycles, where the programming voltage is incremented by a constant fine voltage with a fixed pulse width, or where the programming voltage is fixed and the programming pulse width is varied.

Optionally, the step of determining whether the current through the selected non-volatile memory cell during a read or verification operation is less than or equal to a coarse target threshold value can be performed by applying a fixed bias to terminals of the non-volatile memory cell; measuring and digitizing the current drawn by the selected non-volatile memory cell to produce a digital output bit; and comparing the digital output bit to a digital bit representing a first threshold current.

Optionally, the step of determining whether the current through the selected non-volatile memory cell during a read or verification operation is less than or equal to a coarse target threshold value can be performed by applying a fixed bias to terminals of the non-volatile memory cell; measuring and digitizing the current drawn by the selected non-volatile memory cell to produce a digital output bit; and comparing the digital output bit to a digital bit representing a first threshold current.

Optionally, the step of determining whether the current through the selected non-volatile memory cell during a read or verification operation is less than or equal to a coarse target threshold value can be performed by applying an input to a terminal of the non-volatile memory cell; modulating the current drawn by the selected non-volatile memory cell with an output pulse to produce a modulated output; digitizing the modulated output to produce digital output bits; and comparing the digital output bits to digital bits representing a first critical current.

FIG. 42 depicts a third example of the coarse programming operation 3505, which is an absolute calibration method 4200. The absolute calibration method begins (step 4201). The cell is programmed with a preset starting value v ₀ (step 4202). The control gate voltage (VCGRx) of the cell is measured at a current value I _target and the control gate voltage (VCGRx) is stored (step 4203). A new desired voltage v ₁ is determined based on the stored control gate voltage and the current target and offset value I _target +I _offset (step 4204). For example, the new desired voltage _v1 can be calculated as follows: _v1 = _v0 +(VCGBIAS-stored VCGR), where VCGBIAS is the preset read control gate voltage at the maximum target current, for example =~1.5V, and the stored VCGR is the read control gate voltage measured in step 4203.

The cell is then programmed using vi. When i=1, the voltage _v1 from step 4204 is used. When i>1, the voltage _vi = _vi-1 + _vincrement is used. _vincrement can be determined from a lookup table that stores _vincrement values relative to target current values. Next, a verification operation is performed in which a read operation is performed on the selected cell and the current drawn through the selected cell ( _Icell ) is compared to _ICT (step 4206). If _Icell is less than or equal to _ICT (which is a critical value here), then the absolute calibration method 4200 is complete and the precise programming method 3506 can begin. If I _cell is not less than or equal to I _CT , steps 4205 - 4206 are repeated and i is incremented.

A single weight verification method can be used to verify whether a unit has achieved a weight target as a result of a programmed operation. A memory unit is selected for verification operation, and the output of the memory unit is then verified by the verification mechanism described below with respect to Figures 43-49.

The differential weight verification method can be used to determine whether a differential cell (formed by 2 cells, where the stored value is the difference between the values stored in the 2 cells) has reached the weight target due to the programmed operation. Two cells associated with the differential weight are selected for the verification operation. The output difference between the cells is then verified by the verification mechanism described below with respect to Figures 43-49. For example, if cell ₁ stores the w+ value and cell ₂ stores the w- value, then w=w+-w- is the differential weight. Cell ₁ and Cell ₂ are the two cells selected for the verification operation to determine whether the weight w has reached the target. Alternatively, verify the w- value of cell ₂ first, and then verify the differential weight w. Alternatively, verify the w+ value of cell ₁ first, and then verify the differential weight w. Alternatively, verify the w+ and w- values of cell ₁ and cell ₂ in the first operation, and then verify the difference weight w in the second operation.

43 depicts a VMM system 4300. Current to voltage converter and analog to digital converter block 4301 receives current from VMM array 3401, typically from a bit line or source line in VMM array 3401, and provides an output to verification circuit 4302. Current to voltage converter and analog to digital converter block 4301 and verification circuit 4302 together are output blocks coupled to VMM array 3401 to generate a voltage during a verification operation of VMM array 3401 and to generate a digital output during a read operation of VMM array 3401. Each current to voltage converter in block 4301 converts a current into a voltage. The analog-to-digital converter in block 4301 is reconfigured (e.g., in the manner described below with respect to FIG. 44A ) for use during a verification operation (also referred to as a read-verify operation). During the verification operation, reference array 4304 is used to generate all N possible current targets (e.g., 32 values between 3-96 nA in increments of 3 nA). Each of the N possible current targets corresponds to one of the N weighted targets stored in a corresponding reference memory cell in reference array 4304. Alternatively, a main reference current generator 4305 is used to generate all N current targets. A reference current from a reference array 4304 or a main reference current generator 4305 is provided to a reference voltage generator 4303, which includes a voltage DAC and utilizes the voltage DAC to convert the reference current into a reference voltage, wherein there are N possible voltages corresponding to N possible values. For example, for a 5-bit cell, there are 32 reference voltages corresponding to 32 current targets. An appropriate voltage value among the N voltage values is selected for comparison by the verification circuit 4302 with the voltage provided by the corresponding current to voltage converter 4301. This comparison is a verification operation performed by the verification circuit 4302. Therefore, the weights can be verified after being programmed into the VMM array 3401. In this method, verification is done by comparing the output voltage from the memory cell with the reference voltage from the voltage reference generator 4303.

Alternatively, the cell current is verified directly using a reference current digital-to-analog converter (IDAC) instead of a current-to-voltage converter, which means comparing the cell current to a reference current. With this approach, delays and variations are typically larger due to the settling time of low-current (e.g., a few nA) circuits.

44A depicts a neuron output ITV+ADC+verification circuit 4488, which includes a current to voltage converter (ITV) 4401, a successive approximation register (SAR) analog to digital converter (ADC) 4402, and a verification circuit 4403. The verification circuit 4403 includes a comparator 4404 (which is also used to generate a digital output during a read or read neuron operation), a reference voltage selection circuit 4406, and a verification register 4405. The verification register 4405 is a data output register of the SAR ADC 4402, which is used here to perform a verification function, but can also be used to generate a digital output during a read or read neuron operation. The current-to-voltage converter 4401 and the SAR analog-to-digital converter 4402 are examples of implementations of the current-to-voltage converter and the analog-to-digital converter 4301 in Figure 43, and the verification circuit 4403 is an example of implementation of the verification circuit 4302 in Figure 43.

The current-to-voltage converter 4401 and the SAR analog-to-digital converter 4301 can be used during a read or neural read operation. However, the current-to-voltage converter 4401 and the SAR analog-to-digital converter 4301 can also be used during a verification operation, where the weights (meaning one of N possible weight values) that have been programmed into the non-volatile memory cells within the VMM array are verified.

The current to voltage converter 4401 receives current from the VMM array from a single selected cell and converts that current to a voltage. The current to voltage conversion may be accomplished by a plurality of resistors ITV (RITV) 4490R or a plurality of capacitors ITV (CITV) 4490C. One of N possible reference voltages is provided to the verification circuit 4403 via a verification register 4405 and a verification reference voltage selection circuit 4406. The verification register may be, for example, an 8-bit register that is used to select one voltage reference level from 256 voltage reference levels in the verification reference voltage selection circuit 4406. The verification reference voltage as an input to the verification reference voltage select circuit 4406 (via the verification reference voltage line) is provided by a global verification reference voltage generator as shown in Figure 45. The comparator 4404 then compares the voltage from the current to voltage converter 4401 to which of the N possible reference voltages to indicate whether the cell is storing the correct value. The capacitors and SAR logic in the SAR ADC 4402 are not used during the verification operation. In one example, the control circuit closes switch S1A in SAR ADC 4402 to provide the positive output Vinp of ITV 4401 directly to the non-inverting input of comparator 4404, and opens a switch (unnamed) and closes a switch (unnamed) to provide the output of the verification reference voltage selection circuit 4406 to the inverting input of comparator 4404.

The ITV+ADC+verification circuit 4488 can be used for single weight verification operations as well as differential weight verification operations. For single weight verification operations, only one input from one unit is required. The output voltage of ITV 4401 is proportional to the unit current value and is verified against the reference voltage level provided by the verification reference voltage selection circuit 4406. For differential weight verification operations, two inputs from two units are the two inputs to the ITV+ADC+verification circuit 4488, and the output voltage of ITV 4401 (e.g., Vinp) is proportional to the difference between the two unit currents and is verified against the reference voltage level provided by the verification reference voltage selection circuit 4406.

The total offset compensation of the ITV+ADC+verification circuit 4488 can be trimmed by using the offset trim of the comparator 4404. This offset can be further trimmed by trimming the resistor 4490R or capacitor 4490C of the ITV circuit 4401.

In another example, the overall gain compensation of the ITV+ADC+verification circuit 4488 can be trimmed by trimming the resistor 4490R or the capacitor 4490C of the ITV circuit 4401.

An alternative offset compensation method can be accomplished in the time domain by using an ITV 4401 with a capacitor 4490C. The ITV 4401 uses a variable width pulse with a reference current input to achieve integration of the capacitor 4490C to produce an output voltage. A comparator 4404 compares the output voltage from the ITV 4401 to the reference voltage. The parameters of the variable width pulse are stored, for example, in the digital domain by a counter (not shown) or in the analog domain by a table (not shown) that stores the analog voltage for each ITV (the analog voltage is converted from the variable pulse input). The controller (not shown) uses this information to enable the ITV with an enable signal (not shown) for authentication operations.

FIG44B depicts a comparator and offset circuit 4490 that can be used to replace the comparator 4404 in FIG44A to add the additional function of offset compensation. The comparator and offset circuit 4490 includes a comparator 4491, its comparison inputs VINP and VINN (which can be the same signals shown in FIG44A) to generate an output COMPOUT and its complement COMPOUTB. A calibration circuit 4492 can be adjusted to provide an offset voltage VON to the comparator 4491, and a calibration circuit 4493 can be adjusted to provide an offset voltage VOP to the comparator 4491.

FIG45 depicts a reference voltage generator 4500. The reference voltage generator 4500 is an example implementation of the reference voltage generator 4303. In one example, a reference array 4304 (not shown) provides a maximum current from a reference cell, where the current represents the highest possible weight (of N possible weights) that can be stored in a non-volatile memory cell. The current-to-voltage converter 4501 converts the maximum current provided from the reference cell to a high verification reference voltage, which can be accomplished using a resistor current-to-voltage converter (RITV) 4511 or a capacitor current-to-voltage converter (CITV) 4510.

This voltage is then used, for example, by resistor string 4504, which here includes N-1 resistors in series, to generate 32 verification reference voltages, for example, for a 5-bit cell. In another example, the reference array provides a reference current that is converted to a reference voltage, for example, the reference current can be a mid-range value that is appropriately converted to all N reference voltages (for example, using an ITV circuit through a current ratioed mirror, through trimmed resistor values, or through trimmed capacitor values). The ITV4501 shown in the figure uses a differential operational amplifier. This ITV is a copy of the sub-circuit ITV 4301 in the VMM system 4300, also shown as ITV 4401 in the ITV+ADC+verification circuit 4488. The differential operational amplifier (op amp) is a copy of the regional differential op amp 4480 in Figure 44A, so that the N global references can track regional voltages from ITV4301 that vary with PVT (process, supply or temperature). Alternatively, ITV4501 can be based on a single-ended op amp. Resistors 4511x and capacitors 4510x are trimmable to adjust the range and compensate for mismatch or offset changes. Resistor string 4504 is trimmable to adjust the range of VN to V1, shift the range of VN to V1 up/down, or adjust the regional value VN to V1.

In another example, a constant bias current (eg, from an IDAC) is used instead of a reference current from a reference array.

Bias current or reference current from a reference array is adjustable to achieve a target value. They also compensate for PVT (process, power or temperature) variations.

The total global offset and mismatch compensation of the ITV+ADC+verification circuit 4488 can be trimmed by using the offset and mismatch trimming of the reference voltage generator 4500. This can be done by adjusting the bias current circuit 4512 or trimming resistors 4511a and 4511b, capacitors 4510a and 4510b, or resistor string 4504.

A buffer 4502 is used to buffer a high verification reference voltage, which represents, for example, the 32nd level (L31) of the 32 reference voltage levels (L0-L31) of one 5-bit cell, to drive the resistor string 4504 at one end of the resistor string 4504. In another example, a buffer 4503 is provided to buffer a low verification reference voltage VREF2, which corresponds to the first level (L0) of the 32 levels (L0-L31) of one 5-bit cell, and the buffer 4503 provides that voltage to one end of the resistor string 4504. For example, the high verification reference voltage may be 900 mV, and the low verification reference voltage may be 300 mV. The voltage ladder (resistor string) 4504 generates N voltages ranging from V0 to VN, which represent the N possible values that can be stored in the VMM array. Those reference voltages are then used by the verification circuit 4403 in FIG. 44A. In another example, the input voltage and/or output voltage of the buffers 4502 and 4503 are trimmed to adjust the range and calibrate any offset, such as buffer offset. Alternatively, the ITV 4501 can directly drive the resistor string 4504 to provide 32 reference levels (L0-L31).

In one example, K verification reference voltage lines may be used to provide N different voltages. For example, for a 5-bit cell, 32 verification reference voltage lines are required to feed the verification reference voltage selection circuit 4406, and for a 6-bit cell, 64 verification reference voltage lines are required. The 32 reference voltage lines are each used twice to provide 64 verification reference voltages by time multiplexing their use, such that the 32 lines provide a first set of 32 voltages during a first verification period and a second set of 32 voltages during a second verification period. This method can be extended by using four verification periods to provide 128 voltages for a 7-bit cell, or using eight verification periods to provide 256 voltages for an 8-bit cell, etc., but is not limited to this.

In one example, the bias voltages at the array inputs used to verify operation (e.g., CG bias and EG bias) are generated by a reference array so that these bias voltages adapt with temperature to keep the array current as constant as possible.

46-50 illustrate examples of reference arrays that may be used with reference array 4304 of FIG. 43 .

FIG. 46 depicts a physical array 4600. The physical array 4600 includes an array of non-volatile memory cells. The non-volatile memory cells may optionally include stacked gate flash memory cells or split gate flash memory cells. The physical array 4600 is divided into two types of arrays—the VMM array 3401 (as in FIG. 34 ) and the reference array 4304. In one example, the VMM array 3401 and the reference array 4304 share the same bit lines. In another example, the VMM array 3401 and the reference array 4304 use separate sets of bit lines, where the two sets of bit lines are separated.

47 depicts a physical array 4700 that is divided into two arrays—VMM array 3401 and reference array 4304. In one example, VMM array 3401 and reference array 4304 share one or more sets of horizontal lines, such as word lines, control gate lines, and erase lines. In another example, VMM array 3401 and reference array 4304 do not share any sets of horizontal lines.

FIG48 depicts an example where reference array 4304 and VMM array 3401 are located in separate physical arrays. For example, there may be a substrate separation or active diffusion separation between the two arrays. Physical array 4801 includes VMM array 3401, and physical array 4802 includes reference array 4304. VMM array 3401 and reference array 4304 do not share any bit lines, word lines, control gate lines, or erase lines.

FIG49 depicts an example of a reference array 4304. Here, the reference array 4304 includes a plurality of sub-reference arrays, for example, sub-reference arrays 4901-0, 4901-1, ..., 4901-(n-1), and 4901-n. Therefore, the reference array 4304 includes n+1 different sub-reference arrays. Different sub-reference arrays may have different characteristics, which enables each sub-reference array to have a different I-V curve characteristic from other sub-reference arrays. For example, each sub-reference array may differ in one or more of the following dimensions: (1) width of control gate lines of transistors in each reference array; (2) width of word lines of transistors in each reference array; (3) width of floating gates of transistors in each reference array; (4) total width of non-volatile memory cells in each reference array; (5) shallow trench isolation (STI) spacing within each reference array; (6) other characteristics. Furthermore, each sub-reference array may differ in one or more device implant conditions or doping characteristics (e.g., well implant conditions, source implant conditions, drain implant conditions, but not limited thereto).

FIG50 depicts another example of a reference array 4304. Here, the reference array 4303 includes a plurality of sub-reference arrays, for example, sub-reference arrays 5001-0, 5001-1, ..., 5001-(n-1), and 5001-n, and 5002-0, 5002-1, ..., 5002-(n-1), and 5002-n. Therefore, the reference array 4304 includes 2*(n+1) different sub-reference arrays, which means that the number is twice that of FIG49. As in FIG49, different sub-reference arrays in FIG50 can have different characteristics, which makes each sub-reference array have different characteristics of I-V curves from other sub-reference arrays. For example, the sub-reference arrays may differ in one or more of the following dimensions: control gate width, word line width, floating gate width, total width of non-volatile memory cells in the array, STI spacing, and device implantation conditions, but are not limited thereto.

It should be noted that, as used herein, the terms "above" and "on" both include "directly on" (without intervening materials, elements, or spaces disposed therebetween) and "indirectly on" (with intervening materials, elements, or spaces disposed therebetween). Similarly, the term "adjacent" includes "directly adjacent" (without intervening materials, elements, or spaces disposed therebetween) and "indirectly adjacent" (with intervening materials, elements, or spaces disposed therebetween), "mounted to" includes "directly mounted to" (without intervening materials, elements, or spaces disposed therebetween) and "indirectly mounted to" (with intervening materials, elements, or spaces disposed therebetween), and "electrically coupled to" includes "directly electrically coupled to" (without intervening materials or elements electrically connecting the elements together) and "indirectly electrically coupled to" (with intervening materials or elements electrically connecting the elements together). For example, forming a component "over a substrate" may include forming the component directly on the substrate without intervening materials/components therebetween, as well as forming the component indirectly on the substrate with one or more intervening materials/components therebetween.

12: semiconductor substrate 14: source region 16: drain region 18: channel region 20: floating gate 22: word line terminal 24: bit line 28: control gate 30: erase gate 31: digital to analog converter 32: vector matrix multiplication (VMM) array 32a: VMM array 32b: VMM array 32c: VMM array 32d: VMM array 32e: VMM array 33: Non-volatile memory cell array 34: Erase gate and word line gate decoder 35: Control gate decoder 36: Bit line decoder 37: Source line decoder 38: Differential adder 39: Activation function block 210: Memory cell 3 10: 4-gate memory cell 4 10: 3-gate memory cell 5 10: Stacked gate memory cell 7 10: Memory cell 900: Neuron VMM Array 901: Memory array of non-volatile memory cells 902: Reference array of non-volatile reference memory cells 903: Control gate 904: Erase gate 1000: Neuron VMM Array 1001: Reference array of first non-volatile reference memory cells 1002: Reference array of second non-volatile Reference array of reference memory unit 1003: Memory array of non-volatile memory unit 1014: Multiplexer 1100: Neuron VMM array 1101: Reference array of first non-volatile reference memory unit 1102: Reference array of second non-volatile reference memory unit 1103: Non-volatile Memory array 1200 of a nonvolatile memory unit: neuron VMM array 1201: reference array of a first nonvolatile reference memory unit 1202: reference array of a second nonvolatile reference memory unit 1203: memory array of a nonvolatile memory unit 1204: stacked transistor 1205: Multiplexer 1212: Multiplexer 1300: Neuron VMM array 1301: Reference array of first non-volatile reference memory unit 1302: Reference array of second non-volatile reference memory unit 1303: Memory array of non-volatile memory unit 1314: Multiplexer 1400: LSTM 1401: unit 1402: unit 1403: unit 1404: unit 1500: LSTM unit 1501: sigmoid function device 1502: sigmoid function device 1503: sigmoid function device 1504: tanh device 1505: tanh device 1506: multiplication device 1507: multiplication device 1508: multiplication device 1509: Addition device 1600: LSTM unit 1601: VMM array 1602: Activation function block 1700: LSTM unit 1701: VMM array 1702: Activation function block 1703: Multiplication device 1704: Register 1705: Register 1706: Register 1707: Register 1708: Addition device 1709: Multiplexer 1710: Multiplexer 1800: GRU 1801: Unit 1802: Unit 1803: Unit 1804: Unit 1900: GRU unit 1901: sigmoid function device 1902: sigmoid function device 1903: tanh device 1904: multiplication device 1905: multiplication device 1906: multiplication device 1907: addition device 1908: complementary device 2000: GRU unit 2001: VMM array 2002: activation function block 2100: GRU unit 2101: VMM array 2102: activation function block 2103: multiplication device 2104: multiplexer 2105: addition device 2106: temporary storage 2107: register 2108: register 2109: complementary device 2200: neuron VMM array 2300: neuron VMM array 2400: neuron VMM array 2500: neuron VMM array 2600: neuron VMM array 2700: neuron VMM array 2701-1-2701-N: bit line control gate 2800: neuron VMM array 2900: neuron VMM array 3000: neuron VMM array 3100: VMM system 3101: summing circuit 3102: summing circuit 3210: VMM system 3211: first array 3212: second array 32 13: Summing circuit 3300: VMM system 3301: Array 3302: Array 3303: Summing circuit 3304: Summing circuit 3305: Summing circuit 3306: Summing circuit 3307: Summing circuit 3308: Summing circuit 3400: VMM system 3401: VMM array 3402: Column decoder 3403: High voltage decoder 3404: Row decoder 3405: Bit line driver 3406: Input circuit 3407: Output circuit 3408: Control logic 3409: Bias generator 3410: High voltage generation block 3411: Charge pump 3412: Charge pump regulator 3413: High voltage Level generator 3414: Algorithm controller 3415: Analog circuit 3416: Control engine 3417: Test control logic 3418: Static random access memory (SRAM) block 3701: Sequence 3702: Sequence 3703: Sequence 3801: Sequence 3802: Sequence 3803: Sequence 4001: Current source 4002: Memory unit 4300: VMM system 4301: Current to voltage converter and analog to digital converter block 4302: Verification circuit 4303: Reference voltage generator 4304: Reference array 4305: Main reference current generator 4401: Current to voltage converter (ITV) 4402: Continuous Approximation Register (SAR) Analog to Digital Converter (ADC) 4403: Verification Circuit 4404: Comparator 4405: Verification Register 4406: Reference Voltage Selection Circuit 4480: Regional Differential Operational Amplifier 4488: Neuron Output ITV+ADC+Verification Circuit 4490: Comparator and Offset Circuit 4490C: Capacitor ITV (CITV) 4490R: Resistor ITV (RITV) 4491: Comparator 4492: Calibration circuit 4493: Calibration circuit 4500: Reference voltage generator 4501: Current to voltage converter 4502: Buffer 4503: Buffer 4504: Resistor string 4510: Capacitor current to voltage converter (CITV) 4510a: Capacitor 4510b: Capacitor 4511: Resistor current to voltage converter (RITV) 4511a: resistor 4511b: resistor 4512: bias circuit 4600: physical array 4700: physical array 4801: physical array 4802: physical array 4901-0: sub-reference array 4901-1: sub-reference array 4901-(n-1): sub-reference array 4901-n: sub-reference array 5001-0: sub-reference array 5001-1: sub-reference array 5001-2: sub-reference array 5001-3: sub-reference array 5001-4: sub-reference array 5001-5: sub-reference array 5001-6: sub-reference array 5001-7: sub-reference array 5001-8: sub-reference array 5001-9: sub-reference array 5001-10: sub-reference array 5001-11: sub-reference array 5001-12: sub-reference array 5001-13: sub-reference array 5001-14: sub-reference array 5001-15: sub-reference array 01-1: sub-reference array 5001-(n-1): sub-reference array 5001-n: sub-reference array 5002-0: sub-reference array 5002-1: sub-reference array 5002-(n-1): sub-reference array 5002-n: sub-reference array BL0-BLN: bit line BLR0: terminal BLR1: terminal BLR2: terminal BLR3: terminal c ₀ : Unit state vector c ₁ : Unit state vector c ₂ : Unit state vector c(t-1): Unit state vector c(t): Unit state vector C1: Layer C2: Layer C3: Layer CB1: Synapse CB2: Synapse CB3: Synapse CB4: Synapse CG0: Voltage input (control gate line) CG1: Voltage input (control gate line) CG2: Voltage input (control gate line) CG3: Voltage input (control gate line) CG ₀ -CG _M : Control gate line COMPOUT: Output COMPOUTB: Complement of output COMPOUT EG0: EG line EG1: EG line EGR0: EG line EGR1: EG line h ₀ : Output vector h ₁ : Output vector h ₂ : Output vector h ₃ : Output vector h(t-1): Output vector h(t): Output vector INPUT ₀ -INPUT _M : Input INPUT ₀ -INPUT _N : Input OUTPUT ₀ -OUTPUT _N : Output P1: Excitation function P2: Excitation function S0: Input layer S1: Layer S2: Layer S3: Output layer S1A: Switch SL0: Source line SL1: Source line SL2: Source line SL3: Source line VINN: Input Vinp: Positive output of ITV 4401 VINP: Input VON: Offset voltage VOP: Offset voltage VREF2: Low verification reference voltage WL0: Voltage input (word line) WL1: voltage input (word line) WL2: voltage input (word line) WL3: voltage input (word line) WL ₀ -WL _M : word line WLA0: word line WLA1: word line WLA2: word line WLA3: word line WLB0: word line WLB1: word line WLB2: word line WLB3: word line x ₀ : input vector x ₁ : input vector x ₂ : input vector x ₃ : input vector x(t): input vector

FIG1 is a diagram illustrating an artificial neural network.

FIG. 2 depicts a prior art split gate flash memory cell.

FIG. 3 depicts another known split-gate flash memory cell.

FIG. 4 depicts another prior art split gate flash memory cell.

FIG. 5 depicts another prior art split gate flash memory cell.

FIG6 is a diagram illustrating different levels of an exemplary artificial neural network utilizing one or more non-volatile memory arrays.

FIG7 is a block diagram illustrating a VMM system.

FIG8 is a block diagram illustrating an exemplary artificial neural network for use with one or more VMM systems.

FIG9 depicts another specific example of a VMM system.

FIG10 depicts another specific example of a VMM system.

Figure 11 depicts another specific example of a VMM system.

Figure 12 depicts another specific example of a VMM system.

Figure 13 depicts another specific example of a VMM system.

Figure 14 depicts the long-term and short-term memory system for learning skills.

FIG15 depicts an exemplary unit for use in a long short-term memory system.

FIG16 depicts an exemplary implementation of the unit of FIG15.

FIG17 depicts another exemplary implementation of the unit of FIG15.

Figure 18 depicts the gated recursive unit system of the learning technique.

FIG. 19 depicts an exemplary cell for use in a gated recurrent cell system.

FIG. 20 depicts an exemplary implementation of the unit of FIG. 19 .

FIG. 21 depicts another exemplary implementation of the unit of FIG. 19 .

Figure 22 depicts another example of a VMM system.

Figure 23 depicts another example of a VMM system.

Figure 24 depicts another example of a VMM system.

Figure 25 depicts another example of a VMM system.

Figure 26 depicts another example of a VMM system.

Figure 27 depicts another example of a VMM system.

Figure 28 depicts another example of a VMM system.

Figure 29 depicts another example of a VMM system.

Figure 30 depicts another example of a VMM system.

Figure 31 depicts another example of a VMM system.

Figure 32 depicts another example of a VMM system.

Figure 33 depicts another example of a VMM system.

Figure 34 depicts another example of a VMM system.

Figures 35A and 35B depict individual programming methods.

Figure 36 depicts the search and execute method.

Figure 37 depicts the precise stylized approach.

Figure 38 depicts the precise stylized approach.

Figure 39 depicts the adaptive calibration method.

Figure 40 depicts the calibration circuit.

Figure 41 depicts the adaptive calibration method.

Figure 42 depicts the absolute calibration method.

Figure 3 depicts a VMM system including verification circuitry.

Figure 44A depicts an example verification circuit.

FIG4B depicts an example comparator circuit with offset compensation.

Figure 45 depicts the reference voltage generator.

Figure 46 depicts an entity array including a reference array.

Figure 47 depicts an entity array including a reference array.

FIG. 48 depicts a physical array including a VMM array and another physical array including a reference array.

FIG. 49 depicts a reference array comprising a plurality of reference sub-arrays.

FIG. 50 depicts a reference array comprising a plurality of reference sub-arrays.

C1: Layer

C2: Layer

C3: Layer

CB1: contact

CB2:Touch

CB3:Touch

CB4:Touch

P1: incentive function

P2: incentive function

S1: Layer

S2: Layer

S3: Output layer

Claims (32)

A system includes: a vector matrix multiplication array including a plurality of non-volatile memory cells arranged in columns and rows, each of which is capable of storing one of N possible levels corresponding to one of N possible currents; and a plurality of output blocks for receiving current from respective rows of the vector matrix multiplication array and generating voltage during a verification operation of the vector matrix multiplication, and generating digital output during a read operation of the vector matrix multiplication array. A system as claimed in claim 1, wherein the plurality of output blocks convert current from the rows of the array into voltage using a plurality of resistors or a plurality of capacitors. The system of claim 1, comprising: A reference voltage generator for generating one of N voltages during the verification operation. The system of claim 3, comprising: A verification circuit for comparing a voltage from the reference voltage generator with a voltage from one of the plurality of output blocks. A system as claimed in claim 4, wherein the verification circuit generates a digital output indicating a result of the comparison. The system of claim 3, wherein the reference voltage generator generates one of the N voltages in response to a current received from a reference array. A system as in claim 3, wherein the reference voltage generator generates one of the N voltages in response to a current received from a main reference current generator. The system of claim 3, wherein the reference voltage generator comprises: a current-to-voltage converter for converting a maximum current among the N possible currents into a maximum voltage; and a resistor string for generating N voltages ranging between the maximum voltage and a minimum voltage. A system as claimed in claim 8, wherein the resistor string comprises (N-1) resistors connected in series. The system of claim 8, comprising a first buffer for providing the maximum voltage to a first end of the resistor string. The system of claim 10, comprising a second buffer for providing the maximum voltage to a second end of the resistor string. A system includes: a current-to-voltage converter for converting a current from a vector matrix array into a voltage; a continuous approximation register analog-to-digital converter for receiving the voltage from the current-to-voltage converter and generating a digital output during a read operation; and a verification circuit for receiving the voltage from the current-to-voltage converter and comparing the voltage to a reference voltage during a verification operation. A system as claimed in claim 12, wherein the reference voltage is provided by a reference voltage generator, the reference voltage generator comprising: a current-to-voltage converter for converting a maximum current among N possible currents into a maximum voltage; and a resistor string for generating N voltages ranging between the maximum voltage and a minimum voltage. A system as claimed in claim 13, wherein the resistor string comprises (N-1) resistors connected in series. The system of claim 13, comprising a first buffer for providing the maximum voltage to a first end of the resistor string. The system of claim 15, comprising a second buffer for providing the maximum voltage to a second end of the resistor string. A system includes: a vector matrix multiplication array including a plurality of non-volatile memory cells arranged in columns and rows, each of which is capable of storing one of N possible voltages corresponding to one of N possible currents; and a plurality of current-to-voltage converters and a plurality of verification circuits for receiving current from the rows of the array and generating voltage during a verification operation of the vector matrix multiplication array and generating digital output during a read operation of the vector matrix multiplication array. The system of claim 17, comprising: A reference voltage generator for generating one of N voltages during the verification operation of the vector matrix multiplication array. The system of claim 18, comprising: A comparator for comparing a voltage from the reference voltage generator with a voltage from one of the plurality of current-to-voltage converters. A system as claimed in claim 19, wherein the comparator performs offset calibration. A system as in claim 20, wherein the offset calibration is performed in the time domain. A system as in claim 17, wherein the current-to-voltage converter uses a plurality of resistors or a plurality of capacitors to convert current from the rows of the array into a voltage. A system includes: a vector matrix multiplication array including a plurality of non-volatile memory cells arranged in columns and rows, each of which is capable of storing one of N possible levels corresponding to one of N possible currents; and a plurality of output blocks for receiving differential currents from the rows of the vector matrix multiplication array and generating voltages during a verification operation of the vector matrix multiplication array. The system of claim 23 further comprises: A verification circuit for generating an output. A system as claimed in claim 23, wherein the plurality of output blocks include respective current-to-voltage converters for converting current from rows of the array of the vector matrix multiplication array into voltage using a plurality of resistors or a plurality of capacitors, respectively. A system as in claim 25, wherein the plurality of output blocks include respective analog-to-digital converters for converting the voltages from the current-to-voltage converters into digital outputs, and wherein the verification operation utilizes the digital outputs. A method, comprising: receiving a differential current according to the formula w=(w+)–(w-), wherein (w+) is received from a first row of a vector matrix multiplication array and (w-) is received from a second row of the vector matrix multiplication array; and verifying the differential current against a reference current. A system includes: a vector matrix multiplication array including a plurality of non-volatile memory cells arranged in columns and rows, each of which is capable of storing one of N possible levels corresponding to one of N possible currents; and a plurality of reference voltage generators generating K reference voltages on K reference voltage lines, wherein K < N, and wherein the K reference voltage lines are verified for the N possible currents in a time multiplexed manner. The system of claim 28, comprising: A plurality of current-to-voltage converters for receiving current from rows of the array of the vector matrix multiplication array and generating voltage during a verification operation of the vector matrix multiplication array. A system as in claim 29, wherein the current-to-voltage converter uses a plurality of resistors or a plurality of capacitors to convert current from the rows of the array of the vector matrix multiplication array into voltage. A system as claimed in claim 28, comprising: A verification circuit for generating a comparison output. A system as claimed in claim 28, comprising: an analog-to-digital converter for generating a comparison output.