TW202407579A - Artificial neural network comprising a three-dimensional integrated circuit - Google Patents

Abstract

Numerous examples are disclosed of an artificial neural network comprising a three-dimensional integrated circuit. In one embodiment, a three-dimensional integrated circuit for use in an artificial neural network comprises a first die comprising a first vector by matrix multiplication array and a first input multiplexor, the first die located on a first vertical layer; a second die comprising an input circuit, the second die located on a second vertical layer different than the first vertical layer; and one or more vertical interfaces coupling the first die and the second die; wherein during a read operation, the input circuit provides an input signal to the first input multiplexor over at least one of the one or more vertical interfaces, the first input multiplexor applies the input signal to one or more rows in the first vector by matrix multiplication array, and the first vector by matrix multiplication array generates an output.

Description

Artificial neural network containing three-dimensional integrated circuits

[Priority claim] This application claims U.S. Provisional Patent Application No. 63/328,126, filed on April 6, 2022 and titled "Artificial Neural Network Comprising a Three-Dimensional Integrated Circuit". Priority is granted to U.S. Patent Application No. 17/848,371, filed on June 23, 2022, and titled "Artificial Neural Networks Including Three-Dimensional Integrated Circuits."

Revealed many examples of artificial neural networks including three-dimensional integrated circuits.

Artificial neural networks simulate biological neural networks (the central nervous system of animals, especially the brain) and are used to estimate or estimate functions that may depend on a large number of inputs and are often unknown. Artificial neural networks typically consist of layers of interconnected "neurons" that exchange messages with each other.

Figure 1 illustrates an artificial neural network, where circles represent inputs or layers of neurons. Connections (called synapses) are represented by arrows and have numerical weights that can be tuned based on experience. This allows the neural network to adapt to the input and learn. Typically, neural networks include multiple input layers. There are typically one or more layers of interneurons and a layer of output neurons that provide the output of the neural network. Neurons at each level make decisions individually or collectively based on the data received from synapses.

One of the major challenges in the development of artificial neural networks for high-performance information processing is the lack of appropriate hardware technology. In fact, practical neural networks rely on an extremely large number of synapses to achieve high connectivity between neurons, which means extremely high computational parallelism. In principle, this complexity could be achieved using digital supercomputers or clusters of specialized graphics processing units. However, in addition to high cost, these methods also suffer from moderate energy efficiency compared to biological networks, mainly because biological networks perform low-precision analog calculations and therefore consume much less energy. CMOS analog circuits have been used in artificial neural networks, but most synapses implementing CMOS are too large due to the large number of neurons and synapses.

The applicant previously disclosed an artificial (analog) neural network using one or more non-volatile memory arrays as synapses in US Patent Application Publication 2017/0337466A1, which is incorporated by reference. . The non-volatile memory array operates analogously to a neural memory and contains non-volatile memory cells arranged in columns and rows. The neural network includes: a first plurality of synapses configured to receive a first plurality of inputs and to generate a first plurality of outputs from the first plurality of inputs; and a first plurality of neurons configured to to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each of the memory cells includes: a source region and a drain region formed in the semiconductor substrate, and the channel region is between the source region and the drain region. extending between the drain regions; a floating gate mounted above and insulated from a first portion of the channel region; and a non-floating gate mounted above a second portion of the channel region and insulated from the second Partially insulated. Each of the plurality of memory cells stores a weight value corresponding to the number of electrons on the floating gate. A plurality of memory cells multiply a first plurality of inputs by the stored weight values to generate a first plurality of outputs. non-volatile memory cells

Non-volatile memories are well known. For example, U.S. Patent 5,029,130 (the "'130 patent"), which is incorporated herein by reference, discloses an array of split-gate non-volatile memory cells, which is a type of flash memory cell. This memory cell 210 is shown in Figure 2. Each memory cell 210 includes a source region 14 and a drain region 16 formed in the semiconductor substrate 12, with a channel region 18 located between the source region and the drain region. Floating gate 20 is formed over and insulated from (and controls the conductivity of) a first portion of channel region 18 and over a portion of source region 14 . Wordline terminal 22 (which is typically coupled to the wordline) has a first portion disposed over and insulated from (and controls the conductivity of) a second portion of channel region 18 ; and a second portion extending on and above the floating gate 20 . The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by gate oxide. Bit line 24 is coupled to drain region 16 .

Memory cell 210 is erased (with electrons removed from the floating gate) by placing a high positive voltage on word line terminal 22, which causes the electrons on floating gate 20 to pass through the Fowler -Nordheim; FN) tunnels from the floating gate 20 through the intermediate insulator to the word line terminal 22.

Memory cell 210 is programmed by applying source side injection (SSI) of hot electrons by placing a positive voltage on word line terminal 22 and a positive voltage on source region 14 (The electrons are placed on the floating gate). Electron current will flow from the drain region 16 toward the source region 14 . When electrons reach the gap between word line terminal 22 and floating gate 20, they will be accelerated and heated. Some of the heated electrons will be injected through the gate oxide onto the floating gate 20 due to the attractive electrostatic force from the floating gate 20 .

Memory cell 210 is read by placing a positive read voltage on drain region 16 and word line terminal 22 (this turns on the portion of channel region 18 below the word line terminal). If floating gate 20 is positively charged (i.e., the electrons are erased), then the portion of channel region 18 below floating gate 20 will also turn on, and current will flow across channel region 18, which is sensed as erased. Divide or "1" status. If floating gate 20 is negatively charged (i.e., programmed electronically), then the portion of the channel region below floating gate 20 is mostly or completely disconnected, and current will not flow across channel region 18 (or there will be (very little current flows), this is sensed as a programmed or "0" state.

Table 1 describes typical voltage and current ranges that may be applied to the terminals of memory cell 210 for performing read, erase, and program operations: Table 1: Operation of flash memory cell 210 of Figure 2 wL BL SL read 2-3V 0.6-2V 0V Erase ~11-13V 0V 0V stylized 1-2V 10.5-3μA 9-10V

Other split-gate memory cell configurations are known, which are other types of flash memory cells. For example, FIG. 3 depicts a four-gate memory cell 310 that includes a source region 14 , a drain region 16 , a floating gate 20 over a first portion of channel region 18 , a second portion of channel region 18 An upper select gate 22 (typically coupled to word line WL), a control gate 28 above floating gate 20 , and an erase gate 30 above source region 14 . This configuration is described in US Patent 6,747,310, which is incorporated herein by reference for all purposes. Here, except floating gate 20, all gates are non-floating gates, which means that these gates are electrically connected or can be electrically connected to a voltage source. Programming is performed by heated electrons from channel region 18 injecting themselves onto floating gate 20 . Erasing is performed by electron tunneling from floating gate 20 to erase gate 30.

Table 2 depicts typical voltage and current ranges that may be applied to the terminals of memory cell 310 for performing read, erase, and program operations: Table 2: Operation of flash memory cell 310 of Figure 3 WL/SG BL CG EG SL read 1.0-2V 0.6-2V 0-2.6V 0-2.6V 0V Erase -0.5V/0V 0V 0V/-8V 8-12V 0V stylized 1V 0.1-1μA 8-11V 4.5-9V 4.5-5V

Figure 4 depicts a three-gate memory cell 410, which is another type of flash memory cell. The memory cell 410 is the same as the memory cell 310 of FIG. 3 , except that the memory cell 410 does not have a separate control gate. The erase operation (whereby erasure is performed through the use of the erase gate) and the read operation are similar to those of Figure 3, except that no control gate bias is applied. Programming operations also occur without controlled gate bias, and therefore, higher voltages are applied to the source lines during programming operations to compensate for the lack of controlled gate bias.

Table 3 depicts typical voltage and current ranges that may be applied to the terminals of memory cell 410 for performing read, erase, and program operations: Table 3: Operation of flash memory cell 410 of Figure 4 WL/SG BL EG SL read 0.7-2.2V 0.6-2V 0-2.6V 0V Erase -0.5V/0V 0V 11.5V 0V stylized 1V 0.2-3μA 4.5V 7-9V

Figure 5 depicts a stacked gate memory cell 510, which is another type of flash memory cell. Memory cell 510 is similar to memory cell 210 of Figure 2, except that floating gate 20 extends over the entire channel region 18, and control gate 22 (which will be coupled to the word line here) Extends above the floating gate 20 and is separated by an insulating layer (not shown in the figure). Erasing is performed by FN tunneling of electrons from the FG to the substrate. Programming is performed by channel hot electron (CHE) injection in the area between channel region 18 and drain region 16, by electrons from the source. Region 14 flows toward drain region 16 and the read operation is similar to the read operation for memory cell 210 with a higher control gate voltage.

Table 4 describes typical voltage ranges that may be applied to the terminals of memory cell 510 and substrate 12 for performing read, erase, and program operations: Table 4: Operation of flash memory cell 510 of Figure 5 CG BL SL substrate read 2-5V 0.6-2V 0V 0V Erase -8 to -10V/0V FLT FLT 8-10V/15-20V stylized 8-12V 3-5V 0V 0V

The methods and approaches described herein may be applied to other non-volatile memory technologies such as, but not limited to, FINFET split gate flash or stacked gate flash, NAND flash, silicon-oxide-nitride- Oxide-silicon (SONOS, charge trapping in nitride), metal-oxide-nitride-oxide-silicon (MONOS, metal charge trapping in nitride), resistive random access memory (ReRAM), Phase change memory (PCM), magnetic ram (MRAM), ferroelectric ram (FeRAM), charge trapping (CT) memory, carbon tube (CN) memory, dual-level or multi-level one-time programmable (OTP) ) and related electronic ram (CeRAM).

To utilize a memory array containing one of the non-volatile memory cell types described above in artificial neural networks, two modifications are made. First, the lines are organized so that each memory cell can be programmed, erased, and read individually without adversely affecting the memory state of other memory cells in the array, as explained further below. Second, a sequential (analogous) stylization of memory cells is provided.

Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can independently and continuously change from a fully erased state to a fully erased state with minimal interference to other memory cells. Completely stylized state, and vice versa. This means that the cell memory effectively analogues or at least can store one of many discrete values (such as 16 or 64 different values), which allows for extremely precise and individual tuning of all memory cells in the memory array. , and this makes memory arrays ideal for storing and tuning synaptic weights in neural networks. Neural network using non-volatile memory cell arrays

Figure 6 conceptually illustrates a non-limiting example of a neural network utilizing a non-volatile memory array of embodiments of the present invention. This example uses a non-volatile memory array neural network for a face recognition application, but any other suitable application can be implemented using a non-volatile memory array based neural network.

S0 is the input layer, which for this example is a 32×32 pixel RGB image with 5-bit precision (that is, three 32×32 pixel arrays, one for each color R, G, and B, and each pixel is 5 bit precision). Synapse CB1 traveling from input layer S0 to layer C1 applies a different set of weights in some cases and shares weights in other cases, and scans the input image with a 3×3 pixel overlapping filter (kernel), shifting the filter by 1 pixels (or more than 1 pixel, as specified by the model). Specifically, the values of 9 pixels in a 3×3 portion of the image (i.e., called the filter or kernel) are provided to synapse CB1 where the 9 input values are multiplied by the appropriate weights, and after summing the outputs of their multiplications, a single output value is determined by the first synapse CB1 and provided for generating a pixel of one of the feature maps of layer C1. The 3×3 filter is then shifted one pixel to the right within the input layer S0 (i.e., a row of three pixels is added on the right and a row of three pixels is discarded on the left), thereby positioning this new filter Nine of the pixel values are provided to synapse CB1, where they are multiplied by the same weight and a second single output value is determined by the associated synapse. This process continues for all three colors and for all bits (precision values) until the 3x3 filter is scanned across the entire 32x32 pixel image of the input layer S0. The process is then repeated using different sets of weights to generate different feature maps for layer C1 until all feature maps for layer C1 have been calculated.

In this example, there are 16 feature maps in layer C1, each feature map has 30×30 pixels. Each pixel is a new feature pixel extracted from the input multiplied by the kernel, and therefore each feature map is a two-dimensional array, and therefore in this example, layer C1 constitutes 16 layers of the two-dimensional array (it should be remembered that in this article The layers and arrays mentioned are logical relationships, not necessarily physical relationships - that is, the arrays are not necessarily oriented in a physical two-dimensional array). Each of the 16 feature maps in layer C1 is generated from one of sixteen different sets of synaptic weights applied to the filter sweep. C1 feature maps can all target different aspects of the same image feature, such as boundary recognition. For example, a first image (generated using a first set of weights, common to all scans used to generate this first image) can identify rounded edges, and a second image (generated using a second set of weights different from the first set of weights) Produces) that can identify rectangular edges, or the aspect ratio of certain features, etc.

The activation function P1 (pooling) is applied before entering layer S1 from layer C1, which pools values from consecutive non-overlapping 2×2 regions in each feature map. The purpose of the pooling function P1 is to average nearby positions (or a maximum function can also be used), for example to reduce the dependence of edge positions and reduce the data size before entering the next stage. At layer S1, there are sixteen 15x15 feature maps (ie, sixteen different arrays of 15x15 pixels each). The synapse CB2 from layer S1 into layer C2 scans the image in layer S1 using a 4×4 filter, where the filter is shifted by 1 pixel. At layer C2, there are 22 12×12 feature maps. The activation function P2 (pooling) is applied before entering layer S2 from layer C2, which pools values from consecutive non-overlapping 2×2 regions in each feature map. At layer S2, there are 22 6×6 feature maps. An activation function (pooling) is applied at the synapses CB3 from layer S2 into layer C3, where each neuron in layer C3 is connected to each graph in layer S2 via a respective synapse in CB3. At layer C3, there are 64 neurons. Synapse CB4 from layer C3 into output layer S3 completely connects C3 to S3, that is, every neuron in layer C3 is connected to every neuron in layer S3. The output at S3 includes 10 neurons, among which the highest output neuron determines the category. This output may, for example, indicate the identification or classification of the content of the original image.

Each synaptic layer is implemented using an array, or a portion of an array, of non-volatile memory cells.

Figure 7 is a block diagram of an array that may be used for this purpose. Vector matrix multiplication (VMM) array 32 includes non-volatile memory cells and serves as a synapse between one layer and the next (such as CB1, CB2, CB3, and CB4 in Figure 6). Specifically, the VMM array 32 includes a non-volatile memory cell array 33, an erase gate and a word line gate decoder 34, a control gate decoder 35, a bit line decoder 36 and a source line decoder. Decoders 37 decode respective inputs of the non-volatile memory cell array 33. Inputs to VMM array 32 may come from erase gate and word line gate decoders 34 or from control gate decoders 35. In this example, source line decoder 37 also decodes the output of non-volatile memory cell array 33. Alternatively, bit line decoder 36 may decode the output of non-volatile memory cell array 33.

The non-volatile memory cell array 33 serves two purposes. First, it stores the weights to be used by the VMM array 32. Next, the non-volatile memory cell array 33 effectively multiplies the input by the weight stored in the non-volatile memory cell array 33 and adds the results according to the output line (source line or bit line) to produce Output, which will be the input to the next layer or the input to the final layer. By performing multiply and add functions, non-volatile memory cell array 33 eliminates the need for separate multiply and add logic circuits, and is also power efficient due to its in-memory computation.

The output of the non-volatile memory cell array 33 is supplied to a differential summer (such as a summing operational amplifier or summing current mirror) 38 which sums the output of the non-volatile memory cell array 33 to produce a single value for that convolution. Difference summer 38 is configured to perform a summation of positive and negative weights.

The summed output value of the difference summer 38 is then supplied to an activation function block 39 which rectifies the output. The activation function block 39 can provide sigmoid, hyperbolic tangent (tanh) or ReLU functions. The rectified output values of activation function block 39 become elements of the feature map of the next layer (eg, C1 in Figure 6) and are then applied to the next synapse to produce the next feature map or final layer. Therefore, in this example, the non-volatile memory cell array 33 forms a plurality of synapses (which receive input from the previous neuron layer or from an input layer such as an image database), and the sum operation The amplifier 38 and the activation function block 39 constitute a plurality of neurons.

The inputs to VMM array 32 in Figure 7 (WLx, EGx, CGx, and optionally BLx and SLx) may be analog levels, binary levels, or digital bits (in which case a DAC is provided to convert the digital bits to the appropriate input analog levels), and the output may be analog levels, binary levels, or digital bits (in which case an output ADC is provided to convert the output analog levels to digital bits).

8 is a block diagram depicting the use of numerous layers of VMM array 32, labeled here as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in Figure 8, the input denoted as Inputx is converted from digital to analog by a digital-to-analog converter 31 and provided to the input VMM array 32a. The converted analog input can be voltage or current. Input D/A conversion of the first layer may be performed by using a function or LUT (look-up table) that maps input input x to the appropriate analog level for the matrix multiplier input to VMM array 32a. Input conversion may also be performed by an analog-to-analog (A/A) converter to convert an external analog input into a mapped analog input to the input VMM array 32a.

The output produced by the input VMM array 32a is provided as an input to the next VMM array (hidden level 1) 32b, which in turn produces an output provided to the next VMM array (hidden level 2) 32c input, etc. The various layers of VMM array 32 serve as different synaptic and neuronal layers of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e may be a separate physical non-volatile memory array, or multiple VMM arrays may utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays may utilize Overlapping portions of identical physical non-volatile memory arrays. The example shown in Figure 8 contains five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c) and two fully connected layers (32d, 32e) . Those skilled in the art should understand that this is illustrative only and the system may alternatively include more than two hidden layers and more than two fully connected layers. Vector matrix multiplication (VMM) array

Figure 9 depicts a neuronal VMM array 900 that is particularly suitable for memory cells 310 as shown in Figure 3 and serves as part of the synapses and neurons between the input layer and the next layer. VMM array 900 includes a memory array 901 of non-volatile memory cells and a reference array 902 of non-volatile reference memory cells (at the top of the array). Alternatively, another reference array can be placed at the bottom.

In VMM array 900, control gate lines, such as control gate lines 903, run in the vertical direction (thus, reference array 902 in the column direction is orthogonal to control gate lines 903), and erase gate lines, such as The erasure gate line of 904 extends in the horizontal direction. Here, the input to the VMM array 900 is provided on the control gate lines (CG0, CG1, CG2, CG3) and the output of the VMM array 900 appears on the source lines (SL0, SL1). In one instance, only even columns are used, and in another instance, only odd columns are used. The current placed on each source line (SL0, SL1 respectively) performs a summation function on all currents from the memory cells connected to that particular source line.

As described herein for neural networks, the non-volatile memory cells of VMM array 900, ie, memory cells 310 of VMM array 900, may be configured to operate in sub-threshold regions.

The non-volatile reference memory cells and non-volatile memory cells described in this article are biased in weak inversion (sub-threshold): Ids = Io * e ^{(Vg- Vth) /nVt} = w * Io * e ^(Vg)/nVt , where w = e ^{(- Vth)/nVt} where Ids is the drain to source current; Vg is the gate voltage on the memory cell; Vth is the memory The threshold voltage of the cell; Vt is the thermal voltage = k*T/q, where k is the Boltzmann constant, T is the temperature in Kelvin, and q is the electron charge; n is the slope Factor = 1 + (Cdep/Cox), where Cdep = capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer; Io is the memory cell current at the gate voltage equal to the threshold voltage, and Io is Proportional to (Wt/L)*u*Cox* (n-1) * Vt ² , where u is the carrier mobility of the memory cell, and Wt and L are the width and length respectively.

For an I-to-V logarithmic converter that uses a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert input current to input voltage: Vg= n*Vt*log [Ids/wp*Io] Among them, wp is the w of the reference or peripheral memory cell.

For a memory array used as a vector matrix multiplier VMM array with current input, the output current is: Iout = wa * Io * e ^(Vg)/nVt , that is, Iout = (wa/wp) * Iin = W * Iin W = e ^{(Vthp - Vtha)/nVt} Here, wa = w of each memory cell in the memory array. Vthp is the effective threshold voltage of the peripheral memory cell, and Vtha is the effective threshold voltage of the main (data) memory cell. It should be noted that the threshold voltage of a transistor is a function of the substrate back bias voltage, and the substrate back bias voltage, denoted Vsb, can be modulated to compensate for various conditions at this temperature. The threshold voltage Vth can be expressed as: Vth = Vth0 + γ (SQRT |Vsb – 2*ϕF) - SQRT |2* ϕF |) where Vth0 is the threshold voltage with zero substrate bias, ϕF is the surface potential, and γ System effect parameters.

The word lines or control gates can be used as inputs to the memory cells for input voltages.

Alternatively, the flash memory cells of the VMM arrays depicted herein can be configured to operate in the linear region: Ids =β* (Vgs-Vth)*Vds; β= u*Cox*Wt/L W = α (Vgs-Vth) This means that the weight W in the linear region is proportional to (Vgs-Vth).

Word lines or control gates or bit lines or source lines may be used as inputs to memory cells operating in the linear region. Bit lines or source lines can be used as the output of the memory cell.

For I to V linear converters, memory cells (such as reference memory cells or peripheral memory cells) or transistors operating in the linear region can be used to linearly convert input/output currents into input/output voltages .

Alternatively, the memory cells of the VMM array described herein can be configured to operate in the saturation region: Ids = * β* (Vgs-Vth) ² ; β = u*Cox*Wt/L Wα (Vgs-Vth) ² , which means that the weight W is proportional to (Vgs-Vth) ² .

Word lines, control gates, or erase gates can be used as inputs to memory cells operating in the saturation region. Bit lines or source lines can be used as the output of the output neuron.

Alternatively, the memory cells of the VMM arrays depicted herein may be used in all regions or combinations thereof (subcritical regions, linear regions, or saturation regions) of each layer or layers of a neural network.

Other examples of VMM array 32 of Figure 7 are described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in that application, source lines or bit lines can be used as neuron outputs (current summation outputs).

Figure 10 depicts a neuronal VMM array 1000 that is particularly suitable for memory cells 210 as shown in Figure 2 and serves as a synapse between an input layer and the next layer. The VMM array 1000 includes a memory array 1003 of non-volatile memory cells, a reference array 1001 of first non-volatile reference memory cells, and a reference array 1002 of second non-volatile reference memory cells. Reference arrays 1001 and 1002 arranged in the row direction of the array are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2 and BLR3 into voltage inputs WL0, WL1, WL2 and WL3. In practice, the first and second non-volatile reference memory cells are diode-connected through-multiplexers 1014 (only partially described) with current inputs flowing into the multiplexers. The reference cells are tuned (eg, programmed) to a target reference level. The target reference level is provided by a reference small array matrix (not shown).

Memory array 1003 serves two purposes. First, it stores the weights to be used by the VMM array 1000 on its respective memory cells. Secondly, the memory array 1003 effectively causes the inputs (i.e., the current inputs provided in the terminals BLR0, BLR1, BLR2, and BLR3) to be converted by the reference arrays 1001 and 1002 into input voltages to be supplied to the word lines WL0, WL1, WL2 and WL3) are multiplied by the weights stored in memory array 1003, and all results (memory cell currents) are then summed to produce an output on the respective bit lines (BL0 to BLN), which output will be Input to the next layer or input to the final layer. By performing multiply and add functions, memory array 1003 eliminates the need for separate multiply and add logic circuits and is also power efficient. Here, voltage inputs are provided on word lines WL0, WL1, WL2, and WL3, and outputs appear on respective bit lines BL0 through BLN during read (inference) operations. The current placed on each of the bit lines BL0 through BLN performs a summation function on the currents from all non-volatile memory cells connected to that particular bit line.

Table 5 depicts the operating voltages and currents for VMM array 1000. The rows in the table indicate the voltages placed on: character lines for selected cells, character lines for unselected cells, bit lines for selected cells, bit lines for unselected cells. Bit lines for selected cells, source lines for selected cells, and source lines for unselected cells. The columns indicate read, erase, and program operations. Table 5: Operation of the VMM array 1000 of Figure 10 wL WL - not selected BL BL - not selected SL SL - not selected read 1-3.5V -0.5V/0V 0.6-2V (Ineuron) 0.6V-2V/0V 0V 0V Erase ~5-13V 0V 0V 0V 0V 0V stylized 1-2V -0.5V/0V 0.1-3uA Vinh ~2.5V 4-10V 0-1V/FLT

Figure 11 depicts a neuronal VMM array 1100 that is particularly suitable for memory cells 210 as shown in Figure 2 and serves as part of the synapses and neurons between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of non-volatile memory cells, a reference array 1101 of first non-volatile reference memory cells, and a reference array 1102 of second non-volatile reference memory cells. Reference arrays 1101 and 1102 extend in the column direction of VMM array 1100 . The VMM array is similar to VMM 1000 except that in VMM array 1100 the word lines run in a vertical direction. Here, inputs are provided on word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3) and outputs appear on source lines (SL0, SL1) during read operations. The current placed on each source line performs a summation function on all currents from the memory cells connected to that particular source line.

Table 6 depicts the operating voltages and currents for VMM array 1100. The rows in the table indicate the voltages placed on: character lines for selected cells, character lines for unselected cells, bit lines for selected cells, bit lines for unselected cells. Bit lines for selected cells, source lines for selected cells, and source lines for unselected cells. The columns indicate read, erase, and program operations. Table 6: Operation of VMM array 1100 of Figure 11 wL WL - not selected BL BL - not selected SL SL - not selected read 1-3.5V -0.5V/0V 0.6-2V 0.6V-2V/0V ~0.3-1V (Ineuron) 0V Erase ~5-13V 0V 0V 0V 0V SL-Suppression (~4-8V) stylized 1-2V -0.5V/0V 0.1-3uA Vinh ~2.5V 4-10V 0-1V/FLT

Figure 12 depicts a neuronal VMM array 1200 that is particularly suitable for the memory cells 310 shown in Figure 3 and serves as part of the synapses and neurons between the input layer and the next layer. VMM array 1200 includes a memory array 1203 of non-volatile memory cells, a reference array 1201 of first non-volatile reference memory cells, and a reference array 1202 of second non-volatile reference memory cells. Reference arrays 1201 and 1202 are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2 and BLR3 into voltage inputs CG0, CG1, CG2 and CG3. In practice, the first and second non-volatile reference memory cells are diode-connected through multiplexers 1212 (only partially shown) with current inputs flowing into the multiplexers via BLR0, BLR1, BLR2 and BLR3 middle. Multiplexers 1212 each include a respective multiplexer 1205 and cascade transistor 1204 to ensure that a bit line (such as BLRO) of each of the first and second non-volatile reference memory cells during a read operation constant voltage on. The reference cells are tuned to the target reference level.

Memory array 1203 serves two purposes. First, it stores the weights that will be used by the VMM array 1200. Second, memory array 1203 effectively provides inputs (current inputs) to terminals BLR0, BLR1, BLR2, and BLR3, where reference arrays 1201 and 1202 convert these current inputs into input voltages for supply to control gates (CG0, CG1 , CG2 and CG3) are multiplied by the weights stored in the memory array, and all results (cell currents) are then summed to produce an output, which appears on BL0 to BLN and will be the input to the next layer or to Inputs to the final layer. By executing the multiply and add functions, the memory array eliminates the need for separate multiply and add logic circuits and is also power efficient. Here, the inputs are provided on the control gate lines (CG0, CG1, CG2 and CG3), and the outputs appear on the bit lines (BL0 to BLN) during read operations. The current placed on each bit line performs on all currents from the memory cells connected to that particular bit line Sum function.

The VMM array 1200 performs unidirectional tuning for the non-volatile memory cells in the memory array 1203 . That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is placed on the floating gate (causing an incorrect value to be stored in the cell), the cell is erased and the sequence of partially programmed operations begins again. As shown, two columns sharing the same erase gate (such as EG0 or EG1) are erased together (this is known as a page erase), and thereafter, each cell is partially programmed until reaching the floating gate. to the desired charge.

Table 7 depicts the operating voltages and currents for VMM array 1200. The rows in the table indicate the voltages placed on the word lines for selected cells, the word lines for unselected cells, the bit lines for selected cells, and the bit lines for unselected cells. Bit lines for cells, control gates for selected cells, control gates for unselected cells in the same sector as the selected cell, control gates for unselected cells in different sectors from the selected cell Control gate for unselected cells, erase gate for selected cells, erase gate for unselected cells, source line for selected cells, and source for unselected cells polar line. The columns indicate read, erase, and program operations. Table 7: Operation of VMM array 1200 of Figure 12 wL WL-not selected BL BL-not selected CG CG - Same sector not selected CG-Not selected EG EG-not selected SL SL-not selected read 1.0-2V -0.5V/0V 0.6-2V (Ineuron) 0V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0V 0V Erase 0V 0V 0V 0V 0V 0-2.6V 0-2.6V 5-12V 0-2.6V 0V 0V stylized 0.7-1V -0.5V/0V 0.1-1uA Vinh(1-2V) 4-11V 0-2.6V 0-2.6V 4.5-5V 0-2.6V 4.5-5V 0-1V

Figure 13 depicts a neuronal VMM array 1300 that is particularly suitable for memory cells 310 as shown in Figure 3 and serves as part of the synapses and neurons between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of non-volatile memory cells, a reference array 1301 of first non-volatile reference memory cells, and a reference array 1302 of second non-volatile reference memory cells. The EG lines EGR0, EG0, EG1 and EGR1 run vertically, while the CG lines CG0, CG1, CG2 and CG3 and the SL lines WL0, WL1, WL2 and WL3 run horizontally. The VMM array 1300 is similar to the VMM array 1400 except that the VMM array 1300 implements bidirectional tuning, where each individual cell may be fully erased, partially programmed, and partially erased to achieve float as needed due to the use of separate EG lines. The required amount of charge on the gate. As shown, reference arrays 1301 and 1302 convert input currents in terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 to be applied to the memory cells in the column direction (via two The polar body-connected reference cells perform actions through the multiplexer 1314). The current outputs (neurons) are in bit lines BL0 through BLN, where each bit line sums all currents from the non-volatile memory cells connected to that particular bit line.

Table 8 depicts the operating voltages and currents for VMM array 1300. The rows in the table indicate the voltages placed on the word lines for selected cells, the word lines for unselected cells, the bit lines for selected cells, and the bit lines for unselected cells. Bit lines for cells, control gates for selected cells, control gates for unselected cells in the same sector as the selected cell, control gates for unselected cells in different sectors from the selected cell Control gate for unselected cells, erase gate for selected cells, erase gate for unselected cells, source line for selected cells, and source for unselected cells polar line. The columns indicate read, erase, and program operations. Table 8: Operation of VMM array 1300 of Figure 13 wL WL-not selected BL BL– not selected CG CG – Same sector not selected CG-Not selected EG EG-not selected SL SL-not selected read 1.0-2V -0.5V/0V 0.6-2V (Ineuron) 0V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0V 0V Erase 0V 0V 0V 0V 0V 4-9V 0-2.6V 5-12V 0-2.6V 0V 0V stylized 0.7-1V -0.5V/0V 0.1-1uA Vinh(1-2V) 4-11V 0-2.6V 0-2.6V 4.5-5V 0-2.6V 4.5-5V 0-1V

Figure 22 depicts a neuronal VMM array 2200 that is particularly suitable for memory cells 210 as shown in Figure 2 and serves as part of the synapses and neurons between the input layer and the next layer. In VMM array 2200, inputs INPUT ₀ , ..., INPUT _N are received on bit lines BL ₀ , ..., BL _N respectively, and outputs OUTPUT ₁ , OUTPUT ₂ , OUTPUT ₃ and OUTPUT ₄ are respectively generated on source line SL ₀ , SL ₁ , SL ₂ and SL ₃ .

Figure 23 depicts a neuronal VMM array 2300 that is particularly suitable for memory cells 210 as shown in Figure 2 and serves as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , INPUT ₁ , INPUT ₂ and INPUT ₃ are received on source lines SL ₀ , SL ₁ , SL ₂ and SL ₃ respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on On the bit lines BL ₀ , ..., BL _N.

Figure 24 depicts a neuronal VMM array 2400 that is particularly suitable for memory cells 210 as shown in Figure 2 and serves as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., _WLM, respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N .

Figure 25 depicts a neuronal VMM array 2500 that is particularly suitable for memory cells 310 as shown in Figure 3 and serves as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., _WLM, respectively, and outputs OUTPUT _{0 ,} ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N .

Figure 26 depicts a neuronal VMM array 2600 that is particularly suitable for memory cells 410 as shown in Figure 4 and serves as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , . . . , INPUT _n are received on vertical control gate lines CG ₀ , . . . , _CGN respectively, and outputs OUTPUT ₁ and OUTPUT ₂ are generated on source lines SL ₀ and SL ₁ .

Figure 27 depicts a neuronal VMM array 2700 that is particularly suitable for memory cells 410 as shown in Figure 4 and serves as part of the synapses and neurons between the input layer and the next layer. In this example, the inputs INPUT ₀ ,..., INPUT _N are respectively received on the gates of the bit line control gates 2701-1, 2701-2,..., 2701-(N-1) and 2701-N. The poles are respectively coupled to bit lines BL ₀ , ..., BL _N . Example outputs OUTPUT ₁ and OUTPUT ₂ are generated on source lines SL ₀ and SL ₁ .

Figure 28 depicts a neuronal VMM array 2800 that is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells as shown in Figure 7 710, and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., _WLM , and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL, respectively. ₀ ,...,BL _N on.

Figure 29 depicts a neuron VMM array 2900 that is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells as shown in Figure 7 710, and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG ₀ , ..., _CGM . The outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical source lines SL ₀ , ..., SL _N respectively, where each source line SL _i is coupled to the source lines of all memory cells in row i.

Figure 30 depicts a neuronal VMM array 3000 that is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells as shown in Figure 7 710, and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG ₀ , ..., _CGM . The outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical bit lines BL ₀ , ..., BL _N respectively, where each bit line BL _i is coupled to the bit lines of all memory cells in row i. long short term memory

Prior technology includes a concept called long short-term memory (LSTM). LSTM units are often used in neural networks. LSTM allows a neural network to remember information for any predetermined time interval and use that information in subsequent operations. It is known that the LSTM unit includes cells, input gates, output gates and forgetting gates. The three gates regulate the flow of information into and out of the cell and the time interval during which information is remembered in the LSTM. VMM is especially suitable for LSTM cells.

Figure 14 depicts an example LSTM 1400. LSTM 1400 in this example includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives an input vector x ₀ and generates an output vector h ₀ and a cell state vector c ₀ . Cell 1402 receives the input vector x ₁ , the output vector (hidden state) h ₀ from cell 1401 , and the cell state c ₀ from cell 1401 , and generates an output vector h ₁ and a cell state vector c ₁ . Cell 1403 receives the input vector x ₂ , the output vector (hidden state) h ₁ from cell 1402, and the cell state c ₁ from cell 1402, and generates an output vector h ₂ and a cell state vector c ₂ . Cell 1404 receives the input vector x ₃ , the output vector (hidden state) h ₂ from cell 1403, and the cell state c ₂ from cell 1403, and produces an output vector h ₃ . Additional cells can be used, and the LSTM with four cells is only an example.

Figure 15 depicts an example implementation of LSTM cell 1500, which may be used with cells 1401, 1402, 1403, and 1404 in Figure 14. The LSTM cell 1500 receives the input vector x(t), the cell state vector c(t-1) from the previous cell, and the output vector h(t-1) from the previous cell, and generates the cell state vector c( t) and the output vector h(t).

LSTM cell 1500 includes sigmoid function components 1501, 1502, and 1503, each of which applies a number between 0 and 1 to control how much each component of the input vector is allowed to pass through the output vector. LSTM cell 1500 also includes hyperbolic tangent components 1504 and 1505 for applying the hyperbolic tangent function to the input vector, multiplier components 1506, 1507, and 1508 for multiplying the two vectors together, and Addition component 1509 for adding two vectors together. The output vector h(t) may be provided to the next LSTM cell in the system, or may be accessed for other purposes.

Figure 16 depicts an LSTM cell 1600, which is an example of an implementation of the LSTM cell 1500. For the convenience of the reader, the same numbers from LSTM cell 1500 are used in LSTM cell 1600. Sigmoid function components 1501, 1502 and 1503 and hyperbolic tangent component 1504 each include a plurality of VMM arrays 1601 and activation function blocks 1602. Therefore, it can be seen that the VMM array is particularly suitable for LSTM cells used in some neural network systems. The multiplier components 1506, 1507 and 1508 and the adding component 1509 are implemented digitally or analogously. Activation function block 1602 may be implemented digitally or analogously.

An alternative to LSTM cell 1600 (and another example of an implementation of LSTM cell 1500) is shown in Figure 17. In FIG. 17 , the S-shaped function components 1501, 1502 and 1503 and the hyperbolic tangent component 1504 share the same physical hardware (VMM array 1701 and activation function block 1702) in a time multiplexing manner. The LSTM cell 1700 also contains a multiplier component 1703 for multiplying two vectors together, an addition component 1708 for adding two vectors together, and a hyperbolic tangent component 1505 (which contains the activation function block 1702 ), a register 1707 for storing i(t) when the value i(t) is output from the S-type function block 1702, and a register 1707 for storing the value f(t) * c(t-1) via the multiplexer 1710 A register 1704 for storing the value when it is output from the multiplier component 1703, a register 1705 for storing the value i(t) * u(t) when it is output from the multiplier component 1703 via the multiplexer 1710, And a register 1706 for storing the value o(t)*c~(t) when it is output from the multiplier component 1703 via the multiplexer 1710 and the multiplexer 1709.

LSTM cell 1600 contains multiple sets of VMM arrays 1601 and respective activation function blocks 1602 , while LSTM cell 1700 contains only one set of VMM arrays 1701 and activation function blocks 1702 , the VMM arrays 1701 and the activation function Block 1702 is used to represent multiple layers in an instance of LSTM cell 1700. LSTM cell 1700 will require less space than LSTM 1600 because LSTM cell 1700 will require 1/4 of the space for VMM and activation function blocks compared to LSTM cell 1600.

It is further understood that an LSTM cell will typically contain multiple VMM arrays, each of which requires functionality provided by certain circuit blocks external to the VMM array, such as summer and activation function blocks and high voltage generation blocks. Providing separate circuit blocks to each VMM array would require a large amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the circuitry required outside the VMM array itself. gated recursive unit

Analogous VMM implementations can be used in gated recurrent unit (GRU) systems. GRU is the gate control mechanism in recurrent neural networks. GRU is similar to LSTM, except that GRU cells usually contain fewer components than LSTM cells.

Figure 18 depicts an example GRU 1800. GRU 1800 in this example includes cells 1801, 1802, 1803, and 1804. Cell 1801 receives the input vector _x0 and produces the output vector _h0 . Cell 1802 receives input vector x ₁ , output vector h ₀ from cell 1801, and generates output vector h ₁ . Cell 1803 receives the input vector x ₂ and the output vector (hidden state) h ₁ from cell 1802, and generates the output vector h ₂ . Cell 1804 receives the input vector x ₃ and the output vector (hidden state) h ₂ from cell 1803 and produces an output vector h ₃ . Additional cells may be used, and a GRU with four cells is an example only.

Figure 19 depicts an example implementation of a GRU cell 1900 that may be used in cells 1801, 1802, 1803, and 1804 of Figure 18. GRU cell 1900 receives the input vector x(t) and the output vector h(t-1) from the previous GRU cell, and generates the output vector h(t). GRU cell 1900 includes sigmoid function components 1901 and 1902, each of which applies a number between 0 and 1 to components from the output vector h(t-1) and the input vector x(t). The GRU cell 1900 also includes a hyperbolic tangent component 1903 for applying a hyperbolic tangent function to an input vector, a plurality of multiplier components 1904, 1905, and 1906 for multiplying two vectors together, and a plurality of multiplier components 1904, 1905, and 1906 for An additive component 1907 that adds vectors together and a complementary component 1908 that subtracts the input from 1 to produce the output.

Figure 20 depicts GRU cell 2000, which is an example of an implementation of GRU cell 1900. For the convenience of the reader, the same numbers from GRU cell 1900 are used in GRU cell 2000. As can be seen in Figure 20, the sigmoid function components 1901 and 1902 and the hyperbolic tangent component 1903 each include a plurality of VMM arrays 2001 and activation function blocks 2002. Therefore, it can be seen that the VMM array is particularly suitable for GRU cells used in some neural network systems. The multiplier components 1904, 1905, 1906, the adding component 1907 and the complementary component 1908 are implemented digitally or analogously. Activation function block 2002 may be implemented digitally or analogously.

An alternative to GRU cell 2000 (and another example of an implementation of GRU cell 1900) is shown in Figure 21. In Figure 21, GRU cell 2100 utilizes VMM array 2101 and activation function block 2102, which when configured as a sigmoid function applies a number between 0 and 1 to control each component of the input vector. The amount allowed to pass through the output vector. In Figure 21, the S-shaped function components 1901 and 1902 and the hyperbolic tangent component 1903 share the same physical hardware (VMM array 2101 and activation function block 2102) in a time multiplexing manner. GRU cell 2100 also contains a multiplier block 2103 for multiplying two vectors together, an addition block 2105 for adding two vectors together, and a complementary block for subtracting the input from 1 to produce the output. 2109. Multiplexer 2104. A register 2106 for saving the value h(t-1) * r(t) when it is output from the multiplier component 2103 via the multiplexer 2104. -1) The register 2107 that holds the value of *z(t) when it is output from the multiplier component 2103 via the multiplexer 2104, and is used to store the value h^(t) * (1-z(t)) via the multiplexer 2104 When the processor 2104 outputs from the multiplier block 2103, the register 2108 holds the value.

GRU cell 2000 contains multiple sets of VMM arrays 2001 and activation function blocks 2002, while GRU cell 2100 contains only one set of VMM arrays 2101 and activation function blocks 2102, which is used to represent the example of GRU cell 2100 of multiple layers. GRU cell 2100 will require less space than GRU cell 2000 because GRU cell 2100 will require 1/3 of the space for VMM and activation function blocks compared to GRU cell 2000.

It is further understood that a GRU system will typically contain multiple VMM arrays, each of which requires functionality provided by certain circuit blocks external to the VMM array, such as summer and activation function blocks and high voltage generation blocks. Providing separate circuit blocks to each VMM array would require a large amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the circuitry required outside the VMM array itself.

The input to the VMM array can be an analog level, a binary level, a pulse, a time-modulated pulse, or a digital bit (in which case a DAC is required to convert the digital bit to the appropriate input analog level), and The output can be an analog level, a binary level, a timing pulse, a pulse, or a digital bit (in this case, an output ADC is required to convert the output analog level into a digital bit).

Generally speaking, for each memory cell in the VMM array, each weight W can be implemented by a single memory cell or a differential cell or two hybrid memory cells (the average of 2 cells). In the case of differential cells, two memory cells are required to implement the weight W as a differential weight (W = W+ - W-). In two hybrid memory cells, two memory cells are required to implement the weight W as the average of the two cells.

Figure 31 depicts VMM system 3100. In some examples, the weight W stored in the VMM array is stored as a differential pair W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). In the VMM system 3100, half of the bit lines are designated as W+ lines, that is, the bit lines connected to the memory cells that will store the positive weight W+, and the other half of the bit lines are designated as W- lines, also That is, the bit lines connected to memory cells implementing negative weight W-. The W- lines are interspersed with the W+ lines in an alternating manner. The subtraction operation is performed by summing circuits that receive current from the W+ and W- lines, such as summing circuits 3101 and 3102. The output of the W+ line and the output of the W- line are combined, effectively giving W = W+ - W- for each pair of (W+, W-) cells for all pairs of (W+, W-) lines. Although the above has been described with respect to W- lines being interspersed among W+ lines in an alternating manner, in other examples, the W+ lines and W- lines can be arbitrarily located anywhere in the array.

Figure 32 depicts another example. In the VMM system 3210, the positive weight W+ is implemented in the first array 3211 and the negative weight W- is implemented in the second array 3212, the second array 3212 is separated from the first array, and the resulting weights are obtained by the summation circuit 3213 put together appropriately.

Figure 33 depicts VMM system 3300. The weight W stored in the VMM array is stored as a differential pair W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). VMM system 3300 includes array 3301 and array 3302. Half of the bit lines in each of arrays 3301 and 3302 are designated as W+ lines, that is, the bit lines connected to the memory cells that will store the positive weight W+, and half of the bit lines in each of arrays 3301 and 3302 The other half of the bit lines are designated as W- lines, that is, the bit lines connected to memory cells implementing negative weight W-. The W- lines are interspersed with the W+ lines in an alternating manner. The subtraction operation is performed by summing circuits that receive current from the W+ and W- lines, such as summing circuits 3303, 3304, 3305, and 3306. The outputs of the W+ line and the W- line from each array 3301, 3302 are combined together to effectively obtain W = for each pair of (W+, W-) cells of all pairs of (W+, W-) lines. W+ - W-. Additionally, the W values from each array 3301 and 3302 may be further combined via summing circuits 3307 and 3308 such that each W value is the result of the W value from array 3301 minus the W value from array 3302, meaning that from The final result of the sum circuits 3307 and 3308 is the difference of the two difference values.

Each non-volatile memory cell used in an analog neural memory system is erased and programmed to maintain a very specific and precise amount of charge, or number of electrons, in the floating gate. For example, each floating gate should hold one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, 64, 128, and 256.

Figure 34 depicts a block diagram of VMM system 3400. VMM system 3400 includes VMM array 3401, column decoder 3402, high voltage decoder 3403, row decoder 3404, bit line driver 3405, input circuit 3406, output circuit 3407, control logic 3408 and bias generator 3409. The VMM system 3400 further includes a high voltage generation block 3410 that includes a charge pump 3411, a charge pump regulator 3412, and a high voltage analog precision level generator 3413. VMM system 3400 further includes (programmed/erased, or weight-tuned) algorithm controller 3414, analog circuitry 3415, control engine 3416 (which may include but is not limited to special functions such as arithmetic functions, activation functions, embedded microcontrollers controller logic), and test control logic 3417.

Input circuitry 3406 may include circuitry such as a digital-to-analog converter (DAC), a digital-to-pulse converter (DPC, a digital-to-time modulated pulse converter), an analog-to-analog converter (AAC, such as a current-to-voltage converter, logarithmic converter), pulse-to-analog converter (PAC), or any other type of converter. Input circuitry 3406 may implement one or more of a normalization, a linear or nonlinear scaling up/down function, or an arithmetic function. Input circuit 3406 may implement a temperature compensation function for the input level. Input circuit 3406 may implement an activation function such as ReLU or sigmoid.

Output circuitry 3407 may include circuits such as analog-to-digital converters (ADCs to convert neuron analog outputs into digital bits), analog-to-analog converters (AACs such as current-to-voltage converters, logarithmic converters) , analog to pulse converter (APC, analog to time modulated pulse converter), or any other type of converter. Output circuit 3407 may implement an activation function, such as a rectified linear activation function (ReLU) or sigmoid. The output circuit 3407 may implement statistical normalization, regularization, scaling/gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shift, logarithm) on the neuron output. one or more. The output circuit 3407 may implement a temperature compensation function for the neuron output or the array output (such as the bit line output) in order to keep the power consumption of the array approximately constant or to improve the array (neuron) output, such as by keeping the IV slope approximately the same. accuracy.

As applications of artificial neural networks become more complex, there is an increasing need for larger VMM arrays. At the same time, there is a need to efficiently use space within a packaged integrated circuit and save as much power as possible while still maintaining accuracy so that each of the N different weights still stores and reads appropriately.

Various examples are described for providing artificial neural network systems that include three-dimensional integrated circuits that include one or more VMM arrays.

3D VMM system architecture

Figure 35 depicts a 3D VMM system 3500 that includes a plurality of dies, such as dies 3501, 3502, 3503, 3504, 3505, and 3506, that are vertically stacked within a package 3522 to form a packaged integrated circuit. 3D VMM system 3500 includes certain functional blocks that are functionally similar to those included in VMM system 3400 in Figure 34, but the blocks may be located on different dies. Here, components contained in dies 3501 and 3502 share components contained in dies 3503, 3504, 3505, and 3506.

In this embodiment, die 3501 contains respective VMM arrays 3507 (functionally similar to VMM 3401 in Figure 34), respective input multiplexers 3509, respective column buffers 3523 (which can provide, for example, sample and hold buffering voltage to the array input), respective high voltage decoders 3508 (similar in function to high voltage decoder 3403 in Figure 34), and respective neuron circuits 3510 (which may perform, for example, but not limited to, scaling of the array output current function, min/max limit function, differential output conversion, buffering). Input multiplexer 3509 receives the analog input signal and applies the analog input signal to VMM array 3507 from which neuron circuit 3510 receives an analog output signal representative of the neuron output. The output signals can be sent to other blocks within the 3D VMM system 3500.

Die 3502 also contains respective VMM arrays 3507, respective input multiplexers 3509, respective column buffers 3522, respective high voltage decoders 3508, and respective neuron circuits 3510.

In this embodiment, two dies (die 3501 and 3502) contain respective VMM arrays 3507, but it is understood that additional dies containing respective VMM arrays may be included.

Die 3503 contains a high voltage generator 3511 (similar in function to high voltage generating block 3410 in Figure 34), analog circuitry 3512 (similar in function to analog circuitry 3415 in Figure 34), and temperature compensation circuitry 3513. The 3D VMM system 3500 may have thermal challenges that the 2D VMM system 3400 does not. Because dies 3501, 3502, 3503, 3504, 3505, and 3506 are stacked in a vertical configuration and contain different types of circuitry, each die may experience different thermal operating conditions during operation. For example, some dies will become hotter than other dies, and the rate of temperature increase may vary among the dies. This introduces the possibility of inaccuracies due to thermal changes. Temperature compensation circuit 3513 compensates for temperature changes experienced among various dies. Optionally, one or more thermal sensors are located on each respective die to provide temperature data to temperature compensation circuitry 3513. Temperature compensation circuit 3513 then changes the trim or configuration settings to compensate for any changes in temperature. The temperature compensation circuit 3513 also compensates for temperature changes of each die, such as compensating for changes in cell current within temperature, such that the resulting bit line (neuron) currents are approximately the same in temperature. Temperature compensation circuitry 3513 is also used in the neuron circuit 3510, DAC, and ADC circuits so that the operating dynamic ranges (eg, the output range of the DAC, the input range of the ADC, the output range of the neuron circuit 3510) are approximately the same across temperatures.

Die 3504 contains input circuitry 3514 (functionally similar to input circuitry 3406), which includes address decoding circuitry 3524, column registers 3525 (holding active input values for array columns), and digital-to-analog converters (DACs) 3515 . DAC 3515 receives digital signals from register 3525 and converts them into analog signals.

Die 3505 contains analog-to-digital converter (ADC) 3516. ADC 3516 receives analog signals and converts them into digital signals.

Die 3506 contains digital circuitry 3517, static random access memory (SRAM) 3518, register 3519, physical I/O connection 3520, digital accelerator 3531 and network-on-chip (NOC) 3715. Die 3506 provides control functions for other die. Digital circuitry 3517 may include digital logic, microcontrollers, single instruction multiple data (SIMD) processors, and processors. SRAM 3518 and register 3519 may be used to store system information and fabric information used by digital circuit 3517 or other circuits or blocks in 3D VMM system 3500. Physical I/O connections 3520 provide an IO interface to components external to the VMM system 3500 (such as an external processing unit) or to another package 3522. Digital accelerator 3531 is used in certain neural networks or certain layers within a neural network where additional processing may be required, such as when there are small activation sizes, where the weights stored in the cells are dynamic and not fixed, The MAC operation needs to be performed but is not limited to this. NOC 3715 provides network routing functions within 3D VMM system 3500, such as by generating control signals to route signals from one block to another.

Each of the plurality of dies is connected to one or more other dies of the plurality of dies via a vertical interface 3521 , which respectively connects two or more dies together. In one embodiment, vertical interface 3521 is implemented as a through silicon via (TSV).

During a read operation of 3D VMM system 3500, digital input is received through input circuit 3514. The digital input enables column register 3525, which stores the activation input and in response to the digital input applies the selected activation input to DAC 3515, which converts the digital output from column register 3525 into respective analog signals. Analog signals generated by DAC 3515 are provided by input circuitry 3514 via one or more vertical interfaces 3521 to input multiplexer 3509 and column buffer 3523 on one or more of dies 3501, 3502, which are then Applying the signal to one or more columns in the respective VMM array 3507 causes an output to be generated by the VMM array 3507. The output from the respective VMM array 3507 is received by a respective neuron circuit 3510 which provides a buffering function to drive the parasitic capacitance of one or more vertical interfaces 3521 to which it is connected. Neuron circuit 3510 provides analog signals via one or more vertical interfaces 3521 to ADC 3516 on die 3505, which converts the analog signals into digital signals. Alternatively, the analog signal can bypass the ADC 3516 and remain in analog form. The output of ADC 3516 is provided via physical I/O 3520 to components external to 3D VMM system 3500 (such as a processing unit or graphics processing unit), or applied to respective VMM arrays 3507 (representing another layer in an artificial neural network). Enter. Alternatively, the analog signal from the neuron circuit 3510 may bypass the ADC 3516 and remain in analog form and applied as input to the respective VMM array 3507.

Figure 36 depicts a 3D VMM system 3600 including a package 3622. The 3D VMM system 3600 is similar to the 3D VMM system 3500 and contains many of the same components, except for some differences in the placement of the components and certain additional components. The same items as in Figure 35 contain the same number of items as in Figure 36. Here, components contained in dies 3601 and 3602 share components contained in dies 3603, 3604, 3605, and 3606.

3D VMM system 3600 includes a plurality of dies, such as dies 3601, 3602, 3603, 3604, 3605, and 3606, which are vertically stacked in a common package 3522 to form a packaged integrated circuit.

In this embodiment, die 3601 contains individual VMM arrays 3507, individual input multiplexers 3509, individual registers 3524 (holding active input values for array columns), individual column buffers 3523, individual A separate high voltage multiplexer 3608 and a separate line multiplexer 3610.

Die 3602 also contains individual VMM arrays 3507, individual input multiplexers 3509, individual registers 3524, individual column buffers 3523, individual high voltage multiplexers 3608, and row multiplexers 3610.

In this embodiment, two dies (die 3601 and 3602) contain respective VMM arrays 3507, but it is understood that additional dies containing respective VMM arrays may be included.

Die 3603 contains a high voltage generator 3511, an analog circuit system 3512, a temperature compensation circuit 3513 and a high voltage multiplexer 3608.

Die 3604 contains input circuitry 3614 including address decoding 3524 and DAC 3515.

Die 3605 contains neuron circuit 3510 and ADC 3516.

Die 3606 contains digital circuitry 3517, SRAM 3518, register 3519 and physical I/O connections 3520.

Respective dies of the plurality of dies are connected to one or more other dies of the plurality of dies via respective vertical interfaces 3521 .

During a read operation of 3D VMM system 3600, input circuit 3614 receives input and receives an address. Address decoder 3524 decodes addresses and provides input to respective column registers 3525 corresponding to the decoded addresses via one or more vertical interfaces 3521. The output of the respective column register 3525 is coupled to a DAC 3515 through one or more vertical interfaces 3521. The DAC 3515 converts the digital output received from the column register 3525 into an analog signal and passes through one or more vertical interfaces 3521. Interface 3521 provides analog signals to respective input multiplexers 3509 and column buffers 3523. Column buffers 3523 apply buffered analog signals to column inputs of respective VMM arrays 3507 for selected columns, such as array control gates or word lines. Outputs from respective VMM arrays 3507, such as from array bit lines, are received by respective row multiplexers 3610, and the outputs of respective row multiplexers 3610 provide signals to the die via one or more vertical interfaces 3521 The ADC 3516 on the 3605 converts analog signals into digital signals. Alternatively, the signal can bypass the ADC 3516 and remain in analog form. The output of the ADC 3516 is provided to the digital circuit 3517 on the die 3606 (which can perform activation functions, pooling functions or other network functions) through one or more vertical interfaces 3521. The output of the digital circuit 3517 can be provided through the respective vertical interfaces. 3521 is provided via physical I/O 3520 on die 3606 to components external to the 3D VMM system 3600 (such as a processing unit or graphics processing unit) or to input circuitry 3614 on die 3604 as to another respective VMM array 3507 (represents the input of another layer in the artificial neural network).

Figure 37A depicts a 3D VMM system 3700 that includes a plurality of dies in two or more vertical stacks. In this embodiment, the first vertical stack includes dies 3701, 3702, 3703, 3704, 3705, and 3706, and the second vertical stack includes dies 3707, 3708, 3709, 3710, 3711, and 3712, all contained in a common package 3522 to form a single package integrated circuit. In one embodiment, the dies in the first vertical stack are physically separate dies from the dies in the second vertical stack. In another embodiment, the die in the first vertical stack and the die in the second vertical stack are the same physical die (meaning, for example, die 3701 and 3707 are the same die). Here, components contained in dies 3701 and 3702 share components contained in dies 3703, 3704, 3705, and 3706, and components contained in dies 3707 and 3708 share components contained in dies 3709, 3710, 3711, and 3712. components.

In the embodiment shown, die 3701 and die 3707 each contain a respective VMM array 3507, a respective array input 3729 (which includes a respective input multiplexer 3509, a respective address decoder 3713, a respective column register 3525 and respective column buffers 3523), respective high voltage multiplexers 3608 and respective neuron circuits 3510.

Die 3702 and die 3708 also each include a respective VMM array 3507, a respective array input 3729 (which includes a respective input multiplexer 3509, a respective address decoder 3713, a respective column register 3525 and a respective column buffer 3523), respective high voltage multiplexers 3608 and respective neuron circuits 3510.

In this embodiment, four dies (die 3701, 3702, 3707, and 3708) contain respective VMM arrays 3507, although it is understood that additional dies containing VMM arrays may be included.

Die 3703 and die 3709 each include a respective high voltage decoder 3714, a respective high voltage generator 3511, a respective analog circuitry 3512, and a respective temperature compensation circuit 3513.

Die 3704 and die 3710 each contain respective input circuits 3514 including respective DACs 3515 .

Die 3705 and die 3711 each contain a respective ADC 3516.

Die 3706 and die 3712 each include respective digital circuits 3517, respective SRAM 3518, respective registers 3519, respective physical I/O connections 3520, and respective network on-chip (NOC) connections 3715.

Each of the plurality of dies is connected to one or more other dies within the plurality of dies via one or more vertical interfaces 3521 or horizontal interfaces 3716 that respectively connect two or more dies together. grain. In one embodiment, the vertical interfaces 3521 are through silicon vias (TSVs) respectively. In one embodiment, horizontal interfaces 3716 are respectively redistribution layer (RDL) connections.

During read operations of the 3D VMM system 3700, digital inputs are received by respective input circuits 3514, which convert the digital inputs to analog signals via their respective DACs 3515, and the outputs of the respective DACs 3515 are Respective vertical interfaces 3521 and/or horizontal interfaces 3716 are coupled to column registers 3525 of respective array inputs 3729 . The outputs of column registers 3525 of respective array inputs 3729 are provided to input multiplexers 3509 and column buffers 3523, which then apply signals to one or more columns in VMM array 3507. The output from the respective VMM array 3507 is received by a respective neuron circuit 3510, which provides a buffering function to drive parasitic capacitance connected to one or more of its vertical interfaces 3521 or horizontal interfaces 3716. Neuron circuitry 3510 provides buffered analog signals via one or more vertical interfaces 3521 or horizontal interfaces 3716 to respective ADCs 3516, which convert the analog signals into digital signals. The output of the ADC 3516 is provided to a respective digital circuit 3517 (which performs an activation function, a pooling function or a network function), and the output of the respective digital circuit 3517 can be provided external to the 3D VMM system 3700 via a respective physical I/O 3520 (such as a processing unit or graphics processing unit) or to a respective input circuit 3514 to be converted by a respective DAC 3515, and the output of the respective input circuit 3514 is coupled to another VMM array via a respective physical I/O 3520 3507 (representing another layer in the artificial neural network) or to another package 3522.

Figure 37B depicts a 3D VMM system 3750 similar to the 3D VMM system 3750, except that it has another type of VMM array, shown as VMM array 3557 on die 3758, which contains static RAM cells or dynamic RAM cell.

Figure 38 depicts a 3D VMM system 3800 that includes a plurality of dies in two vertical stacks. It is possible to have more than two stacks. In this embodiment, the first vertical stack includes dies 3801, 3802, 3803, 3804, 3805, and 3806, and the second vertical stack includes dies 3807, 3808, 3809, 3810, 3811, and 3812, all contained in a common package 3522 to form a single package integrated circuit. In one embodiment, the dies in the first vertical stack are physically separate dies from the dies in the second vertical stack. In another embodiment, the die in the first vertical stack and the die in the second vertical stack are the same physical die (meaning, for example, die 3801 and 3807 are the same die). Here, components contained in dies 3801 and 3802 share components contained in dies 3803, 3804, 3805, and 3806, and components contained in dies 3807 and 3808 share components contained in dies 3809, 3810, 3811, and 3812. components.

In the embodiment shown, die 3801, 3802, 3807, and die 3808 each include a respective VMM array 3507, a respective array input 3729, a respective high voltage multiplexer 3608, a respective row multiplexer 3610, and Individual array input circuit 3729. Respective array input 3729 includes input multiplexer 3509 and address decoder 3713, column register 3524 and column buffer 3523.

In this embodiment, four dies (die 3801, 3802, 3807, and 3808) contain VMM arrays, but it is understood that additional dies containing VMM arrays may be included.

Die 3803 and die 3809 respectively include a high voltage decoder 3714, a high voltage generator 3511, an analog circuit system 3512 and a temperature compensation circuit 3513.

Die 3804 and die 3810 each contain input circuitry 3514 including a DAC 3515.

Die 3805 and die 3811 contain ADC 3516 and neuron circuit 3510 respectively.

Die 3806 and die 3812 respectively include digital circuits 3517, SRAM 3518, registers 3519, physical I/O connections 3520, and NOC connections 3715.

The plurality of dies are respectively connected to one or more other dies within the plurality of dies via one or more vertical interfaces 3521 or horizontal interfaces 3716 respectively connecting two or more dies together. In one embodiment, the respective vertical interfaces 3521 are implemented as through silicon vias (TSVs). In one embodiment, the respective horizontal interfaces 3716 are implemented as redistribution layer (RDL) connections.

During read operations of 3D VMM system 3800, digital input is received by input circuitry 3514, which converts the digital input to analog form using DAC 3515 and provides the analog signal via respective vertical interface 3521 and/or horizontal interface 3716 to respective array input circuit 3729. Array input circuit 3729 receives the analog signal from the input circuit and the address, and then applies the analog signal to the selected column in VMM array 3507 in response to the address. Output from VMM array 3507 is received by row multiplexer 3610, which provides analog signals via one or more vertical interfaces 3521 or horizontal interface 3716 to ADC 3516, which converts the analog signals to digital signals. . Alternatively, the analog signal can bypass the ADC 3516 and remain in analog form. The output of the ADC 3516 is provided to the digital circuit 3517 (which can perform activation functions, pooling functions or other network functions) via one or more vertical interfaces 3521 or horizontal interface 3716, and the output of the digital circuit 3517 can be via physical I/O 3520 provides input circuitry 3514 to components external to the 3D VMM system 3800 (such as a processing unit or graphics processing unit) or to another VMM array 3507 (representing another layer in the artificial neural network) or to another via physical I/O 3520 3522 in one package.

Figure 39A depicts a 3D VMM system 3900 that includes a plurality of dies in two vertical stacks. In this embodiment, the first vertical stack includes dies 3901, 3902, 3903, and 3904, and the second vertical stack includes dies 3905, 3906, 3907, and 3908, all contained in common package 3522 to form a single packaged area. body circuit. In one embodiment, the dies in the first vertical stack are physically separate dies from the dies in the second vertical stack. In another embodiment, the die in the first vertical stack and the die in the second vertical stack are the same physical die (meaning, for example, die 3901 and 3905 are the same die). Here, components contained in dies 3901 and 3902 share components contained in dies 3903 and 3904, and components contained in dies 3905 and 3906 share components contained in dies 3907 and 3908.

In the embodiment shown, die 3901, 3902, 3905, and die 3906 contain VMM array 3507, array input 3729, high voltage multiplexer 3608, and row multiplexer 3610, respectively.

In this embodiment, four dies (die 3901, 3902, 3903, and 3904) contain VMM arrays, but it is understood that additional dies containing VMM arrays may be included.

Die 3903 and die 3907 respectively include a high voltage decoder 3714, a high voltage generator 3511, an analog circuit system 3512, a temperature compensation circuit 3513, and an input circuit 3514 including a DAC 3515, a neuron circuit 3510, and an ADC 3516.

Die 3904 and die 3908 respectively include digital circuit 3517, SRAM 3518, register 3519, physical I/O connection 3520 and NOC connection 3715.

The plurality of dies are respectively connected to one or more other dies within the plurality of dies via one or more vertical interfaces 3521 or horizontal interfaces 3716 respectively connecting two or more dies together.

During read operations of 3D VMM system 3900, digital input is received by input circuitry 3514, which converts the digital input to analog form using DAC 3515 and provides the analog signal via respective vertical interface 3521 and/or horizontal interface 3716 to respective array input circuit 3729. Array input circuit 3729 receives the analog signal from the input circuit and the address, and applies the analog signal to the selected column in VMM array 3507 in response to the received address. Output from VMM array 3507 is received by row multiplexer 3610, which provides analog signals via one or more vertical interfaces 3521 or horizontal interface 3716 to ADC 3516, which converts the analog signals to digital signals. . Alternatively, the analog signal can bypass the ADC 3516 and remain in analog form. The output of ADC 3516 is provided to digital circuitry 3517 via respective vertical interface 3521 and/or horizontal interface 3716, and then to components external to 3D VMM system 3900 (such as a processing unit or graphics processing unit) via physical I/O 3520 or to Input circuitry 3514 to be applied as an input to VMM array 3507 (representing another layer in the artificial neural network) or to another package 3522 via physical I/O 3520.

Figure 39B depicts a 3D VMM system 3950 similar to the 3D VMM system of Figure 39A, except that dies 3951, 3952, 3955, and 3956 now have both input circuitry 3514 and array input 3729. 3D VMM system 3950 includes package 3522 and dies 3951, 3952, 3953, 3954, 3955, 3956, 3957, and 3958. The plurality of dies are respectively connected to one or more other dies within the plurality of dies via one or more vertical interfaces 3521 or horizontal interfaces 3716 respectively connecting two or more dies together.

Figure 39C depicts a 3D VMM system 3980 that includes a plurality of dies in two vertical stacks. In this embodiment, the first vertical stack includes dies 3981, 3982, 3983, and the second vertical stack includes dies 3984, 3985, and 3986, all contained in co-package 3522 to form a single packaged integrated circuit. In one embodiment, the dies in the first vertical stack are physically separate dies from the dies in the second vertical stack. In another embodiment, the die in the first vertical stack and the die in the second vertical stack are the same physical die (meaning, for example, die 3981 and 3984 are the same die). Here, components included in die 3981 and 3982 share components included in die 3983, and components included in die 3984 and 3985 share components included in die 3986.

In the illustrated embodiment, die 3981, 3982, 3984, and die 3985 contain VMM array 3507, high voltage block 3991, input block 3990, output block 3992, and analog block 3993, respectively. Input block 3990 may include input circuit 3514, DAC 3515, and array input circuit 3729. High voltage block 3991 may include high voltage multiplexer 3608 and high voltage decoder 3714. Output block 3992 may include row multiplexer 3610, neuron circuit 3510, and ADC 3516. Analog block 3993 may include a high voltage generator 3511, analog circuitry 3512, and temperature compensation circuitry 3513.

Die 3983 and die 3986 each contain digital circuitry 3517, SRAM 3518, registers 3519, physical I/O connections 3520, digital accelerator 3521 (digitally used for multiply-accumulate (MAC) functions), and NOC connections 3715.

The plurality of dies are respectively connected to one or more other dies within the plurality of dies via one or more vertical interfaces 3521 or horizontal interfaces 3716 , each of which connects two or more dies together.

Figure 40 depicts a 3D VMM system 4000 that includes a plurality of dies in two vertical stacks. In this embodiment, the first vertical stack includes dies 4001, 4002, 4003, and 4004, and the second vertical stack includes dies 4005, 4006, 4007, and 4008, all contained in co-package 3522 to form a single packaged area. body circuit. In one embodiment, the dies in the first vertical stack are physically separate dies from the dies in the second vertical stack. In another embodiment, the die in the first vertical stack and the die in the second vertical stack are the same physical die (meaning, for example, die 4001 and 4005 are the same die). Here, components contained in dies 4001 and 4002 share components contained in dies 4003 and 4004, and components contained in dies 4005 and 4006 share components contained in dies 4007 and 4008.

In the embodiment shown, dies 4001, 4002, 4005, and 4006 respectively contain a VMM array 3507, an array input 4029 (which includes an input multiplexer 3509 and/or a decoder 3713 - not shown), a high voltage multiplexer 3608 and line multiplexer 3610.

In this embodiment, four dies (die 4001, 4002, 4003, and 4004) contain VMM arrays, but it is understood that additional dies containing VMM arrays may be included.

Die 4003 and die 4007 respectively include a high voltage decoder 3714, a high voltage generator 3511, an analog circuit system 3512, a temperature compensation circuit 3513 and a neuron circuit 3510.

Die 4004 and die 4008 respectively include digital circuit 3517, SRAM 3518, register 3519, physical I/O connection 3520 and NOC connection 3715.

Unlike VMM system 3900, VMM system 4000 does not contain DAC 3515 and ADC 3516 because the input and output remain in analog form and do not convert between analog and digital forms.

The plurality of dies are respectively connected to one or more other dies within the plurality of dies via one or more vertical interfaces 3521 or horizontal interfaces 3716 respectively connecting two or more dies together.

During read operations of the 3D VMM system 4000, analog inputs (such as voltages, currents, or timing-based entities such as a series of pulses) are received by respective array inputs 4029, which in turn apply signals to respective One or more columns in VMM array 3507. Outputs from respective VMM arrays 3507 are received by row multiplexers 3610 which provide analog signals (such as voltage, current, or timing-based entities) via one or more vertical interfaces 3521 or horizontal interfaces 3716 to Separate neuron circuits 3510 that provide buffered signals via physical I/O 3520 to a component external to the 3D VMM system 4000 (such as a processing unit or graphics processing unit) or to another package 3522 or to another VMM Array input 4029 for array 3507 (representing another layer in the artificial neural network).

41-44 depict additional details regarding an example configuration of VMM array 3507. 41-44 depict 3D VMM systems 4100, 4200, 4300, and 4400, respectively. Each of the 3D VMM systems 4100, 4200, 4300, and 4400 includes a first die on a common package (not shown). VMM array 3507-1 and VMM array 3507-2 on the second die. VMM arrays 3507-1 and 3507-2 contain non-volatile memory cell arrays configured in m+1 columns and n+1 rows, respectively. A respective column is coupled to one of the control gate lines labeled CG0...CGm, and a respective row is coupled to one of the bit lines labeled BL0...BLn. The cell is located at the intersection of the bit line and the control gate line. For example, cell 4101mn is located in column m and row n and is coupled to CGm and BLn in VMM array 3507-1, and cell 4102mn is located in column m and row n and is coupled to VMM array 3507-2 CGm and BLn. Because the VMM arrays 3507-1 and 3507-2 are located on different dies, they can be manufactured using different semiconductor processes as appropriate. Regardless of whether the same or different semiconductor processes are used to fabricate VMM arrays 3507-1 and 3507-2, cells in VMM array 3507-1 may store a different number of bits than cells in VMM array 3507-2. For example, a cell in VMM array 3507-1, such as cell 4101mn, can store i bits, and a cell in VMM array 3507-2, such as cell 4102mn, can store j bits, where i and j is an integer with different values. For example, i can be 3 (meaning that the cells in the VMM array 3507-1 each store a 3-bit value), and j can be 5 (meaning that the cells in the VMM array 3507-2 each store a 5-bit value). value).

In Figure 41, inputs are provided individually to VMM arrays 3507-1 and 3507-2 via control gate lines, and outputs are obtained individually on bit lines. Figure 41 depicts example cells 4101mn and 4102mn. Optionally, the outputs may be combined elsewhere, such as by using ADC 3516 (shown in the previous figure) to convert the outputs into digital form, and using digital circuitry 3517 (shown in the previous figure) to sum the outputs.

In Figure 42, inputs are provided individually to VMM arrays 3507-1 and 3507-2 via control gate lines. The output of VMM array 3507-1 is obtained on a first set of bit lines, and the output of VMM array 3507-2 is obtained on a second set of bit lines by effectively converting the output signal to The respective vertical interfaces 3521 added together in analog form are coupled to the second set of bit lines. Optionally, the combined analog output can be digitized elsewhere if desired, such as by converting it to digital form using an ADC 3516 (shown in the previous figure).

In Figure 43, input is provided to VMM array 3507-1 via a first set of control gate lines, and to VMM array 3507-2 via a second set of control gate lines, with each Individual vertical interfaces 3521 are coupled to the second set of control gate lines, and the outputs are obtained individually on the bit lines. Optionally, the outputs may be combined elsewhere, such as by using ADC 3516 (shown in the previous figure) to convert the outputs into digital form, and using digital circuitry 3517 (shown in the previous figure) to sum the outputs.

In Figure 44, input is provided collectively to VMM arrays 3507-1 and 3507-2 via vertical interface 3521 via control gate lines. The outputs are obtained on the bit lines, which are combined together via a vertical interface 3521, which effectively adds the signals together in analog form. Optionally, the combined analog output can be digitized elsewhere if desired, such as by converting it to digital form using an ADC 3516 (shown in the previous figure).

Figures 45-48 depict embodiment structural layout options for die, vertical interface, and horizontal interface.

Figure 45 depicts a 3D VMM system 4500 including a first set of dies (dies 4501, 4502, 4503, and 4504) configured in a vertical configuration and connected by a vertical interface 3521. The second die set (die 4505, 4506, 4507 and 4508). Here one die can be connected to two other dies through respective ones of vertical interfaces 3521.

Figure 46 depicts a 3D VMM system 4600 that includes four die levels configured in a vertical staggered configuration, where the first level includes dies 4601 and 4602, the second level includes dies 4603, 4604, and 4605, and the third level includes Dies 4606 and 4607, and the fourth level includes die 4608, 4609, and 4610. Dies in different levels are connected to die in the level above and to die in the level below through respective vertical interfaces 3521. Here one die can be connected to four other dies through each of the vertical interfaces 3521.

Figure 47 depicts a 3D VMM system 4700 that includes four die levels configured in a vertical staggered configuration, where the first level includes dies 4701 and 4702, the second level includes dies 4703, 4704, and 4705, and the third level includes Dies 4706 and 4707, and the fourth level includes die 4708, 4709, and 4710. Dies in different levels are connected by respective vertical interfaces 3521, and dies in the same level are connected by respective horizontal interfaces 3716. Here, one die can be connected to six other dies through respective ones of vertical interfaces 3521 and through respective ones of horizontal interfaces 3716 .

Figure 48 depicts an embodiment of the physical layout of a 3D VMM system 4800, with only connector 4801 shown. Connector 4801 is located in the die and connects to one or more vertical interfaces 3521 and horizontal interfaces 3716 . As can be seen, the die and interface can be configured so that the connectors 4801 are positioned in a vertically staggered configuration. Circuit used in 3D VMM system

Figures 49-55 depict circuits used in 3D VMM systems such as those previously described.

Figure 49 depicts a neuron circuit 4900 that may be used with the previously described neuron circuit 3510. Specifically, neuron circuit 3510 may include embodiments of neuron circuit 4900 for each bit line in VMM array 3507. Neuron circuit 4900 includes a p-channel metal oxide semiconductor (PMOS) transistor 4901 and an operational amplifier 4902 configured as shown. One terminal of PMOS transistor 4901 is attached to a voltage source. The other terminal of PMOS transistor 4901 is attached to the gate of PMOS transistor 4910 and to the bit lines in VMM array 3507 and the non-inverting terminal of operational amplifier 4902. The output of operational amplifier 4902 is connected to the inverting input of operational amplifier 4902. The current drawn by bit line I-BL is generated from the voltage V_IBL output by operational amplifier 4902. Op amp 4902 acts as a buffer and V_IBL will maintain its level regardless of the load to which it may be attached. For example, if V_IBL is provided to the vertical interface 3521, the vertical interface 3521 may have parasitic capacitance or parasitic current. Neuron circuit 4900 will maintain the output voltage V_IBL regardless of load changes. Thus, this is a neuronal current buffer circuit. In another specific example, neuronal current (bitline current) can be amplified or reduced by a current mirror before entering the circuit.

Figure 50 depicts a neuron circuit 5000 that may be used with the previously described neuron circuit 3510. Specifically, neuron circuit 3510 may include embodiments of neuron circuit 5000 for each bit line in VMM array 3507. Neuron circuit 5000 includes a controlled switch 5001, a reference memory cell 5002, and an operational amplifier 5003 configured as shown. The current I-BL drawn by the bit line produces a voltage at the non-inverting terminal of operational amplifier 5003. When switch 5001 is closed, feedback VNEUOUT from the output of operational amplifier 5003 is provided to the control gate terminal of reference memory cell 5002. Due to the inherent characteristics of operational amplifiers, operational amplifier 5003 will modify the output voltage VNEUOUT until the voltage on its non-inverting terminal is equal to the voltage VREF on its inverting terminal. This is done via feedback to the control gate terminal of the reference memory cell 5002. Neuron circuit 5000 will maintain output voltage VNEUOUT regardless of loads such as parasitic capacitance of vertical interface 3521 that may receive the voltage. Neuron circuit 5000 uses memory cells 5002 to convert neuron current I-BL into voltage VNEUOUT. Therefore, it is a memory cell-based current-to-voltage converter using op amps and feedback.

Figure 51 depicts a force and sense (F/S) drive circuit 5100 (where force is induced by an op amp 5103 with input VIN at the positive terminal, and sensing occurs at the target node, node 5101 or 5102, target The voltage at the node is fed back to the negative terminal of the amplifier, and this causes the voltage to be equal to the input voltage VIN) by the action of the amplifier, which can be used in neuron circuit 3510 or elsewhere to accurately deliver the neuron's voltage output through varying loads. F/S drive circuit 5100 includes an operational amplifier 5103 and controlled switches 5104, 5105, 5106, and 5107 configured as shown. Vin is received at the non-inverting input of op amp 5103, and the output of op amp 5103 is called VOUT. When closed, switches 5104 and 5106 respectively connect the output of operational amplifier 5103 and the inverting input of operational amplifier 5103 as VOUTFB to node 5101 in the first die via vertical interface 3521. When closed, switches 5105 and 5107 respectively connect the output of operational amplifier 5103 and the inverting input of operational amplifier 5103 as VOUTFB to node 5102 in the second die via vertical interface 3521. Driver circuit 5100 provides accurate voltage VOUT to node 5101 and node 5102 regardless of any loading effects caused by vertical interface 3521 or by the first and second dies. This is because the sense node (drive node) 5101 or 5102 is fed back to the inverting input of the operational amplifier 5103.

Figure 52 depicts a neuron circuit 5200 that may be used with the previously described neuron circuit 3510. Specifically, neuron circuit 3510 may include an example of differential neuron circuit 5200 for each pair of differential bit lines in VMM array 3507, such as in an example where one row stores W+ values and the other row stores W- value, each pair of W+ and W- represents the stored weight.

Differential neuron circuit 5200 includes operational amplifier 5201 configured as shown, variable integrating resistors 5202 and 5203, controlled switches 5204, 5205, 5206 and 5207, and sample and hold (and/or integrating) capacitors 5208 and 5209. The differential neuron circuit 5200 receives differential current BLw+ from the W+ bit line and BLw- from the W- bit line, respectively, and outputs voltages Vout+ and Vout-. The output voltages Vout+ = (BLw+) * R and Vout- = (BLw-) * R, where the variable integrating resistors 5202 and 5203 each have a value equal to R. Capacitors 5208 and 5209 act as respective sample and hold (S/H) capacitors with resistors 5202 and 5203 removed from the circuit by opening controlled switches 5206, 5207 and input current by opening controlled switches 5204, 5205 Maintains output voltage when turned off. The control circuit (not shown in the figure) controls the opening and closing of switches 5204, 5205, 5206 and 5207 to provide integration time. Optionally, the differential output voltages Vout+ and Vout- may be input to an ADC 3516, which converts the differential output voltages Vout+ and Vout- into a set of digital output bits Doutx. Optionally, the circuit can use capacitors 5208 and 5209 as integrating capacitors to integrate the neuron current to convert the current into voltage, Vout = time * neuron / capacitance. In the case of using the integrating capacitor method, neuron scaling is provided by variable resistors 5292 and 5203 or variable integration time and/or variable capacitance.

Figure 53 depicts a sample and hold buffer 5300 that may be used with the previously described input circuit 3514. Specifically, input circuit 3514 may include an instance of sample and hold buffer 5300 for each column in VMM array 3507 . Sample and hold buffer 5300 includes controlled switch 5301, capacitor 5302, and buffer 5303. Buffer 5303 may be a single buffer formed from an operational amplifier. During operation, switch 5301 is closed, which allows an analog value (eg, from DAC 3515) to be stored in capacitor 5302. This value may then be output from buffer 5303, which drives the column input of VMM array 3507. Capacitor 5302 may be an actual capacitor, or it may be an intrinsic capacitor found in electrical wires.

Figure 54 depicts a differential sequential address register (SAR) analog-to-digital converter (ADC) 5400 that may be used with the previously described ADC 3516.

The differential continuous address register analog-to-digital converter 5400 converts an analog input or a differential analog input to a digital output through all possible quantization levels to identify the appropriate digital output.

Differential Continuous Address Register Analog-to-Digital Converter 5400 includes binary capacitive digital-to-analog converter (CDAC) 5401, binary CDAC 5402 (complementary to CDAC 5401), comparator 5403, and SAR logic and registers 5404.

The differential continuous address register analog-to-digital converter 5400 receives differential current inputs Vinp and Vinn. The SAR logic and register 5404 cycles through all possible combinations of digital bits, which in turn controls the switches in CDACs 5401 and 5402 to couple the voltage source to the capacitor. When the output of the comparator 5403 is inverted, the SAR logic and the digital bit combination in the register 5404 are combined as a digital output (Digital Output) and output. Optionally, the SAR logic and register 5404 generate an additional 1-bit digital output DMAJ in the digital output, which is "1" when the majority of the bits in the digital value are "1" and when the corresponding digital value If most of the bits are not "1", it will be "0".

Figure 55 depicts a reference current based SAR ADC circuit 5500 that may be used with the previously described ADC 3516. Specifically, ADC 3516 may include one or more instances of ADC circuit 5500, where ADC circuit 5500 converts current I-BL into a digital value digital output (Digital Output). The ADC circuit 5500 includes a binary current block 5501, a switch 5502, a comparator 5503, and a SAR logic and register 5504. The binary current block 5501 provides a binary reference current for comparison with the bit line input current using a binary search method, which means searching from MSB (most significant bit) to LSB (least significant bit).

It should be noted that as used herein, the terms "on" and "on" both include "directly on" (without intervening materials, components or spaces) and "Indirectly on" (with intermediate materials, components or spaces installed therebetween). Likewise, the term "adjacent" includes "directly adjacent" (without intervening materials, components, or spaces therebetween) and "indirectly adjacent" (with intervening materials, components, or spaces between them), and "mounted to" includes "directly adjacent" "mounted to" (without intervening materials, components or spaces therebetween) and "indirectly mounted to" (with intermediate materials, components or spaces therebetween), and "electrical coupling" includes "direct electrical coupling to" ( "without intervening materials or components electrically connecting the components together" and "indirectly electrically coupled to" (with intervening materials or components electrically connecting the components together). For example, forming a component "over" a substrate may include forming the component directly on the substrate without intervening materials/components, as well as indirectly forming the component on the substrate with one or more intermediate materials/components therebetween.

12: Semiconductor substrate 14: Source area 16: Drain area 18: Channel area 20: Floating gate 22: Word line terminal/select gate 24, I-BL: Bit line 28: Control gate 30: Wipe Except gate 31, 3515: Digital to analog converter 32, 32a, 32b, 32c, 32d, 32e: Vector matrix multiplication array 33: Non-volatile memory cell array 34: Erase gate and word line gate Decoder 35: Control gate decoder 36: Bit line decoder 37: Source line decoder 38: Differential summer/summing operational amplifier 39, 1602, 1702, 2002, 2102: Activation function block 210, 310, 410, 510, 710: Memory cells 900, 1000, 1100, 1200, 1300, 2200, 2300, 2400, 2500, 2600, 2700, 2800, 2900, 3000: neuron VMM array 901, 1003, 1103, 1203, 1303: memory array 902 ,1001,1002,1101,1102,1201,1202,1301,1302: Reference array 903: Control gate line 904: Erase gate line 1014,1212,1314: Diode connected through multiplexer 1204: String Stacked transistor 1205, 1709, 1710, 2104: Multiplexer 1400: VMM array/LSTM 1401, 1402, 1403, 1404, 1801, 1802, 1803, 1804, 4101mn, 4102mn: Cell 1500, 1600, 1700: LSTM cell Yuan 1501, 1502, 1503, 1901, 1902: S-shaped function component 1504, 1505, 1903: Hyperbolic tangent component 1506, 1507, 1508, 1703, 1904, 1905, 1906, 2103: Multiplier component 1509, 1708, 1907, 2105: Addition component 1601, 1701, 2001, 2101, 3401, 3507, 3557, 3507-1, 3507-2: VMM array 1704, 1705, 1706, 1707, 2106, 2107, 2108: Temporary register 1800: Gate transfer Return unit 1900, 2000, 2100: GRU cell 1908, 2109: complementary component 2701-1, 2701-2, 2701-(N-1), 2701-N: bit line control gate 3100, 3210, 3300, 3400 : VMM system 3101, 3102, 3213, 3303, 3304, 3305, 3306, 3307, 3308: summation circuit 3211: first array 3212: second array 3301, 3302: array 3402: column decoder 3403, 3508, 3714: High voltage decoder 3404: row decoder 3405: bit line driver 3406, 3514: input circuit 3407: output circuit 3408: control logic 3409: bias generator 3410: high voltage generation block 3411: charge pump 3412: charge pump Regulator 3413: High voltage analog precision level generator 3414: Algorithm controller 3415, 3512: Analog circuit system 3416: Control engine 3417: Test control logic 3500, 3600, 3700, 3750, 3800, 3900, 3950, 3980, 4000, 4100, 4200, 4300, 4400, 4500, 4600, 4700, 4800: 3D VMM system 3501, 3502, 3503, 3504, 3505, 3506, 3601, 3602, 3603, 3604, 3605, 3606, 3701, 3702, 370 3 ,3704,3705,3706,3707,3708,3709,3710,3711,3712,3758,3801,3802,3803,3804,3805,3806,3807,3808,3809,3810,3811,3812,3901,3902,39 03 ,3904,3905,3906,3907,3908,3951,3952,3953,3954,3955,3956,3957,3958,3981,3982,3983 3984,3985,3986,4001,4002,4003,4004,4005,40 06, 4007,4008,4501,4502,4503,4504,4505,4506,4507,4508,4601,4602,4603,4604,4605,4606,4607,4608,4609,4610,4701,4702,4703,4704,470 5, 4706, 4707, 4708, 4709, 4710: Die 3509: Input multiplexer 3510, 4900, 5000, 5200: Neuron circuit 3511: High voltage generator 3513: Temperature compensation circuit 3516: Analog to digital converter 3517: Digital Circuit 3518: Static random access memory 3519: Register 3520: Physical I/O connection 3521: Vertical interface/digital accelerator 3522, 3622: Package 3523: Column buffer 3524: Address decoding circuit/address decoder/ Column register 3525: Column register 3531: Digital accelerator 3608: High voltage multiplexer 3610: Row multiplexer 3713: Address decoding circuit/address decoder 3715: Network single chip 3716: Horizontal interface 3729, 4029: Array input 3990: Input block 3991: High voltage block 3992: Output block 3993: Analog block 4801: Connector 4901: P-channel metal oxide semiconductor transistor 4902, 5003, 5103, 5201: Operational amplifier 5001 ,5104,5105,5106,5107,5204,5205,5206,5207,5301: controlled switch 5002: reference memory cell 5100: drive circuit 5101,5102: node 5202,5203: variable integrating resistor 5208,5209 :Sample and Hold Capacitor 5300: Sample and Hold Buffer 5302: Capacitor 5303: Buffer 5400: Differential Continuous Address Register Analog to Digital Converter 5401: Binary Capacitive Digital to Analog Converter 5402: Binary CDAC 5403 , 5503: Comparator 5404, 5504: SAR logic and register 5500: SAR ADC circuit based on reference current 5501: Binary current block 5502: Switch BL0, BL1, BL2, BL3, BLn, BLN: Bit line BLR0 ,BLR1,BLR2,BLR3: terminal BLw+, BLw-: differential current c ₀ ,c ₁ ,c ₂ , c(t-1),c(t): cell state vector C1,C2,C3,S1,S2, S3: Layer CB1, CB2, CB3, CB4: Synapse CG0, CG1, CG2, CG3, CG _M-1 , CG _M , CGm, CGn: Voltage input/control gate voltage DMAJ: Additional 1-bit digital output Doutx: Digital output bit set EG0, EG1: erase gate/EG line h ₀ , h ₁ , h ₂ , h ₃ , h (t-1), h (t): output vector INPUT ₀ , INPUT ₁ , INPUT _{N -1,} INPUT _N , INPUT _M-1 , INPUT _M : input OUTPUT ₀ , OUTPUT ₁ , OUTPUT ₂ , OUTPUT ₃ , OUTPUT ₄ , OUTPUT _N-1 , OUTPUT _N : output P1, P2: activation function S0: input layer SL0 , SL1, SL ₂ , SL ₃ : source lines V_IBL, VNEUOUT, VOUT, Vout+, Vout-, VREF: voltage VIN: input voltage Vinp, Vinn: differential current input VOUTFB: inverting input Vth: threshold voltage WL, WL0 ,WL1,WL2,WL3,WL4,WL5,WL6,WL7,WL _M-1 ,WL _M ,WLA0,WLA1,WLA2,WLA3,WLB0,WLB1,WLB2,WLB3: word line x ₀ ,x ₁ ,x ₂ ,x ₃ ,x(t): input vector

Figure 1 is a diagram illustrating an artificial neural network.

Figure 2 depicts prior art separation of gate flash memory cells.

Figure 3 depicts another prior art split gate flash memory cell.

Figure 4 depicts another prior art split gate flash memory cell.

Figure 5 depicts another prior art split gate flash memory cell.

Figure 6 is a diagram illustrating different levels of an example artificial neural network utilizing one or more non-volatile memory arrays.

Figure 7 is a block diagram of an exemplary VMM system.

Figure 8 is a block diagram illustrating an example artificial neural network utilizing one or more VMM systems.

Figure 9 depicts another example of a VMM system.

Figure 10 depicts another example of a VMM system.

Figure 11 depicts another example of a VMM array.

Figure 12 depicts another example of a VMM system.

Figure 13 depicts another example of a VMM system.

Figure 14 depicts a prior art long short term memory system.

Figure 15 depicts an example cell used in a long short-term memory system.

Figure 16 depicts an example implementation of the cell of Figure 15.

Figure 17 depicts another example implementation of the cell of Figure 15.

Figure 18 depicts a prior art gated recursive unit system.

Figure 19 depicts an example cell used in a gated recursive cell system.

Figure 20 depicts an example implementation of the cell of Figure 19.

Figure 21 depicts another example implementation of the cell of Figure 19.

Figure 22 depicts another example of a VMM system.

Figure 23 depicts another example of a VMM system.

Figure 24 depicts another example of a VMM system.

Figure 25 depicts another example of a VMM system.

Figure 26 depicts another example of a VMM system.

Figure 27 depicts another example of a VMM system.

Figure 28 depicts another example of a VMM system.

Figure 29 depicts another example of a VMM system.

Figure 30 depicts another example of a VMM system.

Figure 31 depicts another example of a VMM system.

Figure 32 depicts another example of a VMM system.

Figure 33 depicts another example of a VMM system.

Figure 34 depicts an example of a 2D VMM system.

Figure 35 depicts an example of a 3D VMM system.

Figure 36 depicts an example of a 3D VMM system.

Figures 37A and 37B depict examples of 3D VMM systems.

Figure 38 depicts an example of a 3D VMM system.

Figures 39A, 39B, and 39C depict examples of 3D VMM systems.

Figure 40 depicts an example of a 3D VMM system.

Figure 41 depicts an example of a 3D VMM system.

Figure 42 depicts an example of a 3D VMM system.

Figure 43 depicts an example of a 3D VMM system.

Figure 44 depicts an example of a 3D VMM system.

Figure 45 depicts an example of a 3D VMM system.

Figure 46 depicts an example of a 3D VMM system.

Figure 47 depicts an example of a 3D VMM system.

Figure 48 depicts an example of a 3D VMM system.

Figure 49 depicts an example of a neuron circuit.

Figure 50 depicts an example of a neuron circuit.

Figure 51 depicts an example of a driver circuit.

Figure 52 depicts an example of a differential neuron circuit.

Figure 53 depicts an example of a sample and hold buffer.

Figure 54 depicts an example of an ADC circuit.

Figure 55 depicts an example of an ADC circuit.

C1:Layer

C2:Layer

C3:Layer

CB1: synapse

CB2: synapse

CB3: synapse

CB4: synapse

P1: activation function

P2: activation function

S1:Layer

S2:Layer

S3:Layer

Claims (53)

A three-dimensional integrated circuit used in an artificial neural network, which includes: a first die including a first vector matrix multiplication array and a first input multiplexer, the first die being located on a first vertical layer; a second die including an input circuit, the second die being on a second vertical layer different from the first vertical layer; and one or more vertical interfaces coupling the first die and the second die; During a read operation, the input circuit provides an input signal to the first input multiplexer through at least one of the one or more vertical interfaces, and the first input multiplexer applies the input signal to to one or more columns in the first vector matrix multiplication array, and the first vector matrix multiplication array produces an output. The three-dimensional integrated circuit of claim 1, wherein the second die includes a digital-to-analog converter for converting a digital input into an analog input provided to the input circuit as the input signal. The three-dimensional integrated circuit of claim 1, wherein the first die includes a neuron circuit for buffering the output. The three-dimensional integrated circuit of claim 1, wherein the first die includes a circuit for sending the output to the second die or a third die through at least one of the one or more vertical interfaces. A row of multiplexers. For example, the three-dimensional integrated circuit of claim 1 includes: a third die including an analog-to-digital converter for converting the output from the first die to a digital output, the third die being located on a different layer than the first vertical layer and the second vertical layer One of the layers is on the third vertical layer. For example, the three-dimensional integrated circuit of claim 5 includes: A fourth die including a high voltage generator, analog circuitry and a temperature compensation circuit, the fourth die being located at one of the different first vertical layers, the second vertical layer and the third vertical layer On the fourth vertical level. For example, the three-dimensional integrated circuit of claim 6 includes: A fifth die, which includes a second vector matrix multiplication array, a second input multiplexer, a high voltage decoder and a neuron circuit, the fifth die is located in a different position from the first vertical layer, the On a fifth vertical layer of the second vertical layer, the third vertical layer and the fourth vertical layer. For example, the three-dimensional integrated circuit of claim 1 includes: A third die including a second vector matrix multiplication array and a second input multiplexer is located on the first vertical layer. A three-dimensional integrated circuit as claimed in claim 1, wherein the vector matrix multiplication array includes a plurality of non-volatile memory cells. The three-dimensional integrated circuit of claim 9, wherein the plurality of non-volatile memory cells include stacked gate flash memory cells. The three-dimensional integrated circuit of claim 9, wherein the plurality of non-volatile memory cells include split-gate flash memory cells. A method that contains: providing an input signal via an input circuit on a first die to an input multiplexer on a second die via one or more vertical interfaces; applying the input signal to one or more columns in a neural network array via the input multiplexer; and Generate an output through the neural network array; The first crystal grain and the second crystal grain are located on different vertical layers. The method of claim 12, wherein the neural network array includes a plurality of non-volatile memory cells. The method of claim 13, wherein the plurality of non-volatile memory cells include stacked gate flash memory cells. The method of claim 13, wherein the plurality of non-volatile memory cells include split gate flash memory cells. A device comprising: a first die including a first vector matrix multiplying array containing a plurality of non-volatile memory cells arranged in columns and rows, the first die being located in a first vertical layer above; a second die including a second vector matrix multiply array containing a plurality of non-volatile memory cells arranged in columns and rows, the second die being located in a different vertical layer than the first on the second vertical layer; and one or more vertical interfaces coupling the first die and the second die; wherein during a programming operation, one or more non-volatile memories in the first array A cell can store i bits, and one or more non-volatile memory cells in the second array can store j bits, where i ≠ j . The device of claim 16, wherein the first die is manufactured according to a first semiconductor process, and the second die is manufactured according to a second semiconductor process that is different from the first semiconductor process. Such as the device of claim 16, which includes: a set of first element lines coupled to the first vector matrix multiplication array; and A second set of bit lines, different from the first set of bit lines, is coupled to the second vector matrix multiplication array. Such as the device of claim 18, which includes: a first set of control gate lines coupled to the first vector matrix multiplication array; and A second set of control gate lines, different from the first set of control gate lines, is coupled to the second vector matrix multiplication array. The device of claim 19, wherein the first set of control gate lines is coupled to the second set of control gate lines through a respective vertical interface. The device of claim 18, wherein the first set of bit lines is coupled to the second set of bit lines through a respective vertical interface. Such as the device of claim 21, which includes: a first set of control gate lines coupled to the first vector matrix multiplication array; and A second set of control gate lines coupled to the second vector matrix multiplication array. The device of claim 22, wherein the first set of control gate lines is coupled to the second set of control gate lines through a respective vertical interface. The device of claim 16, wherein the plurality of non-volatile memory cells in the first die and the plurality of non-volatile memory cells in the second die respectively include stacked gate fast Flash memory cells. The device of claim 16, wherein the plurality of non-volatile memory cells in the first die and the plurality of non-volatile memory cells in the second die respectively include split gate fast Flash memory cells. A method comprising: storing a value comprising i bits in a non-volatile memory cell in a first neural network array in a first die; and storing a value comprising a non-volatile memory cell in a first neural network array in a first die; A value of i bits in the memory cell is stored in a second neural network array in a second die, where i ≠ y ; wherein the first die and the second die are located in different on a vertical layer. A device containing: a first die including a first vector matrix multiplying array containing a plurality of respective non-volatile memory cells arranged in columns and rows, the first die being located on a first vertical layer; A second die including a second vector matrix multiply array containing a plurality of respective non-volatile memory cells arranged in columns and rows, the second die being located in a second layer different from the first layer on the second vertical floor; and A third die including one or more of a digital-to-analog converter and an analog-to-digital converter. The device of claim 27, wherein the analog-to-digital converter is a capacitor-based continuous approximate register analog-to-digital converter. The device of claim 27, wherein the analog-to-digital converter is a continuous approximate register analog-to-digital converter based on a reference current. The device of claim 27, wherein the respective plurality of non-volatile memory cells in the first vector matrix multiplication array and the second vector matrix multiplication array comprise stacked gate flash memory cells. The device of claim 27, wherein the respective plurality of non-volatile memory cells in the first vector matrix multiplication array and the second vector matrix multiplication array comprise split gate flash memory cells. A device containing: a first die including a first vector matrix multiplying array containing a plurality of non-volatile memory cells arranged in columns and rows, the first die being located on a first vertical layer; A second die including a second vector matrix multiplying array of non-volatile memory cells arranged in columns and rows, the second die being located in a second vertical layer different from the first on a vertical level; and A third die containing a neuron circuit. The device of claim 32, wherein the neuron circuit includes: A p-channel metal oxide semiconductor transistor including a first terminal coupled to a voltage source, a gate, and a second terminal coupled to the gate and a neuron; and An operational amplifier including a non-inverting input coupled to the second terminal and the gate of the p-channel metal oxide semiconductor transistors, an inverting input and coupled to the inverting input in response to The current from the neuron produces a voltage output. The device of claim 32, wherein the neuron circuit includes: a switch; a reference memory cell including a bit line terminal, a source line terminal and a control gate terminal coupled to a neuron; and An operational amplifier including an inverting input coupled to a reference voltage, a non-inverting input coupled to the bit line terminal of the reference memory cell and switchably coupled to the Reference is made to the output of one of the control gate terminals of the memory cell. The device of claim 32, wherein the neuron circuit includes: An operational amplifier including an inverting input, a non-inverting input and a first output coupled to a first output node and a second output coupled to a second output node; a first variable integrating resistor switchably coupled between the first output node and the inverting input of the operational amplifier via a first switch; a second variable integrating resistor switchably coupled between the second output node and the non-inverting input of the operational amplifier via a second switch; a first capacitor switchably coupled between a first input current from a bit line and the first output node via a third switch; and A second capacitor switchably coupled via a fourth switch between a second input current from the bit line and the second output node. The device of claim 35, wherein the first input current and the second input current are a differential current signal, and the first output node and the second output node include a differential voltage signal. The device of claim 36, wherein the first input current is received from a W+ bit line and the second input current is received from a W- bit line. The device of claim 36 includes an analog-to-digital converter to convert the differential voltage signal into a set of digital output bits. The device of claim 32, wherein the plurality of non-volatile memory cells in the first vector matrix multiplication array and the second vector matrix multiplication array comprise stacked gate flash memory cells. The device of claim 32, wherein the plurality of non-volatile memory cells in the first vector matrix multiplication array and the second vector matrix multiplication array comprise split gate flash memory cells. A device containing: a first die including a first vector matrix multiplying array containing a plurality of respective non-volatile memory cells arranged in columns and rows, the first die being located on a first vertical layer; a second die including a second vector matrix multiply array containing a plurality of respective non-volatile memory cells arranged in columns and rows, the second die being located in a different vertical layer than the first on the second vertical level; and A third die that includes digital circuitry including one or more of a microcontroller, digital logic, or a single instruction multiple data processor. The device of claim 41, wherein the third die includes a digital accelerator. The device of claim 41, wherein the third die includes a static random access memory. The device of claim 41, wherein the third die includes physical input/output connections. The device of claim 41, wherein the third die includes a register. The device of claim 41, wherein the respective plurality of non-volatile memory cells in the first vector matrix multiplication array and the respective plurality of non-volatile memory cells in the second vector matrix multiplication array The cells include stacked gate flash memory cells. The device of claim 41, wherein the respective plurality of non-volatile memory cells in the first vector matrix multiplication array and the respective plurality of non-volatile memory cells in the second vector matrix multiplication array The cells include split-gate flash memory cells. A device containing: a first vertical layer including: a first vector matrix multiplication array including a respective plurality of non-volatile memory cells arranged in columns and rows; and a second vector matrix multiplication array including a column and a plurality of respective non-volatile memory cells arranged in rows; one or more respective horizontal interfaces coupled to the first vector matrix multiplication array and the second vector matrix multiplication array; a second vertical layer including: a third vector matrix multiplication array including a respective plurality of non-volatile memory cells arranged in columns and rows; and a fourth vector matrix multiplication array including a column and a plurality of respective non-volatile memory cells configured in rows; and One or more respective horizontal interfaces couple the third vector matrix multiplication array and the fourth vector matrix multiplication array. The device of claim 48, wherein the first vector matrix multiplication array is located on a first die, the second vector matrix multiplication array is located on a second die, and the third vector matrix multiplication array is located on a third die. on the die and the fourth vector matrix multiplication array is located on a fourth die. The device of claim 49, wherein the first die is vertically aligned with the third die, and the second die is vertically aligned with the fourth die. The device of claim 49, wherein the first die and the second die are vertically staggered with the third die and the fourth die. The device of claim 48, wherein the respective plurality of non-volatile elements in the first vector matrix multiplication array, the second vector matrix multiplication array, the third vector matrix multiplication array and the fourth vector matrix multiplication array The memory cells include stacked gate flash memory cells. The device of claim 48, wherein the respective plurality of non-volatile elements in the first vector matrix multiplication array, the second vector matrix multiplication array, the third vector matrix multiplication array and the fourth vector matrix multiplication array The memory cells include split gate flash memory cells.