JP2021527886A - Configurable in-memory computing engine, platform, bit cell, and layout for it - Google Patents

Abstract

Various embodiments include systems, methods, architectures, mechanisms or devices for providing programmable or pre-programmed in-memory computing operations. The in-memory computing architecture reshapes the sequence of received data words to form a massively parallel bit-by-bit input signal, multiple CIM channel output signals, thereby multi-bit output words. An input signal and storage using a single-bit internal circuit and signal in a CIM array with a reshaping buffer configured to supply the sequence and thereby a sequence of multi-bit output words. A control circuit configured to perform multi-bit computing operations on a signal and near-memory computing configured to provide a sequence of multi-bit output words as a result of computing. With a pass.

Description

This application is filed on June 18, 2018, US Provisional Patent Application No. 62 / 686,296, and on July 24, 2018, US Provisional Patent Application No. 62 / 702,629, 2018. Claiming the interests of US Provisional Patent Application No. 62 / 754,805 filed on November 2, and US Provisional Patent Application No. 62 / 756,951 filed on November 7, 2018, these The entire US provisional patent application is incorporated herein by reference in its entirety.

The present invention relates to the fields of in-memory computing and matrix-vector multiplication.

In recent years, Charge-domain in-memory computing has emerged as a robust and scalable way to perform in-memory computing. Here, the arithmetic operation in the memory bit cell typically supplies the result as a charge using voltage-charge conversion via a capacitor. Therefore, the bit cell circuit includes the proper switching of the local capacitor in a given bit cell, and that local capacitor is also properly coupled and coupled with the other bit cell capacitors. Generates aggregated calculation results for the entire cell. In-memory computing is well suited to implement matrix-vector products, where matrix elements are stored in a memory array and vector elements are broadcast in parallel across the memory array.

Provisional Patent Application No. 62 / 555,959, "Analog Switched-Capacitor Neural Network", filed September 8, 2017

Various shortcomings in the prior art are addressed by systems, methods, architectures, mechanisms, or devices that provide programmable or pre-programmed in-memory computing operations.

One embodiment includes a reshaping buffer configured to reshape a sequence of received data words to form a massively parallel bit-by-bit input signal, and a first compute-in. A bit cell configured to receive the above-mentioned massively parallel bit-by-bit input signals via a memory (CIM) array dimension and one or more stored signals via a second CIM array dimension. A CIM array and a plurality of CIMs in which each of the plurality of bit cells associated with a common storage signal forms a CIM channel configured to supply a respective output signal. Using an analog-digital converter (ADC) circuit configured to supply a sequence of multi-bit output words by processing the channel output signal, and a single-bit internal circuit and signal to the CIM array. A control circuit configured to perform multi-bit computing operations on the inputs and stored signals and a near memory configured to supply a sequence of multi-bit output words as a result of computing. -Provides an in-memory computing architecture with a computing path.

Further objects, advantages and novel features of the present invention will be described in part in the following description and will be partially apparent to those skilled in the art at the time of consideration of the following description or may be learned by practicing the present invention. The objects and advantages of the present invention may be realized and achieved by means and combinations specifically indicated in the appended claims.

The accompanying drawings, which are incorporated herein and constitute a portion of the present specification, illustrate examples of the invention, with a general description of the invention described above, as well as a detailed description of the examples below. , Play a role in explaining the principles of the present invention.

It is a figure which illustrates the typical structure of an in-memory computing architecture. It is a high-level block diagram of the example architecture by one example. FIG. 2 is a high-level block diagram of an exemplary compute-in-memory unit (CIMU) suitable for use in the architecture of FIG. A high-level block diagram of an Input-Activation Vector Reshaping Buffer (IA BUFF) suitable for use in the architecture of FIG. 2 according to an embodiment is illustrated. FIG. 2 is a high level block diagram of a CIMA read / write buffer suitable for use in the architecture of FIG. 2 according to an embodiment. FIG. 2 is a high level block diagram of a Near Memory Datapath (NMD) module suitable for use in the architecture of FIG. 2 according to an embodiment. FIG. 2 is a high level block diagram of a direct memory access (DMA) module suitable for use in the architecture of FIG. 2 according to an embodiment. FIG. 2 is a high-level block diagram of examples with different digitization / weighting of CIMA channels suitable for use in the architecture of FIG. FIG. 2 is a high-level block diagram of examples with different digitization / weighting of CIMA channels suitable for use in the architecture of FIG. It is a flow chart of the method by one Example. It is a circuit diagram of a multiplication bit cell. It is a circuit diagram of three M-BCs configured to execute an XNOR function. It is a circuit diagram of M-BC by one Example. It is a figure which illustrates the example IC of M-BC of FIG. It is a block diagram of a bit cell having a switch type coupling structure. It is a block diagram of a bit cell having a non-switch type coupling structure. It is a circuit diagram of the bit cell circuit which has a non-switch type coupling structure by one Example. It is a circuit diagram of the two-way interleaving of the layout of a bit cell according to one embodiment.

It should be understood that the accompanying drawings are not necessarily illustrated to a constant scale and show some simplified representation of various features that explain the basic principles of the invention. Specific design features of a series of operations as disclosed herein, including, for example, specific dimensions, orientations, arrangements, and shapes of various illustrated components, are specific intended applications and uses. It will be partially judged by the environment. Certain features of the described examples have been magnified or distorted compared to other features for ease of visualization and clear understanding. In particular, for clarity and explanation, light features may be shown darkly.

Before discussing the invention in more detail, it should be understood that the invention is not limited to the particular embodiments described and can therefore, of course, vary. Also, since the scope of the present invention is limited only by the appended claims, the terms used herein are for the sole purpose of describing specific embodiments and are limited. Please understand that it is not intended.

If a range of values is given, each intervening value between the upper and lower bounds of the range, up to one tenth of the minimum scale, and others within the defined range, unless explicitly indicated by the context. It is understood that the value or intervening value is included within the present invention. The upper and lower limits of these smaller ranges can be independently included in the smaller range and are also included within the invention in accordance with the specifically excluded limits of the defined range. .. When the defined range includes one or both of the above limits, the range excluding one or both of those included limits is also included in the present invention.

Unless defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Any method and material similar to or equivalent to that described herein can be used in the practice or testing of the present invention, but a limited number of exemplary methods and materials are described herein. It should be noted that, as used in this specification and the appended claims, the singular forms "a", "an", and "the" unless the context explicitly dictates. ) ”Contains the plural reference term.

Disclosure examples include systems and methods that allow programmability of highly efficient linear algebra computing architectures for modern VLSI implementations. The computing architecture disclosed herein is referred to as "in-memory computing" and the programmability of this architecture allows it to be widely used, especially across applications for machine learning and artificial intelligence. It becomes. The disclosure approach incorporates a range of configurable features for in-memory computing arrays, thereby similarly dealing with energy and throughput over a wide range of machine learning and artificial intelligence applications. In addition, it will be possible to integrate in a programmable platform.

Linear algebraic operations, especially matrix vector products, have become very prominent, especially in new workloads from machine learning (ML) and artificial intelligence (AI). The dimensions of the matrix and vector can often be very large (usually> 100). Due to the way elements are reused in the above operations, elements are usually stored in embedded or off-chip memory (depending on the number of elements that must be stored). Implementations in modern VLSI technology have shown that, in effect, the energy and delay of accessing data from such memory substantially exceeds the actual computation on the data. This limits the energy / delay reductions achievable with traditional accelerators, thereby separating memory and computation and motivating the in-memory computing paradigm. In the context of in-memory computing systems, raw data is not accessed from memory, but the results of operations on many bits of raw data are accessed, thereby amortizing the energy and delay of access.

The disclosed systems, methods, and parts thereof allow for configurable and programmable high-efficiency linear algebraic computing architectures for modern VLSI implementations and the like, which in turn leads to integrated circuit implementations. The computing architecture disclosed herein is commonly referred to as "in-memory computing" and is used in machine learning, artificial intelligence, and other applications due to the programmability of this architecture. It can be used in a wide range of applications including such matrix vector operations. In various embodiments, the systems, methods, and parts thereof of the present disclosure use hybrid analog / digital computing methods with parallel and serial operation to configure in-memory computing architectures. And enable programmability.

The disclosure approach can generate certain shapes of quantization noise in the middle of the operation, which is controlled by the many architectural features disclosed herein and in some cases standard fixed-point accuracy. It is possible to perform operations that exhibit quantization noise as in computing.

The disclosed system and parts thereof are implemented as integrated circuits, the various features of which are implemented as field programmable gate arrays (FPGAs), as they are simulated using Verilog / transistor level simulations. Was done.

The disclosed approach is incorporated by reference in its entirety as if fully described herein, Provisional Patent Application No. 62 / 555,959, filed September 8, 2017, "Analogue Switched." -Can be applied to a range of in-memory computing architectures, including the architectures disclosed in "Capacitor September Network".

The disclosed approach enables the configurable and programmable nature of in-memory computing, which has been described as a high energy efficiency approach to linear algebra. Such configuration and programmability allow for use in a wide range of applications of the disclosed architecture.

Further disclosure examples provide a bit-parallel / bit-serial approach to multi-bit-in-memory computing. In particular, an in-memory computing architecture in which a memory bit cell performs operations on the 1-b operand is disclosed herein, with multiple bits of one operand mapped to a parallel bit cell. The hardware has been extended to allow operation on multi-bit operands using the bit-parallel / bit-serial (BP / BS) method, and multiple of the other operands. Bits are input consecutively. The disclosed approach is for in-memory computing and digitized output, which is possible over different bit cell output operations in time and space, and is fed to further operations after the operation output bit cells are digitized. On the other hand, in-memory computing in which multi-bit complement operations are performed by the bit parallel / bit serial (BP / BS) method, and the BP / BS approach performs bit-by-bit AND operations using, for example, bit cells. In-memory computing, which uses two's complement representation by use, and the BP / BS approach allows the use of XNOR operations by bit cells, for example 1/0 bits mathematically + 1 /- Consider in-memory computing that uses a different number representation that is taken to have a value of 1.

Configurable in-memory computing engines and platforms Various examples provide in-memory to enable the programmability and virtualization required for expansion into viable applications. Concerning the integration of configurable and hardware support for computing accelerators. In-memory computing typically implements a matrix-vector product, where matrix elements are stored in a memory array and vector elements are broadcast in parallel on the memory array. Some aspects of the embodiment relate to enabling the programmability and configurable nature of such an architecture.

In-memory computing typically includes 1-b representations for matrix elements, vector elements, or both. This is because memory stores data in independent bit cells that are broadcast in a parallel and homogeneous manner, without providing the different binary weighted joins between the bits required for multi-bit operations. be. In the present invention, extensions to multi-bit matrices and vector elements are realized by the bit parallel / bit serial (BPBS) method.

Highly configurable / programmable near-memory computing data paths are included to enable common operations that often involve matrix-vector products. Both allow the operations required to extend from bitwise operations in in-memory computing to multi-bit operations, and for versatility, this supports multi-bit operation and is in-memory. -You are no longer restricted to the 1-b representation unique to memory computing. Because programmable / configurable and multi-bit computing is more efficient in the digital domain, analog-to-digital conversion is performed after in-memory computing in the present invention, and in certain embodiments, others. The multiplexing ratio of is also available, but the configurable data path is multiplexed between the 8 ADC / in-memory computing channels. This is also coordinated with the BPBS method used for multi-bit matrix element support where support up to the 8-b operand is provided in this example.

Since the spatiality of the input vector is common in many linear algebra applications, the present invention incorporates support to enable energy-proportional spatiality control. This is achieved by masking the broadcast of bits from the input vector, which corresponds to the zeroing element (such masking is done for all bits in the bit serial process). This saves computational energy in the memory array as well as broadcast energy.

Assuming a bit-by-bit arithmetic architecture inside for in-memory computing and a digital word architecture outside a typical microprocessor, the data reshaping hardware will be fed when the input vector is fed. It is used for both the arithmetic interface used and the memory interface used when the matrix elements are written and read.

FIG. 1 illustrates a typical structure of an in-memory computing architecture. Given that it consists of a memory array (which may be based on standard bit cells or modified bit cells), in-memory computing has two additional "orthogonal" sets of signals, ie ( Includes 1) input line and (2) storage line. Referring to FIG. 1, a two-dimensional array of bit cells is illustrated, each of the plurality of in-memory computing channels 110 containing a respective column of bit cells, of the bit cells per channel. Each is associated with a common storage line and bit line (column) and their respective input lines and word lines (rows). It should be noted that in the present specification, the columns and rows of signals are relative to each other in order to simply show the row / column relationships within the context of an array of bit cells, such as the two-dimensional array of bit cells illustrated in FIG. Is described as "orthogonal". The term "orthogonal" as used herein is not intended to convey a particular geometric relationship.

A set of signal inputs / bits and stores / bits may be physically combined or separated from existing signals in memory (eg, wordlines, bitlines). To implement a matrix-vector product, matrix elements are first loaded into memory cells. The multiple input vector elements (all possible) are then applied at once via the input line. This causes local arithmetic operations, typically some form of multiplication, to occur in each of the memory bit cells. The result of this arithmetic operation is then driven onto a shared storage line. Thus, the accumulation line represents the result of an operation on a plurality of bit cells activated by the input vector element. This is in contrast to standard memory access, where bit cells are accessed one at a time through the bit line and activated by a single word line.

In-memory computing, as described above, has a number of important attributes. First, the computation is usually analog. This is because the constrained structure of memory and bit cells requires a richer computational model than is achieved by a simple digital switch abstraction. Second, local behavior in a bit cell usually involves an operation using the 1-b representation stored in the bit cell. This is because the bit cells in a standard memory array do not join each other by any binary weighting method, and such a join needs to be achieved by the way bit cells are accessed / read from the end. Because. Hereinafter, extensions relating to in-memory computing proposed in the present invention will be described.

Extensions to near-memory and multi-bit operations In-memory computing has the potential to handle matrix-vector products in ways that traditional digital acceleration cannot handle, while ordinary arithmetic pipelines have matrix-vector products. It may include a range of other actions related to. Such behavior is usually well handled by traditional digital acceleration, but nevertheless, in a suitable architecture, parallelism, high throughput (and thus the need for high communication bandwidth), and in-memory. Placing such acceleration hardware close to in-memory computing hardware can be of great value in order to handle common computational patterns associated with memory computing. The analog-to-digital conversion by the ADC is included after each of the in-memory computing storage lines, as it is preferable that most of the associated operations be performed in the digital domain, and thus the in-memory computing channel. Called. The most important task is to incorporate ADC hardware into the pitch of each in-memory computing channel, and the proper layout approach taken in the present invention makes this possible.

Efficient extension of in-memory computing to support multi-bit matrices and vector elements, each via bit-parallel / bit-serial (BPBS) operations, by incorporating an ADC after each operation channel. Method is possible. Bit-parallel arithmetic involves loading different matrix element bits in different in-memory computing columns. The ADC outputs from the different columns are then appropriately bit-shifted to represent the corresponding bit weights, and digital accumulation is performed on all of the columns to obtain a multi-bit matrix element operation result. Bit-serial operations, on the other hand, apply each bit of a vector element one at a time before digital storage by the next output corresponding to the subsequent input vector bit, and store and store the ADC output each time. Includes proper bit shifting of the output. The BPBS approach, which allows for a mixture of such analog and digital operations, is an analog (1) along with a digital (multi-bit), highly efficient and accurate method, while overcoming the access costs associated with traditional memory operation. The efficiency is high because the high-efficiency and low-precision method of −b) is utilized.

Although the scope of near-memory computing hardware is also conceivable, the details of the hardware incorporated in this embodiment of the present invention will be described below. Eight in-memory computing channels are multiplexed on each near memory computing channel to facilitate the physical layout of the multi-bit digital hardware described above. We can this allow the high parallel operation of in-memory computing to match the throughput with the high frequency operation of digital near memory computing (high parallel analog in-memory computing is digital).・ It operates at a lower clock frequency than near-memory computing). Therefore, each near memory computing channel includes a look-up table (LUT), a fixed nonlinear function implementation, as well as a digital barrel shifter, multiplier, and accumulator. In addition, a configurable finite state machine (FSM) associated with near memory computing hardware is incorporated to control operations across the hardware.

Input Interfacing and Bit Scalability Control To integrate in-memory computing with programmable microprocessors, the internal bitwise behavior and representation is external multi-use used in typical microprocessor architectures. It needs to be properly interfaced with the bit representation. Therefore, the data reshaping buffer is included in both the input vector interface and the memory read / write interface through which the matrix elements are stored in the memory array. Details of the design used for the embodiments of the present invention will be described below. The data reshaping buffer allows for bit-width scalability of input vector elements, while maximizing the bandwidth of data transfer between in-memory computing hardware and external memory and to other architectural blocks. To maintain. This data reshaping buffer serves as a line buffer that receives input-parallel multi-bit data element by element for the input vector and supplies output-parallel single-bit data to all vector elements. Consists of register files.

In addition to word / bit interfacing, it also includes hardware support for convolutions applied to the input vector. Such behavior is prominent in convolutional neural networks (CNNs). In this case, the matrix vector product is performed with only a subset of the new vector elements that need to be supplied (the other input vector elements are buffered and simply shifted appropriately). This alleviates bandwidth constraints for obtaining data for high throughput in-memory computing hardware. In an embodiment of the invention, convolutional support hardware that must perform proper bit-serial ordering of multi-bit input vector elements is due to its properly read output being configurable convolutional stride. It is implemented in a specialized buffer that shifts the data of.

Dimension and Spacity Control For programmability, the hardware considers two additional considerations: (1) matrix / vector dimensions can be variable from application to application, and (2) vectors are sparse in many applications. Must be dealt with by.

For dimensions, in-memory computing hardware often incorporates controls that enable / disable the tile portion of the array, as it consumes energy only at the desired dimension level in the application. However, in the BPBS approach used, the input vector dimension has important implications for computational energy and SNR. For SNR, when performing bitwise operations on each in-memory computing channel, the operation between each input (supplied on the input line) and the data stored in the bit cell is a 1-bit output. The number of individual levels possible on the storage line is equal to N + 1, assuming that Where N is the input vector dimension. This suggests that a log2 (N + 1) bit ADC is needed. However, ADCs have an energy cost that strongly expands or contracts depending on the number of bits. Therefore, in order to reduce the relative contribution of ADC energy, it is advantageous to support a very large but smaller N than log2 (N + 1) bits in the ADC. The result of doing this is that the signal-to-quantization noise ratio (SQNR: signal-to-quantization-noise ratio) of the operation operation is reduced with the number of ADC bits, unlike the standard fixed-precision operation. Is. Therefore, in order to support changing application level dimensions and SQNR requirements, hardware support for configurable input vector dimensions is essential with the corresponding energy consumption. For example, if the reduced SQNR is acceptable, it must support large dimensional input vector segments. On the other hand, if high SQNR must be maintained, lower dimensional input vector segments must be supported and the inner product results from multiple input vector segments that can be combined from different in-memory computing banks. (Therefore, in particular, the input vector dimension can be reduced to the level set by the number of ADC bits to ensure an operation that is ideally matched to standard fixed precision operation). The mixed analog / digital approach made in the present invention makes this possible. That is, the input vector elements can be masked to filter broadcasts only for the desired dimension. This saves broadcast energy and bit cell computational energy in proportion to the input vector dimension.

For spasity, a similar masking approach can be applied to the entire bit serial operation to prevent the broadcast of all input vector element bits corresponding to the zeroing element. We noted that the BPBS approach used was particularly helpful in achieving this. This is because the predicted number of non-zero elements is often known in sparse linear algebra applications, while the input vector dimension can be large. Therefore, the BPBS approach allows us to increase the input vector dimension and still ensure that the number of levels that need to be supported on the storage line is within the ADC resolution, thereby ensuring a high operational SQNR. And. While the predicted number of non-zero elements is known, it is still essential to support a variable number of actual non-zero elements, which can vary from input vector to input vector. The masking hardware simply counts the number of zero-valued elements for a given vector and then applies the corresponding offset to the final product result in the digital domain after BPBS operation, so it is mixed. Easily achieved in the analog / digital approach.

Illustrated Integrated Circuit Architecture FIG. 2 illustrates a high-level block diagram of an exemplary architecture according to an embodiment. In particular, the illustrated architecture of FIG. 2 was implemented as an integrated circuit using VLSI assembly techniques using specific components and functional elements to test the various embodiments herein. It is understood by the inventor that another embodiment with different components (eg, larger or more powerful CPU, memory elements, processing elements, etc.) is within the scope of the present disclosure.

As illustrated in FIG. 2, the architecture 200 includes a central processing unit (CPU) 210 (eg, a 32-bit RISC-V CPU), a program memory (PMEM) 220 (eg, a 128KB program memory), and Expand accessible memory by accessing data memory (DMEM) 230 (eg 128KB data memory) and external memory interface 235 (eg, one or more 32-bit external memory devices (not shown) for illustration purposes). The implementation described herein includes a bootloader module 240 (eg, configured to access an 8KB off-chip EEPROM (not shown)) and various configuration registers 255. Direct memory access (DMA) including a compute-in-memory unit (CIMU) 300 configured to perform in-memory computing and various other functions as usual, and various configuration registers 265. access) module 260, universal asynchronous receiver (UART: Universal Receiver / Transmitter) module 271, general-purpose input / output (GPIO: general purpose input / output) module 273, various timers 27 Includes various support / peripheral modules. Other elements not shown here, such as SoC components (not shown), may also be included in Architecture 200 of FIG.

The CIMU 300 is very well suited for matrix vector products and the like, but other types of operations / calculations may be performed more well by non-CIMU arithmetic units. Thus, in various embodiments, between the CIMU 300 and the near memory so that the selection of the arithmetic unit to which a particular arithmetic and / or function is assigned can be controlled to provide more efficient arithmetic functions. Neighbor-joining is provided.

FIG. 3 illustrates a high level block diagram of an exemplary compute-in-memory unit (CIMU) 300 suitable for use in the architecture of FIG. The following description relates to the architecture 200 of FIG. 2 as well as an exemplary CIMU 300 suitable for use within the context of the architecture 200.

Generally speaking, the CIMU 300 includes various structural elements including, for example, a computation-in-memory array (CIMA) of bit cells configured via various configuration registers. By including, it provides programmable in-memory arithmetic functions such as matrix-vector products. In particular, the illustrated CIMU 300 is configured as a 590 kb, 16 bank CIMU allocated to multiply the input matrix X by the input vector A to obtain the output matrix Y.

Referring to FIG. 3, the CIMU 300 includes a computing in memory array (CIMA) 310, an input activation vector reshaping buffer (IA BUFF) 320, a spurity / AND logic controller 330, and a memory read / write. It is illustrated as including an embedded interface 340, a row decoder / WL driver 350, a plurality of AD converters 360, and a multiplication-shift-accumulated data path (NMD) 370 for near memory computing.

The Computation-in-Memory Array (CIMA) 310 illustrated is arranged as a 4x4 clock-gate 64x (3x3x64) in-memory computing array, thus a total of 256 in-memory. A 256x (3x3x256) computation-in that has a memory computing channel (eg, a memory sequence) and includes 256 ADC360s to support that in-memory computing channel. Includes memory array.

The IA BUFF 320 operates, for example, to receive a sequence of 32-bit data words and reshapes those 32-bit data words into a sequence of high-dimensional vectors suitable for processing by the CIMA 310. Note that 32-bit, 64-bit, or any other width of the data word may be reshaped to match the available size or selected size of the compute-in-memory array 310. If the compute-in-memory array 310 itself is configured to work with respect to high-dimensional vectors, it may have 2-8 bits, 1-8 bits or other sizes and they in parallel throughout the array. Includes elements to which. Further, although the matrix vector product operation described in the present specification is illustrated as utilizing the entire CIMA 310, only a part of the CIMA 310 is used in various embodiments. Moreover, in various other embodiments, the CIMA 310 and associated logic circuits are adapted to implement interleaved matrix vector product operations, and parallel parts of the matrix are processed simultaneously by their respective parts of the CIMA 310.

In particular, the IA BUFF320 reshapes a sequence of 32-bit data words into a highly parallel data structure that may be added to the CIMA310 at one time (or at least in large chunk units) and properly ordered in a bit-serial fashion. do. For example, a 4-bit operation with eight vector elements may be associated with a higher dimensional vector of data elements of 2000 n bits or more. The IA BUFF 320 forms this data structure.

As shown, the IA BUFF 320 receives the input matrix X as, for example, a sequence of 32-bit data words and resizes / rearranges the sequence of received data words according to the size of the CIMA 310. For example, it is configured to supply a data structure containing 2303 n-bit data elements. Each of those 2303 n-bit data elements, along with each masking bit, is transmitted from the IA BUFF 320 to the spurity / AND logical controller 330.

The spasity / AND logical controller 330 is configured to receive, for example, 2303 n-bit data elements and their respective masking bits and call the spurity function accordingly, in which case the zero-valued data elements (each masking). -Those indicated by bits) are not propagated to the CIMA 310 for processing. In this way, the energy specially required for the processing of such bits by the CIMA 310 is saved.

In operation, the CPU 210 loads the PMEM 220 and boot loader 240 via a direct data path implemented in a standard way. The CPU 210 may access the DMEM230, IA BUFF320, and memory read / write buffer 340 via a direct data path implemented in a standard way. All of these memory modules / buffers, CPU 210 and DMA module 260 are connected by the AXI bus 281. The chip configuration module and other peripheral modules are grouped by the APB bus 282 and attached to the AXI bus 281 as slaves. The CPU 210 is configured to write to the PMEM 220 via the AXI bus 281. The DMA module 260 accesses the DMEM230, IA BUFF320, memory read / write buffer 340, and NMD370 via a dedicated data path, and is accessible to others via the AXI / APB bus, eg, according to the DMA controller 265. It is configured to access all of the memory space. The CIMU 300 performs the BPBS matrix vector product described above. Further details of the above and other embodiments will be described below.

Thus, in various embodiments, when the CIMA receives vector information, performs a matrix-vector product, and is further processed by another arithmetic function as needed to provide a composite matrix-vector product function. It operates in a bit-serial-bit-parallel (BPBS) fashion to provide some digitized output signal (ie, Y = AX). Generally speaking, the embodiments described herein include a reshaping buffer configured to reshape a sequence of received data words to form a massively parallel bit-by-bit input signal. A bit cell configured to receive the above-mentioned massively parallel bit-by-bit input signals via a first CIM array dimension and one or more stored signals via a second CIM array dimension. A CIM in a compute-in-memory (CIM) array, each of which is associated with a common storage signal and forms a CIM channel configured to supply its own output signal. A single-bit internal array, an analog-digital converter (ADC) circuit configured to supply a sequence of multi-bit output words by processing multiple CIM channel output signals, and a single-bit internal to the CIM array. A control circuit configured to use circuits and signals to perform multi-bit computing operations on the inputs and stored signals, and to provide a sequence of multi-bit output words as a result of computing. Provides an in-memory computing architecture that includes a configured near-memory computing path.

Memory Map and Programming Model Because the CPU 210 is configured to directly access the IA BUFF 320 and the memory read / write buffer 340, these two memory spaces are particularly for structured data such as array / matrix data. Looks similar to DMEM230 in terms of latency and energy, from a user programming perspective. In various embodiments, the memory read / write buffer 340 and CIMA310 are used as normal data memory when in-memory computing features are not activated or are partially activated. May be done.

FIG. 4 illustrates a high level block diagram of an input activation vector reshaping buffer (IA BUFF) 320 suitable for use in the architecture of FIG. 2 according to one embodiment. The illustrated IA BUFF 320 supports an input activation vector with an element accuracy of 1 to 8 bits, and other accuracy may be considered in various embodiments. According to the bit serial flow mechanism described herein, specific bits of all elements in the input activation vector are broadcast to the CIMA 310 at once for matrix vector product operations. However, the high parallelism of this operation requires that the elements of the high-dimensional input activation vector be provided with maximum bandwidth and minimum energy, otherwise in-memory computing throughput and energy efficiency. The benefits will not be taken advantage of. To achieve this, the input activation reshaping buffer (IA BUFF) 320 may be configured as follows, which allows in-memory computing to be 32 bits (or other bits) of the microprocessor. Width) can be incorporated into the architecture, thereby maximizing the corresponding hardware for 32-bit data transfer for highly parallel internal organizations of in-memory computing.

Referring to FIG. 4, the IA BUFF 320 receives a 32-bit input signal that may include input vector elements with a bit precision of 1 to 8 bits. As a result, the 32-bit input signal is first stored in the 4 × 8-b register 410, which totals 24 (here, represented by registers 410-0 to 410-23). These registers 410 supply their content to an eight register file (denoted as register files 420-0 to 420-8), each with 96 columns, with dimensions up to 3x3x256 = 2304. The input vector it has is placed in a parallel column with its elements. This is done for the 8-b input element, where the 24 4x8-b registers 410 provide 96 parallel outputs to one of the register files 420, and for the 1-b input element. , 24 4x8-b registers 410 provide 1536 parallel outputs for all eight register files 420 (or an intermediate configuration for other bit precision). The height of each register file string is 2x4x8-b, each input vector (element precision up to 8 bits) can be stored in 4 segments, and double when all input vector elements are loaded. Allows buffering. On the other hand, when about 1/3 of the input vector elements are loaded (ie, CNN with 1 stride), one of each of the four register file strings acts as a buffer and the data from the three columns. Allows forward propagation to CIMU for computation.

Thereby, of the 96 columns output by each register file 420, only 72 are selected by their respective circular barrel shift interfaces 430, producing a total of 576 outputs for eight register files 420 at a time. give. These outputs correspond to one of the four input vector segments stored in the register file. Therefore, four cycles are required to load all of the input vector elements into the spurity / AND logic controller 330 in the 1-b register.

A mask bit is generated for each data element while the CPU 210 or DMA 260 writes to the reshaping buffer 320 to take advantage of the spurity in the input activation vector. The masked input activation prevents charge-based arithmetic operations on the CIMA 310, thereby saving computational energy. This mask vector is also stored in the SRAM block and is organized with a 1-bit representation similar to the input activation vector.

The 4 to 3 barrel shifter 430 is used to support VGG style (3x3 filter) CNN operations. When moving to the next filtering operation (convolution reuse), only one of the three input activation vectors needs to be updated, thereby saving energy and improving throughput.

FIG. 5 illustrates a high level block diagram of the CIMA read / write buffer 340 suitable for use in the architecture of FIG. 2 according to one embodiment. The illustrated CIMA read / write buffer 340 is organized as, for example, a 768-bit wide static random access memory (SRAM) block 510, while the illustrated CPU word width is in this example. It is 32 bits and the read / write buffer 340 is used to make an interface connection between them.

The read / write buffer 340 as shown includes a 768-bit write register 511 and a 768-bit read register 512. The read / write buffer 340 typically behaves like a cache for a wide range of SRAM blocks in the CIMA 310, but with some differences. For example, the read / write buffer 340 writes back to the CIMA 310 only when the CPU 210 writes to different lines, but reading different lines does not trigger a write back. If the read address matches the tag of the write register, instead of reading from the CIMA 310, the correction byte (shown as a contaminated bit) in the write register 511 is bypassed to the read register 512.

Storage line analog-to-digital converter (ADC). Each storage line from the CIMA 310 has an 8-bit SAR ADC and fits the pitch of the in-memory computing channel. To save space, a finite state machine (FSM) that controls the bit cycling of the SAR ADC is shared between the 64 ADCs required in each in-memory computing tile. The FSM control logic consists of 8 + 2 shift registers that generate pulses to repeat the reset, sampling, and 8-bit determination steps. Shift register pulses are broadcast to 64 ADCs, they are buffered locally, trigger a local comparator decision, store the bit decision corresponding to the local ADC code register, and then the next. Used to trigger the capacitor-DAC configuration of. The precision metal-oxide-metal (MOM) capacitance may be used to allow miniaturization of the capacitor array of each ADC.

FIG. 6 illustrates a high-level block diagram of the Near Memory Data Path (NMD) Module 600 suitable for use in the architecture of FIG. 2 according to one embodiment, but with other features Digital Near. Memory computing is also available. The NMD module 600 illustrated in FIG. 6 shows a digitally calculated data path after ADC output that supports multi-bit matrix multiplication by the BPBS scheme.

In a particular embodiment, 256 ADC outputs are organized into 8 groups for digital arithmetic flow. This makes it possible to support matrix element configurations up to 8 bits. Therefore, the NMD module 600 includes 32 identical NMD units. Each NMD unit has an 8ADC output 610 and a corresponding bias 621, a multiplier 622/623, a shift number 624 and a multiplexer 610/620 for selecting from storage registers, and 8 for subtracting the global bias and mask count. Adder 631 with unsigned bits and 9-bit signed inputs, signed adder 632 for calculating local bias for neural network tasks, and fixed points for performing scaling. Multiplexer 633, barrel shifter 634 for calculating the exponent of the multiplicand and performing shifts for different bits in the weight element, 32-bit signed adder 635 for performing accumulation, 1, 2, It consists of eight 32-bit storage registers 640 to support weights with 4 and 8-bit configurations and a ReLU unit 650 for neural network application.

FIG. 7 illustrates a high level block diagram of a direct memory access (DMA) module 700 suitable for use in the architecture of FIG. 2 according to one embodiment. The DMA module 700 illustrated has, for example, two channels supporting simultaneous data transfer with different hardware resources and five each of DMEM, IA BUFF, CIMU R / W BUFF, NMD results and AXI4 bus. Includes independent data paths.

のためのＢＰＢＳ方式を図８に示す。ただし、Ｂ_Ａは行列要素ａ_ｍ，ｎのために使用されるビット数に相当し、Ｂ_ｘは入力ベクトル要素ｘ_ｎのために使用されるビット数に相当し、Ｎは入力ベクトルの次元に相当し、これは本実施例のハードウェアにおいて２３０４までとなり得る（Ｍ_ｎはスパーシティ及び次元制御のために使用されるマスク・ビットである）。ａ_ｍ，ｎの複数のビットは並列ＣＩＭＡ列にマッピングされ、ｘ_ｎの複数のビットは直列で入力される。したがって、マルチ・ビット乗算及び蓄積は、両方が本実施例の乗算ビット・セル（Ｍ−ＢＣ）によってサポートされるビット単位のＸＮＯＲ又はビット単位のＡＮＤのいずれかによってイン・メモリ・コンピューティングを介して達成可能である。特に、ビット単位のＡＮＤは、入力ベクトル要素ビットが低いときにその出力が低いままでなければならないという点でビット単位のＸＮＯＲとは異なる。本実施例のＭ−ＢＣは、差分信号として入力ベクトル要素ビットを（一度に１つ）入力することを含む。Ｍ−ＢＣはＸＮＯＲを実装する。ただし真理表における各論理「１」出力は、入力ベクトル要素ビットの真の信号と補数信号とをそれぞれ介してＶ_ＤＤに駆動することによって達成される。したがって、ＡＮＤは補数信号を単にマスキングすることによって容易に達成されるため、出力が低いままとなり、ＡＮＤに対応する真理表を生成する。 Bit Parallel / Bit Serial (BPBS) Matrix Vector Product Multi Bit MVM

The BPBS method for this is shown in FIG. Where B _A corresponds to the number of bits used for the matrix elements am _{, n , B x} corresponds to the number of bits used for the input vector element x _n , and N corresponds to the dimension of the input vector. Correspondingly, this can be up to 2304 in the hardware of this embodiment ( _Mn is a mask bit used for spatiality and dimension control). _{A plurality of bits of am and n} are mapped to a parallel CIMA sequence, and a plurality of bits of _{x n are input in series.} Thus, multi-bit multiplication and storage is via in-memory computing by either bitwise XNOR or bitwise AND, both supported by the multiplication bit cell (M-BC) of this embodiment. Is achievable. In particular, bitwise AND differs from bitwise XNOR in that its output must remain low when the input vector element bits are low. The M-BC of this embodiment includes inputting (one at a time) an input vector element bit as a difference signal. M-BC implements XNOR. However, each logic "1" output in the truth table is achieved by driving the _VDD via the true signal and the complement signal of the input vector element bits, respectively. Therefore, AND is easily achieved by simply masking the complement signal, leaving the output low and producing a truth table corresponding to AND.

Bitwise AND can support standard two's complement representations for multi-bit matrices and input vector elements. This is negative for the column operation that corresponds to the most significant bit (MSB) in the digital domain after the ADC and before adding the digitized output to the output of the operation for other columns. Includes proper application.

に分解して、

を得る。 Bitwise XNORs require minor modifications to the numeric representation. That is, the element bits are mapped to + 1 / -1 instead of 1/0 and require two bits with equivalent LSB weighting to properly represent zero. This is done as follows. First, each B-bit operand (standard two's complement representation) is decomposed into B + 1-bit signed integers. For example, y is B + 1 positive / negative 1 bit

Disassembled into

To get.

Bit-by-bit in-memory computing multiplication may be achieved via logical XNOR operation by mapping 1/0 value bits to + 1 / -1 mathematical values. Therefore, an M-BC that performs a logical XNOR using a difference signal for an input vector element performs signed multi-bit multiplication by bit-weighting and adding the digitized output from a column operation. It can be possible.

M-BC multiplication based on AND and M-BC multiplication based on XNOR represent two options, but other options are possible by using the appropriate number of representations with the logical behavior possible with M-BC. Is. Such alternatives are beneficial. For example, XNOR-based M-BC multiplication is suitable for binarized (1-b) operations, while AND-based M-BC multiplication facilitates integration within a digital architecture. Allows a more standard representation of numbers. In addition, the two approaches generate slightly different signal-to-quantization noise ratios (SQNRs) and can be selected based on application needs.

Heterogeneous Computing Architectures and Interfaces The various examples described herein are when a bit cell (or multiplication bit cell: M-BC) drives an output voltage to a local capacitor that corresponds to the result of an operation. It contemplates various aspects of charge domain in-memory computing. Capacitors from in-memory computing channels (columns) are then coupled to achieve storage through charge redistribution. As mentioned above, such capacitors use certain shapes that are very easy to replicate in VLSI and other processing, such as via wiring that is simply coupled via an electric field because they are in close proximity to each other. It may be formed. Thus, the local bit cell formed as a capacitor stores the charge representing one or zero, while locally adding up all the charges of a large number of capacitors or bit cells to form the basis in the matrix vector product. It enables the implementation of operating multiplication and accumulation / addition functions.

The various examples described above provide an improved bit cell-based architecture, computing engine, and platform, which is beneficial. Matrix-vector product is an operation that is not efficiently performed by standard digital processing or digital acceleration. Therefore, performing this one type of computation in memory offers significant advantages over existing digital designs. However, various other types of operations are performed using digital design.

Various embodiments connect / interface to traditional digital computing architectures and platforms, such as the above bit cell based architectures, computing engines, platforms, etc., to form heterogeneous computing architectures. Consume a mechanism for doing so. In this way, other arithmetic operations that are well suited for traditional computer processing are processed through traditional computer architecture, while operations that are well suited for bit cell architecture processing (eg, matrix vector processing). The operation is processed as described above. That is, various embodiments provide a computing architecture that includes a highly parallel processing mechanism as described herein, and by connecting this mechanism to multiple interfaces, a more conventional digital computing architecture. Can be outer-coupled to. This allows the digital computing architecture to line up directly and efficiently with the in-memory computing architecture, with the two in close proximity to minimize the overhead of data migration between the two. It is possible to be placed. For example, machine learning applications include 80% to 90% matrix vector operations, while 10% to 20% other types of operations / operations are still performed. By combining in-memory computing as described herein with near-memory computing, which is more conventional in architecture, the resulting system is significantly configured to perform many types of processing. Realize the possibilities. Therefore, various examples contemplate near-memory digital computation in combination with in-memory computing as described herein.

The in-memory operations described herein are massively parallel but single-bit operations. For example, often only one bit is stored in a bit cell. It is 1 or 0. The signal driven by the bit cell is usually an input vector (ie, each matrix element is multiplied by each vector element in a 2D vector multiplication operation). This vector element is placed on a signal that is also digital and has only one bit so that the vector element is also one bit.

Various examples extend a matrix / vector from a one-bit element to a multi-bit element using a bit-parallel / bit-serial approach.

8A and 8B illustrate high-level block diagrams of various examples of CIMA channel digitization / weighting suitable for use in the architecture of FIG. In particular, FIG. 8A illustrates a digital binary weighting and addition embodiment similar to that described above for various other figures. FIG. 8B modifies various circuit elements to allow the use of a smaller number of analog-to-digital converters than the embodiment of FIG. 8A and / or the other embodiments described herein. An example of analog / binary weighting and addition is illustrated.

As mentioned above, in various embodiments, a bit cell compute-in-memory (CIM) array is input in bits in parallel via a first CIM array dimension (eg, a row in a 2D CIM array). It is intended to be configured to receive a signal and receive one or more stored signals via a second CIM array dimension (eg, a row of 2D CIM arrays), in which case a common stored signal. Each of the plurality of bit cells associated with (eg, illustrated as a sequence of bit cells) forms a CIM channel configured to supply its respective output signal. An analog-to-digital converter (ADC) circuit is configured to supply a sequence of multi-bit output words by processing multiple CIM channel output signals. The control circuit is configured to cause the CIM array to perform multi-bit computing operations on input and stored signals using single-bit internal circuits and signals, thereby engaging in operation near. Allow the memory computing path to be configured to provide a sequence of multi-bit output words as the result of the operation.

With reference to FIG. 8A, examples of digital binary weighting and addition that perform ADC circuit functions are illustrated. In particular, the two-dimensional CIMA810A receives matrix input values in the first (row) dimension (ie, via multiple buffers 805) and vector input values in the second (column) dimension, and the CIMA810A varies. It operates according to a control circuit or the like (not shown) so as to supply a channel output signal CH-OUT.

The ADC circuit of FIG. 8A has a binary for each ADC 760 configured to digitize the CIM channel output signal CH-OUT and a respective binary for the digitized CIM channel output signal CH-OUT for each CIM channel. By assigning weights, each shift register 865 configured to form each portion of the multi-bit output word 870 is provided.

Referring to FIG. 8B, analog / binary weighting and addition embodiments that perform ADC circuit functions are illustrated. In particular, the two-dimensional CIMA810B receives matrix input values in the first (row) dimension (ie, through multiple buffers 805) and the vector input values in the second (column) dimension, and the CIMA810B varies. It operates according to a control circuit or the like (not shown) so as to supply a channel output signal CH-OUT.

The ADC circuit of FIG. 8B provides four controllable (or preset) banks within the CIMA810B, such as switches 815-1, 815-2, etc., which are the capacitors formed therein. Implements an analog-binary weighting method for each of one or more subgroups of a channel by acting to combine and / or separate, and each of the channel subgroups provides a single output signal. Thus, only one ADC 860B is required to form each portion of the multi-bit output word by digitizing the weighted analog addition of the CIM channel output signal of each subset of the CIM channel. To.

FIG. 9 illustrates a flow chart of the method according to one embodiment. In particular, method 900 of FIG. 9 relates to various processing operations implemented by architectures, systems, etc. as described herein, in which case the input matrix / vector is calculated in a bit-parallel / bit-serial approach. Expanded to be.

In step 910, the matrix and vector data is loaded into the appropriate memory location.

In step 920, each of the vector bits (MSB to LSB) is processed sequentially. In particular, the vector MSB is multiplied by the matrix MSB, the vector MSB is multiplied by the matrix MSB-1, the vector MSB is multiplied by the matrix MSB-2, and so on, and finally the vector The MSB is multiplied by the LSB of the matrix. The resulting analog charge result is then digitized by vector product from MSB to LSB, and the resulting result is latched. This process is repeated from vector MSB-1, vector MSB-2, etc. to vector LSB until each of the vectors MSB-LSB is multiplied by each of the MSB-LSB elements of the matrix.

At step 930, the bits are shifted to apply the appropriate weighting and the results are added together. Note that in some of the embodiments where analog weighting is used, the shift operation in step 930 is unnecessary.

Various embodiments make it possible to perform very stable and robust operations in circuits used to store data in dense memory. In addition, various embodiments advance the computing engines and platforms described herein by enabling high densities for memory bit cell circuits. Density can be increased due to the more compact layout and also due to the improved compatibility of the layout with the very aggressive design rules (ie, push rules) used for memory circuits. Is. Various examples substantially improve the performance of the processor for machine learning and other linear algebra.

Bit cell circuits that can be used within an in-memory computing architecture are disclosed. The disclosed approach allows for very stable / robust operations in the circuits used to store data in dense memory. Disclosure approaches for robust in-memory computing allow higher densities for memory bit cell circuits compared to known approaches. Density is increased due to the more compact layout and also due to the improved compatibility of the layout with the very aggressive design rules (ie, push rules) used for memory circuits. The disclosed devices can be assembled using standard CMOS integrated circuit processing.

Memory access is a major part of energy and latency in many computational workloads. In standard memory, raw data is accessed row by row, causing communication costs when moving data from a storage point to a calculation point outside the memory array, resulting in memory energy and delay. In-memory computing architectures, on the other hand, access many rows at once and amortize communication costs by accessing the results of operations on many bits of data stored across rows.

While such depreciation reduces energy and delay due to multipliers (ie, roughly the number of rows accessed at the same time), the most important issue is that the computational signal-to-noise ratio (SNR) is also reduced by the corresponding factor. To be done. This is to increase the dynamic range, which generally requires operations on a large number of bits, but with this to suppress SNR within the limited swing of the existing bit line of memory. In particular, most of the computational noise in in-memory architectures is due to the deformation and non-linearity of the operational behavior performed by the bit cells. In standard memory, bit cells provide the output current. It has made current domain computation a natural selection for in-memory computing architectures, with the aim of minimizing changes to standardly used high-density bit cells.

However, the bit cell current is susceptible to changes affecting the bit cell transistor and high levels of non-linearity. This limits the SNR of in-memory computing and thus limits scalability. Improvements are obtained according to various embodiments that use charge domain operations. Here, the arithmetic output from the bit cell is stored as an electric charge on the capacitor. For example, various examples contemplate the use of metal finger capacitors on bit cell transistors, such capacitors that do not generate additional regions and therefore have a high density of bit cells. The structure can be maintained. Important for the operational signal-to-noise ratio is that such capacitors also show very good linearity and high stability in the presence of processing and temperature changes. This has, in effect, increased the scalability of in-memory computing.

While metal finger capacitors can be placed on the bit cell, some circuit changes are required within the bit cell for the charge domain operation of the switched capacitor. The present invention focuses on circuits and layouts to enable high density of charge domain bit cell operations in static random access memory (SRAM). In particular, circuits and layouts that improve compatibility with the aggressive push design rules used for SRAM bit cells will be described.

FIG. 10 illustrates a circuit diagram of a multiplication bit cell. The bit cell 900 of FIG. 9 executes operations such as storing, writing, and reading 1-b data, and can further multiply the stored 1-b data and the 1-b IA / IAb signal (difference). To. Therefore, this structure is called a multiplication bit cell (M-BC). The 1-b multiplication corresponds to the logical XNOR operation. To implement this, as illustrated in FIG. 10, a MIMO transistor is added, coupled to a bit cell storage node and driven by an IA / IAb signal.

FIG. 11 illustrates a circuit diagram of three M-BCs configured to perform the XNOR function. In particular, the operation of the circuit of FIG. 10 is that the M-BC capacitor is unconditionally operated by asserting (1) a capacitor short-circuit switch (TSHORT / TSHORTb) and a discharge NMOS transistor (PRE) for three M-BCs. Discharged, (2) TSHORT / TSHORTb is deasserted, IA / IAb is driven, the M-BC XNOR output is stored on the local M-BC capacitor, (3) IA / IAb is deasserted, and TSHORT is It is asserted and involves accumulating charges from all XNOR results and giving a multiplication-accumulation operation. In addition to the eight M-BC transistors, two additional NMOS / MOSFET transistors are required for each M-BC for TSHORT / TSHORTb mounting. In the previous design, the above NMOS / MOSFET transistors, along with the three M-BCs, were placed outside each M-BC to allow efficient sharing of the nodes.

11A and 11B illustrate the layout of an exemplary integrated circuit (IC) on the 8-transistor M-BC (FIG. 11A) next to the 8-transistor M-BC in a standard SRAM bit cell (FIG. 11B). .. Differences in PCB size and complexity between the two types of bit cells can be seen by inspection.

FIG. 12 illustrates an exemplary IC layout of a group of three M-BCs (also shown with metal finger capacitors placed on top of them) and TSHORT NMOS / MOSFETs. It should be noted that the NMOS transistors for bit cell operations lead to the balanced use of the NMOS and NMOS transistors in the M-BC, which results in the standard 6-transistor (6T) SRAM bit cell. The IC layout is significantly different from that of. This affects the likelihood of push rule compliance.

Various embodiments contemplate a novel M-BC circuit that adds an NMOS transistor for bit cell computation. Thereby, the embodiment provides an IC layout that achieves both high density and proximity as compared to a standard 6T bit cell and also improves push rule compliance.

FIG. 13 illustrates a circuit diagram of an M-BC according to an embodiment. In particular, the M-BC1300 of FIG. 13 implements 1-b charge domain multiplication (XNOR) using an NMOS transistor. Here, the NMOS inputs IA / IAb are low during the unconditional discharge of the local capacitor and then differentially driven for computation.

FIG. 14 illustrates an exemplary layout of the M-BC of FIG. In particular, the layout of FIG. 14 contemplates compactly including an NMOS / NMOS TSHORT switch in a single M-BC (metal finger capacitors are located on the bit cell). In layout 1400 of FIG. 14, the signals WL, IA / IAb run horizontally, while the signals BL, Brab, PA, VDD, and GND run vertically. This layout has approximately twice the area of a standard 6T cell and takes advantage of the opportunity to share some nodes with the surrounding M-BC.

While the disclosed approach uses more area than standard memory bit cell circuits, the majority of bit cells have been demonstrated by push rules, and extensions to this are push rules. Similar to any other structure that has successfully used the rule. The disclosed approach substantially improves the performance of the processor for machine learning and other linear algebra. Such improvements have been experimentally proven for traditional architectures, and the disclosure approach is a substantial advance in that architecture.

As mentioned above, the arithmetic operation in the memory bit cell usually uses voltage-charge conversion via a capacitor to supply the result as a charge. Therefore, the bit cell circuit includes the proper switching of the local capacitor of a given bit cell, in which case the local capacitor is also properly coupled and coupled to the other bit cell capacitors. Generates aggregated operation results across bit cells.

A robust bit-cell and bit-cell layout of charge injection for in-memory computing of reconfigurable charge domains is disclosed herein. The disclosed devices, especially bit cell circuits, can be used within an in-memory computing architecture. The disclosed approach allows reconfigurable operations as well as very stable / robust operations to be performed in circuits used to store data in high density memory. The disclosure approach allows for greater robustness and reconfigurability for in-memory computing than traditional approaches. The disclosed devices may be assembled using standard CMOS integrated circuit processing. The disclosed approach is considered to have significant utility for the semiconductor industry as it can substantially improve the performance of processors for machine learning and other linear algebra.

The approaches disclosed herein are (1) a configuration in which coupling between bit-cell capacitors is achievable without the need for an explicit switch (non-switched coupling structure), and (2) coupled. It relates to two novel aspects of a bit cell circuit in a physical layout (interleaved layout) in which a bit cell is interleaved with other combined bit cells.

The non-switched coupling structure relates to a bit cell that supplies the result of an operation on one of the capacitor plates, in which case the coupling between the capacitors is achieved via the other capacitor plate. This is in contrast to the switch-type coupling structure, where the bit cell circuit usually provides the result of operations on the same capacitor plate that is ultimately coupled to another capacitor via a switch.

FIG. 15A illustrates a block diagram of a bit cell with a switch-type coupled structure, while FIG. 15B illustrates a block diagram of a bit cell with a non-switched combined structure. In both cases, the coupling capacitor must first be reset (removing the charge on the capacitor), such as by shorting the output node to which the capacitor is coupled. After that, the operation f (.) Is executed locally in the bit cell. This is exemplified for two operands a and b, one of which is stored in a bit cell and the other which is supplied to the outside from around the bit cell. However, in general, structures with more operands are possible. The arithmetic operation is then coupled to other capacitors either through a switch on the sample plate (switch-type coupling structure) or without a switch on the other plate (non-switch-type coupling structure). Drives the plate of a local capacitor. Advantageously, the non-switched coupling structure of FIG. 15B avoids the need for a coupling switch in the bit cell, and when implemented by a MOSFET, the switch absorbs / releases a variable amount of charge by the MOSFET. It can be made (depending on the voltage level), which has the potential to reduce the effects of charge injection errors, such as a slight loss of charge domain computation.

FIG. 16 illustrates a circuit diagram of a bit cell circuit having a non-switched coupling structure according to an embodiment. It should be noted that other modifications of this circuit are also possible within the context of the embodiments of the present disclosure. The bit cell 1600 of FIG. 16 is either an XNOR or AND operation between the stored data W / Wb (in a 6-transistor cross-coupled circuit formed by MN1-3 / MP1-2) and the input data IA / IAb. Can be implemented. For example, in the case of an XNOR operation, the IA / IAb can be driven complementarily after a reset, resulting in the lower plate of the local capacitor being pulled up / down according to the IA XNOR W. On the other hand, in the case of AND operation, only IA is driven after reset (IAb remains low), and as a result, the lower plate of the local capacitor is pulled up / down according to IA AND W. Interestingly, this structure allows the total switching energy of the capacitors to be reduced due to the resulting series pull-up / pull-down charging structure among all of the coupling capacitors, as well as the output nodes. The effect of the switch charge injection error can be reduced due to the deletion of the coupling switch in.

FIG. 17 illustrates a two-way interleave circuit diagram of the bit cell layout according to one embodiment. In particular, the interleaved layout of bit cells relates to layouts in which capacitors may be coupled together in more than one set. FIG. 17 illustrates the case of bidirectional interleaving, but higher interleaving is also contemplated in various embodiments. In addition, the capacitors may be located on the side of the row and may actually be located on the bit cell transistor and / or elsewhere adjacent to the bit cell transistor. The advantage of this structure is, among other things, improved configurability. That is, since the output is supplied on two different nodes, the couplings A and B can be used to implement separate operations. Alternatively, couplings A and B can be used to implement joint completion, for example by appropriately combining outputs on different nodes via appropriate peripheral circuits.

Note that the functions illustrated and described herein are hardware using, for example, a general purpose computer, one or more application specific integrated circuits (ASICs), or any other equivalent hardware. Alternatively, it may be implemented by a combination of software and hardware. It is contemplated that some of the steps described herein may be implemented in hardware, for example, as circuits that work with processors to perform various method steps. Some of the functions / elements described herein may be implemented as computer program products, and when computer instructions are processed by an arithmetic unit, the methods or techniques described herein are invoked. Or adapt the operation of the arithmetic unit as provided otherwise. Instructions for invoking the methods of the invention may be stored in tangible and non-temporary computer-readable media such as fixed or removable media or memory, or in memory in an arithmetic unit operating in accordance with the instructions. It may be stored in.

Various modifications may be made to the systems, methods, devices, mechanisms, techniques, and parts thereof described herein with respect to the various drawings, such modifications within the scope of the present invention. It is intended to be. For example, a particular order of steps or arrangement of functional elements is presented in the various embodiments described herein, but in the context of the various examples various other orders of steps or functional elements / Arrangement may be utilized. Further, although the modifications with respect to the examples may be described individually, in various examples, a plurality of modifications may be used at the same time or in order, or a combined modification or the like may be used.

Although specific systems, devices, methodologies, mechanisms, etc. have been disclosed as described above, those skilled in the art will be able to make more modifications than described above without departing from the novel concepts described herein. It will be clear that. Therefore, the new subject matter is not limited except for the gist of this disclosure. Moreover, in interpreting this disclosure, all terms should be interpreted in the broadest possible way consistent with the context. In particular, the term "complying" should be construed as referring to an element, component, or step in a non-exclusive manner, and the referred element, component, or step Indicates that it may exist, be used, or be combined with other elements, components, or steps not explicitly mentioned. Moreover, the references described herein are also part of the present application and are incorporated in their entirety as if they were fully described.

Aspects of the various embodiments are specified in the claims and / or the numbered sections below.

1. 1. A bit cell storage circuit coupled to at least one bit cell arithmetic device and a bit cell capacitor coupled to the bit cell arithmetic apparatus are provided, and the bit cell capacitor is the bit cell. -A bit-cell circuit configuration that is further coupled to one or more additional capacitors without the need for a switch between the capacitors and the additional capacitors.

2. The bit cell circuit configuration according to item 1, wherein the cathode plate of the bit cell capacitor is coupled to the bit cell arithmetic unit.

3. 3. The bit cell circuit configuration according to paragraph 1, wherein the anode plate of the bit cell capacitor is coupled to the additional capacitor.

4. The bit cell circuit configuration according to item 1, wherein the bit cell arithmetic unit is configured to execute an arithmetic operation by two operands.

5. A bit cell circuit configuration as illustrated by FIG.

6. The bit cell circuit configuration according to item 5, wherein an XNOR or AND operation can be implemented between the stored data and the input data by the above configuration.

7. The interleaved layout for at least two bit cell configurations of the first term, wherein the bit cell capacitors are both coupled to form at least two sets.

8. The interleaved layout according to paragraph 7, wherein the combined set of bit cell capacitors is placed on top of one or more bit cell transistors.

9. A charge domain-in-memory computing bit cell that drives one plate of the local capacitor and coupling to the other bit cell capacitor is achieved on the other plate.

10. A charge domain-in-memory computing bit cell that can implement an XNOR or AND operation between stored and input data.

11. Bit cell capacitors are an interleaved layout for charge domain-in-memory computing bit cells that are coupled into different sets.

12. Charge domain-in-memory where X sets exist for interleave in the X direction, where X is an integer greater than 1, by coupling the bit cell capacitors into different sets. -Interleaved layout for computing bit cells.

13. A different set of coupled capacitors is placed on top of the bit cell transistor in a layout for charge domain in memory computing bit cells.

14. A multiplication bit cell (M-BC) configured to perform a charge domain operation between the stored data in the bit cell and the 1-b input signal.

15. The M-BC according to paragraph 14, wherein one or more NMOS transistors are utilized to perform the charge domain operation.

16. The M-BC according to paragraph 14, further comprising a capacitor having a metallic structure arranged on the bit cell.

17. The M-BC according to claim 14, wherein the M-BC is configured to implement a logical operation.

18. The M-BC according to claim 17, wherein the logical operation includes an XNOR operation, a NAND operation, an AND operation, and other logical operations.

19. The M-BC according to paragraph 14, further comprising a layout as illustrated in FIG.

The M-BC according to paragraph 14, further comprising an extended layout of 20.6 T cells.

21. 20. The M-BC according to claim 20, further comprising a transistor having a regular polystructure.

22. 13. The M-BC according to paragraph 14, further comprising a layout as illustrated in FIG.

Although various embodiments incorporating the teachings of the present invention have been illustrated and described in detail herein, one of ordinary skill in the art can readily devise many other modified embodiments incorporating these teachings. be. Thus, although the above relates to various embodiments of the invention, other and further embodiments of the invention can be conceived without departing from its basic scope.

Claims (32)

It is configured to receive massively parallel bit-by-bit input signals through a first compute-in-memory (CIM) array dimension and one or more stored signals through a second CIM array dimension. A CIM array of bit cells, each of which is associated with a common storage signal, forms a CIM channel configured to supply its own output signal. When,
The CIM array includes an in-memory configuration that uses a single-bit internal circuit and signals to perform multi-bit computing operations on the input and storage signals. -Computing architecture. The in-memory computing according to claim 1, further comprising a reshaping buffer configured to reshape the sequence of received data words to form the massively parallel bit-by-bit input signal. architecture. The in-device according to claim 1, further comprising an analog-to-digital converter (ADC) circuit configured to supply a sequence of multi-bit output words by processing the plurality of CIM channel output signals. Memory computing architecture. The in-memory computing architecture of claim 1, further comprising a near-memory computing path configured to provide the sequence of multi-bit output words as a result of computing. The ADC circuit applies binary weighting to each ADC configured to digitize the CIM channel output signal and the digitized CIM channel output signal for each CIM channel. The in-memory computing architecture of claim 3, comprising each shift register configured to form each portion of a multi-bit output word. The ADC circuit digitizes the weighted analog addition of the CIM channel output signal of each subset of the CIM channel into each portion of the multi-bit output word for each of the plurality of subsets of the CIM channel. The in-memory computing architecture of claim 3, comprising each ADC configured to form. Masking the massively parallel bit-by-bit input signal zero-value element so that the multi-bit computing operation avoids processing the massively-parallel bit-by-bit input signal zero-value element. The in-memory computing architecture according to claim 1, further comprising a spatial controller configured as described above. The in-memory computing architecture of claim 1, wherein the input signal and the stored signal are combined with an existing signal in the memory. The in-memory computing architecture of claim 1, wherein the input signal and the stored signal are separated from existing signals in the memory. The in-memory computing architecture of claim 3, wherein each ADC and its respective stored signal form an in-memory computing channel. The near memory computing path according to claim 4, wherein the near memory computing path includes one or more of a digital barrel shifter, a multiplexer, an accumulator, a look-up table, and a non-linear function element. In-memory computing architecture. The multi-bit computing operation of the CIMA includes bit parallel / bit serial (BPBS) computing.
The bit parallel computing is
Load different matrix element bits into each in-memory computing channel and
Each barrel shifter is used to implement the corresponding bit weighting by barrel-shifting the digitized output from said computing channel.
Each accumulator is used to perform digital accumulation on all of the computing channels to generate multi-bit matrix element arithmetic results, including performing digital accumulation.
The bit serial computing is
Each bit of the vector element is applied individually to the stored matrix element bits, and the resulting digitized output is stored.
Using each barrel shifter, barrel-shifting the stored digitized output associated with each vector element bit prior to digital accumulation by the stored digitized output corresponding to subsequent input vector bits. The in-memory computing architecture of claim 10, including. The fourth aspect of claim 4, wherein the near memory computing path is physically aligned with the in-memory computing architecture to increase throughput through the in-memory computing architecture. In-memory computing architecture. The in-memory computing according to claim 1, further comprising one or more configurable finite state machines (FSMs) configured to control the computational operation of the in-memory computing architecture. architecture. The FSM that controls the arithmetic operation is configured to control highly parallel computing hardware used by some or all of a plurality of in-memory computing channels. 14. The in-memory computing architecture according to 14. The in-memory computing architecture of claim 14, wherein the FSM controls operations according to software instructions loaded into local memory. The in-memory computing architecture of claim 2, wherein the reshaping buffer is configured to convert a first precision external digital word into a higher dimensional input vector. The in-memory computing architecture of claim 2, wherein the reshaping buffer is configured to supply the CIMA with bits of the input vector elements in order and in parallel. The in-memory compute according to claim 2, wherein the reshaping buffer is configured such that the aligned input data ensures the desired utilization and throughput of in-memory computing operations. Input architecture. The in-memory architecture of claim 2, wherein the reshaping buffer is configured to allow reuse and shift of input data according to a convolutional neural network operation. Each bit cell in the (CIM) array of bit cells is
A bit cell storage circuit coupled to at least one bit cell arithmetic unit,
A bit cell capacitor coupled to the bit cell arithmetic unit, the bit cell capacitor corresponds to another bit cell capacitor without providing a switch between the bit cell capacitors. The architecture of claim 1, wherein the architecture comprises a bit cell circuit configuration including a bit cell capacitor, which is further coupled to one or more additional capacitors. 21. The in-memory architecture of claim 21, wherein the bit cell circuit configuration further comprises a cathode plate of the bit cell capacitor coupled to the bit cell arithmetic unit. 21. The in-memory architecture of claim 21, wherein the bit cell circuit configuration further comprises an anode plate of the bit cell capacitor coupled to the additional capacitor. 21. The in-memory architecture of claim 21, wherein the bit cell circuit configuration further includes said bit cell arithmetic unit configured to perform arithmetic operations by two operands. 21. The in-memory architecture of claim 21, wherein the bit cell circuit configuration allows implementation of an XNOR or AND operation between stored data and input data. The bit cell circuit configuration is an interleave for at least two bit cells that allows each corresponding individual operation by combining each bit cell capacitor into at least two separate sets. 21. The in-memory architecture of claim 21, including layout. 24. The in-memory architecture of claim 24, wherein the set of coupled bit cell capacitors is placed on top of one or more bit cell transistors. The bit cell contains an in-memory computing bit cell in the charge domain that drives one plate of each local capacitor, and coupling with other bit cell capacitors is described in each of the above. The in-memory architecture of claim 26, implemented in other plates of local capacitors. 26. The in-memory computing bit cell of the charge domain is configured to implement an XNOR or AND operation between the stored data and the input data. architecture. Charge domain in-memory computing so that bit cell capacitors are coupled into different sets configured to be selectively coupled via one or more peripheral switches. The in-memory architecture of claim 26, wherein an interleaved layout of bit cells is provided. By coupling the bit cell capacitors into different sets, there is an X set for the X direction interleave, and the in-memory of the charge domain such that X is an integer greater than 1. The in-memory architecture of claim 26, wherein an interleaved layout of computing bit cells is provided. 26. A charge domain in-memory computing bit cell layout is provided such that different sets of coupled capacitors are placed on top of the bit cell transistor. In-memory architecture.

Images