TWI811300B - Computational memory device and simd controller thereof - Google Patents

Abstract

An example device includes a plurality of computational memory banks. Each computational memory bank of the plurality of computational memory banks includes an array of memory units and a plurality of processing elements connected to the array of memory units. The device further includes a plurality of single instruction, multiple data (SIMD) controllers. Each SIMD controller of the plurality of SIMD controllers is contained within at least one computational memory bank of the plurality of computational memory banks. Each SIMD controller is to provide instructions to the at least one computational memory bank.

Description

Computing memory device and its SIMD controller (Related applications are cross-referenced)

This application claims priority from US 62/648,074, filed on March 26, 2018, which is incorporated herein by reference. This application is a partial continuation of US 15/903,754 filed on February 23, 2018, which is incorporated herein by reference.

This invention relates to computational memory and neural networks.

Deep learning has proven to be a powerful technology that can perform functions that have long been inhibited by artificial intelligence methods. For example, deep learning can be applied to identifying objects in cluttered images, speech understanding and translation, medical diagnosis, games, and robotics. Deep learning techniques generally apply many layers (hence "depth") of a neural network that is trained (hence "learning") on a task of interest. Once trained, a neural network can perform "inference," that is, deducing outputs from new input data that are consistent with what it has learned.

Neural networks (also called neural nets) perform computations similar to the operations of biological neurons, typically computing weighted sums (or inner products) and exploiting memoryless nonlinearity to modify the results. Often, however, more general features are required, such as memory, multiplicative nonlinearity, and "pooling."

In many computer architecture types, power consumption is very important due to the physical movement of data between memory and processing elements, and is often the primary use of the power supply. This power consumption is generally due to the energy required to charge and discharge the wiring capacitance, which is roughly proportional to the length of the wiring, and therefore to the distance between the memory and the processing element. Therefore, processing the large consumption in such architectures (such as those commonly required by deep learning and neural networks) usually requires relatively large amounts of power. In architectures that are better suited to handle deep learning and neural networks, other drawbacks arise, such as increased complexity, increased processing time, and larger chip area requirements.

An example device includes a plurality of banks of computing memory. Each of the plurality of computing memory banks includes a memory cell array and a plurality of processing elements connected to the memory cell array. The apparatus further includes a plurality of Single Instruction Multiple Data (SIMD) controllers, each of the plurality of SIMD controllers being included in at least one of the plurality of computing memory banks. within the group. Each SIMD controller provides instructions to the at least one computing memory bank.

This and other examples are detailed below.

12N: Processing element

15N: switch

16:Bus

17:Multiplexer

17A: Bit read output

17B: Bit writing

17C:Multiplexer

17D:Multiplexer

18: Temporary register

19: Temporary register

20: Temporary register

21:Global control bus

100:Planning

104: Memory cell array

108:SIMD controller

112:Line

116: Processing components

120:Bit line

124: Column bus

128: Column connection

132: Column bus

140:Unit

142: OK

144:cell

146: column

200:Processing device

202: Row connection

204: Row bus

220: Processing device

222:SIMD controller

240: Processing device

242:Input/output circuit

244:Controller bus

246: Column bus

248: Column bus

260: Internal temporary register

262: Arithmetic Logic Unit (ALU)

264: Communication status

266: Internal status

268: Internal bus

280:Multiplexer

282:Multiplexer

284:Mux

286:Multiplexer

288:Multiplexer

290:ALU

295:ALU

300: Processing components

302:Multiplexer

400: Processing components

402: Σ square

404: Carry block

500: Column bus

502: switch

600: Processing components

700: Processing components

702:Mux

704: OK

800:SIMD controller

802: Instruction memory

804: Row selection

806: Program counter

808:Decoder

900:Controller bus

The first figure is a schematic diagram of a prior art computer system in which processing elements are embedded in memory.

The second figure is a block diagram of a computing memory arrangement according to the present invention.

Figure 3 is a block diagram of a device having a plurality of computing memory banks with processing elements connected by row buses in accordance with the present invention.

Figure 4 is a block diagram of an apparatus having a plurality of computing memory banks with a controller shared among the banks according to the present invention.

Figure 5 is a block diagram of a device having a plurality of computing memory banks with an input/output circuit in accordance with the present invention.

Figure 6 is a block diagram of a processing element according to the present invention.

Figure 7A is a block diagram of an arithmetic logic unit of a processing element according to the present invention.

Figure 7B is a block diagram of another arithmetic logic unit of the processing element according to the present invention.

Figure 7C is a block diagram of another arithmetic logic unit of the processing element according to the present invention.

Figure 8 is an exemplary arithmetic operation table of an arithmetic logic unit according to the present invention.

Figure 9 is a diagram of a segmented bus of a computing memory array in accordance with the present invention.

Figure 10 is a diagram of an internal busbar of a processing element according to the present invention.

Figure 11 is a diagram of a one-bit processing element used in the present invention, which is suitable for general processing of one-bit values.

Figure 12 is a diagram of a one-bit processing element used in the present invention, which has nearest neighbor communication in the column direction.

Figure 13 is a diagram of a one-bit processing element used in the present invention, which performs two operations for each memory read.

Figure 14 is a diagram of a multi-bit processing element used in the present invention with arithmetic load generator enhancement and reduced memory usage.

Figure 15 is a diagram of a processing element in accordance with the present invention, in which an opcode multiplexer is enhanced to function as a column bus.

Figure 16 is a diagram of a processing element according to the present invention, which has dedicated summing and carrying operations so that the bus can be used for communication at the same time.

Figure 17 is a diagram of a processing element according to the present invention having a segmented switch Column bus.

Figure 18 is a diagram of a processing element according to the present invention with closest neighbor communication in the row direction.

Figure 19 is a diagram of a processing element according to the present invention having a second multiplexer connected to a row of busses.

Figure 20 is a diagram of a controller in accordance with the present invention, which is operable to drive column addresses and operation codes, and to load and store instructions in the memory of its associated column.

Figure 21 is a diagram of a plurality of controllers interconnected by a row of buses in accordance with the present invention. Each controller can operate to control a computing memory array and can operate together to allow instruction memory of sharing.

Figure 22 is a diagram of a plurality of controllers in accordance with the present invention, each further operable to decode compressed coefficient data, and operable together to allow sharing and reuse of instruction memory as coefficient memory.

Figure 23 is an exemplary layout of a computational memory for image pixel data according to the present invention, as well as a diagram of related encoding and core output data for the first layer of a neural network.

Figure 24 is an exemplary layout of a computational memory for color pixel data according to the present invention, and a diagram of data for a convolutional layer of a neural network.

Figure 25 is an exemplary layout diagram of a computational memory for data pooling in a neural network in accordance with the present invention.

The techniques described in this article can handle large amounts of inner products and related neural network calculations with flexible low-precision arithmetic, power-efficient communications, and local storage and decoding of instructions and coefficients.

The computations involved in deep learning can be thought of as the interaction of memory and processing elements. Memory is required for input data, weights for weighted sums, intermediate results passed between layers, control and connection information, and other functions. Data in memory is processed in processing elements or PEs, such as a general-purpose computer's CPU, a Turing machine's table, or a graphics processor's processor, and returned to memory.

Deep learning and neural networks can benefit from low-power designs that perform various types of calculations in an energy-efficient manner. Low-power implementation encourages use in mobile or isolated devices, where reduced battery power consumption is important, as well as mass-scale use, where the need for cooling of processing and memory components can be the limiting factor.

In "Computational RAM: A Memory-SIMD Hybrid ," Elliott explains "matching a bit [processing element] to a narrow spacing of memory and constraining "Pitch-matching narrow 1-bit [processing elements] to the memory and restricting communications to one-dimensional interconnects ". This design aims to reduce the distance between memory and processing elements to the micron range, where the chip-to-die distance required by traditional computer architecture is in the millimeter or centimeter range (thousands or tens of thousands times larger). Ilett summarizes early work, including the early academic work of Loucks, Snelgove, and Zaky, going back to "VASTOR: A Microprocessor-Based Associative Vector for Small Applications"" VASTOR: a microprocessor-based associative vector processor for small-scale applications ", published in Intl.Conf.on Parallel Processing, August 1980, pages 37-46. Elite named this technology "C*RAM" or "Computational Random Access Memory (RAM)".

The design of ELITE and other details of extremely simple processing components requires pitch matching, including the circuitry required for one-dimensional communication. It is also possible to slightly relax the spacing matching constraints from the one-to-one correspondence of memory rows to PEs, for example, allowing each PE to occupy the width of four memory rows. This reduces PEs amount, and may be necessary, or more practical, for very compact memories.

In US Patent No. 5,546,343, Ilett and Snigoff describe the use of a multiplexer as an arithmetic and logic unit (ALU) operable to perform a function on any three-dimensional state of a processing element. As shown in the first figure, in this design type a single off-chip controller is used.

In Computational RAM: Implementation and Bit-Parallel Architecture, Cojocaru describes grouping single-bit processing elements to allow multi-bit computation. Specialized hardware was added to speed up binary arithmetic, and registers were added to reduce memory access requirements.

Yeap describes a suitable C*RAM bit processing element in the document "Design of a VASTOR processing element suitable for VLSI layout" (A.H. Yeap, M.A.Sc., University of Toronto, 1984).

In the article " Computational*RAM Implementations of Vector Quantization for Image and Video Compression", Le describes a method suitable for using computational RAM for image and video compression. algorithm.

The above implementations still fall short in several aspects for low-power deep learning applications. First, their one-dimensional communication makes it difficult to process large two-dimensional images with many channels. In addition, their complex operation codes are generally too large and therefore power-hungry for common arithmetic operations. Operation codes and communication buses take up a lot of chip area. In addition, their processing elements cannot perform permutation or correlation mapping, lookup tables, or processor-specific operations. Additionally, these approaches tend to rely on off-chip controllers, which consume significant amounts of power when communicating with the computing RAM itself. Finally, they are typically pure single-instruction streaming, multi-data streaming devices that can handle consistent operations on large data sets well, but cannot share their processing resources when several smaller tasks are required.

In view of these and other shortcomings of past attempts, the techniques described in this article aim to improve Computational memory, designed to handle large numbers of inner products and related neural network calculations with flexible low-precision arithmetic, provide power-efficient communications, and provide local storage and decoding of instructions and coefficients.

The second figure illustrates a computing memory array 100, referred to as C*RAM, according to an embodiment of the present invention. The computing memory bank 100 includes a memory cell array 104 and a plurality of processing elements 116 connected to the memory cell array 104 .

The computing memory bank 100 further includes a single instruction multiple data (SIMD) controller 108 included within the computing memory bank 100 . SIMD controller 108 provides instructions, and optionally data, to computing memory array 100 . In this particular embodiment, SIMD controller 108 is provided to only one compute memory bank 100 . In other embodiments, the SIMD controller 108 may be shared among several computing memory banks 100 .

Furthermore, in this particular embodiment, the memory cell array 104 is generally rectangular in shape, and the SIMD controller 108 is located near the narrow end of the array. The SIMD controller 108 may be located on either side of the memory cell array 104, that is, on the right side or on the left side, as shown. This provides a space-efficient arrangement of the memory cell array 104, the SIMD controller 108, and the processing elements 116 such that the arrays 100 can be arranged in a rectangular or square configuration, providing efficient operation on a semiconductor substrate or wafer. layout.

Each cell 140 of memory cell array 104 includes a row 142 of memory cells 144 . Cell 144 can be configured to store one bit of information. Cells 144 at the same relative position in a plurality of rows 142 may form a column 146 of cells 144 . The cells 140 of the memory cell array 104 may also be arranged in columns, where a column of cells 140 includes a plurality of columns 146 of cells 144 . In this particular embodiment, each row 142 of cells 144 is connected to a different processing element 116 by a bit line 120 . In other embodiments, multiple rows 142 of cells 144 are connected to each different processing element 116 via bit lines 120 .

Memory cell array 104 is connected to SIMD via one or more column select lines 112 Controller 108, which is also called word line. The SIMD controller 108 may output a signal on the select line 112 to select a column 146 of cells 144 . Accordingly, the SIMD controller 108 may address a column 146 of the memory array 104 via the column select line 112 so that the selected bits in the row 142 are available to the processing element 116 via the bit line 120 .

SIMD controller 108 may include instruction memory loaded from memory cell array 104 .

In this specific embodiment, the memory cell array 104 is a static random access memory (SRAM). For example, each memory cell 144 may be formed of six transistors, such as metal-oxide-semiconductor field effect transistors (MOSFETs), and may be referred to as a 6T memory.

In other embodiments, other types of memory may be used, such as dynamic RAM, ferroelectric RAM, magnetic RAM, or a combination of different types of memory, 1T, 2T, 5T, etc. SRAM memory cells can be used. Memories that are particularly suitable for use in the present invention are memories that have column addressing simultaneously enabling corresponding bits in the rows, and memories that are constructed in a layout that matches the spacing of the SIMD controller 108 and processing elements 116 .

Memory cell array 104 may be divided into subsets with different access energy costs. For example, a "heavy" subset may have memory cells with greater capacitance due to longer bit lines, which therefore utilize more power to access, but with increased density. The "light" subset may have lower capacitance memory cells that use less power to access, but have lower density. Therefore, when a heavy subset is used to store information that is accessed less frequently (such as coefficients and codes), and a light subset is used for information that is accessed more frequently (such as intermediate results), the power Consumption and space efficiency can be improved.

The processing elements 116 are arranged along the width of the memory cell array 104 and are physically located proximate the memory cell array 104 . The processing elements 116 may be arranged in a linear array and addressed sequentially. In this particular embodiment, each processing element 116 is connected to and aligned with a row 142 of the memory cell array 104 .

Addressing of processing elements 116 may be big-endian or little-endian, and may start from the left or right based on implementation preference.

Processing elements 116 may be structurally identical to each other. A large number of relatively simple and structurally identical processing elements 116 can be beneficial for applications in neural networks, since neural networks typically need to process a large number of coefficients. In this context, substantial structural similarity means that small differences in implementation requirements (eg, hardware wiring addresses and different connections of the final processing elements) are to be expected. The repetitive and simplified array of processing elements 116 can reduce design complexity and increase space efficiency in neural network applications.

Each processing element 116 may include registers and an ALU. The registers may include internal registers for performing operations with the processing element 116 and communication registers for communication status with other processing elements 116 . Each processing element 116 may further include communication status provided by one or more other processing elements 116 . The ALU is configured to execute an arbitrary function, such as a function of one or more operands defined by an opcode.

Processing elements 116 may be connected to the SIMD controller 108 via any number and arrangement of column buses 124, 132. Column buses 124 , 132 are operable to pass information between any SIMD controller 108 and a plurality of processing elements 116 in one or two directions. Column bus 132 may provide a level of segmentation such that a subset of processing elements 116 may communicate via column bus 132 . Segmentation of the column bus 132 may be permanent, or may be initiated by a switch to be turned on or off by the SIMD controller 108 or a processing element 116 . Column buses 124, 132 may be provided with latches that may enable data replacement, local operations, and similar functions. Although illustrated as linear, column bus bars 124, 132 may include any number of lines. Column buses 124, 132 may be connected to the ALU of processing element 116 to facilitate computing while data is read from and written to buses 124, 132.

For example, the plurality of column buses 124 and 132 may include an operand bus 124 and a general column bus 132 . Operand bus 124 may be used to pass operand selections from SIMD controller 108 to processing elements 116 such that each processing element 116 is Selected local operands perform the same operation. Universal column bus 132 may carry carry data and opcode information to supplement operand selection carried by operand bus 124 .

Processing element column connections 128 may be provided to directly connect processing elements 116 such that a given processing element 116 can communicate directly with an adjacent or remote processing element 116 in the array 100 . Column connections 128 may allow sharing of status information, such as sum and carry values, and address information. Column connections 128 can facilitate column shifting, which can be unidirectional (left or right) or bidirectional (left and right), and further can be circular. Column connections 128 may be configured to serve as shifts to adjacent processing elements 116 (eg, the next bit in either/both directions), and to end processing elements 116 (eg, eight bits apart in either/both directions). or some other number of bits of processing elements 116). One or more registers of processing element 116 may be used to store information received via column connection 128 .

Processing element column connection 128 may provide a ripple link for ALU pass-through, summation, or other output. These values do not need to be latched and are based on the values in the local registers of the processing element 116 and the values received from any bus. Dynamic logic (which can be precharged to high bits) can be used so that when the input decreases monotonically, the ripple function can decrease monotonically; Carry is a monotonically increasing function, which can be effectively low, so that the carry is initially high. (i.e. pre-charged) and can be discharged to low level, but will not turn to high level for summing.

There are at least four types of communications between processing elements 116 within array 100 . First, synchronous communications can be performed using column connections 128 and associated communications registers. Second, asynchronous communication over a ripple pass-type link can be performed via column connection 128 , and two such links can carry information in opposite directions in a linear array of processing elements 116 . Both links are available for multi-bit arithmetic (such as passing left or right marches and reverse sign-extended marches), and can be used for search and max-pooling type operations. Third, processing element 116 can write information to column bus 132 and this information can be read by SIMD controller 108 or by another processing element 116 . For example, one group of processing elements 116 may write information to a segmented column bus 132 , which may then be processed by the SIMD controller 108 or by another group. Component 116 to read. Fourth, processing elements 116 in adjacent banks 100 can communicate synchronously. In various embodiments, any or more of these four types of communications may be implemented.

From the above, it can be seen that the computing memory array 100 is a space-efficient unit of controllable computing memory, which is suitable for being reproduced in the same or substantially the same form into a space-efficient pattern. Operations on data stored in the memory cell array 104 can be performed by adjacent processing elements 116 so that the operations can be performed in parallel while reducing the energy spent on transferring data back and forth between the processor and the memory. Or reach the minimum.

The third figure illustrates a specific embodiment of a processing device 200 that may be constructed from a plurality of computing memory banks (eg, bank 100). Each computing memory bank 100 includes an array of memory cells 104 and a plurality of processing elements 116, as described elsewhere herein.

A plurality of SIMD controllers 108 are provided to the computing memory bank 100 to provide instructions (and optionally data) to the bank 100 . In this particular embodiment, each bank 100 includes its own different SIMD controller 108 . This provides finer-grained control than having a single controller shared by all platoons. The operation of the SIMD controller 108 may be coordinated in a master/slave scheme, an interrupt/wait scheme, or the like.

Any number of arrays 100 may be provided for the processing device 200 . The size of each row group 100 and the arrangement of the row groups 100 may be selected to provide the width (W) and height (H) dimensions of the device 200 to increase or maximize layout efficiency, such as efficient use of silicon, while also Reduce or minimize the distance between processing and memory, thereby reducing or minimizing power requirements. The arrays 100 may be arranged as a linear array and may be addressed sequentially.

The addressing of the row group 100 may be big-endian or little-endian, and may start from the top or bottom depending on implementation preference.

Apparatus 200 may include processing element row connections 202 to connect processing elements 116 in different rows 100 such that a given processing element 116 can communicate with another processing element in an adjacent or end row 100 Component 116 communicates directly. Row connections 202 facilitate row shifting, which may be unidirectional (up or down) or bidirectional (up and down), and further may be circular. One or more registers of processing element 116 may be used to store information received via row connection 202 .

The device 200 may include a row bus 204 to connect any number of the processing elements 116 of the computing memory bank 100 . In this particular embodiment, a row 142 of memory spans the bank 100 , and each processing element 116 associated with the same row 142 is connected by a row bus 204 . Although depicted as one line, row bus 204 may include any number of lines. Any number and arrangement of row buses 204 may be provided.

Processing elements 116 in different row groups 100 may communicate with each other via the row bus 204 . Row bus 204 is operable to pass information between connected processing elements 116 in one or two directions. Row bus 204 may pass opcode information to supplement information passed by other paths, such as operand bus 124 within each row group 100 . Any number and arrangement of row buses 204 may be provided. A given row bus may provide a given degree of segmentation such that a subset of the processing elements 116 in a corresponding row 142 may communicate via the row bus. The segmentation of the row bus 204 may be permanent, or may be switch enabled and toggled on or off by the SIMD controller 108 or processing element 116 .

Column buses 132 connecting processing elements 116 within bank groups 100 and row buses 204 connecting processing elements 116 between bank groups 100 may allow controlled two-dimensional communication of data and instructions within processing device 200 . This enhances the processing of large images, which can be mapped to rectangular or square areas to reduce or minimize communication distance and therefore power requirements. Thus, the controlled two-dimensional communication provided by buses 132, 204 may allow efficient implementation of neural networks that process images or similar information.

Additionally, configuring the SIMD controller 108 to match the height (H) of the rows 100 may allow multiple control rows 100 to be placed in a space-efficient manner, one above the other, tilted in the row direction. This allows the production of almost square arrays, which facilitates packaging even though the individual row groups 100 are opposite is very wide (i.e., in the column dimension or width W) in its height (i.e., in the main part of the row dimension or total height H). This can be useful for a variety of practical RAM circuits, and for having a large number of processors to amortize the area and power costs of a single SIMD controller 108.

In an exemplary implementation, referring to Figures 2 and 3, the processing device 200 includes 32 computing memory banks 100, each having a memory cell array 104 containing 4096 rows of memory. Within each bank 100 , each row 142 contains 192 memory bits connected to a processing element 116 .

It should be apparent from the above that the processing device 200 may include a stacked structure of the computing memory array 100 to increase processing capabilities and allow for large-scale parallel operations while still maintaining a space-efficient overall layout of the array 100, and Reduce or minimize the energy expended in transmitting data back and forth between rows 100. The advantages of a single row group 100 can be replicated in the row direction, and in addition, a communication method can be provided for the row group 100 .

Figure 4 illustrates a specific embodiment of a processing device 220 constructed from a plurality of compute memory banks (eg, bank 100). The processing device 220 is similar to other devices described herein, and repeated descriptions are omitted for clarity. Reference may be made to the related descriptions of other specific embodiments, in which the same reference numerals represent the same components.

A plurality of SIMD controllers 108, 222 are provided to the computing memory bank 100 to provide instructions (and optionally data) to the bank 100. In this particular embodiment, SIMD controller 222 is included within at least two computing memory banks 100 . That is, the SIMD controller 222 may be shared by multiple computing memory banks 100 . Any number of other banks 100 may include dedicated, or shared SIMD controllers 108, 222.

Choosing the ratio of banks 100 that share controllers 222 to banks 100 that have their own dedicated controllers 108 may allow for balanced utilization of processing elements to be implemented, which drives toward increasing the number of dedicated controllers 108 and thus enabling good area utilization. and power efficiency to deal with smaller problems, which is towards increasing Increase the number of shared controllers 222 driven to limit duplication.

Figure 5 illustrates a specific embodiment of a processing device 240 constructed from a plurality of compute memory banks (eg, bank 100). The processing device 240 is similar to other devices described herein, and repeated descriptions are omitted for clarity. Reference may be made to the related descriptions of other specific embodiments, in which the same reference numerals represent the same components.

Among the plurality of computing memory banks 100, at least one bank 100 includes an input/output circuit 242 for software-driven input/output. In this particular embodiment, the lowermost bank 100 includes an input/output circuit 242 connected to its SIMD controller 108 . The software-driven input/output may be provided by another device, such as a general-purpose processor, co-located with the processing device 240 in the same larger device, such as a tablet, smart phone, wearable device, etc. The input/output circuit 242 may include a serial peripheral interface (SPI), a double data rate (DDR) interface, a mobile industrial processor interface (MIPI), a peripheral component interconnect express (PCIe), etc. Any number of input/output circuits 242 may be provided to support any number of such interfaces.

Input/output circuitry 242 may be configured to cause SIMD controller 108 to perform operations. SIMD controller 108 may be configured to cause input/output circuitry 242 to perform operations.

The input/output circuitry 242 may be configured to reset the SIMD controller 108 and to read and write the registers of the SIMD controller 108 . Through the registers of the SIMD controller 108, the input/output circuit 242 allows the SIMD controller 108 to perform operations, including writing instructions to the memory. Therefore, the initialization process may include resetting the SIMD controller 108, writing the startup code to the bottom of the instruction memory, and releasing the reset, at which point the startup code is executed.

Additionally, a plurality of SIMD controllers 108 may be connected to a controller bus 244 to provide intercommunication between the SIMD controllers 108 . Input/output circuitry 242 may also be connected to controller bus 244 for communication with SIMD controller 108, and such connection may be made through the SIMD controller 108 of its array 100, as described, or directly. Controller bus 244 allows Sharing of data and instructions, and coordination of processing operations.

Any number of controller busses 244 may be provided. Controller bus 244 may be segmented to any suitable extent. For example, the first controller bus 244 can be a full-height bus that connects all SIMD controllers 108; the second controller bus 244 can be segmented into two half-height busses that connect all SIMD controllers 108. The SIMD controller 108 is divided into two groups; and the third and fourth controller busses 244 can each be divided into four segments. Therefore, different groups of SIMD controllers 108 can be defined to coordinate operations. A given SIMD controller 108 can subscribe to any connected controller bus 244.

When the SIMD controller 108 is to operate in a master/slave scheme, the SIMD controller 108 operating as a slave does nothing but relay sequences from its master/slave controller 244 to its connected computing memory bank 100 Don't do it either. The SIMD controller 108 operating as slave devices will be inactive with the pointer register, loop counter, stack and instruction memory.

Additionally, in this particular embodiment, a plurality of universal column buses 246 , 248 are provided to connect the processing elements 116 and the SIMD controller 108 in each bank group 100 . Column buses 246 , 248 may include a main column bus 246 unidirectionally oriented from the SIMD controller 108 to all processing elements 116 in the bank 100 and between groups of processing elements 116 in the bank 100 A segmented column bus 248 for local bidirectional communication. The main column bus 246 connects the SIMD controller 108 to the processing elements 116 of each bank 100 to distribute operation codes and data. Segmented column bus 248 is used for local operations such as substitutions and in-pipeline processing element transfers of information.

It should be understood from the above description that the controller bus 244 provides operational configuration flexibility of the computing memory array 100 . Additionally, input/output circuitry 242 allows SIMD controller 108 to manage and coordinate the operation of device 240 .

Figure 6 illustrates a specific embodiment of a processing element 116 that may be used in a computational memory bank (eg, bank 100).

The processing element 116 includes an internal register 260, an arithmetic logic unit (ALU) 262, Communication status 264 and internal status 266. Internal registers 260 and communication status 264 are connected to a row of memory 142 via an internal bus 268 (which may be a differential bus). The internal register 260 can be implemented as a contact 6T memory cell, where in addition to the standard outputs to the bit lines, the state of the register can also be read directly by external circuitry.

Internal bus 268 can be written to and read from memory rows 142, internal registers 260, ALU 262, and communication status 264.

The internal register 260 may include a plurality of general registers (eg, R0, R1, R2, R3), a plurality of static registers (eg, X, Y), accessible to adjacent processing elements 116 A plurality of communication registers (for example, Xs, Ys), and a masking bit (for example, K). Internal register 260 may be connected to internal bus 268 to be written to, or to other registers, and to communicate information with ALU 262 . The SIMD controller 108 can control which internal registers 260 are to be written to and read from, and whether the shadow bit K is to be overwritten.

Internal register 260 may be configured for performing arithmetic operations with ALU 262, such as sums and differences. Generally speaking, the internal register 260 can be used to calculate any function.

Static registers X, Y may be configured via communication buffers that are associated with static registers The processors Xs, Ys are used to provide information to adjacent processing elements 116 in the same bank or in different banks. The communication status 264 of the connected processing element 116 is obtained from the local communication registers Xs, Ys. Accordingly, ALU 262 may be configured to transfer data (eg, perform shifting) in a synchronous or pipeline manner between connected processing elements 116 . The SIMD controller 108 can provide a strobe pulse specific to the communication registers Xs, Ys, so that the strobe pulse can be skipped and its power can be saved. The mask bit K in the processing element 116 protects the static registers X and Y in the same processing element 116 but not the communication registers Xs and Ys.

In this example, communication registers Xs, Ys are readable by adjacent processing elements 116 in the same bank, and communication register Ys is readable by processing elements 116 in the same row in adjacent banks Read. That is, the registers Xs and Ys can transmit information in the column direction through, for example, the column connection 128, and the register Ys can transmit information in the row direction through, for example, the row connection 202. Other examples are contemplated, such as limiting register Ys to row communications and using register Xs only for column communications.

The communication registers Xs, Ys can be implemented as slave latch stages so that their values can be used by other processing elements 116 without creating race conditions.

Shadow bit K may be configured to disable all write operations (eg, writes to memory row 142, register 260, and/or column buses 246, 248) unless it is overridden by the connected SIMD controller 108. Masking bit K can be configured to disable writeback when high; this can include masking bit K disabling itself, so that unless masking bit K is overwritten, subsequent writes to masking bit K will disable writeback in the linear array. More and more processing elements 116 are coming. This has the implementation advantage that the shaded bit K can be established exactly like other bits, and the programming advantage that the shaded bit K implements nested conditional states (i.e., "if" statements) without increasing complexity.

ALU 262 may include multiple levels of multiplexers (eg, two levels). ALU 262 may be configured to select inputs from internal registers 260, communication state 264, and internal state 266, and allow arbitrary functions to be calculated on these inputs. Functions may be defined from information passed via buses 246, 248.

The communication status 264 includes information based on communication registers (eg, Xs, Ys) of other processing elements 116 . Communication status 264 may be used for shifting and similar operations.

Communication status 264 may include X-adjacent states Xm, Xp from communication registers Xs of adjacent processing elements 116 in the same bank 100 . The communication status Xm may be the value of the register Xs of the adjacent processing element 116 with a lower address (ie, "m" means "minus"); The value of the register Xs of the adjacent processing element 116 (ie, "p" means "plus"). The X-neighbor states Xm, Xp at each end of the linear array of processing elements 116 may be set to a specific value, such as zero. In other embodiments, on processing element 116 The X-adjacent states Xm,

To facilitate greater capability for column-based communications within a bank 100 , communication states 264 may include other X-adjacent states Yxm, Yxp from communication registers Ys of adjacent processing elements 116 of the same bank 100 . The communication status Yxm may be the value of the register Ys of the adjacent processing element 116 with a lower address; the communication status Yxp may be the value of the register Ys of the adjacent processing element 116 with a higher address. The other X-neighbor states Yxm, Yxp at each end of the linear array of processing elements 116 may be set to a specific value, such as zero. In other embodiments, the other X-neighbor states Yxm, Yxp at each end of the linear array of processing elements 116 may be wired to obtain their values from the communication register Ys at the opposite end, such that the values Can "roll".

Communication status 264 may include X-remote status Xm8, The communication status Xm8 may be the value of the register Xs of the processing element 116 with an 8-bit lower address. The communication state Xp8 may be the value of the register Xs of the processing element 116 with an address that is 8 bits higher. The X-distal states Xm8, Xp8 near each end of the linear array of processing elements 116 may be set to a specific value, such as zero. In other embodiments, the X-remote states Xm8, Allows the value to "roll" a fixed address distance.

The communication status 264 may include Y-adjacent states Ym, Yp from the communication register Ys of the processing element 116 in the same row of the adjacent bank 100 . The communication status Ym may be the value of the register Ys of the corresponding processing element 116 with a lower address in the adjacent bank 100 . The communication status Yp may be the value of the register Ys of the corresponding processing element 116 with a higher address in the adjacent bank 100 . Fixed thresholds or rolling can be implemented, as explained above.

SIMD controller 108 may be configured to access X-remote states Xp8, The registers Xs and Ys of the last processing element 116 in the linear array of processing elements are configured so that the static registers X and Y values of the last and adjacent processing elements 116 can be read.

Communication state 264 may further include a carry input Ci and another input Zi, which may represent sign extension.

The carry input Ci may fluctuate asynchronously from the carry output Co of an adjacent processing element 116 . The carry input Ci of the last row may be provided by the SIMD controller 108 . When the bank 100 is divided into two halves, the carry input Ci of the end-most row of each half of the bank 100 may be provided by the SIMD controller 108 . The carry input Ci can be expected to decrease monotonically with time.

The sign extension input Zi may ripple asynchronously from the sum Z of adjacent processing elements 116 in the opposite direction to the carry ripple. As opposed to the last row of carry input Ci, the last row of sign extension input Zi may be provided by the SIMD controller 108 . When the row group 100 is divided into two halves, the sign extension input Zi for the end-most row of each half of the row group 100 may be provided by the SIMD controller 108 . The sign-extended input Zi can be expected to decrease monotonically with time. The input Zi can also be used to fluctuate an arbitrary function.

SIMD controller 108 may be configured to read carry output Co from one end of the linear array of processing elements 116 and read output Zo (eg, a sign-extended output) at an opposite end of the linear array of processing elements 116 .

A given processing element's communication status 264 may be implemented as an endpoint of direct connections 128 to other processing elements 116 .

Internal state 266 may include address bit An, high-order bit HB, and low-order bit LB. The address bits An, high-order bits HB, and low-order bits LB may be used to place a processing element 116 in the context of a plurality of processing elements 116 in a linear array of processing elements 116 .

The address bits are hard-coded such that each processing element 116 is uniquely addressable within array 100 . In the example of 4096 processing elements per bank, 12 address bits (A0-A11) can be used. In other embodiments, the address bit An may be stored in a temporary register, And can be configured by SIMD controller 108.

The SIMD controller 108 may select a precision level for the array 100 and the upper bits HB and lower bits LB may be derived from the selected accuracy level. The accuracy level selection may identify to the processing element 116 which address bit An is to be referenced for calculating the upper bit HB and the lower bit LB. SIMD controller 108 may make accuracy level selections by passing an accuracy signal to all processing elements 116 in array 100 . The accuracy signal may indicate which address bit An is to be used as the accuracy indicating address bit An of the array 100 . The accuracy signal may be a single hot signal on a number of lines equivalent to the number of address bits An, or it may be a coded signal, such as a 4-bit signal, which uniquely identifies an address bit An. .

The high-order bits HB and low-order bits LB may group processing elements 116 for multi-bit arithmetic. These groups can be fixed in size and are powers of 2.

The low bit LB defines the lowest bit in the group. When the accuracy indication address bit An of the processing element 116 is not set (eg, 0), and the accuracy indication address bit An of the next processing element 116 in the low-order bit direction is set (eg, 1), The lower bit LB is set (for example, set to 1) in a specific processing element 116 .

The high bit HB defines the highest bit in the group. When the accuracy indication address bit An of processing element 116 is set (eg, 1) and the accuracy indication address bit An of the next processing element 116 in the upper bit direction is not set (eg, 0), The high-order bit HB is set (for example, set to 1) in a specific processing element 116 .

Only one of the high-order bit HB and the low-order bit LB needs to be calculated. If processing element 116 has its upper bit HB set, then the lower bit LB of the next processing element 116 can be set. On the contrary, if the processing element 116 does not have its high-order bit HB set, then the low-order bit LB of the next processing element 116 should not be set.

Used to set the address and set the accuracy via the high-order bit HB and the low-order bit LB The technique works with both big-endian and little-endian conventions.

The high-order bits HB and low-order bits LB can be used to limit the propagation of the carry input Ci and the sign extension input Zi to meet the operational accuracy of the array 100 .

As shown in Figure 7A, the ALU 262 may include two levels of multiplexers. The first level may include multiplexers 280, 282, 284, while the second level may include multiplexers 286, 288. For space and energy efficiency, multiplexers can be implemented as dynamic logic. SIMD controller 108 may provide a clock to gate the multiplexers.

The first level multiplexers 280 , 282 , 284 may be configured to provide selection bits (eg, three selection bits) to the second level multiplexers 286 , 288 based on input from the operand bus 124 . The first level multiplexers 280, 282, 284 can be configured for single hot inputs such that one of the inputs can be selected. Inputs to the first level multiplexers 280, 282, 284 may include any of the various bits available at the processing element 116, such as internal registers 260, communication status 264, and internal status 266. The outputs to the second level multiplexers 286, 288 may include differential signals. The first level multiplexers 280, 282, 284 may be implemented using parallel N-type metal oxide semiconductor logic (NMOS) devices.

Internal registers 260, communication status 264, and internal status 266 may be provided as inputs to allow execution of an arbitrary function. For example, registers X, Y and R1-R4 and communication states Xp, Yp, Xm, Ym, Xp8, A11 can be used to assign specific values to specific processing elements, for flipping multi-bit values, etc. There are no specific restrictions on any functions that can be executed.

Second level multiplexers 286, 288 may include a main bus multiplexer 286 and a segment bus multiplexer 288. Primary bus multiplexer 286 may be configured to receive input via primary column bus 246 , such as a truth table from SIMD controller 108 , which may be 8 bits. Segment bus multiplexer 288 may be configured to receive input via segment column bus 246 , such as a truth table from SIMD controller 108 , which may be 8 bits. The second level multiplexers 286, 288 compute an arbitrary function, It can be defined through buses 246, 248. This function operates on an operand (eg, 3 bits) selected by the first level multiplexer 280, 282, 284 and provided as a selection input to the second level multiplexer 286, 288. NMOS switch trees driven by differential signals from the first level multiplexers 280, 282, 284 can be used to implement the second level multiplexers 286, 288.

Status information from the processing elements 116 including the ALU 262 is provided to the first level multiplexers 280, 282, 284 whose control inputs are provided to the bank 100 by the associated SIMD controller 108 via the operand bus 124. All processing elements 116. Thus, operations can be performed across all processing elements 116 via operand bus 124 based on the operands selected using SIMD controller 108 , and the operations can be performed across the entire array 100 via primary column bus 246 Shared operations or other information, and/or locally shared operations or other information on segmented column bus 246 .

ALU 262 may be used to write to buses 204, 246, 248. The bus lines used for writing are selected by the outputs (ie, 3-bit outputs) of the first level multiplexers 280, 282, 284 to select one of the eight lines.

Figure 7B illustrates ALU 290 according to another specific embodiment. ALU 290 is similar to ALU 262 except that it does not provide fixed address bits A0-A11 as inputs to the first level multiplexers 280, 282, 284. ALU 290 is a simpler ALU that does not allow functions related to the address of processing element 116 . It is contemplated that various other ALUs may be used as the input subset shown for ALU 262.

Figure 7C illustrates an ALU 295 according to another specific embodiment. ALU 295 is similar to ALU 262, except that it provides a selectable address bit An instead of fixed address bits A0-A11 as input to the first level multiplexer 280. Therefore, ALU 295 can access a selected address bit for its calculation.

The eighth figure illustrates one of the exemplary arithmetic operations tables of the ALU 262 showing the truth table for the carry output Co and the sum Z. The example operation is addition, and other operations can be performed directly.

The first level multiplexers 280, 282, 284 may provide operand values (eg, the values of registers R0 and R1) and carry inputs Ci to the second level multiplexers 286, 288 via the main column bus 246, respectively. A sum Z truth table is received, and a carry output Co truth table is received via segmented column bus 248 . Therefore, the second level multiplexers 286, 288 can calculate the sum Z and the carry output Co.

The truth table of the carry output Co and the sum Z can be regarded as the operation code of addition. In this example, the addition operation code in hexadecimal is 0x2b 0x69. The opcode part 0x2b is the carry output Co truth table (i.e. bits 0010 1011 of the Co row read from bottom to top), while the opcode part 0x69 is the sum Z truth table (i.e. Z read from bottom to top Row bits 0110 1001). The carry output Co opcode portion 0x2b and the sum Z opcode portion 0x69 are provided to the segmented column bus 248 and main column bus 246 respectively, such that the second level multiplexers 286, 288 transfer the first level multiplexer The operands provided by 280, 282, and 284 are added to the output sum Z and carry Co.

The carry may be propagated through the group of processing elements 116 as carry input Ci to carry output Co. Carry propagation is delineated at positions selected by the SIMD controller 108 as a power of 2, which delimitation is available to the processing element 116 for the upper bit HB and the lower bit LB.

Figure 9 illustrates a specific embodiment of segmented bus 248. Each input of segment bus multiplexer 288 of each processing element 116 may be connected to a corresponding line of segment bus 248 . The segment bus 248 can be preset high in each segment by the SIMD controller 108 and then left floating so that any enabled segment bus multiplexer 288 can pull the line low and then latch.

SIMD controller 108 has access to the end-most segment and can be configured to read and write the end-most segment. This can be used to pump data from the memory cell array 104 to the SIMD controller 108, for example to load controller code from main memory. Information specific to processing element 116 may be similarly distributed from SIMD controller 108 . The array 100 including an input/output circuit 242 may use this mechanism for input/output.

Segmented bus 248 can also be used to perform lookup tables, where processing elements 116 set their own opcodes, since segmented bus 248 can be written to and read locally.

Figure 10 illustrates a specific embodiment of the internal busbars of the processing element 116, as well as exemplary implementation details of the processing element 116. Heavy and light memory cells and internal registers can be implemented using sense-amplifier structures as shown in the figure.

Figures 11-14 illustrate conventional processing elements suitable for use in the computing memory array 100 of the present invention. While certain structures/functions of the processing elements are known, their application to the computing memory array 100 is considered part of the present invention.

Figure 11 illustrates a prior art processing element 12N that can be used as the processing element 116 in the computing memory bank 100 of the present invention. Processing element 12N contains an ALU implemented as an 8:1 multiplexer 17. The output lines of the multiplexer 17 are connected to the data inputs of the registers 18 (ie, static register X) and 19 (ie, static register Y), and to the write enable register 20 and the bit write Input 17B (which may be provided to a row 142 in the memory cell array 104). Bit read output 17A may be provided to line 142 along with the data outputs of registers 18 and 19 to address multiplexer 17 and thereby select the eight operations to be input to it from a global control bus 21 Which of the code lines is connected to its output. In this manner, multiplexer 17 is used to calculate an arbitrary function of the bit values at 17A, 18 and 19. This arbitrary function may be defined by a truth table represented by 8-bit values on the global control bus 21. Global control bus 21 may be column bus 132, as described elsewhere herein.

Writing to startup register 20 allows conditional execution. For example, by disabling writing in some processing elements 12N but not others, the same instructions can be executed in all processing elements 12N, but writing is selectively enabled. Therefore, one can do this by initiating a write (which is done by calculating the write enable as a condition for all processing elements 12N), then executing the THEN block, then inverting the write enable in all processing elements 12N, and then executing The "ELSE" block handles the condition ("IF") that causes the execution of the "THEN" or "ELSE" block.

In addition to providing the 8-bit truth table of the ALU, the global control bus 21 can also provide clock signals "Write W/E)" so that ALU data can be clocked into registers 18, 19 and 20. The bus 21 may further provide control signals "Group Write" and "Write", which allow external input data to be written into the memory without using the ALU. This external input data may be driven from, for example, a 16-bit data bus 16 onto line 17B through switch 15N. Data bus 16 can also be used to load registers 18 and 19 via this path.

Figure 12 illustrates a bit processing element of a conventional technique proposed by Elliott, which has closest neighbor communication in the column direction. This processing element can be suitably used as the processing element 116 in the computing memory array 100 of the present invention. This processing element adds secondary inputs and outputs to the X and Y registers, allowing each The register can be loaded from the output of the ALU to its left ("right shift") or both.

Figure 13 illustrates a conventional one-bit processing device disclosed in US Pat. No. 5,546,343, which can perform two operations per memory read. This processing element may be adapted as the processing element 116 of the computing memory array 100 of the present invention. The global control bus can be doubled to 16 bits wide, so it can carry two 8-bit truth tables. Multiplexers 17C and 17D simultaneously compute three local status bits X, Y and two functions of memory. X and Y values can be calculated simultaneously.

Figure 14 illustrates a conventional multi-bit processing element proposed by Cojocaru. This processing element includes carry generator enhancements for arithmetic and has less memory usage. This processing element can be suitably used as the processing element 116 in the computing memory array 100 of the present invention. An obvious feature is that the X and Y registers have been generalized into register banks, in this case there are two registers each (such as X and AX), and the memory has been treated similarly A type of register arrangement in which one register ("M") is replaced by a bit read from memory. Read-only bits can also be processed as registers in a register array. For low-power applications, there may be a need to cache data in low-power registers rather than repeatedly referencing high-power memory. Note that the left-right nearest neighbor communication described elsewhere in this article can also be used with this structure.

A further enhancement here is the addition of a "Carry" block, which has an input "Carry-in" from an adjacent processing element, which can be combined with data from the X and Y register arrays, and which The resulting "Carry Out" will be passed to the next processing element in the opposite direction as appropriate. Registers S and B can be used to suppress carry propagation ("S") and replace it with a given bit "B". For example, if register S is set up to suppress carry propagation in every four processing elements, and the carry is replaced with a "0", the effect is to queue from a compute memory with N single-bit processing elements A system with N/4-bit processing elements is produced in 100. If you need to perform 8-bit calculations four bits at a time in a group of four processing elements, you can add a path for storing "Carry-Out" in the local processing element.

Figure 14 also illustrates a conventional segmented bus in which a register T can be used to enable or disable connections between adjacent bus segments (labeled "Bus-Official Segments"). tie segment)") switch. This allows a single bus to be split into any number of smaller local busses.

Figure 15 illustrates a processing element 300 according to the present invention. The processing element 300 may be used as the processing element 116 in the computing memory array 100 of the present invention. Processing element 300 is similar to other devices described herein, and repeated description is omitted for clarity. Reference may also be made to the relevant descriptions of other specific embodiments, in which the same reference numerals represent the same components.

Processing element 300 includes an opcode multiplexer 302 configured as a column-wise bus. Multiplexer 302 is used for bidirectional communication. Area-efficient multiplexers can be implemented using switch trees without adding complexity. X and Y registers (R0 and R1) are provided and are also bidirectional on the port connected to the multiplex side of multiplexer 302. Register tri-state and sense amplifier types are available Used for X and Y registers. In various other embodiments of the present invention, bidirectional multiplexer 302 is combined with other features described herein, such as register banks, double-operand or carry-enhanced processing elements, carry suppression, and the like.

If space is at a premium, or if space needs to be supplemented when increasing communication bandwidth, making the multiplexer 302 bidirectional allows the column bus 132 to be eliminated.

Figure 16 illustrates a processing element 400 according to the present invention, which has dedicated sum and carry operations so that the column-oriented bus can be used for communication simultaneously. The processing element 400 may be used as a processing element 116 in the computing memory array 100 of the present invention. Processing element 400 is similar to other devices described herein, and repeated description is omitted for clarity. Reference may be made to the related descriptions of other specific embodiments, in which the same reference numerals represent the same components.

Σ(sigma) block 402 operates to calculate the bit sum of its three inputs X, Y, and M. Carry block 404 may operate to simultaneously calculate carry bits. Both sums and carries are written back to any combination of X, Y, M (memory) and W (write enable) registers, which can be implemented as memory banks. At the same time, column bus 132 can be read into X, Y, M, or W, or a single column bus line selected by three X, Y, M can be driven from X, Y, M, or W. Any register can be implemented as a register bus. In addition, arithmetic blocks can be driven and multiplexers can be addressed by different registers from these register files. In addition, latching of multiplexer address or arithmetic inputs can be provided. Column bus bits can be addressed independently of arithmetic operations.

Figure 17 illustrates a processing element 400 having a column bus 500 with segmented switches 502. In some embodiments, switch 502 may be controlled by a register in an associated processing element 400 . In other embodiments, switch 502 is controlled directly by SIMD controller 108 of computing memory bank 100 .

Figure 18 illustrates a processing element 600 with closest neighbor communication in the row direction in accordance with the present invention. The processing element 600 may be used as the computing memory array 100 of the present invention. processing element 116. Processing element 600 is similar to other devices described herein, and repeated description is omitted for clarity. Reference may also be made to the relevant descriptions of other specific embodiments, in which the same reference numerals represent the same components.

Closest neighbor communication in the row direction may be combined with closest neighbor communication in the column direction. In some embodiments, X and Y are single registers, and a 2:1 multiplexer selects whether registers X and Y pass data in the column or row direction. In other embodiments, X and Y are register banks, and different registers within register banks X and Y can be configured by adjacent processing elements 600 in the column and row directions.

Figure 19 illustrates a processing element 700 having a second multiplexer 702 connected to a row bus 704. The processing element 700 may be used as the processing element 116 in the computing memory array 100 of the present invention. Processing element 700 is similar to other devices described herein, and repeated description is omitted for clarity. Reference may also be made to the relevant descriptions of other specific embodiments, in which the same reference numerals represent the same components.

Figure 20 illustrates a SIMD controller 800 operable to drive column addresses and operation codes, and to load and store instructions in an associated array of memory cells 104. The SIMD controller 800 may be used as the SIMD controller 108 in the computing memory array 100 of the present invention. SIMD controller 800 is similar to other devices described herein, and repeated description is omitted for clarity. Reference may also be made to the relevant descriptions of other specific embodiments, in which the same reference numerals represent the same components.

SIMD controller 800 includes instruction memory 802, row selector 804, program counter 806, and decoder 808. Decoder 808 decodes instructions and may further include a decompressor to decompress instructions and/or data, which may be stored in compressed form to save memory.

The SIMD controller 800 is configured to obtain instructions from the array of memory cells 104 as needed. The retrieved instructions may be stored in instruction memory 802 . Instructions may instruct control circuits required for processing elements and their associated buses, as well as memory for selecting processing elements List of addresses required for the data.

During the execution of fetching instructions from memory other than memory cell array 104, a "Harvard architecture" may be implemented, in which instructions and (optionally) data available from memory cell array 104 are fetched in parallel. Conversely, since some calculations are data-heavy and others are instruction-heavy, it may be advantageous to load instructions from arrays of arrays of memory cells 104 .

Instruction decoder 808 may be located between instruction memory 802 and memory cell array 104 and processing element 116 .

SIMD controller 800 may address its instruction memory 802 through program counter 806, decode its contents read by decoder 808, and use this information to drive memory cell array 104 and processing element 116. Pipelining can be used to avoid having to wait for instructions to be fetched and decoded before executing. The instruction set may include "OP" instructions, which drive the opcodes and load registers of processing element 116; jump instructions (such as JMP and JSR) that manipulate program counter 806; address registers that may allow indirect and pointer addressing; Loop structures (such as fixed-length loops) and conditional jumps.

Figure 21 illustrates a plurality of SIMD controllers 800 interconnected by a controller bus 900. Each SIMD controller 800 operates to control the computing memory array 100, and the SIMD controllers 800 operate together to allow sharing of instruction memory.

Figure 22 illustrates a plurality of SIMD controllers 800, each of which is further operable to decode compressed coefficient data, and which can operate together to allow sharing of instruction memory and reuse of instruction memory as coefficient memory.

Neural networks usually need to store a large number of coefficients. For example, for the commonly known recognition algorithm AlexNet, it is about 250 million levels. It is expected that coefficients will be stored in a compressed form (e.g., the common special case of storing zero coefficients with a single "0" bit). Decompression may be performed by a computing memory array 100 through the processing element 116 and the array of memory cells 104, or using a SIMD controller. The 800 provides a separate component to execute, such as a decompression engine, to read and decompress a variable-length sequence of compressed numbers.

Coefficient compression can be used for more than just space savings. For example, if the coefficients are zero, the associated multiply-add step of the inner product can simply be skipped, saving both time and power. In addition to, or instead of, decompressing numbers, decompression can be configured to fallback encoding. For example, decompression can be configured to return the address of a subroutine that efficiently handles the special case of a given coefficient (such as zero, as explained above, or a pure bit shift), Along with a register value (such as the number of bits to shift) as an argument to this subroutine.

Decompression may share instruction memory with an instruction decoder, or may be provided in a separate memory. In a large vector scenario, where multiple SIMD controllers 800 run the same encoding with the same coefficients, one controller can perform decompression while the other acts as the master controller.

Figure 23 illustrates an exemplary layout of pixel data for an image and associated encoding and core output data for the first layer of a neural network in various computing memory arrays 100 in accordance with the present invention.

Figure 24 details an exemplary layout of color pixel data in a computational memory array 100 and data for a neural network convolutional layer in accordance with the present invention.

Figure 25 illustrates an example layout of data in a computing memory array 100 for pooling in a neural network.

The above image data is represented by tuples representing pixel coordinates. The example image size is 256x256 pixels.

When the vectors of data to be processed are larger than 100 in a single compute memory array, multiple SIMD controllers issue the same opcodes and controls. This can be done by copying the instructions in the memory of all relevant SIMD controllers and using the synchronization described above to keep them locked together. A given SIMD controller can be configured to act as the master controller, while other SIMD controllers are slaves to it. given controller. Controller busses can facilitate this mode of operation, and controller busses can be segmented so that multiple groups of controllers can operate independently in this manner. Controllers in a group can be programmed to switch master controls so that larger programs can fit into the command memory since it is shared rather than replicated.

In view of the above, it should be understood that computing memory arrays, SIMD controllers, processing elements, and their interconnection buses allow for large amounts of processing with flexible low-precision architectures, power-efficient communications, and local storage and decoding of instructions and coefficients. Processing of inner product and correlation neural network computations.

It should be understood that the features and concepts of the above various examples can also be combined into other examples, which also fall within the scope of the present invention. Furthermore, the drawings are not to scale and may have exaggerated size and shape for illustrative purposes.

100:Planning

104: Memory cell array

108:SIMD controller

112:Line

116: Processing components

120:Bit line

124: Column bus

128: Column connection

132: Column bus

140:Unit

142: OK

144:cell

146: column

Claims (25)

A computing memory device, comprising: a plurality of computing memory banks, each of the plurality of computing memory banks including a memory unit array and a plurality of processes connected to the memory unit array component; and a plurality of Single Instruction Multiple Data (SIMD) controllers, each SIMD controller of the plurality of SIMD controllers being included in at least one computing memory bank of the plurality of computing memory banks. Within; wherein each of the SIMD controllers will provide instructions to the at least one computing memory bank and control the execution of the instructions through the at least one computing memory bank; wherein among the plurality of processing elements each of the processing elements includes a static register and an arithmetic logic unit (ALU) to perform operations with the static register; and wherein each of the processing elements is configured to receive a static register from another processing element The communication state, the ALU is used to perform operations with the static register and the communication state. The device of claim 1 further includes a bus connecting the plurality of processing elements in one of the plurality of computing memory banks. The device of claim 2, wherein the bus is connected to a SIMD controller of the computing memory array, and wherein the bus is configured to carry operational code to the plurality of processing elements. For example, in the device of Item 2 of the patent application scope, the busbar is segmented. For example, the device of Item 1 of the patent application further includes a plurality of buses, each of which is operable to transmit information in one or two directions between the SIMD controller and the plurality of processing elements, wherein the At least one of the busbars is segmented, and one of the busbars At least the other one is unsegmented. The device of claim 1 further includes a bus connecting a processing element of one of the plurality of computing memory banks to the plurality of computing memory banks. Another array of computing memory processing elements. For example, in the device of Item 6 of the patent application scope, the busbar is segmented. The device of claim 1 further includes a plurality of buses, each of which is operable to transmit information in one or two directions between any of the computing memory banks, wherein among the buses At least one of the busbars is segmented, and at least another of the busbars is unsegmented. As in the device of claim 1, each of the SIMD controllers is included in a different computing memory bank among the plurality of computing memory banks. As claimed in the device of claim 1, wherein one SIMD controller of the plurality of SIMD controllers is included in at least two of the computing memory banks of the plurality of computing memory banks. For example, the device of Item 1 of the patent application further includes a bus connected to the plurality of SIMD controllers. The device of claim 1 further includes an input/output circuit connected to the plurality of SIMD controllers. For example, in the device of Item 1 of the patent application, the ALU includes multiple levels of multiplexers. For example, the device of Item 1 of the patent application further includes a bus connecting the plurality of processing elements in a computing memory bank and the SIMD controller, and the bus is used to connect the data from the SIM controller. Operand selections are passed to the ALU for each of the processing elements. For example, the device in item 1 of the patent application further includes a bus connected to a Compute the plurality of processing elements and the SIMD controller in the memory bank, and the bus is used to transfer a function to the ALU of each of the processing elements. For example, the device of Item 1 of the patent application further includes a communication register subordinate to the static register, and the communication register is used to provide communication status to another processing element. The device of claim 1 further includes at least one direct connection between each of the plurality of processing elements and at least one other processing element. For example, in the device of claim 17, the at least one direct connection is used to provide the communication status. The device of claim 17, wherein the at least one direct connection allows sharing of status information, the status information including shipping and signature information. A SIMD controller having an instruction memory configured to load from memory in one or more banks of computational memory, each of said banks including the memory and configured to utilize the memory The memory performs a plurality of processing elements operating in parallel; wherein the SIMD controller provides instructions to the at least one computing memory bank and controls execution of the instructions through the at least one computing memory bank. A plurality of SIMD controllers, each of said SIMD controllers including instruction memory configured to load from memory in one or more banks of computational memory, each of said banks of computational memory including the memory and a plurality of processing elements for performing parallel operations using the memory, wherein at least one of the SIMD controllers includes a decompressor operable to stream variable lengths from its instruction memory data and generates decompression information, the decompression information including one or both of the coefficients and a fixed-length representation of the instruction, the plurality of SIMD controller devices connected by a bus to share the decompression information; each of which The SIMD controller will provide instructions to the at least one computing memory bank and control execution of the instructions through the at least one computing memory bank. A computing memory device, comprising: a plurality of computing memory banks, each of the plurality of computing memory banks including a memory unit array and a plurality of processes connected to the memory unit array component; and a plurality of Single Instruction Multiple Data (SIMD) controllers, each SIMD controller of the plurality of SIMD controllers being included in at least one computing memory bank of the plurality of computing memory banks. Within; wherein each of the SIMD controllers will provide instructions to the at least one computing memory bank and control the execution of the instructions through the at least one computing memory bank; wherein among the plurality of processing elements Each of the processing elements includes a static register and an arithmetic logic unit (ALU) to perform operations with the static register; and a communication register subordinate to the static register, the communication register The register is used to provide communication status to another processing element. A computing memory device, comprising: a plurality of computing memory banks, each of the plurality of computing memory banks including a memory unit array and a plurality of processes connected to the memory unit array component; and a plurality of Single Instruction Multiple Data (SIMD) controllers, each SIMD controller of the plurality of SIMD controllers being included in at least one computing memory bank of the plurality of computing memory banks. wherein each of the SIMD controllers will provide instructions to the at least one computing memory bank and control the execution of the instructions by the at least one computing memory bank; and a bus that will all A processing element of a computing memory bank of the plurality of computing memory banks is connected to a processing element of another computing memory bank of the plurality of computing memory banks. pieces. A computing memory device, comprising: a plurality of computing memory banks, each of the plurality of computing memory banks including a memory unit array and a plurality of processes connected to the memory unit array component; and a plurality of Single Instruction Multiple Data (SIMD) controllers, each SIMD controller of the plurality of SIMD controllers being included in at least one computing memory bank of the plurality of computing memory banks. Within, each of the SIMD controllers will provide instructions to the at least one computing memory bank and control the execution of the instructions through the at least one computing memory bank; and a plurality of buses, each The busses are operable to pass information in one or two directions between any of the computing memory banks, wherein at least one of the busses is segmented and at least one other of the busses is not segmented. A computing memory device, comprising: a plurality of computing memory banks, each of the plurality of computing memory banks including a memory unit array and a plurality of processes connected to the memory unit array component; and a plurality of Single Instruction Multiple Data (SIMD) controllers, each SIMD controller of the plurality of SIMD controllers being included in at least one computing memory bank of the plurality of computing memory banks. Within; wherein each of the SIMD controllers will provide instructions to the at least one computing memory bank and control the execution of the instructions through the at least one computing memory bank; a bus connecting the Multiple SIMD controllers.