TW202122993A - Memory-based processors - Google Patents

Abstract

In some embodiments, an integrated circuit may include a substrate and a memory array disposed on the substrate, where the memory array includes a plurality of discrete memory banks. The integrated circuit may also include a processing array disposed on the substrate, where the processing array includes a plurality of processor subunits, each one of the plurality of processor subunits being associated with one or more discrete memory banks among the plurality of discrete memory banks. The integrated circuit may also include a controller configured to implement at least one security measure with respect to an operation of the integrated circuit and take one or more remedial actions if the at least one security measure is triggered.

Description

Memory processor

priority

This application claims the priority of the following: U.S. Provisional Application No. 62/886,328 filed on August 13, 2019; U.S. Provisional Application No. 62/907,659 filed on September 29, 2019; February 2020 U.S. Provisional Application No. 62/971,912 filed on July 7; and U.S. Provisional Application No. 62/983,174 filed on February 28, 2020. The aforementioned application is incorporated herein by reference in its entirety.

The present invention generally relates to devices for facilitating memory-intensive operations. In particular, the present invention relates to a hardware chip including processing elements coupled to a dedicated memory bank. The present invention also relates to equipment for improving the power efficiency and speed of memory chips. In particular, the present invention relates to a system and method for implementing partial renewal or even no renewal on a memory chip. The present invention also relates to memory chips with selectable sizes and dual port capabilities on the memory chips.

As both processor speed and memory size continue to increase, the significant limitation on effective processing speed is the von Neumann bottleneck. The Von Neumann bottleneck is caused by the throughput limitation caused by the conventional computer architecture. In particular, compared to the actual operation performed by the processor, the data transfer from the memory to the processor often encounters a bottleneck. Therefore, the number of clock cycles used to read and write the memory increases significantly with memory-intensive processing procedures. These clock cycles result in a lower effective processing speed. This is because reading and writing to the memory consumes clock cycles. Isochronous cycles cannot be used to perform operations on data. In addition, the computing bandwidth of the processor is generally greater than the bandwidth of the bus used by the processor to access the memory.

These bottlenecks are particularly obvious for the following: memory-intensive processing procedures, such as neural networks and other machine learning algorithms; database construction, index search and query; and include more reads and writes than data processing operations Other tasks of operation.

In addition, the rapid growth in the amount and granularity of available digital data has created opportunities for the development of machine learning algorithms and has empowered new technologies. However, this also brings thorny challenges to the field of database and parallel computing. For example, the emergence of social media and the Internet of Things (IoT) generates digital data at record rates. This new data can be used to generate algorithms for a variety of purposes, ranging from new advertising techniques to more precise control methods for industrial processing procedures. However, new data is difficult to store, process, analyze, and dispose of.

New data resources can be huge, sometimes on the order of petabytes to zetta bytes. In addition, the growth rate of these data resources may exceed the data processing capacity. Therefore, data scientists have committed to parallel data processing techniques to meet these challenges. In order to improve computing power and handle large amounts of data, scientists have tried to establish systems and methods that can perform parallel-intensive operations. However, these existing systems and methods cannot keep up with the data processing requirements. This is often because the technologies used are limited by the requirements of these technologies for additional resources for data management, separate data integration, and segmented data analysis.

In order to facilitate the manipulation of large data sets, engineers and scientists are now trying to improve the hardware used to analyze data. For example, new semiconductor processors or chips (such as those described herein) can be incorporated in a single substrate manufactured with a technology more suitable for memory operations rather than arithmetic operations The memory and processing functions are specifically designed for data-intensive tasks. It is possible to meet new data processing requirements by using integrated circuits specifically designed for data-intensive tasks. However, this new approach to data processing for large data sets needs to solve new problems in chip design and manufacturing. For example, if new chips designed for data-intensive tasks are manufactured using manufacturing technology and architecture for general-purpose chips, these new chips will have poor performance and/or unacceptable The yield rate. In addition, if the new chips are designed to operate using the current data processing method, the new chips will have poor performance because the current method can limit the ability of the chip to handle parallel operations.

The present invention describes solutions for alleviating or overcoming one or more of the problems set forth above and other problems in the prior art.

In some embodiments, an integrated circuit may include a substrate and a memory array disposed on the substrate, wherein the memory array includes a plurality of discrete memory groups. The integrated circuit may also include a processing array disposed on the substrate, wherein the processing array includes a plurality of processor subunits, each of the plurality of processor subunits and the plurality of discrete memories One or more discrete memory groups in the group are associated. The integrated circuit may also include a controller configured to implement at least one safety measure relative to an operation of the integrated circuit and take one or more remedies if the at least one safety measure is triggered action.

The disclosed embodiments may also include a method for protecting an integrated circuit from tampering, wherein the method includes using a controller associated with the integrated circuit to implement at least one security measure relative to the operation of the integrated circuit and at least one security measure One or more remedial actions are taken when the measure is triggered, and the integrated circuit includes: a substrate; a memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; and a processing array, It is arranged on a substrate, the processing array includes a plurality of processor sub-units, each of the plurality of processor sub-units is related to one or more of the plurality of discrete memory groups United.

The disclosed embodiment may include an integrated circuit including: a substrate; a memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; a processing array arranged on the substrate, the processing array Includes a plurality of processor sub-units, each of the plurality of processor sub-units is associated with one or more of the plurality of discrete memory groups; and The controller is configured to: implement at least one security measure relative to the operation of the integrated circuit; wherein the at least one security measure includes copying code in at least two different memory portions.

In some embodiments, a distributed processor memory chip is provided, which includes: a substrate; a memory array arranged on the substrate; a processing array arranged on the substrate; a first communication port; and a second communication port . The memory array may include a plurality of discrete memory groups. The processing array may include a plurality of processor subunits, each of the plurality of processor subunits being associated with one or more of the plurality of discrete memory groups. The first communication port can be configured to establish a communication connection between the distributed processor memory chip and an external entity other than another distributed processor memory chip. The second communication port can be configured to establish a communication connection between the distributed processor memory chip and the first additional distributed processor memory chip.

In some embodiments, a method for transferring data between a first distributed processor memory chip and a second distributed processor memory chip may include: using a first distributed processor memory chip and a second distributed processor memory chip. The controller associated with at least one of the distributed processor memory chips determines whether the first processor subunit among the plurality of processor subunits placed on the first distributed processor memory chip is ready to The data is transferred to the second processor sub-unit included in the second distributed processor memory chip; and after determining that the first processor sub-unit is ready to transfer the data to the second processor sub-unit, use the control The clock enabling signal controlled by the device starts the transmission of data from the first processor sub-unit to the second processor sub-unit.

In some embodiments, a memory unit may include: a memory array including a plurality of memory banks; at least one controller configured to control at least one of the read operations relative to the plurality of memory banks Aspect; at least one zero value detection logic unit, which is configured to detect multi-bit zero values stored in a specific address of a plurality of memory groups; and wherein the at least one controller and the at least one zero The value detection logic unit is configured to return the zero value indicator to one or more circuits outside the memory unit in response to the zero value detection performed by the at least one zero value detection logic.

Some embodiments may include a method for detecting a zero value in a specific address of a plurality of discrete memory groups, which includes: receiving and reading bits stored in the plurality of discrete memory groups from a circuit outside the memory unit Request for data in the address; in response to the received request, the controller activates the zero value detection logic unit to detect the zero value in the received address; and responds to the zero value performed by the zero value detection logic unit The value is detected and the zero value indicator is transmitted to the circuit by the controller.

Some embodiments may include a non-transitory computer-readable medium that stores a set of instructions that can be executed by a controller of a memory unit to enable the memory unit to detect a zero value in a specific address of a plurality of discrete memory groups, the The method includes: receiving a request to read data stored in the addresses of a plurality of discrete memory groups from a circuit outside the memory unit; in response to the received request, the controller activates a zero-value detection logic unit to detect Receiving the zero value in the address; and in response to the zero value detection performed by the zero value detection logic unit, the controller transmits the zero value indicator to the circuit.

In some embodiments, a memory unit may include: one or more memory banks; a bank controller; and an address generator; wherein the address generator is configured to assign the associated memory bank to be accessed The current address in the current row is provided to the group controller to determine the predicted address of the next row to be accessed in the associated memory group, and the read operation relative to the current row associated with the current address is completed Previously, the predicted address was provided to the group controller.

In some embodiments, a memory unit may include: one or more memory groups, wherein each of the one or more memory groups includes a plurality of rows; the first row of controllers is configured to Control the first subset of the plurality of rows; the second row controller, which is configured to control the second subset of the plurality of rows; a single data input terminal, which is used to receive the data to be stored in the plurality of rows; And a single data output terminal, which is used to provide data retrieved from multiple rows.

In some embodiments, a distributed processor memory chip may include: a substrate; a memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; a processing array, arranged on the substrate, The processing array includes a plurality of processor subunits, and the processor subunits Each of the units is associated with a corresponding dedicated memory group in the plurality of discrete memory groups; the first plurality of buses each connects one of the plurality of processor subunits to its corresponding And a second plurality of buses, each of which connects one of the plurality of processor subunits to the other of the plurality of processor subunits. At least one of the memory banks may include at least one DRAM memory pad disposed on the substrate. At least one of the processor units may include one or more logic components associated with at least one memory pad. At least one memory pad and one or more logic components can be configured to act as cache memory for one or more of the plurality of processing subunits.

In some embodiments, a method for executing at least one instruction in a distributed processor memory chip may include: retrieving one or more data values from a memory array of the distributed processor memory chip; A data value is stored in a register formed in the memory pad of the memory chip of a distributed processor; and one or more data stored in the register is accessed according to at least one instruction executed by the processor element Value; wherein the memory array includes a plurality of discrete memory groups arranged on a substrate; wherein the processor element is a processor subunit included in a plurality of processor subunits in the processing array arranged on the substrate, Each of the processor sub-units is associated with a corresponding dedicated memory group in a plurality of discrete memory groups; and the register is provided by a memory pad arranged on the substrate.

Some embodiments may include a device that includes: a substrate; a processing unit disposed on the substrate; and a memory unit disposed on the substrate, wherein the memory unit is configured to store data to be accessed by the processing unit Data, and where the processing unit includes a memory pad configured to act as a cache for the processing unit.

The processing system is expected to process the increased amount of information at an extremely high rate. For example, the fifth generation (5G) mobile Internet is expected to receive a large amount of information streams and process these information streams at an increased rate.

The processing system may include one or more buffers and a processor. Applied by the processor The processing operation may have a certain latency and this may require a lot of buffers. A large number of buffers can be costly and/or area consuming.

Sending a large amount of information from the buffer to the processor may require a high-bandwidth connector and/or a high-bandwidth bus between the buffer and the processor, which can also increase the cost and area of the processing system.

There is an increasing need to provide efficient processing systems.

The processing system is expected to process the increased amount of information at an extremely high rate. For example, the fifth generation (5G) mobile Internet is expected to receive a large amount of information streams and process these information streams at an increased rate.

The processing system may include one or more buffers and processors. The processing operations applied by the processor may have a certain latency and this may require a large amount of buffers. A large number of buffers can be costly and/or area consuming.

Sending a large amount of information from the buffer to the processor may require a high-bandwidth connector and/or a high-bandwidth bus between the buffer and the processor, which can also increase the cost and area of the processing system.

There is an increasing need to provide efficient processing systems.

A decomposed server includes multiple subsystems, and each subsystem has a unique role. For example, a decomposed server may include one or more switching subsystems, one or more computing subsystems, and one or more storage subsystems.

One or more computing subsystems and one or more storage subsystems are coupled to each other through one or more switching subsystems.

The arithmetic subsystem may include multiple arithmetic units.

The switching subsystem may include multiple switching units.

The storage subsystem may include multiple storage units.

The bottleneck of this decomposed server is the bandwidth required to transmit information between subsystems.

When the execution requires all (or at least most) computing units in different computing subsystems This is especially true when sharing information units among distributed calculations (such as graphics processing units).

Suppose there are N arithmetic units participating in the sharing, N is a very large integer (for example, at least 1024), and each of the N arithmetic units must send the information unit to all other arithmetic units (and receive information from all other arithmetic units) unit). Under these assumptions, approximately N×N transmission processing procedures of the information unit need to be executed. Mass transfer processing procedures are time-consuming and energy-consuming, and will significantly limit the throughput of the disaggregated server.

There is an increasing need to provide efficient decomposed servers and efficient ways to perform distributed processing.

The database includes many entries, and these entries include multiple fields. Database processing usually includes executing one or more queries that include one or more filter parameters (for example, identifying one or more related fields and one or more related field values) and also includes a Or multiple operating parameters, the one or more operating parameters can determine the type of operation to be performed, the variable or constant to be used in the application operation, and the like.

For example, a database query can request a statistical operation (operation parameter) to be performed on all records in the database, and a certain field has a value within a predefined range (filter parameter). For another example, the database query may request deletion of (operation parameter) records with a certain field less than the threshold value (screening parameter).

Large databases are usually stored in storage devices. In order to respond to queries, the database is sent to the memory unit, usually one database section followed by another database section.

The entries of the database section are sent from the memory unit to the processor that does not belong to the same integrated circuit as the memory unit. These entries are then processed by the processor.

For each database section of the database stored in the memory unit, the processing includes the following steps: (i) select the record of the database section; (ii) send the record from the memory unit to the processor; (iii) ) Filter records through the processor to determine whether the records are relevant; and (iv) execute on related records One or more additional operations (summing, applying any other mathematical operations and/or statistical operations).

The screening process ends after all records are sent to the processor and the processor determines which records are relevant.

Under the condition that the relevant entries of the database section are not stored in the processor, these relevant records need to be sent to the processor for further processing after the screening phase (application of operations after processing).

When multiple processing operations follow a single screening, the result of each operation can be sent to the memory unit and then sent to the processor again.

This processing procedure is bandwidth-consuming and time-consuming.

There is an increasing need to provide efficient ways to perform database processing.

Word embedding is a collective term for a collection of language modeling and feature learning techniques in natural language processing (NLP), in which words or phrases from a vocabulary are mapped to a vector of elements. Conceptually, it involves mathematical embedding (www.wikipedia.org) from a space with many dimensions per word to a continuous vector space with much lower dimensions.

The methods for generating this mapping include neural networks, dimensionality reduction of word co-occurrence matrices, probability models, interpretable knowledge base methods, and explicit representations based on the context in which words appear.

Word and phrase embeddings have been shown to improve the effectiveness of NLP tasks such as grammar analysis and sentiment analysis when used as basic input representations.

Sentences can be segmented into words or phrases, and each segment can be represented by a vector. Sentences can be represented by a matrix, which includes all vectors representing words or phrases of the sentence.

The vocabulary that maps words to vectors can be stored in a memory unit (such as dynamic random access memory (DRAM)), which can be accessed using words or phrases (or indexes representing words).

These accesses can be random access, which reduces the throughput of DRAM. In addition, these Access can saturate the DRAM, especially when a large amount of access is fed into the DRAM.

In particular, the words included in the sentence are usually quite random. Even when using DRAM bursts, accessing the memory mapped DRAM memory will usually result in lower performance of random access. This is because usually during bursts, DRAM memory bank entries (different when accessed at the same time) Only one of a small part of the multiple entries in the memory group will store entries related to a certain sentence.

Therefore, the throughput of DRAM memory is low and non-continuous.

Retrieve every word or phrase of a sentence from the DRAM memory under the control of the host computer. The host computer is outside the integrated circuit of the DRAM memory and must control the representation of each word or area based on the knowledge of the position of the word Each acquisition of each vector of a segment is a time-consuming and resource-intensive task.

Data centers and other computerized systems are expected to process and exchange increased amounts of information at extremely high rates.

The exchange of increased amounts of data can be a bottleneck for data centers and other computerized systems, and can allow such data centers and other computerized systems to use only part of their capabilities.

FIG. 96A illustrates an example of the prior art database 12010 and the prior art server motherboard 12011. The database may include multiple servers, and each server includes multiple server motherboards (also denoted as "CPU+Memory+Network"). Each server motherboard 12011 includes a CPU 12012 (such as but not limited to Intel’s XEON), which receives traffic and is connected to a memory unit 12013 (represented as RAM) and multiple database accelerators (DB accelerators) 12014 .

The DB accelerator is optional, and the DB acceleration operation can be executed by the CPU 12012.

All traffic flows through the CPU, and the CPU can be coupled to the DB accelerator via a link with a relatively limited bandwidth (such as PCIe).

A large amount of resources are dedicated to the delivery of information units among multiple server motherboards.

There is an increasing need to provide efficient data centers and other computerized systems.

The size of artificial intelligence (AI) applications such as neural networks has increased significantly. In response to The increased size of the neural network each serves as multiple servers of the AI acceleration server (including the server motherboard) for performing neural network processing tasks, such as but not limited to training. An example of a system including multiple AI acceleration servers arranged in different racks is shown in FIG. 1.

In a typical training session, a large number of images are processed at the same time to provide a large number of values, such as loss. A large number of values are transmitted between different AI acceleration servers and result in an exceptional amount of traffic. For example, some neural network layers can be calculated across multiple GPUs located in different AI acceleration servers, and may need to be aggregated on a network that consumes bandwidth.

The transmission of exceptional traffic requires ultra-high bandwidth, which may not be feasible or cost-effective.

Figure 97A illustrates a system 12050 including subsystems. Each subsystem includes: a switch 12051 for connecting to an AI acceleration server 12052 with a server motherboard 12055, which includes RAM memory (RAM 12056) , Central Processing Unit (CPU) 12054, Network Interface Card (NIC) 12053, and CPU 12054 is connected (via PCIe bus) to multiple AI accelerators 12057 (such as graphics processing unit, AI chip (AI ASIC), FPGA and its Similar). NICs are coupled to each other (for example, by one or more switches) through a network (using, for example, Ethernet, UDP links, and the like), and these NICs may be able to transport more than required by the system. High bandwidth.

There is an increasing need to provide efficient AI computing systems.

In accordance with other disclosed embodiments, the non-transitory computer-readable storage medium can store program instructions that are executed by at least one processing device and perform any of the methods described herein.

The general description above and the detailed description below are only illustrative and explanatory, and are not limited by the scope of the patent application.

38: line

100: CPU

110: processing unit

120a: processor subunit

120b: processor subunit

130: Cache memory

140a: shared memory

140b: shared memory

200: GPU

210: Processing Unit

220a: processor subunit

220b: processor subunit

220c: processor subunit

220d: processor subunit

220e: processor subunit

220f: processor subunit

220g: processor subunit

220h: processor subunit

220i: processor subunit

220j: processor subunit

220k: processor subunit

220l: processor subunit

220m: processor subunit

220n: processor subunit

220o: processor subunit

220p: processor subunit

230a: Cache memory

230b: Cache memory

230c: Cache memory

230d: Cache memory

250a: shared memory

250b: shared memory

250c: shared memory

250d: shared memory

300: hardware chip

300': hardware chip

310a: Processing group

310b: Processing group

310c: Processing group

310d: Processing group

320a: logic and control subunit

320b: logic and control subunit

320c: logic and control subunit

320d: logic and control subunit

320e: logic and control subunit

320f: logic and control subunit

320g: logic and control subunit

320h: logic and control subunit

330a: Dedicated memory instance

330b: Dedicated memory example

330c: Dedicated memory instance

330d: Dedicated memory instance

330e: Dedicated memory instance

330f: Dedicated memory example

330g: Dedicated memory example

330h: Dedicated memory instance

340a: control

340b: control

340c: control

340d: control part

350: host

350a: processor subunit

350b: processor subunit

350c: processor subunit

350d: processor subunit

360a: bus

360b: bus

360c: bus

360d: bus

360e: bus

360f: bus

400: Processing program

410: Processing Group

420: Dedicated memory instance

430: processor subunit

440: processing element

450: address generator

460: control

500: Illustrative processing program for executing special commands

510: Processing Group

520: Dedicated memory instance

530: processing components

600: Processing group

610: processor subunit

620: Memory/Memory Components

630: Bus

640: processing element

650: accelerator/MUX

660: Input multiplexer (MUX)/DEMUX

670: output DEMUX

710: Substrate

720a: memory bank

720b: Memory bank

720c: memory bank

720d: memory bank

720e: memory bank

720f: memory bank

720g: memory bank

720h: memory bank

730a: processor subunit

730b: processor subunit

730c: processor subunit

730d: processor subunit

730e: processor subunit

730f: processor subunit

730g: processor subunit

730h: processor subunit

740a: bus

740b: bus

740c: bus

740d: bus

740e: bus

740f: bus

740g: busbar

740h: bus

750a: busbar

750b: bus

750c: bus

750d: bus

750e: bus

750f: bus

750g: busbar

750h: busbar

750i: bus

750j: bus

760: Architecture

770a: memory chip

770b: Memory chip

770c: memory chip

770d: memory chip

800: Method for compiling a series of instructions

810: step

820: step

830: step

840: step

850: step

900: Group

910: column decoder

920: Global Sensing Amplifier

930-1: Pad

930-2: Pad

940-1: Pad

940-2: Pad

950: word line

960: bit line

1000: pad

1010-1: Regional amplifier

1010-2: Area amplifier

1010-x: Area amplifier

1020-1: word line driver

1020-2: Word line driver

1020-x: word line driver

1030-1: Cell

1030-2: Cell

1030-3: Cell

1040: word line

1050: bit line

1100: group/subgroup

1110: group column decoder

1120: Group row decoder

1121a: bit line

1121b: bit line

1130a: Subgroup Controller

1130b: Subgroup controller

1130c: Subgroup Controller

1131a: bus

1131b: bus

1131c: bus

1140a: solver

1140b: solver

1140c: solver

1150a: logic

1150b: logic

1150c: logic

1160a: decoder

1160b: decoder

1160c: decoder

1160d: decoder

1160e: decoder

1160f: decoder

1160g: decoder

1160h: decoder

1160i: decoder

1170a: subgroup

1170b: subgroup

1170c: subgroup

1180a: decoder

1180b: decoder

1180c: decoder

1181a: bit line

1181b: bit line

1190a-1: pad

1190a-2: pad

1190a-x: pad

1190b-1: Pad

1190b-2: pad

1190b-x: pad

1190c-1: pad

1190c-2: pad

1190c-x: pad

1200: memory subgroup

1210: Memory Controller

1220a: fuse and comparator

1220b: fuse and comparator

1230a: column decoder

1230b: column decoder

1240a: pad

1240b: pad

1250a: Line decoder

1250b: Line decoder

1251: bit line

1253: bit line

1260a-1: cell

1260a-2: cell

1260a-x: cell

1260b-1: cell

1260b-2: cell

1260b-x: cell

1300: memory chip

1301: substrate

1302: Address Manager

1304: memory array

1304(a,a): memory group/memory block/memory instance

1304(z,z): memory group/memory instance

1306: memory logic

1308: business logic

1310: Redundant business logic/redundant logic block/redundant business block

1312: Do not start the switch

1314: Start switch

1400: Redundant logical block collection

1402: address bus

1404: data bus

1500: logical block

1504: get circuit

1506: decoder

1508: register

1510: arithmetic unit

1512: Copy arithmetic unit

1514: switch circuit

1516: switch circuit

1518: write back circuit

1602: logical block

1602(a): logical block

1602(b): logical block

1602(c): logical block

1604: Fuse identification

1604(a): logical block

1604(b): logical block

1604(c): logical block

1614: address bus

1616: command line

1618: data line

1702: unit

1702(a): unit

1702(b): The failed unit

1702(c): unit

1712: unit

1712(a): unit

1712(b): unit

1712(c): unit

1722: switch circuit

1722(a): Switching circuit

1722(b): Switching circuit

1722(c): Switch circuit

1728: switch circuit

1728(a): Switch circuit

1728(c): Switch circuit

1730: sample circuit

1730(a): Sample circuit

1730(b): Sample circuit

1730(c): Sample circuit

1804: I/O block

1806: unit

1808: switch box

1810: connection box

1902(a): unit

1902(b): unit

1902(c): unit

1902(d): unit

1902(e): unit

1902(f): unit

1904(a): configuration switch

1904(b): configuration switch

1904(c): configuration switch

1904(d): configuration switch

1904(e): configuration switch

1904(f): configuration switch

1904(g): configuration switch

1904(h): configuration switch

2000: Redundant block enablement process

2002: steps

2004: steps

2006: steps

2008: steps

2010: steps

2100: Address assignment process

2102: step

2104: step

2106: step

2108: step

2110: steps

2112: steps

2114: step

2116: step

2118: step

2120: step

2122: step

2200: processing device

2202: The first memory block

2204: second memory block

2210: Memory Controller

2212: Configuration Manager

2214: logical block/unit

2216: accelerator/unit

2216(a): accelerator

2216(n): accelerator

2218: line

2220: line

2230: host

2300: processing device

2302: MAC unit

2304: Configuration Manager

2306: Memory Controller

2308(a): memory block

2308(b): memory block

2308(c): memory block

2308(d): memory block

2500: Memory configuration processing program

2502: step

2504: step

2506: step

2508: step

2510: steps

2512: step

2514: step

2600: Memory read process

2602: step

2604: step

2606: step

2608: step

2614: step

2616: step

2618: step

2620: steps

2700: Execute the handler

2702: step

2704: step

2706: step

2708: step

2710: Step

2712: step

2714: step

2176: step

2718: step

2720: step

2800: memory chip

2801a: memory bank

2803: New controller

2805: Controller

2900: Instance and new controller

2900': New controller for instance

2901: Timer

2903: column counter

2905: effective bit

2907: adder

2909: Data Storage

2911: new gate

3000: Used for partial renewal processing procedures in the memory chip

3010: steps

3020: steps

3030: steps

3100: Process used to determine the renewal of memory chips

3110: step

3120: step

3130: step

3140: step

3200: Process used to determine the renewal of memory chips

3210: steps

3220: steps

3230: steps

3240: step

3250: steps

3300: Instance and new controller

3301: timer

3303: column counter

3305: adder

3307: Data Storage

3400: Process used to determine the renewal of memory chips

3410: steps

3420: step

3430: step

3440: step

3501: Wafer

3503: Die

3504: second zone/memory chip/group

3505: zone/memory chip

3506: memory chip

3506A: Memory Die/Memory Chip

3506B: memory die/memory chip

3506C: Memory chip

3506D: Memory chip

3507: substrate

3511A: Memory Bank

3511B: Memory Bank

3511C: Memory Bank

3511D: Memory Bank

3511E: memory bank

3511F: Memory Bank

3511G: memory bank

3511H: memory bank

3512: busbar or connector

3515A: processor subunit

3515B: processor subunit

3515C: processor subunit

3515D: processor subunit

3515E: processor subunit

3515F: processor subunit

3515G: processor subunit

3515H: Processor subunit/connector

3515I: processor subunit

3515J: processor subunit

3515K: processor subunit

3515L: processor subunit

3515M: processor subunit

3515N: processor subunit

3515O: processor subunit

3515P: processor subunit

3516A: Connector

3516B: Connector

3516C: Connector

3516D: Connector

3517: zone/memory chip

3521: Input and output (IO) controller/IO control module

3521A: IO controller

3521B: IO controller

3521: IO controller/IO control module

3522: IO controller/IO control module

3523: IO control module

3524: IO controller

3530: input and output bus/bus line

3530A: Input and output bus

3530B: input and output bus

3531A: branch

3531B: branch

3532: memory chip

3533: line

3540: Die

3542A: Input/Output Controller

3542B: Input/Output Controller

3546: Die

3554: Fuse/fuse element

3554A: Fuse

3554B: Fuse

3555: Fuse/fuse element

3556: Fuse/fuse element

3557: fuse/fuse element

3601: domain

3602: domain

3603: domain

3611: bus line

3612: bus line

3613: bus line

3711: glue logic / logic circuit / glue logic unit

3713: Group

3715: Group

3801: Horizontal cutting

3802: wire/cut

3803: Vertical cutting

3804: wire/cut

3806: wire/cut

3811A: District

3811B: District

3811C: District

3820: bus line

3822: District

3824: bus line

3901: Group of memory units

3905: connector

4000: An example process for building a memory chip from a die group

4011: step

4015: steps

4017: step

4100: An example process for manufacturing a memory chip containing multiple dies

4101: handler

4102: handler

4111: Step

4113: Step

4115: step

4117: Step

4119: step

4131: step

4133: Step

4140: Step

4200: Example circuit system

4201: Memory Array

4203: column decoder

4205a: Row multiplexer (``mux'')

4205b: Row multiplexer (``mux'')

4300: Example circuit system

4301: Memory Array

4303: column decoder

4305: Row Multiplexer

4400: Example circuit system

4401: memory array

4403: column decoder

4405: Row decoder (or multiplexer)

4500: method

4600: Example circuit system

4601a: column decoder

4601b: column decoder

4603a: Row multiplexer

4603b: Row multiplexer

4607: Column Control

4609a: Memory pad

4609b: Memory pad

4611a: word line

4611b: word line

4613a: switching element

4613b: switching element

4615a: bit line

4615b: bit line

4700: Process used to provide dual-port access on a single-port memory array or pad

4710: step

4720: step

4730: step

4740: step

4750: A process used to provide dual-port access on a single-port memory array or pad

4760: Step

4770: step

4780: step

4790: step

4800: Example circuit system

4801a: column decoder

4801b: column decoder

4803a: Row multiplexer

4803b: Row multiplexer

4900: memory pad

5000: Example integrated circuit

5001: memory cell

5008: memory cell

5011: Memory read path

5018: Memory read path

5020: output port

5021: bit

5028: bit

5030: Reduced unit

5040: Reading circuit system

5050: Array of memory cells

5100: memory bank

5101: Memory Bank

5102: memory unit

5111: Array

5112: column decoder

5113: Row Multiplexer

5114: main I/O bus

5115: output bus

5116: In-Memory Processing (PIM) logic

5117: bus

5118: PIM address bus/address line bus

5119: bus

5130: instance method

5132: step

5134: step

5140: Memory chip

5140(1): Part of the memory bank

5140(2): Part of the memory bank

5140(3): Part of the memory bank

5140(4): part of the memory bank

5140(5): Part of the memory bank

5140(6): Part of the memory bank

5141: Memory pad and associated logic

5142: Memory pad and associated logic

5143: Memory pad and associated logic

5144: Memory pad and associated logic

5145: Memory pad and associated logic

5146: Memory pad and associated logic

5147: bus

5150(10): Memory pad

5150(2): Memory pad

5150(3): memory pad

5150(4): memory pad

5150(5): Memory pad

5150(6): Memory pad

5151(1): memory pad

5151(2): memory pad

5151(3): memory pad

5151(4): memory pad

5151(5): Memory pad

5151(6): Memory pad

5152(1): Memory Pad/Global Word Line

5152(2): Memory Pad/Global Word Line

5152(3): Memory Pad/Global Word Line

5152(4): Memory Pad/Global Word Line

5152(5): Memory Pad/Global Word Line

5152(6): Memory Pad/Global Word Line

5152(8): Global word line

5153(1): Delay or isolation circuit

5153(3): Delay or isolation circuit

5154(1): Delay or isolation circuit

5154(3): Delay or isolation circuit

5155(1): Flip-flop

5155(3): Flip-flop

5156(1): Flip-flop

5156(3): Flip-flop

5157(1): switch

5157(3): switch

5157(8): switch

5158(1): switch

5158(3): switch

5158(8): switch

5159(1): inverter gate or buffer

5159(3): inverter gate or buffer

5159'(1): inverter gate or buffer

5159'(3): inverter gate or buffer

5160(1): column control unit

5160(2): unit

5160(3): unit

5170(1): column part enable signal

5170(2): Column part enable signal

5180: Global word line

5190: Method for operating memory unit

5192: Step

5194: step

5200: Tester

5201: switch

5202: Segment/complete wafer

5210: chip (or wafer of chip)/integrated circuit/memory

5211: chip interface

5212: Memory Bank

5213: bus

5214: I/O Controller

5215: Logic Unit/Logic

5216: Fuse interface

5217: bus

5218: Test Unit (TU)

5219: Test Pattern Generator

5221: Write test sequence/first step

5222: Read back test results

5223: Write expected result sequence / second step

5224: Read the fault address to repair/the third step

5225: Programmable Fuse/Fourth Step

5231: test result

5232: test code

5300: Method for testing memory bank

5302: Step

5310: step

5320: steps

5350: Method for testing the memory bank of an integrated circuit

5352: step

5355: step

5358: step

7000: Method for decentralized processing

7001: Method for decentralized processing

7010: steps

7020: steps

7030: steps

7040: steps

7050: steps

7011: memory/processing unit

7012: memory/processing unit

7013: memory/processing unit

7014: Integrated Circuit

7015: Integrated Circuit

7101: Decomposition System

7102: Decomposition System

7103: Decomposition System

7104: Decomposition System

7110: Processing/Memory Subsystem

7120: computing subsystem

7120(1): arithmetic unit PU(1)

7120(n): arithmetic unit PU(n)

7120(N): arithmetic unit PU(N)

7121(1): Partial model update

7121(n): Partial model update

7121(N): Partial model update

7122: updated model

7130: Storage subsystem

7140: Exchange subsystem

7150: accelerator subsystem

7200: Integrated Circuit/Integrated Chip

7210: memory array

7210_1: Discrete memory bank

7210_2: Discrete memory bank

7210_j: dedicated memory bank

7210_J1: Discrete memory bank

7210_Jn: Discrete memory bank

7220: processor subunit

7220_1: processor subunit

7220_2: processor subunit

7220_k: processor subunit

7220_k+1: processor subunit

7220_K: processor subunit

7230: Communication port

7240: Controller

7241: Cyber Attack Detector

7242: Response module

7243: Access Control Rules

7244: program/model operation pattern

7245: tamper detector

7246: Response module

7247: Curve

7250: bus

7260: the first bus

7261: second bus

7270: host computer

7280: host memory

7281: data can be changed

7282: Unchangeable data

7283: command

7450: method

7452: step

7454: step

7500: The first distributed processor memory chip

7500': Second distributed processor memory chip

7500": The third distributed processor memory chip

7500''': Distributed processor memory chip

7500A: Distributed processor memory chip

7500B: Distributed processor memory chip

7500C: Distributed processor memory chip

7500D: Distributed processor memory chip

7500E: Distributed processor memory chip

7500F: Distributed processor memory chip

7500G: Distributed processor memory chip

7500H: Distributed processor memory chip

7500I: Distributed processor memory chip

7500A': Distributed processor memory chip

7500B': Distributed processor memory chip

7500C': Distributed processor memory chip

7510: memory array

7510_1: dedicated memory bank

7510_2: dedicated memory bank

7510_3: dedicated memory bank

7510_4: dedicated memory bank

7510_5: dedicated memory bank

7510_6: dedicated memory bank

7510': memory array

7510": Memory array

7520: Processing array

7520_1: Distributed processor subunit

7520_2: Distributed processor subunit

7520_3: Distributed processor subunit

7520_4: Distributed processor subunit

7520_5: Distributed processor subunit

7520_6: Distributed processor subunit

7520_K: Distributed processor subunit

7520': Processing array

7520": Processing array

7530: the first communication port

7530': Communication port

7530": Communication port

7531: second communication port

7531': Communication port

7531": Communication port

7532: third communication port

7532': Communication port

7532": Communication port

7533: bus

7533': busbar

7534: bus

7534': busbar

7535: bus

7540: Controller

7540': Controller

7540": Controller

7547: Controller and interface module

7548_1: Communication interface

7548_N: Communication interface

7570: Host communication port

7570': Port

7572: chip port/communication port

7580: The first bus

7580': second bus

7600: Distributed processor memory chip

S7710: steps

S7720: steps

S7730: steps

7800: System/memory unit used to detect zero values stored in one or more specific addresses in a plurality of memory groups

7810: memory chip

7811A: Memory Bank

7811B: Memory Bank

7812: IO bus

7820: host

7830: Zero detection logic unit

7830A: Zero indicator line

7830B: Zero indicator line

7831: Internal zero indicator line

7840: bus

7840A: Bus

7840B: bus

7841: bus

7841A: Bus

7911: Memory Bank

7912A: Memory pad

7912B: memory pad

7913A: Memory Pad Controller

7913B: Memory Pad Controller

7914A: Zero detection logic unit

7914B: Zero detection logic unit

7915A: Area sensing amplifier

7915B: Area sensing amplifier

7916: Global Sensing Amplifier

7931A: Zero indicator line

7931B: Zero indicator line

8000: An exemplary method for detecting zero values stored in specific addresses of multiple memory banks

8010: steps

8020: steps

8030: steps

8100: system/memory unit

8180: memory bank

8180A: Memory bank

8180B: memory bank

8181: memory subgroup

8183A: The first sub-group controller

8183B: The second sub-group controller

8191: Group Controller

8192: Current and predicted address generator

8192A: Counter

8192B: current address generator

8192C: Predictive address generator

8193: Cache memory

8280: Dual control memory bank

8290: Data input (DIN)

8291: Group Controller/Column Address (ROW)

8292: Row Address (COLUMN)

8293: The first command input (COMMAND_1)

8294: Second command input (COMMAND_2)

8295: Data output (Dout)

8400: traditional computer architecture

8402: CPU

8406: External memory

8500a: Distributed processor memory chip

8500b: Distributed processor memory chip

8500c: device

8502: substrate

8504: Register file

8510a: Processing group

8510b: Processing group

8510c: Processing group

8520: memory array

8520a: dedicated memory bank

8520b: dedicated memory bank

8520c: dedicated memory bank

8522a: Memory pad

8522b: Memory pad

8522c: memory pad

8524a: Memory pad

8524b: memory pad

8524c: memory pad

8526a: Memory pad

8526b: Memory pad

8526c: memory pad

8530: Processing array

8530a: processor subunit/accelerator

8530b: processor subunit

8530c: processor subunit

8532a: Memory pad/register file

8532b: memory pad

8532c: memory pad

8534a: logic component

8534b: logic component

8534c: logical component

8540a: bus

8540b: bus

8540c: bus

8550a: bus

8550b: bus

8560: substrate

8570: First memory group

8572: second memory bank

8580: processing unit

8582: Register file

8584: processor

8600: Flow chart

8602: step

8604: step

8606: step

9010: memory/processing unit

9011: memory/processing unit

9012: memory/processing unit

9013: memory/processing unit

9014: memory/processing unit

9015: memory/processing unit

9018: host

9019: memory/processing unit

9020: Controller/Logic

9021: internal bus

9022: Bus

9030: logic

9033: Buffer

9039: Status line

9040: memory bank

9050: vector processor

9070: Glossary

9071: Retrieve key

9072: words/phrases

9073: Vector

9100: database query

9101: The final result of the screening operation

9102: partial response

9103: complete response

9210: storage device

9211: Interface

9220: Memory and filtering system

9220(k): database section

9222: Memory cell entry

9224: Screening unit

9224': relevance flag

9225: Processing Unit

9227: memory/processing unit

9228: Memory/Processing System

9229: Arbiter/Memory and Processing System

9240: CPU

9300: Method for accelerating database analysis

9301: Method for accelerating database analysis

9302: Method for accelerating database analysis

9303: Method for accelerating database analysis

9304: Methods of accelerating database analysis

9305: Methods of accelerating database analysis

9310: steps

9314: step

9315: steps

9320: steps

9324: step

9325: Step

9330: steps

9331: step

9332: step

9333: Step

9334: step

9335: Step

9340: steps

9341: Step

9342: steps

9344: step

9351: step

9352: step

9390: steps

9391: step

9400: Method for embedding

9401: Method for embedding

9402: Method for embedding

9410: steps

9420: steps

9430: steps

9431: step

9440: Step

9442: step

10800: Method for distributed processing of at least one information stream

10810: step

10820: step

10830: step

10840: step

10850: step

10900: System

10901: System

10902: system

10903: System

10908: DMA controller

10909: preprocessor

10910: memory/processing unit

10910(1): memory/processing unit

10910(N): Memory/processing unit

10911(1,1): Processing resources

10911(1,2): Processing resources

10911(1,K): Processing resources

10912(1,1): Memory resource

10912(1,2): Memory resources

10912(1,J-1): Memory resource

10912(1,J): Memory resource

10915: link

10920: processor

10931: link

10931(1): link

10931(N): link

10932: link

10932(1): link

10932(N): link

10933: link

11011: Hybrid integrated circuit

11011': Hybrid integrated circuit

11012: Hybrid integrated circuit

11012': Conductor

11013: Hybrid integrated circuit

11013': Hybrid integrated circuit

11014: Hybrid integrated circuit/bus

11014': Hybrid integrated circuit

11015: Hybrid integrated circuit/bus

11015': Hybrid integrated circuit

11016: Hybrid integrated circuit/bus

11016': Hybrid integrated circuit

11017: Package substrate

11017': Hybrid integrated circuit

11018: Intermediary layer

11018': Hybrid integrated circuit

11019: basic grain

11019': Memory processing unit/hybrid integrated circuit

11020: Micro bump

11021: DRAM wafer/DRAM die

11021': memory/processing unit

11022: Second memory controller

11022': Conductor

11023: WOW middle layer

11030: HBM DRAM stack/wafer

11031: The first memory controller

11032: HDM DRAM memory chip/HDM DRAM die/second memory controller

11039: TSV

11040: Wafer/HBM memory chip stacking

11051: processor

11052: L2 cache

11053: memory unit

11061: WOW joint

11062: second chip

11100: Method for memory-intensive processing

11110: steps

11120: step

11130: steps

11140: steps

11150: computer system

11200: Method for database acceleration

11210: steps

11220: steps

11230: steps

11240: steps

11250: steps

11260: steps

11270: steps

11271: steps

11272: steps

11300: A method for operating the database to accelerate the group of integrated circuits

11310: steps

11311: Step

11312: steps

11314: steps

11316: steps

11320: steps

11350: Method for database acceleration

11352: step

11354: step

11355: steps

11356: step

11358: step

11359: step

11510: computing system

11511: Manager

11512: computing node

11513: Management Unit

11520: Device used for database acceleration

11530: Database accelerated integrated circuit

11531: network communication interface/unit

11531(1): The first port of the network communication interface/Ethernet port

11531(2): One or more second ports of network communication interface

11531(4): Ethernet port

11531(5): Serial expansion port

11531(9): PCIe port

11532: the first processing unit

11533: Memory Controller

11534: Large throughput interface

11535: Database acceleration unit

11536: Interconnect

11537: Cryptographic engine

11538: Second-level static random access memory (L2 SRAM)

11540: SATA controller

11545: RDMA unit

11546: remote RAM

11547: Ethernet memory DIMM/database accelerator subunit

11548: Tier 3 L3 memory

11549: DMA engine

11550: Memory resources

11551: Memory processing integrated circuit

11560: storage system

11561: Local storage unit

11563: Non-volatile memory (NVM)

11571: Cache memory

11572: Independent database processing unit

11573: database processing subunit

11574: Reconfigurable array of DB accelerator

11575: Shared memory unit

11576: Configurable link or interconnect

11580: Blade/Group

11590: switch

11601: PCIe switch

11611: exchange system

11612: storage system

11613: computing system

11615: One or more devices used for database acceleration

11621: system

11622: system

11700: Method for database acceleration

11710: steps

11720: steps

11730: steps

11740: steps

11750: steps

11760: steps

12010: database

12011: Server motherboard

12012: CPU

12013: memory unit

12014: Database Accelerator

12020: database

12021: Management Unit

12022: DB accelerator board

12024: processor

12026: memory/processing unit

12031: RDMA engine

12033: DDR controller

12034: DB query database engine

12040: Hybrid system

12042: processor

12043: Memory/Processing Unit (MPU)

12044: compact microcontroller

12049: Cache memory

12050: System

12051: Switch

12052: AI acceleration server

12053: Network Interface Card (NIC)

12054: Central Processing Unit (CPU)

12055: Server motherboard

12056: RAM

12057: AI accelerator

12060: System

12061: Switch

12063: AI processing and network connection unit

12064: Server motherboard

A: Matrix

A1: Data/Shard

A15: Data/Shard

B: Matrix

B1: data/shard

B2: data/sharding

B3: data/sharding

B4: data/sharding

B5: data/sharding

B6: data/shard

B7: data/shard

B8: data/shard

B9: data/sharding

B10: data/shard

B11: data/sharding

B12: data/shard

B13: data/shard

B14: data/shard

B15: data/shard

RD: column decoder

COL: Row decoder

The accompanying drawings incorporated in and forming part of the present invention illustrate various disclosed embodiments. In the schema:

Figure 1 is a diagrammatic representation of a central processing unit (CPU).

Figure 2 is a diagrammatic representation of a graphics processing unit (GPU).

FIG. 3A is a diagrammatic representation of an embodiment of an exemplary hardware chip in accordance with the disclosed embodiment.

FIG. 3B is a diagrammatic representation of another embodiment of an exemplary hardware chip in accordance with the disclosed embodiment.

Figure 4 is a diagrammatic representation of general commands executed by an exemplary hardware chip consistent with the disclosed embodiments.

Figure 5 is a diagrammatic representation of a special command executed by an exemplary hardware chip consistent with the disclosed embodiment.

Figure 6 is a diagrammatic representation of processing groups for use in an exemplary hardware chip consistent with the disclosed embodiments.

Figure 7A is a diagrammatic representation of a rectangular array of processing groups consistent with the disclosed embodiment.

Figure 7B is a diagrammatic representation of an elliptical array of processing groups consistent with the disclosed embodiment.

Figure 7C is a diagrammatic representation of an array of hardware chips in accordance with the disclosed embodiments.

Figure 7D is a diagrammatic representation of another array of hardware chips in accordance with the disclosed embodiments.

FIG. 8 is a flowchart depicting an exemplary method for compiling a series of instructions for execution on an exemplary hardware chip consistent with the disclosed embodiments.

Figure 9 is a diagrammatic representation of the memory bank.

Figure 10 is a diagrammatic representation of the memory bank.

FIG. 11 is a diagrammatic representation of an embodiment of an exemplary memory bank with sub-group controls in accordance with the disclosed embodiments.

FIG. 12 is a diagrammatic representation of another embodiment of an exemplary memory bank with sub-group controls in accordance with the disclosed embodiments.

FIG. 13 is a block diagram of an exemplary memory chip in accordance with the disclosed embodiment.

FIG. 14 is a block diagram of an exemplary set of redundant logic blocks in accordance with the disclosed embodiment.

FIG. 15 is a block diagram of an exemplary logic block in accordance with the disclosed embodiment.

FIG. 16 is a block diagram of an exemplary logic block connected to the bus in accordance with the disclosed embodiment.

FIG. 17 is a block diagram of an exemplary logic block connected in series in accordance with the disclosed embodiment.

FIG. 18 is a block diagram of an exemplary logic block connected in a two-dimensional array in accordance with the disclosed embodiment.

FIG. 19 is a block diagram of an exemplary logic block forming a complex connection in accordance with the disclosed embodiment.

FIG. 20 is an exemplary flowchart illustrating a redundant block enabling processing procedure in accordance with the disclosed embodiment.

FIG. 21 is an exemplary flowchart illustrating an address assignment processing procedure in accordance with the disclosed embodiment.

Figure 22 provides a block diagram of an exemplary processing device in accordance with the disclosed embodiments.

Figure 23 is a block diagram of an exemplary processing device in accordance with the disclosed embodiments.

Figure 24 includes an exemplary memory configuration diagram consistent with the disclosed embodiments.

FIG. 25 is an exemplary flowchart illustrating a memory configuration processing procedure in accordance with the disclosed embodiment.

FIG. 26 is an exemplary flowchart illustrating a memory read processing procedure in accordance with the disclosed embodiment.

FIG. 27 is an exemplary flowchart illustrating the execution of a processing program in accordance with the disclosed embodiment.

Figure 28 shows an example memory chip with a renewed controller in accordance with the present invention.

Figure 29A shows a renewed controller in accordance with an example of the present invention.

Figure 29B shows another example of a renewed controller in accordance with the present invention.

Fig. 30 is an example flow chart of the processing procedure executed by the renewed controller according to the present invention.

FIG. 31 is a flowchart of an example of a processing program implemented by a compiler according to the present invention.

FIG. 32 is a flowchart of another example of a processing program implemented by a compiler according to the present invention.

Figure 33 shows an example of renewing the controller according to the stored pattern configuration according to the present invention.

FIG. 34 is an example flow chart of the processing procedure implemented by the software in the renewed controller according to the present invention.

Figure 35A shows an example wafer including dies in accordance with the present invention.

Figure 35B shows an example memory chip connected to the input/output bus in accordance with the present invention.

Figure 35C shows an example wafer including memory chips arranged in rows and connected to the input and output bus in accordance with the present invention.

FIG. 35D shows two memory chips that are grouped and connected to the input and output bus in accordance with the present invention.

Figure 35E shows an example wafer in accordance with the present invention, which includes dies placed in a hexagonal lattice and connected to an input and output bus.

36A to 36D show various possible configurations of the memory chip connected to the input/output bus in accordance with the present invention.

Figure 37 shows the implementation of the die of the shared glue logic according to the present invention Example grouping.

Figures 38A to 38B show example cutting through the wafer in accordance with the present invention.

FIG. 38C shows an example configuration of the die on a wafer and the configuration of the input/output bus in accordance with the present invention.

Figure 39 shows an example memory chip on a wafer with interconnected processor subunits in accordance with the present invention.

FIG. 40 is an example flow chart of the processing procedure for arranging groups of memory chips from the wafer in accordance with the present invention.

FIG. 41A is a flowchart of another example of a processing procedure for arranging a group of memory chips from a wafer according to the present invention.

41B to 41C are flow charts of an example of a processing procedure for determining a cutting pattern for cutting one or more groups of memory chips from a wafer according to the present invention.

FIG. 42 shows an example of a circuit system in a memory chip that provides dual port access along a row in accordance with the present invention.

FIG. 43 shows an example of a circuit system in a memory chip that provides dual port access along a row in accordance with the present invention.

FIG. 44 shows an example of a circuit system in a memory chip that provides dual-port access along both rows and rows according to the present invention.

Figure 45A shows double read using a replicated memory array or pad.

Figure 45B shows double writing using a replicated memory array or pad.

FIG. 46 shows an example of a circuit system in a memory chip having switching elements for dual-port access along a row in accordance with the present invention.

FIG. 47A is an example flowchart of a process for providing dual-port access on a single-port memory array or pad according to the present invention.

FIG. 47B is an example flowchart of another processing procedure for providing dual-port access on a single-port memory array or pad according to the present invention.

FIG. 48 shows another example of a circuit system in a memory chip that provides dual-port access along both rows and rows in accordance with the present invention.

FIG. 49 shows an example of a switch element for dual-port access in a memory pad according to the present invention.

Figure 50 illustrates an example integrated circuit with reduced cells configured to access partial words in accordance with the present invention.

FIG. 51 illustrates a memory bank for using the reduction unit as described in relation to FIG. 50.

FIG. 52 illustrates a memory bank that uses a reduced unit integrated into the PIM logic in accordance with the present invention.

FIG. 53 illustrates the use of PIM logic to activate the memory bank used to access the partial word switch in accordance with the present invention.

FIG. 54A illustrates a memory bank with a segmented row multiplexer for not starting to access part of words in accordance with the present invention.

FIG. 54B is an example flowchart of a processing procedure for partial word access in memory according to the present invention.

FIG. 55 illustrates a conventional memory chip including multiple memory pads.

Figure 56 illustrates an example memory chip with a startup circuit for reducing power consumption during line disconnection in accordance with the present invention.

FIG. 57 illustrates another example memory chip with a startup circuit for reducing power consumption during line disconnection in accordance with the present invention.

FIG. 58 illustrates yet another example memory chip with a startup circuit for reducing power consumption during line disconnection in accordance with the present invention.

FIG. 59 illustrates an additional example memory chip in accordance with the present invention with a startup circuit for reducing power consumption during online disconnection.

FIG. 60 illustrates an example memory chip with global word lines and regional word lines for reducing power consumption during line disconnection in accordance with the present invention.

Fig. 61 illustrates another example memory chip with global word lines and regional word lines for reducing power consumption during line disconnection in accordance with the present invention.

FIG. 62 is an example flow chart of the processing procedure for sequentially disconnecting the lines in the memory according to the present invention.

Figure 63 illustrates an existing tester used for memory chips.

Figure 64 illustrates another conventional tester used for memory chips.

FIG. 65 illustrates an example of testing a memory chip using logic cells on the same substrate as the memory according to the present invention.

FIG. 66 illustrates another example of testing a memory chip using logic cells on the same substrate as the memory according to the present invention.

FIG. 67 illustrates another example of testing a memory chip using logic cells on the same substrate as the memory according to the present invention.

FIG. 68 illustrates an additional example of testing a memory chip using logic cells on the same substrate as the memory in accordance with the present invention.

FIG. 69 illustrates another example of testing a memory chip using logic cells on the same substrate as the memory according to the present invention.

FIG. 70 is an example flow chart of a processing procedure for testing memory chips in accordance with the present invention.

FIG. 71 is an example flowchart of another processing procedure for testing memory chips in accordance with the present invention.

FIG. 72A is a diagrammatic representation of an integrated circuit including a memory array and a processing array in accordance with an embodiment of the present invention.

FIG. 72B is a diagrammatic representation of a memory area inside an integrated circuit according to an embodiment of the present invention.

Figure 73A is a diagrammatic representation of an integrated circuit with an example configuration of a controller in accordance with an embodiment of the present invention.

Figure 73B is a diagrammatic representation of a configuration for simultaneous execution of a copy model in accordance with an embodiment of the present invention.

Figure 74A is a diagrammatic representation of an integrated circuit with another example configuration of a controller in accordance with an embodiment of the present invention.

FIG. 74B is a flowchart representation of a method of protecting an integrated circuit according to an exemplary disclosed embodiment.

FIG. 74C is a diagrammatic representation of detecting elements located at various points within the chip according to an illustratively disclosed embodiment.

FIG. 75A is a diagrammatic representation of an expandable processor memory system including a plurality of distributed processor memory chips in accordance with an embodiment of the present invention.

FIG. 75B is a diagrammatic representation of an expandable processor memory system including a plurality of distributed processor memory chips in accordance with an embodiment of the present invention.

FIG. 75C is a diagrammatic representation of an expandable processor memory system including a plurality of distributed processor memory chips in accordance with an embodiment of the present invention.

FIG. 75D is a diagrammatic representation of a dual-port distributed processor memory chip in accordance with an embodiment of the present invention.

FIG. 75E is an example timing diagram according to an embodiment of the present invention.

FIG. 76 is an integrated controller and interface module according to an embodiment of the present invention and A diagrammatic representation of the processor memory chips that make up an expandable processor memory system.

FIG. 77 is a flowchart for transferring data between processor memory chips in the expandable processor memory system shown in FIG. 75A according to an embodiment of the present invention.

FIG. 78A illustrates a system for detecting zero values stored in one or more specific addresses of a plurality of memory banks implemented in a memory chip at the chip level in accordance with an embodiment of the present invention.

FIG. 78B illustrates a memory chip for detecting zero values stored in one or more of the specific addresses of a plurality of memory groups at the memory group level in accordance with an embodiment of the present invention.

FIG. 79 illustrates a memory set for detecting zero values stored in one or more of the specific addresses of a plurality of memory pads at the memory pad level in accordance with an embodiment of the present invention.

FIG. 80 is a flowchart illustrating an exemplary method for detecting zero values in specific addresses of a plurality of discrete memory groups according to an embodiment of the present invention.

FIG. 81A illustrates a system for activating the next row associated with a memory bank based on the prediction of the next row in accordance with an embodiment of the present invention.

FIG. 81B illustrates another embodiment of the system of FIG. 81A in accordance with an embodiment of the present invention.

FIG. 81C illustrates the first and second sub-group row controllers of each memory sub-group in accordance with an embodiment of the present invention.

Figure 81D illustrates an embodiment of the next column prediction in accordance with an embodiment of the present invention.

FIG. 81E illustrates an embodiment of a memory bank in accordance with an embodiment of the present invention.

FIG. 81F illustrates another embodiment of the memory bank in accordance with the embodiment of the present invention.

FIG. 82 illustrates a dual-control memory bank for reducing memory row activation penalty in accordance with an embodiment of the present invention.

Figure 83A illustrates the first example of accessing and activating a row of memory banks.

FIG. 83B illustrates a second example of accessing and activating a row of memory banks.

FIG. 83C illustrates a third example of accessing and activating a row of memory banks.

Figure 84 provides a graphical representation of the conventional CPU/register file and external memory architecture.

Figure 85A illustrates an exemplary distributed processor memory chip with memory pads acting as register files in accordance with one embodiment.

FIG. 85B illustrates an exemplary distributed processor memory chip with a memory pad configured to act as a register file in accordance with another embodiment.

FIG. 85C illustrates an exemplary device with a memory pad acting as a register file in accordance with another embodiment.

FIG. 86 provides a flowchart representing an exemplary method for executing at least one instruction in a distributed processor memory chip consistent with the disclosed embodiments.

Figure 87A includes an example of an exploded server;

Figure 87B is an example of distributed processing;

Figure 87C is an example of a memory/processing unit;

Figure 87D is an example of a memory/processing unit;

Figure 87E is an example of a memory/processing unit;

FIG. 87F is an example of an integrated circuit including a memory/processing unit and one or more communication modules;

FIG. 87G is an example of an integrated circuit including a memory/processing unit and one or more communication modules;

Figure 87H is an example of the method;

Figure 87I is an example of the method;

Figure 88A is an example of the method;

Figure 88B is an example of the method;

Figure 88C is an example of the method;

Figure 89A is an example of memory/processing unit and vocabulary;

Figure 89B is an example of a memory/processing unit;

Figure 89C is an example of a memory/processing unit;

Figure 89D is an example of a memory/processing unit;

Figure 89E is an example of a memory/processing unit;

Figure 89F is an example of a memory/processing unit;

Figure 89G is an example of a memory/processing unit;

Figure 89H is an example of a memory/processing unit;

Figure 90A is an example of the system;

Figure 90B is an example of the system;

Figure 90C is an example of the system;

Figure 90D is an example of the system;

Figure 90E is an example of the system;

Figure 90F is an example of the method;

Figure 91A is an example of a memory and screening system, storage device, and CPU;

Figure 91B is an example of memory and processing system, storage device and CPU;

Figure 92A is an example of a memory and processing system, storage device and CPU;

Figure 92B is an example of a memory/processing unit;

Figure 92C is an example of a memory and screening system, storage device and CPU;

Figure 92D is an example of memory and processing system, storage device and CPU;

Figure 92E is an example of memory and processing system, storage device and CPU;

Figure 92F is an example of the method;

Figure 92G is an example of the method;

Figure 92H is an example of the method;

Figure 92I is an example of the method;

Figure 92J is an example of the method;

Figure 92K is an example of the method;

Fig. 93A is a cross-sectional view of an example of a hybrid integrated circuit;

Figure 93B is a cross-sectional view of an example of a hybrid integrated circuit;

Figure 93C is a cross-sectional view of an example of a hybrid integrated circuit;

Figure 93D is a cross-sectional view of an example of a hybrid integrated circuit;

Fig. 93E is a top view of an example of a hybrid integrated circuit;

FIG. 93F is a top view of an example of a hybrid integrated circuit;

Figure 93G is a top view of an example of a hybrid integrated circuit;

Figure 93H is a cross-sectional view of an example of a hybrid integrated circuit;

Figure 93I is a cross-sectional view of an example of a hybrid integrated circuit;

Figure 93J is an example of the method;

Fig. 94A is an example of a storage system, one or more devices, and a computing system;

FIG. 94B is an example of a storage system, one or more devices, and a computing system;

FIG. 94C is an example of one or more devices and computing systems;

Fig. 94D is an example of one or more devices and computing systems;

Figure 94E is an example of a database accelerated integrated circuit;

Figure 94F is an example of a database accelerated integrated circuit;

Figure 94G is an example of a database accelerated integrated circuit;

Figure 94H is an example of a database acceleration unit;

Fig. 94I is an example of the group of blade and database accelerated integrated circuit;

Fig. 94J is an example of a group of accelerated integrated circuits in the database;

Fig. 94K is an example of the group of accelerating integrated circuits in the database;

FIG. 94L is an example of the group of accelerated integrated circuits in the database;

FIG. 94M is an example of a group of accelerated integrated circuits in the database;

Figure 94N is an example of the system;

Figure 94O is an example of the system;

Figure 94P is an example of the method;

Figure 95A is an example of the method;

Figure 95B is an example of the method;

Figure 95C is an example of the method;

Figure 96A is an example of a prior art system;

Figure 96B is an example of the system;

Figure 96C is an example of a database accelerator board;

Figure 96D is an example of a part of the system;

Figure 97A is an example of a prior art system;

Figure 97B is an example of the system; and

Figure 97C shows an example of an AI network interface card.

The following detailed description refers to the accompanying drawings. Wherever convenient, the same reference numbers are used in the drawings and the following description to refer to the same or similar parts. Although several illustrative examples are described herein, modifications, adaptations, and other implementations are possible. For example, the components illustrated in the drawings can be replaced, added, or modified, and the illustrative methods described herein can be modified by replacing, reordering, removing steps, or adding steps to the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. The fact is that the appropriate scope is defined by the scope of the attached patent application.

Processor architecture

As used throughout the present invention, the term "hardware chip" refers to a semiconductor wafer (such as, Silicon or the like) on which one or more circuit elements (such as transistors, capacitors, resistors, and/or the like) are formed. These circuit elements can form processing elements or memory elements. "Processing element" refers to one or more circuit elements that perform at least one logic function (such as arithmetic functions, logic gates, other Boolean operations, or the like) together. The processing element may be a general-purpose processing element (such as a configurable plurality of transistors) or a dedicated processing element (such as a specific logic gate or a plurality of circuit elements designed to perform a specific logic function). "Memory component" refers to one or more circuit components that can be used to store data. "Memory device" can also be referred to as "memory cell". The memory device can be dynamic (so that it needs electricity to be renewed to maintain data storage), static (so that the data persists for at least a period of time after losing power) or non-volatile memory.

The processing elements can be joined to form a processor sub-unit. The "processor subunit" may therefore include the smallest grouping of processing elements that can perform at least one task or instruction (for example, belonging to the processor instruction set). For example, a subunit may include one or more general-purpose processing elements configured to execute instructions together, one or more general-purpose processing elements paired with one or more dedicated processing elements configured to execute instructions in a complementary manner Element, or the like. The processor sub-units can be arranged in an array on a substrate (for example, a wafer). Although the "array" may include a rectangular shape, any configuration of the sub-units in the array may be formed on the substrate.

The memory components can be joined to form a memory bank. For example, the memory bank may include one or more rows of memory elements linked along at least one wire (or other conductive connection). In addition, the memory device can be linked along at least one additional wire in another direction. For example, memory devices can be arranged along word lines and bit lines, as explained below. Although the memory bank can include multiple rows, any configuration of the elements in the group can be used to form the group on the substrate. In addition, one or more groups can be electrically coupled to at least one memory controller to form a memory array. Although the memory array can include rectangular configurations of groups, any configuration of the groups in the array can be formed on the substrate.

As used further throughout the present invention, "bus bar" refers to the interconnection between the components of the substrate Any communication connection. For example, wires or wires (to form electrical connections), optical fibers (to form optical connections), or any other connections for communication between components may be referred to as "bus bars."

Conventional processors pair general-purpose logic circuits with shared memory. The shared memory can store both the instruction set for the execution of the logic circuit and the data used for the execution of the instruction set and generated by the execution of the instruction set. As described below, some conventional processors use a cache system to reduce the delay in executing fetches from shared memory; however, the conventional cache system remains shared. The conventional processor includes a central processing unit (CPU), a graphics processing unit (GPU), various application-specific integrated circuits (ASIC), or the like. Figure 1 shows an example of a CPU, and Figure 2 shows an example of a GPU.

As shown in FIG. 1, the CPU 100 may include a processing unit 110 that includes one or more processor sub-units, such as a processor sub-unit 120a and a processor sub-unit 120b. Although not depicted in FIG. 1, each processor sub-unit may include a plurality of processing elements. In addition, the processing unit 110 may include one or more levels of on-chip cache memory. Such cache memory devices are usually formed on the same semiconductor die as the processing unit 110, instead of being connected to the processor sub-units 120a and 120b via one or more bus bars formed in the substrate, which contains the processor sub-units. Units 120a and 120b and cache memory components. For the first-level (L1) and second-level (L2) caches of conventional processors, the configuration that is directly on the same die rather than connected via a bus is commonly used. Alternatively, in early processors, the backside bus between the sub-units and the L2 cache was used to share the L2 cache between the processor sub-units. The backside busbar is generally larger than the frontside busbar described below. Therefore, because the cache memory is to be shared by all the processor sub-units on the die, the cache memory 130 and the processor sub-units 120a and 120b can be formed on the same die or via one or more backside busses. The bank is communicatively coupled to the processor sub-units 120a and 120b. In both the embodiment without the bus (for example, the cache memory is directly formed on the die) and the embodiment using the backside bus, the cache memory is shared between the processor subunits of the CPU.

In addition, the processing unit 110 communicates with the shared memory 140a and the memory 140b. Lift For example, the memory 140a and 140b may represent a memory bank sharing a dynamic random access memory (DRAM). Although depicted as having two groups, most conventional memory chips include between eight and sixteen memory groups. Therefore, the processor sub-units 120a and 120b can use the shared memory 140a and 140b to store data, which is then operated by the processor sub-units 120a and 120b. However, this configuration causes the bus between the memory 140a and 140b and the processing unit 110 to become a bottleneck when the clock speed of the processing unit 110 exceeds the data transmission speed of the bus. For conventional processors, this is usually the case, resulting in an effective processing speed lower than the specified processing speed based on the clock rate and the number of transistors.

As shown in Figure 2, similar defects also exist in GPUs. The GPU 200 may include a processing unit 210 that includes one or more processor sub-units (e.g., sub-units 220a, 220b, 220c, 220d, 220e, 220f, 220g, 220h, 220i, 220j, 220k, 220l, 220m , 220n, 220o and 220p). In addition, the processing unit 210 may include one or more levels of on-chip cache and/or register files. Such cache memory devices are usually formed on the same semiconductor die as the processing unit 210. In fact, in the example of FIG. 2, the cache memory 210 and the processing unit 210 are formed on the same die and shared among all the processor sub-units, and the cache memories 230a, 230b, 230c, and 230d are formed on the same die. A subset of the processor sub-units and dedicated to the processor sub-units.

In addition, the processing unit 210 communicates with shared memories 250a, 250b, 250c, and 250d. For example, the memories 250a, 250b, 250c, and 250d may represent memory banks sharing DRAM. Therefore, the processor subunits of the processing unit 210 can use the shared memory 250a, 250b, 250c, and 250d to store data, which is then operated by the processor subunits. However, this configuration causes the bus between the memory 250a, 250b, 250c, and 250d and the processing unit 210 to become a bottleneck, which is similar to the bottleneck described above with respect to the CPU.

Overview of disclosed hardware chips

FIG. 3A is a diagrammatic representation depicting an embodiment of an exemplary hardware chip 300. FIG. Hard crystal The slice 300 may include a distributed processor designed to alleviate the bottlenecks described above with respect to CPUs, GPUs, and other conventional processors. The distributed processor may include a plurality of processor subunits spatially distributed on a single substrate. In addition, as explained above, in the distributed processor of the present invention, the corresponding memory groups are also spatially distributed on the substrate. In some embodiments, a distributed processor may be associated with an instruction set, and each of the processor subunits of the distributed processor may be responsible for performing one or more tasks included in the instruction set.

As depicted in FIG. 3A, the hardware chip 300 may include a plurality of processor sub-units, such as logic and control sub-units 320a, 320b, 320c, 320d, 320e, 320f, 320g, and 320h. As further depicted in Figure 3A, each processor sub-unit may have a dedicated memory instance. For example, the logic and control sub-unit 320a is operatively connected to the dedicated memory instance 330a, the logic and control sub-unit 320b is operatively connected to the dedicated memory instance 330b, and the logic and control sub-unit 320c is operatively connected to the dedicated memory instance 330b. The memory instance 330c, the logic and control subunit 320d are operatively connected to the dedicated memory instance 330d, the logic and control subunit 320e is operatively connected to the dedicated memory instance 330e, and the logic and control subunit 320f is operatively connected to For the dedicated memory instance 330f, the logic and control subunit 320g is operatively connected to the dedicated memory instance 330g, and the logic and control subunit 320h is operatively connected to the dedicated memory instance 330h.

Although FIG. 3A depicts each memory instance as a single memory group, the hardware chip 300 may include two or more memory groups as dedicated memory instances for the processor subunits on the hardware chip 300. item. In addition, although FIG. 3A depicts each processor subunit as including both logic components and controls for a dedicated memory bank, the hardware chip 300 can use controls for the memory bank. These controls At least partly separate from these logical components. In addition, as depicted in FIG. 3A, two or more processor sub-units and their corresponding memory components may be grouped into processing groups 310a, 310b, 310c, and 310d, for example. The "processing group" may indicate the spatial difference on the substrate on which the hard chip 300 is formed. Therefore, the processing group can include other controls for the memory group in the group, such as Control elements 340a, 340b, 340c, and 340d. Additionally or alternatively, the "processing group" may refer to a logical grouping for the purpose of compiling program code for execution on the hardware chip 300. Therefore, the compiler for the hardware chip 300 (described further below) can divide the entire instruction set among the processing groups on the hardware chip 300.

In addition, the host 350 can provide commands, data, and other inputs to the hardware chip 300 and read and output from the hardware chip. Therefore, the instruction set can all be executed on a single die, for example, on the die of the surrogate hardware chip 300. In fact, the only communication outside the die can include loading commands to the hardware chip 300, any input sent to the hardware chip 300, and any output read from the hardware chip 300. Therefore, all calculations and memory operations can be performed on the die (on the hardware chip 300) because the processor subunit of the hardware chip 300 communicates with the dedicated memory bank of the hardware chip 300.

FIG. 3B is a diagrammatic representation depicting an embodiment of another exemplary hardware chip 300'. Although depicted as an alternative to the hardware chip 300, the architecture depicted in FIG. 3B can be at least partially combined with the architecture depicted in FIG. 3A.

As depicted in FIG. 3B, the hardware chip 300' may include a plurality of processor sub-units, such as processor sub-units 350a, 350b, 350c, and 350d. As further depicted in FIG. 3B, each processor sub-unit may have a plurality of dedicated memory instances. For example, the processor sub-unit 350a is operatively connected to the dedicated memory instances 330a and 330b, the processor sub-unit 350b is operatively connected to the dedicated memory instances 330c and 330d, and the processor sub-unit 350c is operatively connected to The dedicated memory instances 330e and 330f, and the processor subunit 350d is operatively connected to the dedicated memory instances 330g and 330h. In addition, as depicted in FIG. 3B, the processor sub-units and their corresponding memory components may be grouped into processing groups 310a, 310b, 310c, and 310d, for example. As explained above, the "processing group" may refer to the spatial distinction on the substrate on which the hardware chip 300' is formed and/or the logical grouping for the purpose of compiling program codes for execution on the hardware chip 300'.

As further depicted in Figure 3B, the processor sub-units can communicate with each other via a bus. For example, as shown in FIG. 3B, the processor subunit 350a may communicate with the processor subunit 350b via the bus 360a, communicate with the processor subunit 350c via the bus 360c, and communicate with the processor subunit 350c via the bus 360f. 350d communication. Similarly, the processor sub-unit 350b can communicate with the processor sub-unit 350a via the bus 360a (as described above), with the processor sub-unit 350c via the bus 360e, and with the processor sub-unit 350d via the bus 360d Communication. In addition, the processor sub-unit 350c can communicate with the processor sub-unit 350a via the bus 360c (as described above), communicate with the processor sub-unit 350b via the bus 360e (as described above), and communicate with the processor sub-unit 350b via the bus 360b. The processor subunit 350d communicates. Accordingly, the processor sub-unit 350d can communicate with the processor sub-unit 350a via the bus 360f (as described above), communicate with the processor sub-unit 350b via the bus 360d (as described above), and via the bus 360b (As described above) communicate with the processor sub-unit 350c. Those skilled in the art will understand that fewer buses than those depicted in FIG. 3B can be used. For example, the bus 360e can be eliminated, so that the communication between the processor subunits 350b and 350c is passed through the processor subunits 350a and/or 350d. Similarly, the bus 360f can be eliminated so that the communication between the processor subunit 350a and the processor subunit 350d is passed through the processor subunit 350b or 350c.

In addition, those who are generally familiar with the technology will understand that architectures other than those depicted in FIGS. 3A and 3B can be used. For example, arrays of processing groups each having a single processor subunit and memory instance can be arranged on the substrate. The processor subunit may additionally or alternatively form part of the controller for the corresponding dedicated memory bank, part of the controller for the memory pad of the corresponding dedicated memory, or the like.

In view of the architecture described above, compared to the traditional architecture, the hardware chips 300 and 300' can significantly improve the efficiency of memory-intensive tasks. For example, database operations and artificial intelligence algorithms (such as neural networks) are examples of memory-intensive tasks. For memory-intensive tasks, the traditional architecture is less efficient than the hardware chips 300 and 300'. Therefore, the hardware chips 300 and 300' can be referred to as database accelerator processors and/or artificial intelligence accelerator processors.

Hardware chip revealed by configuration

The hardware chip architecture described above can be configured for code execution. For example, each processor sub-unit can be separated from other processor sub-units in the hardware chip and execute the program code (defining instruction set) individually. Therefore, instead of relying on the operating system to manage multi-threaded processing or using multi-tasking (which is simultaneous rather than parallel), the hardware chip of the present invention can allow the processor sub-units to operate completely in parallel.

Except for the fully parallel implementation described above, at least some of the instructions assigned to each processor subunit may overlap. For example, a plurality of processor subunits on a distributed processor can execute overlapping instructions as an implementation solution for operating systems or other management software, and execute non-overlapping instructions at the same time to execute parallel operations within the context of the operating system or other management software. task.

FIG. 4 depicts an exemplary processing procedure 400 performed by the processing group 410 for executing general commands. For example, the processing group 410 may include a part of the hard chip (for example, the hard chip 300, the hard chip 300', or the like) of the present invention.

As depicted in FIG. 4, commands may be sent to the processor sub-unit 430 paired with the dedicated memory instance 420. The external host (for example, the host 350) may send the command to the processing group 410 for execution. Alternatively, the host 350 may have sent an instruction set including the command for storage in the memory instance 420, so that the processor sub-unit 430 can retrieve the command from the memory instance 420 and execute the retrieved command. Therefore, the command can be executed by the processing element 440, which is a general processing element that can be configured to execute the received command. In addition, the processing group 410 may include a control 460 for the memory instance 420. As depicted in FIG. 4, the control component 460 can perform any reading and/or writing of the memory instance 420 required by the processing element 440 when executing the received command. After executing the command, the processing group 410 can output the result of the command to, for example, an external host or to different processing groups on the same hardware chip.

In some embodiments, as depicted in FIG. 4, the processor sub-unit 430 may further Includes address generator 450. The "address generator" can include a plurality of processing elements, which are configured to determine the addresses used to perform reading and writing in one or more memory banks, and can also be located in The data at the determined address performs an operation (for example, addition, subtraction, multiplication, or the like). For example, the address generator 450 can determine the address used for any read or write to the memory. In one example, the address generator 450 can improve efficiency by overwriting the read value with a new value determined based on the command when the read value is no longer needed. Additionally or alternatively, the address generator 450 can select available addresses for storing results from command execution. This allows the result of scheduling for the next clock cycle to be read out, which is more convenient for external hosts. In another example, the address generator 450 may determine the read and write addresses during multi-cycle calculations such as vector or matrix multiply-accumulate calculations. Therefore, the address generator 450 can maintain or calculate the memory addresses used to read data and write the intermediate results of the multi-cycle calculation, so that the processor subunit 430 can continue processing without storing these memory addresses.

FIG. 5 depicts an exemplary processing procedure 500 performed by the processing group 510 for executing special commands. For example, the processing group 510 may include a part of the hard chip (for example, the hard chip 300, the hard chip 300', or the like) of the present invention.

As depicted in FIG. 5, special commands (eg, multiply and accumulate commands) can be sent to the processing element 530 paired with the special memory instance 520. An external host (for example, the host 350) may send the command to the processing element 530 for execution. Therefore, the command can be executed by the processing element 530 under a given signal from the host. The processing element is a special processing element that can be configured to execute a specific command (including the received command). Alternatively, the processing component 530 may retrieve the command from the memory instance 520 for execution. Therefore, in the example of FIG. 5, the processing element 530 is a multiply-accumulate (MAC) circuit configured to execute MAC commands received from an external host or retrieved from the memory instance 520. After executing the command, the processing group 410 can output the result of the command to, for example, an external host or to different processing groups on the same hardware chip. Although it describes a single command and a single result, it can be received or retrieved and Multiple commands are executed, and multiple results can be combined on the processing group 510 before output.

Although depicted as a MAC circuit in FIG. 5, additional or alternative specialized circuits may be included in the processing group 510. For example, you can implement the MAX read command (which returns the maximum value of the vector), the MAX0 read command (also known as a common function of the rectifier, which returns the entire vector and returns the maximum value of 0), Or the like.

Although depicted separately, the general processing group 410 of FIG. 4 and the specialized processing group 510 of FIG. 5 can be combined. For example, a general processor sub-unit may be coupled to one or more specialized processor sub-units to form a processor sub-unit. Therefore, a general processor subunit can be used for all instructions that cannot be executed by one or more specialized processor subunits.

Those who are familiar with this technology will understand that special logic circuits can be used to handle neural network implementations and other memory-intensive tasks. For example, database query, packet inspection, string comparison, and other functions can improve efficiency when executed by the hardware chip described in this article.

Memory-based architecture for distributed processing

On the hardware chip according to the present invention, the dedicated bus can transfer data between the processor sub-units on the chip and/or between the processor sub-units and their corresponding dedicated memory banks. Using a dedicated bus can reduce the cost of arbitration, because competing requests are impossible or easy to avoid using software rather than hardware.

Figure 6 schematically depicts a diagrammatic representation of a processing group 600. The processing group 600 can be used in a hard chip (for example, a hard chip 300, a hard chip 300', or the like). The processor sub-unit 610 can be connected to the memory 620 via the bus 630. The memory 620 may include a random access memory (RAM) device, which stores data and program codes for the processor sub-unit 610 to execute. In some embodiments, the memory 620 may be an N-way memory (where N is a number equal to or greater than 1, which implies the number of sectors in the interleaved memory 620). Because the processor sub-unit 610 is coupled to the memory 620 dedicated to the processor sub-unit 610 via the bus 630, N can be kept relatively small without damage Execution efficiency. This represents an improvement over the conventional multi-channel register file or cache memory, where a lower N usually leads to lower execution performance, and a higher N usually leads to a large area and power loss.

The size of the memory 620, the number of channels, and the width of the bus 630 can be adjusted according to, for example, the size of the data involved in one or more tasks to meet the requirements of the task and application implementation of the system using the processing group 600 . The memory element 620 may include one or more types of memory known in the art, such as volatile memory (such as RAM, DRAM, SRAM, phase change RAM (PRAM), magnetoresistive RAM (MRAM)) , Resistive RAM (ReRAM) or the like) or non-volatile memory (such as flash memory or ROM). According to some embodiments, a part of the memory element 620 may include a first memory type, and another part may include another memory type. For example, the program code area of the memory device 620 may include a ROM device, and the data area of the memory device 620 may include a DRAM device. Another example of this division is to store the weights of the neural network in flash memory and store the data used for calculation in DRAM.

The processor sub-unit 610 includes a processing element 640, which may include a processor. The processor can be pipelined or non-pipelined, can be a customized reduced instruction set computing (RISC) component or implemented in any commercial integrated circuit (IC) known in the art (such as ARM, ARC, RISC) -V, etc.) on other treatment schemes, as generally understood by those who are familiar with this technology. The processing element 640 may include a controller, which in some embodiments includes an arithmetic logic unit (ALU) or other controller.

According to some embodiments, the processing element 640 that executes the received or stored code may include general processing elements, and is therefore flexible and capable of performing a wide variety of processing operations. When comparing the power consumed during the execution of a specific operation, the non-dedicated circuit system generally consumes more power than the dedicated circuit system for the specific operation. Therefore, when performing specific complex arithmetic calculations, the processing element 640 can consume more power and perform less efficiently than dedicated hardware. Therefore, according to some embodiments, the controller of the processing element 640 may be designed to perform specific operations (for example, addition or "move" operations).

In one example, certain operations may be performed by one or more accelerators 650. Every plus The speed controller may be dedicated and programmed to perform specific calculations (such as multiplication, floating-point vector operations, or the like). By using the accelerator, the average power consumed by each calculation of each processor subunit can be reduced, and the calculation throughput will increase accordingly. The accelerator 650 may be selected according to the application that the system is designed to implement (for example, executing a neural network, executing a database query, or the like). The accelerator 650 can be configured by the processing element 640 and can cooperate with the processing element for reducing power consumption and accelerating calculations and operations. The accelerator may additionally or alternatively be used between the memory and the MUX/DEMUX/input/output ports of the processing group 600 (for example, MUX 650 and DEMUX 660) of the processing group 600 such as smart direct memory access (DMA) peripherals Send data.

The accelerator 650 can be configured to perform a variety of functions. For example, an accelerator can be configured to perform 16-bit floating point calculations or 8-bit integer calculations commonly used in neural networks. Another example of the accelerator function is 32-bit floating point calculations commonly used during the training phase of neural networks. Another example of the accelerator function is query processing, such as query processing in a database. In some embodiments, the accelerator 650 may include specialized processing elements for performing these functions and/or may be configured according to configuration data stored on the memory element 620 so that it can be modified.

The accelerator 650 may additionally or alternatively implement a configurable command code processing list of memory movement to time the movement of data to/from the memory 620 or to/from other accelerators and/or input/output terminals. Therefore, as explained further below, all data movement within the hardware chip using the processing group 600 can use software synchronization instead of hardware synchronization. For example, an accelerator in a processing group (for example, group 600) can send data from its input to its accelerator every tenth cycle, and then output the data in the next cycle, thereby making the information self-processing group The memory is streamed to another memory.

As further depicted in FIG. 6, in some embodiments, the processing group 600 may further include at least one input multiplexer (MUX) 660 connected to its input port and at least one output DEMUX 670 connected to its output port. These MUX/DEMUX can be controlled by control signals (not shown) from one of the processing element 640 and/or from the accelerator 650, and these control signals are based on The current instruction being performed by the processing element 640 and/or the operation performed by the accelerator in the accelerator 650 is determined. In some scenarios, it may be necessary to process group 600 (according to a predefined command from its code memory) to send data from its input port to its output port. Therefore, in addition to each of DEMUX/MUX connected to the processing element 640 and the accelerator 650, one or more of the input MUX (e.g., MUX 660) can also be directly connected to the output DEMUX ( For example, DEMUX 670).

The processing group 600 of FIG. 6 may be arranged in an array to form a distributed processor, for example, as depicted in FIG. 7A. The processing group may be disposed on the substrate 710 to form an array. In some embodiments, the substrate 710 may include a semiconductor substrate such as silicon. Additionally or alternatively, the substrate 710 may include a circuit board, such as a flexible circuit board.

As depicted in FIG. 7A, the substrate 710 may include a plurality of processing groups, such as the processing group 600, disposed thereon. Therefore, the substrate 710 includes a memory array including a plurality of groups, such as groups 720a, 720b, 720c, 720d, 720e, 720f, 720g, and 720h. In addition, the substrate 710 includes a processing array, which may include a plurality of processor subunits, such as subunits 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h.

In addition, as explained above, each processing group may include a processor sub-unit and one or more corresponding memory groups dedicated to the processor sub-unit. Therefore, as depicted in FIG. 7A, each subunit is associated with a corresponding dedicated memory group. For example, the processor subunit 730a is associated with the memory group 720a, and the processor subunit 730b is associated with the memory group 720b. The processor subunit 730c is associated with the memory group 720c, the processor subunit 730d is associated with the memory group 720d, the processor subunit 730e is associated with the memory group 720e, and the processor subunit 730f is associated with the memory group 720e. 720f is associated with, the processor subunit 730g is associated with the memory group 720g, and the processor subunit 730h is associated with the memory group 720h.

In order to allow each processor sub-unit to communicate with its corresponding dedicated memory bank, the board 710 may include a first plurality of buses connecting one of the processor subunits to its corresponding dedicated memory bank. Therefore, the bus 740a connects the processor sub-unit 730a to the memory group 720a, the bus 740b connects the processor sub-unit 730b to the memory group 720b, and the bus 740c connects the processor sub-unit 730c to the memory group 720c, The bus 740d connects the processor sub-unit 730d to the memory group 720d, the bus 740e connects the processor sub-unit 730e to the memory group 720e, and the bus 740f connects the processor sub-unit 730f to the memory group 720f. 740g connects the processor sub-unit 730g to the memory group 720g, and the bus 740h connects the processor sub-unit 730h to the memory group 720h. In addition, in order to allow each processor sub-unit to communicate with other processor sub-units, the substrate 710 may include a second plurality of bus bars connecting one of the processor sub-units to the other of the processor sub-units. In the example of FIG. 7A, the bus 750a connects the processor sub-unit 730a to the processor sub-unit 750e, the bus 750b connects the processor sub-unit 730a to the processor sub-unit 750b, and the bus 750c connects the processor sub-unit 730b. Connected to the processor subunit 750f, the bus 750d connects the processor subunit 730b to the processor subunit 750c, the bus 750e connects the processor subunit 730c to the processor subunit 750g, and the bus 750f connects the processor subunit 730c is connected to the processor sub-unit 750d, the bus 750g connects the processor sub-unit 730d to the processor sub-unit 750h, the bus 750h connects the processor sub-unit 730h to the processor sub-unit 750g, and the bus 750i connects the processor sub-unit. The unit 730g is connected to the processor sub-unit 750g, and the bus 750j connects the processor sub-unit 730f to the processor sub-unit 750e.

Therefore, in the example configuration shown in FIG. 7A, a plurality of logical processor subunits are configured in at least one column and at least one row. The second plurality of bus bars connect each processor subunit to at least one adjacent processor subunit in the same column and to at least one adjacent processor subunit in the same row. Figure 7A can be referred to as "partial block connection".

The configuration shown in Figure 7A can be modified to form a "complete block connection." Full block connections include additional bus bars connecting diagonal processor sub-units. For example, the second plurality of confluences The row may include between the processor sub-unit 730a and the processor sub-unit 730f, between the processor sub-unit 730b and the processor sub-unit 730e, between the processor sub-unit 730b and the processor sub-unit 730g, and the processor sub-unit 730c. Additional buses between the processor sub-unit 730f, between the processor sub-unit 730c and the processor sub-unit 730h, and between the processor sub-unit 730d and the processor sub-unit 730g.

Complete block connection can be used for convolution calculations. In convolution calculations, data and results stored in nearby processor subunits are used. For example, during convolutional image processing, each processor sub-unit may receive a block of the image (such as pixels or pixel groups). In order to calculate the convolution result in detail, each processor subunit can obtain data from all eight neighboring processor subunits, each of which has received the corresponding block. In partial block connections, data from diagonally adjacent processor subunits can be transferred via other adjacent processor subunits connected to the processor subunit. Therefore, the distributed processor on the chip can be an artificial intelligence accelerator processor.

In the specific example of convolution calculation, the N×M image can be divided across a plurality of processor sub-units. Each processor sub-unit can perform convolution with the A×B filter on its corresponding block. In order to perform filtering on one or more pixels on the boundary between blocks, each processor sub-unit may require data from adjacent processor sub-units that have pixels on the same boundary Block. Therefore, the code generated by each processor sub-unit is configured to calculate the convolution, and data from the adjacent sub-unit is obtained from the second plurality of buses. The corresponding commands for outputting data to the second plurality of buses are provided to the sub-unit to ensure proper timing of the required data transmission.

The partial block connection in Figure 7A can be modified to N partial block connections. In this modification, the second plurality of buses can further connect each processor subunit to the four directions along which the bus in FIG. 7A runs (ie, up, down, left, and right). The processor subunits within the threshold distance of the processor subunits (for example, within n processor subunits). Similar modifications can be made to the complete block connection (to produce N complete block connections), so that the second plurality of buses further connects each processor sub-unit The element is connected to the four directions along which the bus of FIG. 7A runs except for the two diagonal directions, which are within the threshold distance of the processor subunit (for example, in n processor subunits). Processor subunit.

Other configurations are possible. For example, in the configuration shown in FIG. 7B, bus 750a connects processor sub-unit 730a to processor sub-unit 730d, bus 750b connects processor sub-unit 730a to processor sub-unit 730b, and bus 750c connects the processor sub-unit 730b to the processor sub-unit 730c, and the bus 750d connects the processor sub-unit 730c to the processor sub-unit 730d. Therefore, in the example configuration shown in FIG. 7B, a plurality of processor subunits are configured in a star pattern. The second plurality of bus bars connect each processor subunit to at least one adjacent processor subunit in the star pattern.

Other configurations (not shown) are possible. For example, a neighbor connection configuration may be used such that a plurality of processor subunits are arranged in one or more rows (e.g., similar to the situation depicted in FIG. 7A). In the adjacent connection configuration, the second plurality of bus bars connect each processor subunit to the left processor subunit in the same row, the right processor subunit in the same row, and the left processor subunit in the same row. Both the square processor subunit and the right processor subunit, etc.

In another example, an N linear connection configuration can be used. In the N linear connection configuration, the second plurality of buses connect each processor subunit to the processor subunit within the threshold distance of the processor subunit (for example, within n processor subunits) . The N linear connection configuration can be used with linear arrays (described above), rectangular arrays (depicted in FIG. 7A), elliptical arrays (depicted in FIG. 7B), or any other geometric arrays.

In yet another example, an N logarithmic connection configuration may be used. In the N logarithmic connection configuration, the second plurality of buses connect each processor subunit to within a threshold distance of the power of two of the processor subunit (for example, within 2 ⁿ processor subunits) The processor subunit. The N-log connected configuration can be used with linear arrays (described above), rectangular arrays (depicted in FIG. 7A), elliptical arrays (depicted in FIG. 7B), or any other geometric arrays.

Any of the connection schemes described above can be combined for use in the same hardware chip. For example, a complete block connection can be used in one zone, and a partial block connection can be used in another zone. In another example, an N linear connection configuration can be used in one zone, while an N complete block connection can be used in another zone.

Instead of dedicated buses between processor subunits of memory chips, or in addition to these dedicated buses, one or more common buses can also be used to interconnect all processor subunits (or processing A subset of the device subunit). It is still possible to avoid conflicts on the shared bus by using the code executed by the processor subunit to time the data transmission on the shared bus, as explained further below. In addition to or instead of the shared bus, a configurable bus can also be used to dynamically connect processor subunits to form a group of processor units connected to separate buses. For example, the configurable bus may include transistors or other mechanisms that can be controlled by the processor sub-unit to direct data transmission to the selected processor sub-unit.

In both FIGS. 7A and 7B, the plurality of processor subunits of the processing array are spatially distributed among the plurality of discrete memory groups of the memory array. In other alternative embodiments (not shown), a plurality of processor subunits may be clustered in one or more regions of the substrate, and a plurality of memory groups may be clustered in one or more other regions of the substrate. In some embodiments, a combination of spatial distribution and clustering (not shown) may be used. For example, one area of the substrate may include a cluster of processor subunits, another area of the substrate may include a cluster of memory banks, and another area of the substrate may include a processing array distributed among the memory banks.

Those skilled in the art will recognize that arranging the processor groups 600 in an array on a substrate is not an exclusive embodiment. For example, each processor sub-unit can be associated with at least two dedicated memory banks. Therefore, the processing group 310a, 310b, 310c, and 310d of FIG. 3B can be used instead of or in combination with the processing group 600 to form a processing array and a memory array. Available include For example, other processing groups (not shown) of three, four, or more than four dedicated memory groups.

Each of the plurality of processor subunits can be configured to independently execute software code associated with a specific application program relative to other processor subunits included in the plurality of processor subunits. For example, as explained below, multiple sub-series of instructions can be grouped into machine code and provided to each processor sub-unit for execution.

In some embodiments, each dedicated memory bank includes at least one dynamic random access memory (DRAM). Alternatively, the memory bank may include a mixture of memory types such as static random access memory (SRAM), DRAM, flash memory, or the like.

In conventional processors, data sharing between processor subunits is usually performed by sharing memory. Shared memory usually requires most of the chip area and/or implementation of a bus managed by additional hardware (such as an arbiter). As described above, this bus bar creates a bottleneck. In addition, the shared memory that can be external to the chip usually includes a cache coherency mechanism and more complex cache memory (for example, L1 cache memory, L2 cache memory, and shared DRAM), so that the accurate and up-to-date The data is provided to the processor sub-unit. As explained further below, the dedicated bus depicted in FIGS. 7A and 7B allows for hardware chips without hardware management (such as an arbiter). In addition, the use of dedicated memory as depicted in Figures 7A and 7B allows the elimination of complex cache layers and coherency mechanisms.

In fact, in order to allow each processor sub-unit to access data calculated by other processor sub-units and/or stored in a memory bank dedicated to other processor sub-units, buses are provided, and the timing of these buses It is dynamically executed using code executed individually by each processor subunit. This situation allows the elimination of most (if not all) bus management hardware as conventionally used. In addition, direct transmission on these buses replaces complex caching mechanisms to reduce latency during memory reading and writing.

Memory-based processing array

As depicted in FIG. 7A and FIG. 7B, the memory chip of the present invention can be operated independently. Alternatively, the memory chip of the present invention can be combined with one such as a memory device (for example, one or more DRAM banks), a system-on-a-chip, a field programmable gate array (FPGA), or other processing and/or memory chips. Or multiple additional integrated circuits are operatively connected. In these embodiments, the tasks in a series of instructions executed by the architecture can be divided between the processor subunits of the memory chip and any processor subunits of additional integrated circuits (for example, by the compiler , As described below). For example, other integrated circuits may include a host (for example, the host 350 of FIG. 3A) that inputs commands and/or data to the memory chip and receives output therefrom.

In order to interconnect the memory chip of the present invention with one or more additional integrated circuits, the memory chip may include a memory interface, such as compliant with the Joint Electron Device Engineering Council (JEDEC) standard or its variants The memory interface of any one of them. One or more additional integrated circuits can then be connected to the memory interface. Therefore, if the one or more additional integrated circuits are connected to the plurality of memory chips of the present invention, data can be shared among the memory chips through the one or more additional integrated circuits. Additionally or alternatively, the one or more additional integrated circuits may include a bus for connecting to the bus on the memory chip of the present invention, so that the one or more additional integrated circuits can be connected to the memory of the present invention. The integrated chip executes the code together. In these embodiments, the one or more additional integrated circuits further assist in distributed processing, even though the additional integrated circuits may be on different substrates from the memory chip of the present invention.

In addition, the memory chips of the present invention can be arranged in an array to form an array of distributed processors. For example, one or more bus bars can connect the memory chip 770a to the additional memory chip 770b, as depicted in FIG. 7C. In the example of FIG. 7C, the memory chip 770a includes a processor subunit and one or more corresponding memory groups dedicated to each processor subunit. For example, the processor subunit 730a is associated with the memory group 720a. , The processor subunit 730b is associated with the memory group 720b, the processor subunit 730e is associated with the memory group 720c, and the processor subunit 730f is associated with the memory group 720d. The bus bar connects each processor sub-unit to its corresponding memory bank. Therefore, the sink The bus 740a connects the processor sub-unit 730a to the memory group 720a, the bus 740b connects the processor sub-unit 730b to the memory group 720b, and the bus 740c connects the processor sub-unit 730e to the memory group 720c, and the bus Row 740d connects the processor subunit 730f to the memory bank 720d. In addition, the bus 750a connects the processor sub-unit 730a to the processor sub-unit 750e, the bus 750b connects the processor sub-unit 730a to the processor sub-unit 750b, and the bus 750c connects the processor sub-unit 730b to the processor sub-unit. Unit 750f, and the bus 750d connects the processor sub-unit 730e to the processor sub-unit 750f. For example, as described above, other configurations of the memory chip 770a may be used.

Similarly, the memory chip 770b includes a processor subunit and one or more corresponding memory groups dedicated to each processor subunit. For example, the processor subunit 730c is associated with the memory group 720e, and the processor subunit 730c is associated with the memory group 720e. The unit 730d is associated with the memory group 720f, the processor subunit 730g is associated with the memory group 720g, and the processor subunit 730h is associated with the memory group 720h. The bus bar connects each processor sub-unit to its corresponding memory bank. Therefore, the bus 740e connects the processor sub-unit 730c to the memory group 720e, the bus 740f connects the processor sub-unit 730d to the memory group 720f, and the bus 740g connects the processor sub-unit 730g to the memory group 720g, And the bus 740h connects the processor sub-unit 730h to the memory bank 720h. In addition, the bus 750g connects the processor sub-unit 730c to the processor sub-unit 750g, the bus 750h connects the processor sub-unit 730d to the processor sub-unit 750h, and the bus 750i connects the processor sub-unit 730c to the processor sub-unit. Unit 750d, and bus 750j connects processor sub-unit 730g to processor sub-unit 750h. For example, as described above, other configurations of the memory chip 770b may be used.

The processor sub-units of the memory chips 770a and 770b can be connected using one or more buses. Therefore, in the example of FIG. 7C, the bus 750e can connect the processor sub-unit 730b of the memory chip 770a with the processor sub-unit 730c of the memory chip 770b, and the bus 750f can connect the processor of the memory chip 770a The sub-unit 730f is connected to the processor sub-unit 730c of the memory 770b. For example, the bus 750e can serve as the input bus to the memory chip 770b (and therefore charge When the output bus of the memory chip 770a), the bus 750f can serve as the input bus to the memory chip 770a (and thus the output bus of the memory chip 770b), or vice versa. Alternatively, the bus bars 750e and 750f can serve as bidirectional bus bars between the memory chips 770a and 770b.

The bus bars 750e and 750f may include direct wires or may be interleaved on high-speed connections to reduce the number of pins used for the inter-chip interface between the memory chip 770a and the integrated circuit 770b. In addition, any of the above-described connection configurations used in the memory chip itself can be used to connect the memory chip to one or more additional integrated circuits. For example, the memory chips 770a and 770b may be connected using complete blocks or partial blocks instead of using only two bus bars as shown in FIG. 7C.

Therefore, although the bus bars 750e and 750f are used for illustration, the architecture 760 may include fewer buses or additional buses. For example, a single bus can be used between the processor sub-units 730b and 730c or between the processor sub-units 730f and 730c. Alternatively, additional busbars may be used, for example, between the processor sub-units 730b and 730d, between the processor sub-units 730f and 730d, or the like.

In addition, although it is depicted as using a single memory chip and additional integrated circuits, a plurality of memory chips can be connected using a bus, as explained above. For example, as depicted in the example of FIG. 7C, memory chips 770a, 770b, 770c, and 770d are connected in an array. Similar to the memory chips described above, each memory chip includes a processor sub-unit and a dedicated memory bank. Therefore, the description of these components is not repeated here.

In the example of FIG. 7C, the memory chips 770a, 770b, 770c, and 770d are connected in a loop. Therefore, the bus 750a is connected to the memory chips 770a and 770d, the bus 750c is connected to the memory chips 770a and 770b, the bus 750e is connected to the memory chips 770b and 770c, and the bus 750g is connected to the memory chips 770c and 770d. Although the memory chips 770a, 770b, 770c, and 770d can be connected using full block connections, partial block connections, or other connection configurations, the example of FIG. 7C allows fewer pins between the memory chips 770a, 770b, 770c, and 770d connection.

Relatively large memory

The embodiments of the present invention can use a dedicated memory that is relatively larger in size than the shared memory of a conventional processor. Using dedicated memory instead of shared memory allows continued efficiency gains without gradual decrease as memory increases. This allows memory-intensive tasks such as neural network processing and database query to be performed more efficiently than in conventional processors. In conventional processors, the increased efficiency gain of shared memory is due to the von Neumann bottleneck. gradually decreases.

For example, in the distributed processor of the present invention, the memory array disposed on the substrate of the distributed processor may include a plurality of discrete memory groups. Each of the discrete memory groups may have a capacity greater than one million bytes; and a processing array disposed on the substrate, the processing array including a plurality of processor subunits. As explained above, each of the processor subunits can be associated with a corresponding dedicated memory group among the plurality of discrete memory groups. In some embodiments, the plurality of processor subunits may be spatially distributed among a plurality of discrete memory groups in the memory array. By using at least one million bytes of dedicated memory instead of a few million bytes of shared cache memory for large CPUs or GPUs, the distributed processor of the present invention can be used in conventional systems due to the CPU And the efficiency that the von Neumann bottleneck in the GPU is impossible to achieve.

Different memories can be used as dedicated memories. For example, each dedicated memory bank may include at least one DRAM bank. Alternatively, each dedicated memory bank may include at least one static random access memory bank. In other embodiments, different types of memory can be combined on a single hardware chip.

As explained above, each dedicated memory can be at least one million bytes. Therefore, the size of each dedicated memory group may be the same, or at least two of the plurality of memory groups may have different sizes.

In addition, as described above, the distributed processor may include: a first plurality of bus bars, each of which connects one of the plurality of processor subunits to a corresponding dedicated memory group; and a second plurality of buses Bus bars, each of which connects one of the plurality of processor subunits to the plurality of The other of a processor subunit.

Synchronization using software

As explained above, the hardware chip of the present invention can use software instead of hardware to manage data transmission. In particular, because the timing of the transmission on the bus, the reading and writing of the memory, and the calculation of the processor sub-unit is set by the sub-series of instructions executed by the processor sub-unit, the hardware of the present invention is The body chip can execute code to prevent conflicts on the bus. Therefore, the hardware chip of the present invention can avoid the conventional hardware mechanism used to manage data transmission (such as the network controller in the chip, the packet parser and the packet transmitter between the processor sub-units, and the bus Arbiter, multiple buses used to avoid arbitration, or the like).

If the hardware chip of the present invention conventionally transmits data, using a bus to connect N processor sub-units will require bus arbitration or wide MUX controlled by the arbiter. In fact, as described above, the embodiments of the present invention can use only wires, optical cables, or the like between the processor sub-units of the bus, wherein the processor sub-units individually execute code to Avoid conflicts on the bus. Therefore, the embodiments of the present invention can save space on the substrate and material cost and efficiency loss (for example, power and time consumption due to arbitration). Compared with other architectures that use a first-in-first-out (FIFO) controller and/or mailbox, the efficiency and space gains are even greater.

In addition, as explained above, in addition to one or more processing elements, each processor sub-unit may also include one or more accelerators. In some embodiments, the accelerator can read and write from the bus instead of the processing element. In these embodiments, additional efficiency can be obtained by allowing the accelerator to transmit data during the same cycle in which the processing element performs one or more calculations. However, these embodiments require additional materials for the accelerator. For example, additional transistors may be required for manufacturing accelerators.

The program code may also consider the internal behavior of the processor sub-units (for example, including processing elements and/or accelerators that form part of the processor sub-units), including timing and latency. For example, compile The processor (as described below) can perform preprocessing that considers timing and latency when generating a sub-series of commands that control data transmission.

In one example, a plurality of processor sub-units may be assigned the task of computing a neural network layer that contains a plurality of neurons that are all connected to a layer before a larger plurality of neurons. Assuming that the data of the previous layer is evenly distributed among a plurality of processor subunits, one way to perform this calculation can be to configure each processor subunit to transmit the data of the previous layer to the main bus in turn, and then Each processor subunit multiplies this data by the weight of the corresponding neuron implemented by the subunit. Because each processor subunit calculates more than one neuron, each processor subunit will transmit the data of the previous layer several times, which is equal to the number of neurons. Therefore, the code of each processor sub-unit is different from the code used in other processor sub-units because the sub-units will be transmitted at different times.

In some embodiments, the distributed processor may include: a substrate (for example, a semiconductor substrate such as silicon and/or a circuit board such as a flexible circuit board); a memory array disposed on the substrate, the memory array It includes a plurality of discrete memory groups; and a processing array disposed on the substrate. The processing array includes a plurality of processor subunits, as depicted in, for example, FIG. 7A and FIG. 7B. As explained above, each of the processor subunits can be associated with a corresponding dedicated memory group among the plurality of discrete memory groups. In addition, as depicted in, for example, FIGS. 7A and 7B, the distributed processor may further include a plurality of bus bars, each of which is connected to one of the plurality of processor subunits To at least another of the plurality of processor subunits.

As explained above, these multiple buses can be controlled by software. Therefore, the plurality of buses may not contain sequential hardware logic components, so that the data transmission between the processor subunits and across the corresponding ones of the plurality of buses is not controlled by the sequential hardware logic components. In one example, the plurality of buses may not contain a bus arbiter, so that data transmission between processor subunits and across corresponding ones of the plurality of buses is not controlled by the bus arbiter.

In some embodiments, as depicted in, for example, FIG. 7A and FIG. 7B, the distributed processor may further include a second plurality of buses, and the second plurality of buses may be one of the plurality of processor subunits. Connect to the corresponding dedicated memory bank. Similar to the plurality of buses described above, the second plurality of buses may not contain sequential hardware logic components, so that the data transfer between the processor subunit and the corresponding dedicated memory bank is not affected by the sequential hardware logic. Component control. In one example, the second plurality of buses may not contain a bus arbiter, so that the data transmission between the processor subunit and the corresponding dedicated memory bank is not controlled by the bus arbiter.

As used herein, the phrase "does not contain" does not necessarily imply that components such as sequential hardware logic components (for example, bus arbiter, arbitration tree, FIFO controller, mailbox, or the like) absolutely do not exist. These components can still be included in the hardware chip described as "excluding" their components. In fact, the phrase "not included" refers to the function of the hardware chip; that is, the hardware chip that "does not contain" the sequential hardware logic component controls the timing of its data transmission without using the sequential hardware logic included in it The component (if present). For example, a hardware chip executes program codes that include a sub-series of instructions that control the data transfer between the processor subunits of the hardware chip, even if the hardware chip includes sequential hardware logic components as a protection against the execution The same is true for auxiliary preventive measures for conflicts in code errors.

As explained above, the plurality of bus bars may include at least one of wires or optical fibers between corresponding ones of the plurality of processor subunits. Therefore, in one example, a distributed processor without sequential hardware logic components may only include wires or fibers, and no bus arbiter, arbitration tree, FIFO controller, mailbox, or the like.

In some embodiments, the plurality of processor subunits are configured to transmit data across at least one of the plurality of buses according to the code executed by the plurality of processor subunits. Therefore, as explained below, the compiler can organize sub-series of instructions, each of which contains code that is executed by a single processor subunit. These sub-series commands can instruct the processor sub-unit when to send data to one of the buses and when to retrieve data from the bus. When this sub-series crosses the decentralized place When the processors are executed in cooperation, the timing of the transmission between the processor sub-units can be controlled by the instructions for transmission and retrieval included in the sub-series. Therefore, the code specifies the timing of data transmission across at least one of the plurality of buses. The compiler can generate program code to be executed by a single processor subunit. In addition, the compiler can generate code to be executed by the group of processor subunits. In some cases, the compiler can treat all the processor subunits together as if the processor subunits are a super processor (for example, a distributed processor), and the compiler can generate the super processor for the super processor defined by it. The code executed by the processor/distributed processor.

As explained above and as depicted in FIGS. 7A and 7B, a plurality of processor subunits may be spatially distributed among a plurality of discrete memory groups in the memory array. Alternatively, a plurality of processor subunits may be clustered in one or more regions of the substrate, and a plurality of memory groups may be clustered in one or more other regions of the substrate. In some embodiments, a combination of spatial distribution and clustering may be used, as explained above.

In some embodiments, the distributed processor may include a substrate (for example, a semiconductor substrate including silicon and/or a circuit board such as a flexible circuit board), the substrate having a memory array disposed thereon, and the memory The array includes a plurality of discrete memory groups. The processing array can also be disposed on the substrate, and the processing array includes a plurality of processor subunits, as depicted in, for example, FIG. 7A and FIG. 7B. As explained above, each of the processor subunits can be associated with a corresponding dedicated memory group among the plurality of discrete memory groups. In addition, as depicted in, for example, FIGS. 7A and 7B, the distributed processor may further include a plurality of bus bars, each of the plurality of bus bars is one of the plurality of processor subunits Connected to the corresponding dedicated memory group among the plurality of discrete memory groups.

As explained above, these multiple buses can be controlled by software. Therefore, a plurality of buses may not contain sequential hardware logic components, so that data between the processor subunit and the corresponding dedicated discrete memory group in the plurality of discrete memory groups and across the corresponding ones in the plurality of buses pass Sending is not controlled by sequential hardware logic components. In one example, the plurality of buses may not contain a bus arbiter, so that data transmission between processor subunits and across corresponding ones of the plurality of buses is not controlled by the bus arbiter.

In some embodiments, as depicted in, for example, FIG. 7A and FIG. 7B, the distributed processor may further include a second plurality of buses, and the second plurality of buses may be one of the plurality of processor subunits. One is connected to at least another of the plurality of processor subunits. Similar to the plurality of buses described above, the second plurality of buses may not contain sequential hardware logic components, so that the data transfer between the processor subunit and the corresponding dedicated memory bank is not affected by the sequential hardware logic. Component control. In one example, the second plurality of buses may not contain a bus arbiter, so that the data transmission between the processor subunit and the corresponding dedicated memory bank is not controlled by the bus arbiter.

In some embodiments, distributed processors can use a combination of software timing components and hardware timing components. For example, a distributed processor may include a substrate (for example, a semiconductor substrate including silicon and/or a circuit board such as a flexible circuit board) having a memory array disposed thereon, the memory array including Multiple discrete memory groups. The processing array can also be disposed on the substrate, and the processing array includes a plurality of processor subunits, as depicted in, for example, FIG. 7A and FIG. 7B. As explained above, each of the processor subunits can be associated with a corresponding dedicated memory group among the plurality of discrete memory groups. In addition, as depicted in, for example, FIGS. 7A and 7B, the distributed processor may further include a plurality of bus bars, each of which is connected to one of the plurality of processor subunits To at least another of the plurality of processor subunits. In addition, as explained above, the plurality of processor sub-units can be configured to execute software that controls the timing of data transmission across the plurality of buses to avoid interfering with the plurality of buses. Data transfer conflicts on at least one of them. In this example, the software can control the timing of data transmission, but the transmission itself can be controlled at least in part by one or more hardware components.

Code division

As explained above, the hardware chip of the present invention can run program codes in parallel across the processor subunits included on the substrate forming the hardware chip. In addition, the hardware chip of the present invention can perform multi-task processing. For example, the hardware chip of the present invention can perform area multitasking processing, where a group of processor subunits of the hardware chip perform one task (for example, audio processing), and the processor subunits of the hardware chip Another group performs another task (for example, image processing). In another example, the hardware chip of the present invention can perform sequential multitasking, wherein one or more processor subunits of the hardware chip perform one task during a first time period and perform another during a second time period. One task. A combination of regional multitasking and sequential multitasking can also be used, so that one task can be assigned to the first group processor subunit during the first time period, and another task can be assigned during the first time period To the second group of processor sub-units, thereafter, a third task can be assigned to the processor sub-units included in the first group and the second group during the second time period.

In order to organize the machine code for execution on the memory chip of the present invention, the machine code can be divided among the processor subunits of the memory chip. For example, the processor on the memory chip may include a substrate and a plurality of processor sub-units disposed on the substrate. The memory chip may further include a plurality of corresponding memory groups disposed on the substrate, each of the plurality of processor sub-units is connected to any other of the plurality of processor sub-units At least one dedicated memory bank shared by the processor sub-units. Each processor subunit on the memory chip can be configured to execute a series of instructions independently of other processor subunits. Each series of instructions can be executed by the following operations: configure one or more general processing elements of the processor subunit according to the code defining the series of instructions and/or according to the code provided in the code defining the series of instructions The sequence of the processor subunit activates one or more special processing elements (for example, one or more accelerators).

Therefore, each series of instructions can define a series of tasks to be executed by a single processor subunit. A single task may include instructions in an instruction set defined by the architecture of one or more processing elements in a processor subunit. For example, the processor sub-unit may include a specific register, and a single task It can push data to the register, obtain data from the register, perform arithmetic functions on the data in the register, perform logical operations on the data in the register, or the like. In addition, the processor subunit can be configured for any number of operands, such as 0 operand processor subunit (also known as "stacker"), 1 operand processor subunit (also known as accumulator Computer), 2-operand processor subunit (such as RISC), 3-operand processor subunit (such as complex instruction set computer (CISC)), or the like. In another example, the processor subunit may include one or more accelerators, and a single task may activate an accelerator to perform a specific function, such as a MAC function, a MAX function, a MAX-0 function, or the like.

The series of commands may further include tasks for reading and writing the dedicated memory bank of the memory chip. For example, a task may include writing a piece of data to the memory bank of the processor subunit dedicated to performing the task, reading a piece of data from the memory bank of the processor subunit dedicated to performing the task, or Its similar. In some embodiments, reading and writing can be performed by the processor sub-unit and the controller of the memory bank in cooperation. For example, the processor sub-unit can perform read or write tasks by sending control signals to the controller to perform read or write. In some embodiments, the control signal may include specific addresses for reading and writing. Alternatively, the processor sub-unit can listen to the memory controller to select an address that can be used for reading and writing.

Additionally or alternatively, reading and writing can be performed by one or more accelerators in cooperation with the controller of the memory bank. For example, the accelerators can generate control signals for the memory controller, which is similar to how the processor sub-units generate control signals, as described above.

In any of the above-described embodiments, the address generator can also be used to guide the reading and writing of specific addresses of the memory bank. For example, the address generator may include processing elements configured to generate memory addresses for reading and writing. The address generator can be configured to generate addresses in order to improve efficiency, for example, by writing the result of a later calculation to the same address as the result of the previous calculation that is no longer needed. Therefore, the address generator can respond to sub-units from the processor (e.g., Commands from the processing elements included therein or from one or more accelerators) or from the processor sub-units to generate control signals for the memory controller. Additionally or alternatively, the address generator can generate addresses based on some configurations or registers, such as generating a nested loop structure, so as to repeat on certain addresses in the memory in a certain pattern.

In some embodiments, each series of instructions may include a set of machine codes that define a corresponding series of tasks. Therefore, the series of tasks described above can be encapsulated in the machine code containing the series of instructions. In some embodiments, as explained below with respect to FIG. 8, the series of tasks can be defined by a compiler that is configured to distribute higher-order series of tasks as a plurality of series of tasks among a plurality of logic circuits. For example, the compiler can generate multiple series of tasks based on the higher-order series of tasks, so that the processor subunits that perform each series of tasks cooperatively perform the same functions as those outlined by the higher-order series of tasks.

As explained further below, the higher-level series of tasks can include a set of instructions written in a human-readable programming language. Correspondingly, the series of tasks of each processor subunit may include a lower-level series of tasks, and each of these tasks includes an instruction set written in machine code.

As explained above with respect to FIGS. 7A and 7B, the memory chip may further include a plurality of bus bars, and each bus bar connects one of the plurality of processor sub-units to at least another of the plurality of processor sub-units. By. In addition, as explained above, data transmission on multiple buses can be controlled by software. Therefore, data transmission across at least one of the plurality of buses can be predefined by the series of instructions included in the processor subunit connected to at least one of the plurality of buses. Therefore, one of the tasks included in the series of commands may include outputting data to one of the buses or obtaining data from one of the buses. These tasks can be performed by one of the processing elements of the processor subunit or by one or more accelerators included in the processor subunit. In the latter embodiment, the processor subunit can perform calculations or send control signals to the corresponding memory bank in the same cycle. During the cycle, the accelerator obtains data from one of the buses or places the data in On one of the bus bars.

In one example, the series of instructions included in the processor subunit connected to at least one of the plurality of buses may include a sending task, the sending task including instructions for connecting to at least one of the plurality of buses A command of the processor subunit to write data to at least one of the plurality of buses. Additionally or alternatively, the series of instructions included in the processor subunit connected to at least one of the plurality of buses may include a receiving task, the receiving task including instructions for connecting to at least one of the plurality of buses A command of the processor sub-unit to read data from at least one of the plurality of buses.

In addition to distributing the code in the processor sub-units or instead of distributing the code in the processor sub-units, data can be divided between the memory groups of the memory chip. For example, as explained above, a distributed processor on a memory chip may include a plurality of processor subunits arranged on the memory chip and a plurality of memory groups arranged on the memory chip. Each of the plurality of memory groups can be configured to store data independent of the data stored in the other of the plurality of memory groups, and one of the plurality of processor subunits It can be connected to at least one dedicated memory group among the plurality of memory groups. For example, each processor subunit can access one or more memory controllers dedicated to one or more corresponding memory banks of the processor subunit, and other processor subunits cannot access these Corresponding one or more memory controllers. Therefore, the data stored in each memory bank can be unique to the dedicated processor subunit. In addition, the data stored in each memory bank can be independent of the memories stored in other memory banks, because the memoryless controller can be shared between memory banks.

In some embodiments, as described below with respect to FIG. 8, the data stored in each of the plurality of memory groups can be defined by a compiler that is configured to distribute the data among the plurality of memories In the body group. In addition, the compiler can be configured to use a plurality of lower-level tasks distributed in the corresponding processor subunits to distribute data defined in a higher-level series of tasks among a plurality of memory banks.

As explained further below, the higher-level series of tasks can include a set of instructions written in a human-readable programming language. Correspondingly, the series of tasks of each processor subunit may include a lower-level series of tasks, and each of these tasks includes an instruction set written in machine code.

As explained above with respect to FIGS. 7A and 7B, the memory chip may further include a plurality of bus bars, and each bus bar connects one of the plurality of processor subunits to one or more of the plurality of memory banks Corresponding dedicated memory bank. In addition, as explained above, data transmission on multiple buses can be controlled by software. Therefore, data transmission across a specific bus of the plurality of buses can be controlled by the corresponding processor subunit connected to the specific bus of the plurality of buses. Therefore, one of the tasks included in the series of commands may include outputting data to one of the buses or obtaining data from one of the buses. As explained above, these tasks can be performed by (i) the processing elements of the processor sub-unit or (ii) one or more accelerators included in the processor sub-unit. In the latter embodiment, the processor subunit can perform calculations or use a bus that connects the processor subunit to other processor subunits in the same cycle. During the cycle, the accelerator is self-connected to one or more One of the buses of the corresponding dedicated memory group obtains the data or places the data on one of the buses.

Therefore, in one example, the series of instructions included in the processor subunit connected to at least one of the plurality of buses may include sending tasks. The sending task may include a processor subunit connected to at least one of the plurality of buses for writing data to at least one of the plurality of buses for storage in one or more corresponding dedicated Commands in the memory group. Additionally or alternatively, the series of instructions included in the processor subunit connected to at least one of the plurality of buses may include receiving tasks. The receiving task may include a processor subunit connected to at least one of the plurality of buses for reading data from at least one of the plurality of buses for storage in one or more corresponding dedicated memories Commands in the body group. Therefore, the sending task and the receiving task in these embodiments may include control signals, and the control signals are sent to one or the other along at least one of a plurality of buses. One of multiple corresponding dedicated memory groups or multiple memory controllers. In addition, the sending task and the receiving task can be performed by one part of the processing subunit (for example, by the processing subunit) simultaneously with the calculation or other tasks performed by another part of the processing subunit (for example, by one or more different accelerators of the processing subunit). One or more accelerators of the sub-units) execute. Examples of this simultaneous execution may include MAC relay commands, where receiving, multiplying, and sending are performed in concert.

In addition to distributing data among memory groups, specific parts of data can also be copied across different memory groups. For example, as explained above, a distributed processor on a memory chip may include a plurality of processor subunits arranged on the memory chip and a plurality of memory groups arranged on the memory chip. Each of the plurality of processor subunits can be connected to at least one dedicated memory group among the plurality of memory groups, and each of the plurality of memory groups can be grouped State to store data independent of the data stored in the others of the plurality of memory groups. In addition, at least some of the data stored in one specific memory group among the plurality of memory groups may include a copy of the data stored in at least another memory group among the plurality of memory groups. For example, the numbers, strings, or other types of data used in this series of commands can be stored in a plurality of memory banks dedicated to different processor subunits instead of being transferred from one memory bank to the memory chip The other processor subunits in it.

In one example, parallel string matching can be copied using the data described above. For example, a plurality of strings can be compared with the same string. The conventional processor can sequentially compare each of the plural character strings with the same character string. On the hardware chip of the present invention, the same character string can be copied across the memory bank, so that the processor sub-unit can compare the divided character string with the copied character string in parallel.

In some embodiments, as described below with respect to FIG. 8, at least some of the data copied across one specific memory group among the plurality of memory groups and at least another memory group among the plurality of memory groups is performed by the compiler By definition, the compiler is configured to copy data across memory banks. In addition, The compiler can be configured to use a plurality of lower-level tasks distributed among the corresponding processor subunits to copy at least some data.

The duplication of data can be applied to specific tasks where the same part of the data is reused across different calculations. By copying these parts of the data, different calculations can be distributed among the processor subunits of the memory chip for parallel execution, and each processor subunit can store these parts of the data in a dedicated memory bank And access the stored part from the dedicated memory bank (instead of pushing and obtaining the parts of the data across the bus connected to the processor subunit). In one example, at least some of the data copied across a specific memory group among the plurality of memory groups and at least another memory group among the plurality of memory groups may include neural network weights. In this example, each node in the neural network can be defined by at least one processor subunit among a plurality of processor subunits. For example, each node may include machine code executed by at least one processor subunit that defines the node. In this example, the duplication of weights allows each processor subunit to execute machine code to at least partially realize the corresponding node, while only accessing one or more dedicated memory banks (rather than executing data with other processor subunits). Send). Because the timing of reading and writing to the dedicated memory bank is independent of other processor sub-units, and the timing of data transfer between processor sub-units requires timing synchronization (for example, using software, as explained above), Therefore, duplicating the memory to avoid data transfer between processor sub-units can further improve the overall execution efficiency.

As explained above with respect to FIGS. 7A and 7B, the memory chip may further include a plurality of bus bars, and each bus bar connects one of the plurality of processor subunits to one or more of the plurality of memory banks Corresponding dedicated memory bank. In addition, as explained above, data transmission on multiple buses can be controlled by software. Therefore, data transmission across a specific bus of the plurality of buses can be controlled by the corresponding processor subunit connected to the specific bus of the plurality of buses. Therefore, one of the tasks included in the series of commands may include outputting data to one of the buses or obtaining data from one of the buses. As explained above, these tasks can be represented by (i) The processing element of the processor sub-unit or (ii) is executed by one or more accelerators included in the processor sub-unit. As explained further above, these tasks may include sending tasks and/or receiving tasks including control signals, which are sent to one or more corresponding dedicated memory banks along at least one of a plurality of buses One or more memory controllers.

FIG. 8 depicts a flowchart of a method 800 for compiling a series of instructions for execution on an exemplary memory chip of the present invention (eg, as depicted in FIGS. 7A and 7B). The method 800 can be implemented by any conventional processor (whether general purpose or special purpose).

The method 800 can be executed as part of a computer program forming a compiler. As used herein, "compiler" refers to a higher-level language (for example, a procedural language such as C, FORTRAN, BASIC or the like; an object-oriented language such as Java, C++, Pascal, Python or the like者; etc.) Any computer program that is converted into a lower-level language (for example, assembly code, object code, machine code, or the like). The compiler may allow humans to program a series of instructions in a human-readable language, and then convert the human-readable language into a machine executable language.

At step 810, the processor may assign tasks associated with the series of instructions to different processor subunits in the processor subunits. For example, the series of instructions can be divided into subgroups, and the subgroups are to be executed in parallel across processor subunits. In one example, the neural network can be divided into its nodes, and one or more nodes can be assigned to separate processor subunits. In this example, each sub-group may include a plurality of nodes connected across different layers. Therefore, the processor subunit can implement: nodes from the first layer of the neural network; nodes from the second layer connected to nodes from the first layer implemented by the same processor subunit; and Its similar. By assigning nodes based on node connections, data transfer between processor subunits can be reduced, which can lead to increased efficiency, as explained above.

As explained above as depicted in FIG. 7A and FIG. 7B, the processor sub-units may be spatially distributed among a plurality of memory banks arranged on a memory chip. Therefore, the assignment of tasks can be at least Part of it is spatial division and logical division.

At step 820, the processor may generate a task for transferring data between multiple pairs of processor sub-units of the memory chip, each pair of processor sub-units connected by a bus. For example, as explained above, such data transmission can be controlled by software. Therefore, the processor subunit can be configured to push data on the bus and obtain data on the bus at the synchronization time. The generated tasks may therefore include tasks for performing this synchronous push and acquisition of data.

As explained above, step 820 may include preprocessing to consider the internal behavior of the processor subunits, including timing and latency. For example, the processor can use the known time and latency of the processor subunit (for example, the time to push data to the bus, the time to obtain data from the bus, the latency between calculation and push or acquisition, Or the like) to ensure that the generated tasks are synchronized. Therefore, the data transfer including at least one push by one or more processor sub-units and at least one acquisition by one or more processor sub-units can occur simultaneously, without being caused by the inter-processor sub-units. Delays are caused by timing differences, latency of processor sub-units, or the like.

At step 830, the processor may group the assigned and generated tasks into a plurality of groups of sub-series of instructions. For example, the sub-series of instructions may each include a series of tasks for a single processor sub-unit to perform. Therefore, each of the plurality of groups of the sub-series of instructions may correspond to a different processor sub-unit of the plurality of processor sub-units. Therefore, steps 810, 820, and 830 may result in dividing the series of commands into a plurality of groups of sub-series commands. As explained above, step 820 can ensure synchronization of any data transmission between different groups.

At step 840, the processor may generate machine code corresponding to each of the plurality of groups of the sub-series of instructions. For example, a higher-level program code representing a sub-series of instructions can be converted into a lower-level program code that can be executed by the corresponding processor subunit, such as machine code.

At step 850, the processor may assign the generated machine code corresponding to each of the plurality of groups of the sub-series instructions to the corresponding processor sub-units of the plurality of processor sub-units according to the division. unit. For example, the processor can mark each sub-series of instructions with an identifier corresponding to the processor sub-unit. Therefore, when the sub-series commands are uploaded to the memory chip for execution (for example, by the host 350 of FIG. 3A), each sub-series can be configured with a correct processor sub-unit.

In some embodiments, the assignment of tasks associated with the series of instructions to different processor subunits in the processor subunit may depend at least in part on two or more of the processor subunits on the memory chip. The spatial proximity between the two. For example, as explained above, the efficiency can be improved by reducing the number of data transfers between processor sub-units. Therefore, the processor can minimize data transfers that move data across more than two of the processor subunits. Therefore, the processor can use the known layout of the memory chip in combination with one or more optimization algorithms (such as the greedy algorithm) in order to assign the sub-series to the processor sub-units in a way that maximizes (at least the area Ground) Proximity transfers and minimize (at least regionally) transfers to non-adjacent processor subunits.

The method 800 may include further optimization of the memory chip of the present invention. For example, the processor may group the data associated with the series of instructions based on the division and assign the data to the memory group according to the grouping. Therefore, the memory groups can store data for the sub-series of instructions assigned to each processor subunit to which each memory group is dedicated.

In some embodiments, grouping data may include determining at least a portion of the data to be copied in two or more of the memory groups. For example, as explained above, some data can be used across more than one sub-series of commands. This data can be copied across memory banks dedicated to a plurality of processor subunits assigned different sub-series commands. This optimization can further reduce data transfer across processor subunits.

The output of the method 800 can be input to the memory chip of the present invention for execution. For example, the memory chip may include a plurality of processor sub-units and a corresponding plurality of memory groups, each of the processor sub-units is connected to at least one memory group dedicated to the processor sub-unit, and the memory The processor subunits of the chip can be configured to execute the machine code generated by the method 800. As above As explained in FIG. 3A, the host 350 can input the machine code generated by the method 800 to the processor sub-unit for execution.

Subgroup and Subcontroller

In the conventional memory group, the controller is set at the group level. Each group includes a plurality of pads, and the plurality of pads are usually arranged in a rectangular manner, but may be arranged in any geometric shape. Each pad includes a plurality of memory cells, and the plurality of memory cells are usually arranged in a rectangular manner, but they can be arranged in any geometric shape. Each cell can store a single data bit (for example, depending on whether the cell is maintained at a high voltage or a low voltage).

Examples of this conventional architecture are depicted in FIG. 9 and FIG. 10. As shown in FIG. 9, at the group level, a plurality of pads (eg, pads 930-1, 930-2, 940-1, and 940-2) may form a group 900. In the conventional rectangular organization, the group 900 can be controlled across a global word line (for example, word line 950) and a global bit line (for example, bit line 960). Therefore, the column decoder 910 can select the correct word line based on the incoming control signal (for example, a request to read from an address, a request to write to an address, or the like), and the global sense amplifier 920 (and /Or the global row decoder (not shown in FIG. 9) can select the correct bit line based on the control signal. The amplifier 920 can also amplify any voltage level from the selected group during the read operation. Although depicted as using column decoders for initial selection and performing amplification along the rows, the group may additionally or alternatively use row decoders for initial selection and performing amplification along the columns.

Figure 10 depicts an example of a pad 1000. For example, the pad 1000 may form part of a memory group such as the group 900 of FIG. 9. As depicted in FIG. 10, a plurality of cells (for example, cells 1030-1, 1030-2, and 1030-3) may form a pad 1000. Each cell may include a capacitor, a transistor, or other circuit system that stores at least one data bit. For example, the cell may include a capacitor or a flip-flop that is charged to indicate "1" and discharged to indicate "0", and the flip-flop has a first state that indicates "1" and a flip-flop that indicates "0". "The second state. The conventional pad may include, for example, 512 bits×512 bits. In embodiments where the pad 1000 forms part of MRAM, ReRAM, or the like, the cell The element may include a transistor, resistor, capacitor, or other mechanism for isolating the ion or part of the material storing at least one data bit. For example, the cell may include an electrolyte ion, a part of chalcogenide glass, or the like having a first state representing "1" and a second state representing "0".

As further depicted in FIG. 10, in the conventional rectangular organization, the pad 1000 can be controlled across a regional word line (e.g., word line 1040) and a regional bit line (e.g., bit line 1050). Therefore, word line drivers (e.g., word line drivers 1020-1, 1020-2, ..., 1020-x) can be based on the data from the controller associated with the memory group (pad 1000 forms part of the memory group) The control signal (for example, a request for reading from the address, a request for writing to an address, and a renew signal) controls the selected word line to perform reading, writing, or renewing. In addition, the regional sensing amplifier (for example, the regional amplifiers 1010-1, 1010-2, ..., 1010-x) and/or the regional row decoder (not shown in FIG. 10) can control the selected positioning element lines to perform reading , Write or renew. The area sense amplifiers can also amplify any voltage level from the selected cell during the read operation. Although depicted as using a word line driver for the initial selection and performing magnification along the rows, the pad may instead use a bit line driver for the initial selection and performing magnification along the columns.

As explained above, a large number of pads are copied to form a memory bank. Memory groups can be grouped to form memory chips. For example, the memory chip may include eight to thirty-two memory banks. Therefore, pairing the processor sub-units with the memory bank on the conventional memory chip can produce only eight to thirty-two processor sub-units. Therefore, embodiments of the present invention may include memory chips with additional sub-group levels. The memory chips of the present invention can then include a processor subunit and a memory subgroup used as a dedicated memory group paired with the processor subunit to form a large number of sub-processors, which can then achieve in-memory operations High parallelism and efficiency.

In some embodiments of the present invention, the global column decoders and global sense amplifiers of the group 900 can be replaced by a sub-group controller. Therefore, the controller of the memory bank can direct the control signal to the appropriate sub-group controller instead of sending the control signal to the global column decoder and the global sense amplifier of the memory bank. Steering can be dynamically controlled or can be hardwired (e.g., via one or more logic gates). In some embodiments, the fuse can be used to prompt the controller of each subgroup or pad to block the control signal or to transfer the control signal to the appropriate subgroup or pad. In such embodiments, a fuse can therefore be used to not activate the faulty subgroup.

In one example of these embodiments, the memory chip may include a plurality of memory banks, each memory bank has a bank controller and a plurality of memory sub banks, and each memory bank has a sub bank decoder And the sub-group row decoder to allow reading and writing of the location on the memory sub-group. Each sub-group may include a plurality of memory pads, and each memory pad has a plurality of memory cells and may have a regional column decoder, a row decoder and/or a regional sense amplifier inside. The sub-group row decoders and the sub-group row decoders can handle read and write requests from the group controller or the sub-group processor subunits used for in-memory operations on the sub-group memory, as follows Described. In addition, each memory sub-group may further have a controller that is configured to determine whether to process the read request and write request from the group controller and/or forward the read request and write request to the next Level (for example, the next level of column decoders and row decoders on the pad), or block these requests, for example, to allow internal processing elements or processor subunits to access memory. In some embodiments, the set of controllers can be synchronized with the system clock. However, these subgroup controllers may not be synchronized with the system clock.

As explained above, the use of sub-groups may allow a larger number of processor sub-units to be included in the memory chip than if the processor sub-units are paired with the memory group of the conventional chip. Therefore, each sub-group may further have a processor sub-unit that uses the sub-group as a dedicated memory. As explained above, the processor sub-unit may include RISC, CISC, or other general-purpose processing sub-units and/or may include one or more accelerators. In addition, the processor subunit may include an address generator, as explained above. In any of the embodiments described above, each processor sub-unit can be configured to use column decoders and row decoders dedicated to the sub-group of the processor sub-unit to access the sub-group , Instead of using the group controller. The processor subunits associated with the subgroups can also handle memory pads (including the decoders and memory redundancy mechanisms described below) and/or determine whether to forward and therefore handle them from the upper level (e.g., Group level or memory level) read or write request.

In some embodiments, the sub-group controller may further include a register for storing the state of the sub-group. Therefore, when the register indicates that the subgroup is in use, if the subgroup controller receives a control signal from the memory controller, the subgroup controller can return an error. In an embodiment where each sub-group further includes a processor sub-unit, if the processor sub-unit in the sub-group is accessing a memory that conflicts with an external request from the memory controller, the register may Prompt an error.

Figure 11 shows an example of another embodiment of a memory bank using a sub-bank controller. In the example of FIG. 11, the group 1100 has a column decoder 1110, a row decoder 1120, and a plurality of memory subgroups (e.g., subgroup 1170a) having subgroup controllers (e.g., controllers 1130a, 1130b, and 1130c). , 1170b and 1170c). The sub-group controllers may include address solvers (for example, solvers 1140a, 1140b, and 1140c), which can determine whether to pass the request to one or more sub-groups controlled by the sub-group controller. group.

The sub-group controllers may further include one or more logic circuits (for example, logic 1150a, 1150b, and 1150c). For example, a logic circuit that includes one or more processing elements may allow one or more operations such as renewing cells in a subgroup, clearing cells in a subgroup, or the like, without having to come from outside the group 1100 The processing request. Alternatively, the logic circuit may include a processor sub-unit, as explained above, so that the processor sub-unit has any sub-group controlled by the sub-group controller as a corresponding dedicated memory. In the example of FIG. 11, the logic 1150a may have the subgroup 1170a as the corresponding dedicated memory, the logic 1150b may have the subgroup 1170b as the corresponding dedicated memory, and the logic 1150c may have the subgroup 1170c as the corresponding dedicated memory. In any of the embodiments described above, the logic circuit may have buses to sub-groups, such as bus bars 1131a, 1131b, or 1131c. As further depicted in FIG. 11, the sub-group controllers may each include a plurality of decoders, such as sub-group column decoders and sub-group row decoders, to allow higher order processing elements or processor sub-units or issuing commands The memory controller reads and writes the address on the memory subgroup. For example, the sub-group controller 1130a It includes decoders 1160a, 1160b, and 1160c, the sub-group controller 1130b includes decoders 1160d, 1160e, and 1160f, and the sub-group controller 1130c includes decoders 1160g, 1160h, and 1160i. Based on the request from the group column decoder 1110, the sub-group controller can use the decoder included in the sub-group controller to select the word line. The described system allows the processing elements or processor subunits of a subgroup to access memory without interrupting other groups and even other subgroups, thereby allowing each subgroup of processor subunits and other subgroups of processor subunits Perform memory operations in parallel.

In addition, each sub-group may include a plurality of memory pads, and each memory pad has a plurality of memory cells. For example, subgroup 1170a includes pads 1190a-1, 1190a-2,..., 1190a-x; subgroup 1170b includes pads 1190b-1, 1190b-2,..., 1190b-x; and subgroup 1170c includes pads 1190c-1, 1190c-2,..., 1190c-3. As further depicted in Figure 11, each sub-group may include at least one decoder. For example, subgroup 1170a includes decoder 1180a, subgroup 1170b includes decoder 1180b, and subgroup 1170c includes decoder 1180c. Therefore, the group row decoder 1120 can select a global bit line (for example, bit line 1121a or 1121b) based on an external request, and the sub-group selected by the group column decoder 1110 can use its row decoder based on the dedicated data from the sub-group According to the area request of the logic circuit, the area bit line (for example, bit line 1181a or 1181b) is selected. Therefore, each processor sub-unit can be configured to use the column decoder and row decoder of the sub-group to access the sub-group dedicated to the processor sub-unit without using the column decoder and the row decoder. Therefore, each processor sub-unit can access the corresponding sub-group without interrupting other sub-groups. In addition, when the request for the sub-group is outside the processor sub-unit, the sub-group decoder can reflect the accessed data to the group decoder. Alternatively, in an embodiment where each sub-group has only one row of memory pads, the regional bit lines may be the bit lines of the pads instead of the bit lines of the sub-groups.

A combination of the following can be used: an embodiment using sub-group column decoders and sub-group row decoders; and the embodiment depicted in FIG. 11. For example, the group column decoder can be eliminated, but the group row decoder is reserved and regional bit lines are used.

Figure 12 shows an example of an embodiment of a memory subgroup 1200 with a plurality of pads. Lift For example, the sub-group 1200 may represent a part of the sub-group 1100 in FIG. 11 or may represent an alternative implementation of the memory group. In the example of FIG. 12, the subgroup 1200 includes a plurality of pads (for example, pads 1240a and 1240b). In addition, each pad may include a plurality of cells. For example, the pad 1240a includes cells 1260a-1, 1260a-2, ..., 1260a-x, and the pad 1240b includes cells 1260b-1, 1260b-2, ..., 1260b-x.

Each pad can be assigned a range of addresses to be assigned to the memory cell of the pad. These addresses can be configured at production time so that the pad can be moved around and so that a faulty pad can be deactivated and left unused (for example, using one or more fuses, as explained further below).

The sub-group 1200 receives read and write requests from the memory controller 1210. Although not depicted in FIG. 12, the request from the memory controller 1210 can be filtered by the controller of the sub-group 1200 and directed to the appropriate pad of the sub-group 1200 for address resolution. Alternatively, at least a portion (for example, higher bits) of the requested address from the memory controller 1210 can be transmitted to all the pads of the subgroup 1200 (for example, the pads 1240a and 1240b), so that only the assigned bits of the pad When the address range includes the address specified in the command, each party can process the complete address and the request associated with that address. Similar to the subgroup guidance described above, the mat determination can be controlled dynamically or can be hardwired. In some embodiments, the fuse can be used to determine the address range of each pad, thereby also allowing the faulty pad to be disabled by assigning an illegal address range. The pad can be de-energized by other common methods or fuse connection in addition or alternatively.

In any of the embodiments described above, each pad of the sub-group may include a column decoder (e.g., column decoder 1230a or 1230b) for selecting the word line in the pad. In some embodiments, each pad may further include a fuse and a comparator (e.g., 1220a and 1220b). As described above, the comparator may allow each pad to determine whether to process an incoming request, and the fuse may allow each pad not to activate in the event of a failure. Alternatively, group and/or subgroup column decoders can be used instead of column decoders in each pad.

Furthermore, in any of the above-described embodiments, the ones included in the appropriate pad The row decoder (e.g., row decoder 1250a or 1250b) can select a regional bit line (e.g., bit line 1251 or 1253). The local bit line can be connected to the global bit line of the memory bank. In the embodiment where the sub-group has its own regional bit line, the regional bit line of the cell can be further connected to the regional bit line of the sub-group. Therefore, the row decoders (and/or sense amplifiers) of the cells can be passed, and then the row decoders (and/or sense amplifiers) of the sub-groups (in the case of sub-groups of row decoders and/or sense amplifiers). In the embodiment) and then read the data in the selected cell through the row decoder (and/or sense amplifier) of the group.

The pads 1200 can be copied and arranged in an array to form a memory group (or memory sub-group). For example, the memory chip of the present invention may include a plurality of memory groups, each memory group has a plurality of memory subgroups, and each memory subgroup has a position for processing a pair of memory subgroups The read and write sub-group controller. In addition, each memory sub-group may include a plurality of memory pads, each memory pad has a plurality of memory cells and has a row decoder and a row decoder (for example, as depicted in FIG. 12 ). The padding decoders and the padding decoders can handle read and write requests from the sub-group controller. For example, the mat decoders can receive all requests and determine whether to process the request based on the known address range of each mat (for example, using a comparator), or the mat decoders can be based on subgroups (or groups) The controller selects the pad and only receives requests within the known address range.

Controller data transfer

In addition to using the processing subunit to share data, any one of the memory chips of the present invention can also use a memory controller (or sub-group controller or pad controller) to share data. For example, the memory chip of the present invention may include: a plurality of memory banks (for example, SRAM bank, DRAM bank or the like), each memory bank has a set of controllers, a row of decoders, and a row of decoders , To allow reading and writing to the location on the memory bank; and a plurality of buses, which connect each controller of the plurality of group controllers to at least another control of the plurality of group controllers Device. The plurality of buses may be similar to the buses connecting the processing sub-units as described above, but the plurality of buses are directly connected to the set of controllers instead of via the processing sub-units. In addition, despite the description as The group controller is connected, but the bus bar can additionally or alternatively connect the sub-group controller and/or the pad controller.

In some embodiments, the plurality of buses can be accessed without interrupting the data transmission on the main bus of the memory bank connected to one or more processor subunits. Therefore, the memory group (or sub-group) can transmit data to the corresponding clock cycle in the same clock cycle as when transferring data to a different memory group (or sub-group) or from a different memory group (or sub-group). The processor subunit or the corresponding processor subunit transmits data. In embodiments where each controller is connected to a plurality of other controllers, the controllers may be configurable for selecting another of the other controllers for sending or receiving data. In some embodiments, each controller may be connected to at least one adjacent controller (for example, multiple pairs of spatially adjacent controllers may be connected to each other).

Redundant logic in memory circuit

The present invention generally relates to a memory chip having a main logic portion for data processing on the chip. The memory chip may include redundant logic parts, and the redundant logic parts may replace defective main logic parts to improve the manufacturing yield of the chip. Therefore, the chip may include on-chip components that allow the logic blocks in the memory chip to be configured based on individual testing of the logic parts. This feature of the chip can improve the yield, because the memory chip with a larger area dedicated to the logic part is more prone to manufacturing failures. For example, a DRAM memory chip with a large redundant logic part may be prone to manufacturing problems, which reduces the yield rate. However, the implementation of redundant logic parts can lead to improved yield and reliability, because this implementation enables manufacturers or users of DRAM memory chips to switch on or off all logic parts while maintaining high parallelism. It should be noted that here and throughout the present invention, examples of certain memory types (such as DRAM) may be identified to facilitate the explanation of the disclosed embodiments. However, it should be understood that under these circumstances, the identified memory type is not intended to be limiting. To be precise, memory types such as DRAM, flash memory, SRAM, ReRAM, PRAM, MRAM, ROM, or any other memory can be used with the disclosed embodiments, even if it is specified in a certain section of the present invention. The same is true for fewer instances.

FIG. 13 is a block diagram of an exemplary memory chip 1300 in accordance with the disclosed embodiments. The memory chip 1300 may be implemented as a DRAM memory chip. The memory chip 1300 can also be implemented as any type of volatile or non-volatile memory, such as flash memory, SRAM, ReRAM, PRAM, and/or MRAM. The memory chip 1300 may include a substrate 1301 in which an address manager 1302 is disposed, a memory array 1304 including a plurality of memory groups 1304 (a, a) to 1304 (z, z), a memory logic 1306, Business logic 1308 and redundant business logic 1310. The memory logic 1306 and the business logic 1308 may constitute a main logic block, and the redundant business logic 1310 may constitute a redundant block. In addition, the memory chip 1300 may include configuration switches, and the configuration switches may include a deactivation switch 1312 and an activation switch 1314. The inactivation switch 1312 and the activation switch 1314 can also be arranged in the substrate 1301. In this application, the memory logic 1306, business logic 1308, and redundant business logic 1310 can also be collectively referred to as "logical blocks".

The address manager 1302 may include column and row decoders or other types of memory auxiliary devices. Alternatively or in addition, the address manager 1302 may include a microcontroller or processing unit.

In some embodiments, as shown in FIG. 13, the memory chip 1300 may include a single memory array 1304, and the memory array may arrange a plurality of memory blocks in a two-dimensional array on the substrate 1301. However, in other embodiments, the memory chip 1300 may include a plurality of memory arrays 1304, and each of the memory arrays 1304 may configure memory blocks in different configurations. For example, the memory blocks (also referred to as memory banks) in at least one of the memory arrays can be arranged in a radial distribution to facilitate the address manager 1302 or the memory logic 1306 to the memory blocks Route between.

The business logic 1308 can be used to perform in-memory operations for applications that have nothing to do with the logic used to manage the memory itself. For example, the business logic 1308 can implement AI-related functions, such as floating point, integer, or MAC operations that are used as a startup function. In addition, the business logic 1308 can implement database-related functions, such as minimum, maximum, sorting, counting, and others. The memory logic 1306 can perform tasks related to memory management, including (but not limited to) read, write, and renew operations. because Therefore, business logic can be added to one or more of the group level, the mat level, or the mat group level. The business logic 1308 may have one or more address outputs and one or more data inputs/outputs. For example, the business logic 1308 can be addressed by column\row lines to the address manager 1302. However, in some embodiments, the logic block can be addressed via data input/output in addition or alternatively.

The redundant business logic 1310 may be a reproduction of the business logic 1308. In addition, the redundant business logic 1310 can be connected to the non-start switch 1312 and/or the start switch 1314, which can include a small fuse\anti-fuse, and is used to enable one of the examples (for example, the default connection Instance) logic disable or enable and enable one of the other logic blocks (for example, default disconnected instance). In some embodiments, as further described with respect to FIG. 15, the redundancy of the block may be regional within a logical block such as business logic 1308.

In some embodiments, the logic blocks in the memory chip 1300 can be connected to a subset of the memory array 1304 through dedicated buses. For example, the set of memory logic 1306, business logic 1308, and redundant business logic 1310 can be connected to the first row of memory blocks in the memory array 1304 (ie, memory blocks 1304(a,a) To 1304(a,z)). The dedicated bus allows the associated logical block to quickly access the data of the memory block without requiring the communication line to be opened via, for example, the address manager 1302.

Each of the plurality of main logic blocks can be connected to at least one of the plurality of memory groups 1304. Also, a redundant block such as a redundant commercial block 1310 can be connected to at least one of the memory instances 1304(a, a) to 1304(z, z). The redundant block can reproduce at least one of a plurality of main logic blocks, such as memory logic 1306 or business logic 1308. The inactivation switch 1312 can be connected to at least one of the plurality of main logic blocks, and the activation switch 1314 can be connected to at least one of the plurality of redundant blocks.

In these embodiments, after detecting a fault associated with one of a plurality of main logic blocks (memory logic 1306 and/or business logic 1308), the switch 1312 can be set without activation State to disable that one of the plurality of main logic blocks. At the same time, the activation switch 1314 can be configured to reproduce one of the plurality of redundant blocks to enable a redundant block, such as the redundant logic block 1310, to reproduce one of the plurality of main logic blocks.

In addition, the activation switch 1314 and the deactivation switch 1312, which can be collectively referred to as "configuration switches", can include external inputs for configuring the state of the switches. For example, the activation switch 1314 can be configured so that the activation signal in the external input generates a closed switching condition, while the deactivation switch 1312 can be configured so that the non-activation signal in the external input generates an open switching condition. In some embodiments, all configuration switches in 1300 can be preset to be disabled, and become activated or enabled after the test prompts that the associated logic block is active and the signal is applied to the external input. Alternatively, in some cases, all configuration switches in 1300 can be preset to be enabled, and can be deactivated or deactivated after the test prompts that the associated logic block is inoperative and the deactivation signal is applied to the external input. To be able to.

Regardless of whether the configuration switch is initially enabled or disabled, after detecting a fault associated with the associated logic block, the configuration switch can disable the associated logic block. Under the condition that the configuration switch is initially enabled, the state of the configuration switch can be changed to disable in order to disable the associated logic block. Under the condition of initially disabling the configuration switch, the state of the configuration switch can be maintained in its disabling state so as to disable the associated logic block. For example, the result of the operability test may indicate that a certain logic block is non-operational or that the logic block cannot be operated within certain specifications. Under these conditions, the logic block can be disabled by not enabling the corresponding configuration switch of the logic block.

In some embodiments, the configuration switch can be connected to two or more logic blocks, and can be configured to select between different logic blocks. For example, the configuration switch can be connected to both the business logic 1308 and the redundant logic block 1310. The configuration switch can enable the redundant logic block 1310 and disable the commercial logic 1308 at the same time.

Alternatively or in addition, at least one of the plurality of main logic blocks (memory logic 1306 and/or business logic 1308) can be connected to a plurality of memory banks or memories through a first dedicated connection Subset of system item 1304. Then, at least one redundant block (such as redundant business logic 1310) that reproduces at least one of the plurality of main logic blocks in the plurality of redundant blocks can be connected to the same plurality by a second dedicated connector A subset of memory groups or instances 1304.

In addition, the memory logic 1306 may have different functions and capabilities than the business logic 1308. For example, although the memory logic 1306 can be designed to enable read and write operations in the memory bank 1304, the business logic 1308 can be designed to perform in-memory operations. Therefore, if the business logic 1308 includes the first business logic block and the business logic 1308 includes the second business logic block (such as the redundant business logic 1310), it is possible to disconnect the defective business logic 1308 and reconnect the redundant Business logic 1310 without losing any ability.

In some embodiments, the configuration switch (including the non-activation switch 1312 and the activation switch 1314) can be a fuse, an anti-fuse or a programmable device (including a one-time programmable device) or other forms of non-volatile memory To implement.

FIG. 14 is a block diagram of an exemplary redundant logical block set 1400 in accordance with the disclosed embodiment. In some embodiments, the redundant logic block set 1400 may be disposed in the substrate 1301. The set of redundant logic blocks 1400 may include at least one of a business logic 1308 and a redundant business logic 1310 connected to the switches 1312 and 1314, respectively. In addition, business logic 1308 and redundant business logic 1310 can be connected to address bus 1402 and data bus 1404.

In some embodiments, as shown in FIG. 14, switches 1312 and 1314 can connect logic blocks to clock nodes. In this way, the configuration switch can connect or disconnect the logic block from the clock signal to effectively activate or deactivate the logic block. However, in other embodiments, the switches 1312 and 1314 can connect the logic block to other nodes for activation or deactivation. For example, the configuration switch can connect the logic block to a voltage supply node (for example, VCC) or to a ground node (for example, GND) or a clock signal. In this way, the logic block can be enabled or disabled by the configuration switches, because the configuration switches can generate an open circuit or cut off the power supply of the logic block.

In some embodiments, as shown in FIG. 14, the address bus 1402 and the data bus 1404 may be on opposite sides of the logical blocks, which are connected in parallel to each of the buses . In this way, the logical block assembly 1400 can facilitate the routing of different on-chip components.

In some embodiments, each of the plurality of inactivation switches 1312 couples at least one of the plurality of main logic blocks to the clock node, and each of the plurality of activation switches 1314 can connect the plurality of At least one of the redundant blocks is coupled to the clock node to allow connection/disconnection of the clock as a simple activation/non-activation mechanism.

The redundant business logic 1310 of the redundant logic block set 1400 allows the designer to select blocks that are worth duplicating based on area and routing. For example, chip designers can choose larger blocks for copying, because larger blocks can be more prone to errors. Therefore, the chip designer can decide to replicate large logic blocks. On the other hand, designers may prefer to copy smaller logic blocks, because smaller logic blocks are easy to copy without significant space loss. In addition, using the configuration in Figure 14, the designer can easily choose to copy logic blocks depending on the statistics of errors in each area.

FIG. 15 is a block diagram of an exemplary logic block 1500 in accordance with the disclosed embodiment. The logic block can be business logic 1308 and/or redundant business logic 1310. However, in other embodiments, the exemplary logic block may describe the memory logic 1306, or other components of the memory chip 1300.

The logic block 1500 presents another embodiment of using logic redundancy in a small processor pipeline. The logic block 1500 may include a register 1508, an extraction circuit 1504, a decoder 1506, and a write-back circuit 1518. In addition, the logic block 1500 may include an operation unit 1510 and a copy operation unit 1512. However, in other embodiments, the logic block 1500 may include other units that do not include the controller pipeline, but include distributed processing elements that include the required business logic.

The arithmetic unit 1510 and the copy arithmetic unit 1512 may include digital circuits capable of performing digital calculations. For example, the operation unit 1510 and the copy operation unit 1512 may include an arithmetic logic unit (ALU) to perform arithmetic and bitwise operations on binary numbers. Alternatively, the arithmetic unit 1510 and copy The arithmetic unit 1512 may include a floating-point unit (FPU) for performing arithmetic on floating-point numbers. In addition, in some embodiments, the arithmetic unit 1510 and the copy arithmetic unit 1512 can implement database-related functions, such as minimum, maximum, counting and comparison operations, and others.

In some embodiments, as shown in FIG. 15, the operation unit 1510 and the copy operation unit 1512 may be connected to the switch circuits 1514 and 1516. When activated, the switching circuits can enable or disable the arithmetic units.

In the logic block 1500, the copy operation unit 1512 can reproduce the calculation unit 1510. In addition, in some embodiments, the size of the register 1508, the extraction circuit 1504, the decoder 1506, and the write-back circuit 1518 (collectively referred to as a regional logic unit) may be smaller than the arithmetic unit 1510. Because larger components are more prone to problems during manufacturing, the designer may decide to reproduce larger units (such as arithmetic unit 1510) rather than smaller units (such as regional logic units). However, depending on the historical yield and error rate, in addition to or instead of copying the large unit (or the entire block), the designer can also choose to copy the area logic unit. For example, the arithmetic unit 1510 may be larger than the register 1508, the extraction circuit 1504, the decoder 1506, and the write-back circuit 1518, and is therefore more prone to errors. The designer can choose to copy the arithmetic unit 1510 instead of other elements in the logic block 1500 or the entire block.

The logic block 1500 may include a plurality of area configuration switches, and each of the plurality of area configuration switches is connected to at least one of the operation unit 1510 or the copy operation unit 1512. When a fault is detected in the arithmetic unit 1510, the area configuration switch can be configured to disable the arithmetic unit 1510 and enable the copy arithmetic unit 1512.

FIG. 16 shows a block diagram of an exemplary logic block connected to the bus in accordance with the disclosed embodiment. In some embodiments, the logical blocks 1602 (which can represent memory logic 1306, business logic 1308, or redundant business logic 1310) can be independent of each other, can be connected via a bus, and can be specifically addressed to these logical areas Block and start externally. For example, the memory chip 1300 may include many logic blocks, and each logic block has an ID code. However, in other embodiments, the logic block 1602 It may represent a larger unit including one or more of the memory logic 1306, the business logic 1308, or the redundant business logic 1310.

In some embodiments, each of the logical blocks 1602 may be redundant with other logical blocks 1602. This complete redundancy in which all blocks can be operated as main or redundant blocks can improve manufacturing yields because the designer can disconnect failed units while maintaining the functionality of the entire chip. For example, a designer may be able to disable logic areas that are prone to errors but maintain similar computing power, because all replicated blocks can be connected to the same address bus and data bus. For example, the initial number of logical blocks 1602 can be greater than the target capacity. Therefore, disabling some logic blocks 1602 will not affect the target capability.

The bus connected to the logic block may include an address bus 1614, a command line 1616, and a data line 1618. As shown in Figure 16, each of the logical blocks can be connected independently of each line in the bus. However, in some embodiments, the logic blocks 1602 may be connected in a hierarchical structure to facilitate routing. For example, each line in the bus can be connected to a multiplexer that routes the line to a different logic block 1602.

In some embodiments, in order to allow external access without knowing the internal chip structure (which may be changed due to enabling and disabling units), each of the logical blocks may include a fuse ID, such as fuse identification Piece 1604. The fuse identification element 1604 may include an array of switches (such as fuses) for determining the ID, and may be connected to the management circuit. For example, the fuse identifier 1604 can be connected to the address manager 1302. Alternatively, the fuse identifier 1604 may be connected to a higher memory address unit. In these embodiments, the fuse identifier 1604 may be configurable for specific addresses. For example, the fuse identification element 1604 may include a programmable non-volatile device that determines the final ID based on the command received from the management circuit.

The distributed processor on the memory chip can be designed to have the configuration depicted in FIG. 16. When the chip wakes up or when the factory test is executed as a BIST test program, the running ID code can be assigned Give the blocks in a plurality of main logic blocks (memory logic 1306 and business logic 1308) that have passed the test protocol. The test program can also assign illegal ID codes to blocks in a plurality of main logic blocks that fail the test protocol. The test program can also assign the running ID code to a block (redundant logic block 1310) among a plurality of redundant blocks that pass the test protocol. Because the redundant block replaces the failed main logical block, the block of the redundant blocks assigned to the running ID code can be equal to or greater than that of the main logical blocks assigned to the illegal ID code. Block, thereby disabling the block. In addition, each of the plurality of main logical blocks and each of the plurality of redundant blocks may include at least one fuse identifier 1604. Also, as shown in FIG. 16, the bus connected to the logic block 1602 may include command lines, data lines, and address lines.

However, in other embodiments, all logic blocks 1602 connected to the bus will start to be disabled and will not have ID codes. Test one by one, each good logic block will get a running ID code, and those logic blocks that are not working will retain illegal IDs, which will disable these blocks. In this way, redundant logic blocks can improve manufacturing yield by replacing blocks that are known to be defective during the test process.

The address bus 1614 can couple the management circuit to each of the plurality of memory groups, each of the plurality of main logic blocks, and each of the plurality of redundant blocks. These connections allow the management circuit to assign an invalid address to one of the plurality of main logic blocks and assign a valid address to one of the plurality of main logic blocks after detecting a failure associated with the main logic block (such as business logic 1308) One of a plurality of redundant blocks.

For example, as shown in FIG. 16A, illegal IDs are configured to all logical blocks 1602(a) to 1602(c) (for example, address 0xFFF). After the test, the logic blocks 1602(a) and 1602(c) are verified to be functional, while the logic block 1602(b) is not functional. In FIG. 16A, the unshaded logic blocks may indicate the logic blocks that successfully passed the functional test, and the shaded logic blocks may indicate the logic blocks that failed the functional test. Therefore, the test program will not have a valid ID for the functional logic block Change to a legal ID, and reserve an illegal ID for the inoperative logical block. As an example, in FIG. 16A, the addresses of logical blocks 1602(a) and 1602(c) are changed from 0xFFF to 0x001 and 0x002, respectively. In contrast, the address of the logical block 1602(b) is still the illegal address 0xFFF. In some embodiments, the ID is changed by programming the corresponding fuse identifier 1604.

Different results from the testing of the logic block 1602 can produce different configurations. For example, as shown in FIG. 16B, the address manager 1302 may initially assign illegal IDs to all logical blocks 1602 (ie, 0xFFF). However, the test result can suggest that the two logic blocks 1602(a) and 1602(b) are working. Under these conditions, testing of the logic block 1602(c) may not be necessary, because the memory chip 1300 may only require two logic blocks. Therefore, in order to minimize test resources, logic blocks can be tested only according to the minimum number of active logic blocks required by the product definition of 1300, so that other logic blocks are not tested. FIG. 16B also shows the unshaded logic blocks representing the tested logic blocks that passed the functional test and the shadow logic blocks representing the untested logic blocks.

In these embodiments, the production tester (external or internal, automatic or manual) or controller that executes BIST at startup can change the illegal ID to the running ID for the functioning logic block under test, and Reserve illegal IDs for untested logical blocks. As an example, in FIG. 16B, the addresses of logical blocks 1602(a) and 1602(b) are changed from 0xFFF to 0x001 and 0x002, respectively. In contrast, the address of the untested logical block 1602(c) is still the illegal address 0xFFF.

Figure 17 is a block diagram of exemplary units 1702 and 1712 connected in series in accordance with the disclosed embodiments. Figure 17 may represent the entire system or wafer. Alternatively, FIG. 17 may represent a block in a chip containing other functional blocks.

Units 1702 and 1712 may represent complete units including a plurality of logic blocks such as memory logic 1306 and/or business logic 1308. In these embodiments, the units 1702 and 1712 may also include components required to perform operations, such as the address manager 1302. However, in other embodiments, the units 1702 and 1712 may represent logical units such as business logic 1308 or redundant business logic 1310.

Figure 17 presents an embodiment in which units 1702 and 1712 may need to communicate between themselves. Under such conditions, the units 1702 and 1712 can be connected in series. However, non-working units can break the continuity between logical blocks. Therefore, when a cell needs to be disabled due to a defect, the connection between the cells may include a bypass option. The bypass option can also be part of the bypass unit itself.

In FIG. 17, cells can be connected in series (e.g., 1702(a) to 1702(c)), and a failed cell (e.g., 1702(b)) can be bypassed when it is defective. These units can be further connected in parallel with the switching circuit. For example, in some embodiments, the cells 1702 and 1712 may be connected with the switch circuits 1722 and 1728, as depicted in FIG. 17. In the example depicted in Figure 17, cell 1702(b) is defective. For example, unit 1702(b) failed the circuit functionality test. Therefore, for example, the activation switch 1314 (not shown in FIG. 17) can be used to disable the unit 1702(b), and/or the switch circuit 1722(b) can be activated to bypass the unit 1702(b) and maintain the logic block. Connectivity between.

Therefore, when a plurality of main units are connected in series, each of the plurality of units can be connected in parallel with a parallel switch. After detecting a fault associated with one of the plurality of units, a parallel switch connected to that one of the plurality of units can be activated to connect two of the plurality of units.

In other embodiments, as shown in FIG. 17, the switch circuit 1728 may include one or more sampling points that will cause one or more cycles to be delayed to maintain synchronization between different rows of cells. When a unit is disabled, a short circuit between adjacent logic blocks may cause synchronization errors with other calculations. For example, if the task requires data from both row A and row B, and each of A and B is carried by a series of independent units, disabling the unit will result in a row that will require further data management Desynchronization between. To prevent desynchronization, the sample circuit 1730 can simulate the delay caused by the de-energizing unit 1712(b). However, in some embodiments, the parallel switch may include an anti-fuse instead of the sampling circuit 1730.

18 is a method of exemplary units connected in a two-dimensional array in accordance with the disclosed embodiment Block diagram. Figure 18 may represent the entire system or wafer. Alternatively, FIG. 18 may represent a block in a chip containing other functional blocks.

The unit 1806 may represent an autonomous unit including a plurality of logic blocks such as memory logic 1306 and/or business logic 1308. However, in other embodiments, the unit 1806 may represent a logical unit such as business logic 1308. When convenient, the discussion of FIG. 18 may refer to the elements identified in FIG. 13 (eg, memory chip 1300) and discussed above.

As shown in FIG. 18, the units may be configured in a two-dimensional array, where the unit 1806 (which may include or represent one or more of the memory logic 1306, the business logic 1308, or the redundant business logic 1310) passes through the switch box 1808 and The connection boxes 1810 are interconnected. In addition, in order to control the configuration of the two-dimensional array, the two-dimensional array may include an I/O block 1804 in the periphery of the two-dimensional array.

The connection box 1810 can be a programmable and reconfigurable device that can respond to signals input from the I/O block 1804. For example, the connection box can include a plurality of input pins from the unit 1806 and can also be connected to the switch box 1808. Alternatively, the connection box 1810 may include a group of switches that connect the pins of the programmable logic cell with routing trajectories, and the switch box 1808 may include a group of switches that connect different trajectories.

In some embodiments, the connection box 1810 and the switch box 1808 can be implemented by configuration switches such as switches 1312 and 1314. In these embodiments, the connection box 1810 and the switch box 1808 can be configured by the production tester or BIST executed when the wafer is started.

In some embodiments, the connection box 1810 and the switch box 1808 can be configured after testing the circuit functionality of the unit 1806. In these embodiments, the I/O block 1804 can be used to send test signals to the unit 1806. Depending on the test result, the I/O block 1804 can send programmed signals to configure the connection box by disabling the units 1806 that have not passed the test protocol and enabling the units 1806 that have passed the test protocol. 1810 and switch box 1808.

In these embodiments, a plurality of main logic blocks and a plurality of redundant blocks may be two The dimension grid is arranged on the substrate. Therefore, each of the plurality of main units 1806 and each of the plurality of redundant blocks (such as the redundant business logic 1310) can be interconnected by the switch box 1808, and the input blocks can be placed on the two-dimensional grid. In each row of the grid and the periphery of each row.

FIG. 19 is a block diagram of an exemplary unit for complex connection in accordance with the disclosed embodiment. Figure 19 can represent the entire system. Alternatively, FIG. 19 may represent a block in a chip containing other functional blocks.

The complex connection of Figure 19 includes units 1902(a) to 1902(f) and configuration switches 1904(a) to 1904(f). The unit 1902 may represent an autonomous unit including a plurality of logic blocks such as memory logic 1306 and/or business logic 1308. However, in other embodiments, the unit 1902 may represent a logic unit such as memory logic 1306, business logic 1308, or redundant business logic 1310. The configuration switch 1904 may include any one of a non-activation switch 1312 and an activation switch 1314.

As shown in Figure 19, the complex connection may include cells 1902 in two planes. For example, a complex connection may include two independent substrates separated on the z-axis. Alternatively or in addition, the unit 1902 may be arranged in both surfaces of the substrate. For example, for the purpose of reducing the area of the memory chip 1300, the substrate 1301 may be disposed in two overlapping surfaces and connected to the configuration switch 1904 in a three-dimensional configuration. The configuration switch may include an inactivation switch 1312 and/or an activation switch 1314.

The first plane of the substrate may include the "main" unit 1902. These blocks can be preset to be enabled. In these embodiments, the second plane may include "redundant" cells 1902. These units can be preset to be disabled.

In some embodiments, the configuration switch 1904 may include an anti-fuse. Therefore, after testing the unit 1902, the block can be connected to the block of active units by switching some anti-fuse to "always on" and disabling the selected unit 1902, even if the units are on different planes. The same is true in China. In the example presented in Figure 19, one of the "main" units (unit 1902(e)) does not work. Figure 19 shows the inactive or untested blocks as shaded blocks, and the tested or active blocks can be It is unshaded. Therefore, the configuration switch 1904 is configured so that one of the logical blocks in a different plane (e.g., cell 1902(f)) becomes active. In this way, even if one of the main logic blocks is defective, the memory chip still works by replacing the spare logic unit.

FIG. 19 additionally shows that one of the cells 1902 in the second plane (ie, 1902(c)) is not tested or enabled because the main logic block functions. For example, in Figure 19, two main units 1902(a) and 1902(d) pass the functional test. Therefore, unit 1902(c) has not been tested or enabled. Therefore, FIG. 19 shows the ability to specifically select the logic block that becomes active depending on the test result.

In some embodiments, as shown in FIG. 19, not all cells 1902 in the first plane may have corresponding spare or redundant blocks. However, in other embodiments, all units may be redundant with each other to achieve complete redundancy, where all units are primary or redundant. In addition, although some implementations may follow the star network topology depicted in FIG. 19, other implementations may use parallel connections, series connections, and/or coupling different components in parallel or in series with the configuration switch.

FIG. 20 is an exemplary flowchart illustrating a redundant block enabling processing program 2000 in accordance with the disclosed embodiment. The enabling processing program 2000 can be implemented for the memory chip 1300 and particularly for the DRAM memory chip. In some embodiments, the processing program 2000 may include the following steps: testing at least one circuit functionality of each of the plurality of logic blocks on the substrate of the memory chip; identifying the plurality of main logic blocks based on the test results The faulty logic block; test at least one circuit functionality of at least one redundant or additional logic block on the substrate of the memory chip; at least one faulty logic block is disabled by applying an external signal to the inactivation switch And by applying the external signal to the activation switch to energize the at least one redundant block, the activation switch is connected to the at least one redundant block and arranged on the substrate of the memory chip. The following description of FIG. 20 further details each step of the processing program 2000.

The processing program 2000 may include testing a plurality of logic blocks such as business block 1308 (Step 2002) and a plurality of redundant blocks (for example, redundant commercial block 1310). Testing can be performed before packaging using, for example, a probing station for on-wafer testing. However, step 2000 can also be performed after packaging.

The test in step 2002 may include applying a finite sequence of test signals to each logic block in the memory chip 1300 or a subset of the logic blocks in the memory chip 1300. These test signals may include operations that request zero or one to be expected. In other embodiments, the test signal may request to read a specific address in the memory bank or write to a specific memory bank.

The test technique can be implemented in step 2002 to test the response of the logic block under the iterative process. For example, the test may involve testing the logical block by transmitting a command to write data into the memory bank and then verifying the integrity of the written data. In some embodiments, the test may include repeating the algorithm using reversal data.

In an alternative embodiment, the test of step 2002 may include running a model of a logic block to generate a target memory image based on the test instruction set. Then, the same command sequence can be executed on the logic block in the memory chip, and the result can be recorded. The simulated residual memory image can also be compared with the image obtained from the self-test, and any mismatch can be marked as a fault.

Alternatively, in step 2002, the test may include shadow modeling, in which a diagnosis may be generated, but the result may not be predicted. The reality is that tests using shadow modeling can run in parallel on both the memory chip and the simulation. For example, when a logic block in a memory chip completes a command or task, the simulation can be sent to execute the same command. Once the logic blocks in the memory chip have completed these instructions, the architectural states of the two models can be compared. If there is a mismatch, a fault is indicated.

In some embodiments, all logic blocks (including, for example, each of memory logic 1306, business logic 1308, or redundant business logic 1310) may be tested in step 2002. However, in other embodiments, only a subset of the logical blocks may be tested in different test rounds. For example, in In the first test round, only the memory logic 1306 and associated blocks can be tested. In the second round, only the business logic 1308 and associated blocks can be tested. In the third round, depending on the results of the first two rounds, the logic block associated with the redundant business logic 1310 can be tested.

The processing procedure 2000 may continue to step 2004. In step 2004, the failed logical block can be identified, and the failed redundant block can also be identified. For example, a logic block that fails the test of step 2002 can be identified as a faulty block in step 2004. However, in other embodiments, only certain faulty logic blocks can be identified initially. For example, in some embodiments, only logical blocks associated with the business logic 1308 can be identified, and a failed redundant block can only be identified if a failed redundant block is needed to replace the failed logical block. In addition, identifying the faulty block may include writing identification information of the identified faulty block on the memory bank or non-volatile memory.

In step 2006, the faulty logic block can be disabled. For example, using a configuration circuit, the faulty logic block can be disabled by disconnecting the faulty logic block from the clock, ground, and/or power node. Alternatively, the faulty logic block can be disabled by configuring the connection box to avoid the configuration of the logic block. Moreover, in other embodiments, the faulty logic block can be disabled by receiving an illegal address from the address manager 1302.

In step 2008, the redundant block that replicates the failed logical block can be identified. Even if some logical blocks have failed, in order to support the same capability of the memory chip, in step 2008, redundant blocks that are available and can replicate the failed logical block can be identified. For example, if the logic block that performs vector multiplication is determined to be faulty, in step 2008, the address manager 1302 or the on-chip controller can identify the available redundant logic block that also performs vector multiplication.

In step 2010, the redundant blocks identified in step 2008 can be energized. Compared with the disabling operation of step 2006, in step 2010, the identified redundant blocks can be energized by connecting the identified redundant blocks to the clock, ground, and/or power nodes. Alternatively, the identified redundant block can be energized by configuring the connection box with a configuration connecting the identified redundant block. Also, in other implementations In an example, the identified redundant block can be enabled by receiving the operating address during the execution time of the test program.

FIG. 21 is an exemplary flowchart illustrating an address assignment processing program 2100 in accordance with the disclosed embodiment. The address assignment process 2100 can be implemented for the memory chip 1300 and particularly for the DRAM memory chip. As described with respect to FIG. 16, in some embodiments, the logic blocks in the memory chip 1300 can be connected to the data bus and have address identification items. The processing program 2100 describes an address assignment method that disables the failed logic block and enables the logic block that passes the test. The steps described in the processing procedure 2100 will be described as being executed by the production tester or the BIST executed when the chip is started; however, other components of the memory chip 1300 and/or external devices may also execute one or more of the processing procedures 2100 Steps.

In step 2102, the tester can disable all logical blocks and redundant blocks by assigning illegal identification items to each logical block at the chip level.

In step 2104, the tester can execute the test protocol of the logic block. For example, the tester can run the test method described in step 2002 for one or more of the logic blocks in the memory chip 1300.

In step 2106, depending on the result of the test in step 2104, the tester can determine whether the logic block is defective. If the logical block is not defective (step 2106: No), the address manager can assign the running ID to the tested logical block in step 2108. If the logical block is defective (step 2106: Yes), the address manager 1302 can reserve an illegal ID for the defective logical block in step 2110.

In step 2112, the address manager 1302 can choose to reproduce the redundant logic block of the defective logic block. In some embodiments, the redundant logic block that reproduces the defective logic block may have the same components and connections as the defective logic block. However, in other embodiments, the redundant logic block may have different components and/or connections than the defective logic block, but can perform equivalent operations. For example, if the defective logic block is designed to perform vector multiplication, the selected redundant logic block will be able to Perform vector multiplication even if the selected redundant logic block does not have the same architecture as the defective cell.

In step 2114, the address manager 1302 may test the redundant block. For example, the tester can apply the testing technique applied in step 2104 to the identified redundant blocks.

In step 2116, based on the result of the test in step 2114, the tester can determine whether the redundant block is defective. In step 2118, if the redundant block is not defective (step 2116: No), the tester may assign a running ID to the identified redundant block. In some embodiments, the processing procedure 2100 may return to step 2104 after step 2118 to generate an iterative loop for testing all logic blocks in the memory chip.

If the tester determines that the redundant block is defective (step 2116: Yes), then in step 2120, the tester can determine whether the additional redundant block is available. For example, the tester can query the memory bank for information about available redundant logic blocks. If the redundant logic block is available (step 2120: Yes), the tester can return to step 2112 and identify a new redundant logic block that reproduces the defective logic block. If the redundant logic block is not available (step 2120: No), then in step 2122, the tester can generate an error signal. The error signal may include information on defective logical blocks and defective redundant blocks.

Coupled memory bank

The disclosed embodiments of the present invention also include distributed high-performance processors. The processor may include a memory controller that interfaces with the memory bank and the processing unit. The processor can be configurable to expedite the delivery of data to the processing unit for calculation. For example, if the processing unit requires two data instances to perform tasks, the memory controller can be configured so that the communication line independently provides access to information from the two data instances. The disclosed memory architecture attempts to minimize the hardware requirements associated with complex cache and complex register file solutions. Generally, the processor chip includes a cache hierarchy that allows the core to work directly with the register. However, the cache operation requires a considerable die area and consumes additional power. The disclosed memory architecture avoids the problem by adding logical components to the memory Avoid using the cache hierarchy.

The disclosed architecture also realizes the strategic (or even optimized) placement of data in the memory bank. Even if the memory bank has a single port and high potential, the disclosed memory architecture can also achieve high performance and avoid memory access bottlenecks by strategically positioning data in different blocks of the memory bank. With the goal of providing a continuous stream of data to the processing unit, the compilation optimization step can determine how the data should be stored in the memory bank for a specific or general task. Then, the memory controller that interfaces the processing unit and the memory bank can be configured to authorize access to specific processing units when they need data to perform operations.

The configuration of the memory chip can be performed by a processing unit (for example, a configuration manager) or an external interface. The configuration can also be written by the compiler or other SW tools. In addition, the configuration of the memory controller can be based on the available ports in the memory bank and the organization of the data in the memory bank. Therefore, the disclosed architecture can provide a constant data stream or simultaneous information from different memory blocks to the processing unit. In this way, computing tasks in the memory can be quickly processed by avoiding latency bottlenecks or cache memory requirements.

In addition, the data stored in the memory chip can be configured based on the compilation optimization step. Compilation may allow the creation of processing routines in which the processor efficiently assigns tasks to processing units without the delays associated with memory latency. The compilation can be executed by the compiler and transmitted to the host connected to the external interface in the substrate. Generally, the high latency and/or low port number of certain access patterns will cause data bottlenecks in the processing units that require data. However, the disclosed compilation can locate the data in the memory bank in a way that enables the processing unit to continuously receive the data even in the case of unfavorable memory types.

In addition, in some embodiments, the configuration manager can send a message to the required processing unit based on the calculation required by the task. Different processing units or logic blocks in a chip can have specialized hardware or architectures for different tasks. Therefore, depending on the task to be performed, a processing unit or a group of processing units can be selected to perform the task. The memory controller on the substrate can be configurable to send data or authorize access according to the selection of processing sub-units to improve the data transfer rate. For example, based on compilation optimization and memory Memory architecture, when processing units are needed to perform tasks, the processing units can be authorized to access the memory bank.

In addition, the chip architecture may include on-chip components that facilitate data transmission by reducing the time required to access data in the memory bank. Therefore, the present invention describes a chip architecture for a high-performance processor capable of performing specific or general tasks using simple memory instances along with compilation optimization steps. Memory instances can have high random access latency and/or low port numbers, such as those used in DRAM devices or other memory-oriented technologies, but the disclosed architecture can be realized by self-memory The continuous (or almost continuous) data flow from the body group to the processing unit overcomes these shortcomings.

In this application, simultaneous communication may refer to communication within a clock cycle. Alternatively, simultaneous communication may refer to sending information within a predetermined amount of time. For example, simultaneous communication may refer to communication within a few nanoseconds.

Figure 22 provides a block diagram of an exemplary processing device in accordance with the disclosed embodiments. FIG. 22A shows the first embodiment of the processing device 2200, in which the memory controller 2210 uses a multiplexer to connect the first memory block 2202 and the second memory block 2204. The memory controller 2210 can also be connected to at least one configuration manager 2212, a logic block 2214, and multiple accelerators 2216(a) to 2216(n). 22B shows a second embodiment of the processing device 2200, in which the memory controller 2210 uses a bus to connect the memory blocks 2202 and 2204, the bus connects the memory controller 2210 and at least one configuration manager 2212, a logic Block 2214 and multiple accelerators 2216(a) to 2216(n). In addition, the host 2230 may be external to the processing device 2200 and connected to the processing device via, for example, an external interface.

The memory blocks 2202 and 2204 may include DRAM pads or pad groups, DRAM banks, MRAM\PRAM\RERA1M\SRAM cells, flash memory pads, or other memory technologies. The memory blocks 2202 and 2204 may alternatively include non-volatile memory, flash memory devices, resistive random access memory (ReRAM) devices, or magnetoresistive random access memory (MRAM) devices.

The memory blocks 2202 and 2204 may additionally include a plurality of memory cells, which Several memory cells are arranged in columns and rows between a plurality of word lines (not shown) and a plurality of bit lines (not shown). The gate of each row of memory cells can be connected to each of the plurality of word lines. Each row of memory cells can be connected to each of the plurality of bit lines.

In other embodiments, the memory area (including memory blocks 2202 and 2204) is constructed by simple memory instances. In this application, the term "memory instance" can be used interchangeably with the term "memory block". Memory instances (or blocks) may have undesirable characteristics. For example, the memory may be a single-port memory only and may have a high random access latency. Alternatively or in addition, the memory may be inaccessible during row and row changes and face data access problems related to, for example, capacitor charging and/or circuit system settings. However, by allowing dedicated connections between memory instances and processing units and configuring data in a way that takes into account the characteristics of blocks, the architecture presented in FIG. 22 still facilitates parallel processing in memory devices.

In some device architectures, the memory instance can include several ports to facilitate parallel operation. However, in these embodiments, when the data is compiled and organized based on the chip architecture, the chip can still achieve improved performance. For example, the compiler can improve the efficiency of access in the memory area by providing instructions and organizing data placement. Therefore, even if a single-port memory is used, the memory area can still be easily accessed.

In addition, the memory blocks 2202 and 2204 can be multiple types of memory in a single chip. For example, the memory blocks 2202 and 2204 can be eFlash and eDRAM. Also, the memory block may include DRAM with ROM instances.

The memory controller 2210 may include logic circuits for handling memory access and returning the results to the rest of the module. For example, the memory controller 2210 may include an address manager and a selection device such as a multiplexer to send data between the memory block and the processing unit or authorize access to the memory block. Alternatively, the memory controller 2210 may include a double data rate (DDR) memory controller for driving DDR SDRAM, where the data is on the rise of the system's memory clock Edge and falling edge transmission.

In addition, the memory controller 2210 can constitute a dual-channel memory controller. The incorporation of dual-channel memory can facilitate the memory controller 2210 to control the parallel access lines. The parallel access lines can be configured to have the same length to facilitate data synchronization when multiple lines are used in combination. Alternatively or in addition, the parallel access lines may allow access to multiple memory ports of the memory bank.

In some embodiments, the processing device 2200 may include one or more multiplexers connectable to the processing unit. The processing units may include a configuration manager 2212, a logic block 2214, and an accelerator 2216 that can be directly connected to the multiplexer. In addition, the memory controller 2210 may include at least one data input terminal from a plurality of memory groups or blocks 2202 and 2204, and at least one data output terminal connected to each of the plurality of processing units. With this configuration, the memory controller 2210 can simultaneously receive data from the memory bank or memory blocks 2202 and 2204 via two data input ports, and simultaneously transmit the received data to at least the A selected processing unit. However, in some embodiments, at least one data input terminal and at least one data output terminal can be implemented in a single port to allow only read or write operations. In these embodiments, a single port can be implemented as a data bus including data lines, address lines, and command lines.

The memory controller 2210 can be connected to each of a plurality of memory blocks 2202 and 2204, and can also be connected to the processing unit via, for example, a selection switch. The processing units on the substrate (including the configuration manager 2212, the logic block 2214, and the accelerator 2216) can also be independently connected to the memory controller 2210. In some embodiments, the configuration manager 2212 may receive prompts for tasks to be executed, and in response, configure the memory controller 2210, accelerator 2216, and/or the configuration stored in the memory or externally supplied. Or logical block 2214. Alternatively, the memory controller 2210 can be configured by an external interface. This task may require at least one operation that can be used to select at least one selected processing unit from a plurality of processing units. Alternatively or in addition, the selection may be based at least in part on the ability of the selected processing unit to perform at least one operation. In response, the memory controller 2210 can authorize the storage of the memory bank Access, or use a dedicated bus and/or pipeline memory access to send data between at least one selected processing unit and at least two memory banks.

In some embodiments, the first memory block 2202 of the at least two memory blocks can be arranged on the first side of the plurality of processing units; and the second memory group 2204 of the at least two memory blocks can be It is arranged on the second side of the plurality of processing units opposite to the first side. In addition, the selected processing unit (for example, accelerator 2216(n)) used to perform the task can be configured to be accessed during the clock cycle when the communication line to the first memory bank or the first memory block 2202 is open The second memory group 2204. Alternatively, the selected processing unit may be configured to transmit data to the second memory block 2204 during the clock cycle when the communication line is opened to the first memory block 2202.

In some embodiments, the memory controller 2210 can be implemented as an independent component, as shown in FIG. 22. However, in other embodiments, the memory controller 2210 may be embedded in the memory area or may be placed along the accelerators 2216(a) to 2216(n).

The processing area in the processing device 2200 may include a configuration manager 2212, a logic block 2214, and accelerators 2216(a) to 2216(n). The accelerator 2216 may include multiple processing circuits with predefined functions and may be defined by a specific application. For example, the accelerator may be a vector multiplication accumulation (MAC) unit or a direct memory access (DMA) unit for memory movement between processing modules. The accelerator 2216 may also be able to calculate its own address and request data from the memory controller 2210 or write data to the memory controller. For example, the configuration manager 2212 may send a message to at least one of the accelerators 2216 that the accelerator has access to the memory bank. Then, the accelerator 2216 can configure the memory controller 2210 to deliver data or authorize access to the accelerator itself. In addition, the accelerator 2216 may include at least one arithmetic logic unit, at least one vector processing logic unit, at least one string comparison logic unit, at least one register, and at least one direct memory access device.

The configuration manager 2212 may include a digital processing circuit for configuring the accelerator 2216 and executing execution of command tasks. For example, the configuration manager 2212 can be connected to the memory controller 2210 And each of the plurality of accelerators 2216. The configuration manager 2212 may have its own dedicated memory to save the configuration of the accelerator 2216. The configuration manager 2212 can use the memory bank to retrieve commands and configurations via the memory controller 2210. Alternatively, the configuration manager 2212 can be programmed via an external interface. In some embodiments, the configuration manager 2212 can be implemented by a reduced instruction set computer on chip (RISC) or a complex CPU on chip with its own cache memory level. In some embodiments, the configuration manager 2212 can also be omitted, and the accelerator can be configured via an external interface.

The processing device 2200 may also include an external interface (not shown). The external interface allows access to the memory from the upper level (the memory bank controller, which receives commands from the external host 2230 or the on-chip host processor), or from the external host 2230 or on-chip host processor to the memory To access. This external interface allows the programmable configuration manager 2212 and the program code to be written to the memory via the memory controller 2210 for later use by the configuration manager 2212 or the units 2214 and 2216 themselves. Accelerator 2216. However, the external interface can also directly program the processing unit without routing through the memory controller 2210. In the case that the configuration manager 2212 is a microcontroller, the configuration manager 2212 can allow the autonomous memory of the program code to be loaded into the controller area memory via an external interface. The memory controller 2210 can be configured to interrupt tasks in response to requests received from the external interface.

The external interface may include a plurality of connectors associated with the logic circuit, and the connectors provide a glueless interface to various components on the processing device. The external interface may include: data I/O input terminal for data reading and output terminal for data writing; external address output terminal; external CE0 chip selection pin; low effective chip selector; byte Enable pin; pin used for waiting state of memory cycle; write enable pin; output enable valid pin; and read write enable pin. Therefore, the external interface has required input terminals and output terminals to control the processing procedure and obtain information from the processing device. For example, the external interface can comply with the JEDEC DDR standard. Alternatively or in addition, the external interface may comply with other standards, such as SPI\OSPI or UART.

In some embodiments, the external interface can be disposed on the chip substrate and can be connected to external The host 2230. The external host can access the memory blocks 2202 and 2204, the memory controller 2210, and the processing unit through the external interface. Alternatively or in addition, the external host 2230 can read and write to the memory, or can send a message to the configuration manager 2212 via read and write commands to perform operations, such as starting a processing program and/or stopping a processing program . In addition, the external host 2230 can directly configure the accelerator 2216. In some embodiments, the external host 2230 can directly perform read/write operations on the memory blocks 2202 and 2204.

In some embodiments, the configuration manager 2212 and the accelerator 2216 can be configured to use a direct bus to connect the device area and the memory area depending on the target task. For example, when the subset of accelerators can perform operations required for task execution, the subset of accelerators 2216 can be connected to the memory instance 2204. By performing this separation, it is possible to ensure that the dedicated accelerator obtains the bandwidth (BW) required by the memory blocks 2202 and 2204. In addition, this configuration with a dedicated bus allows the large memory to be split into smaller instances or blocks. This is because connecting the memory instances to the memory controller 2210 allows even in situations with high rank latency You can also quickly access the data in different memories. In order to achieve parallelization of connections, the memory controller 2210 can be connected to each of the two memory instances with a data bus, an address bus, and/or a control bus.

The aforementioned inclusions of the memory controller 2210 can eliminate the requirement for the cache memory hierarchy or complex register files in the processing device. Although the cache hierarchy can be added to obtain the added capabilities, the architecture in the processing device 2200 allows the designer to add enough memory blocks or instances based on processing operations, and manage these instances accordingly. No need to cache the memory hierarchy. For example, the architecture in the processing device 2200 can eliminate the requirement for the cache memory hierarchy by implementing pipelined memory access. In pipeline memory access, the processing unit can receive a continuous data stream in each cycle in which certain data lines can be opened (or activated) and other data lines receive or transmit data. Due to line changes, continuous data flow using independent communication lines can achieve improved execution speed and minimal latency.

In addition, the architecture disclosed in FIG. 22 implements pipelined memory access, which may change Data is organized in a small number of memory blocks and the power loss caused by line switching is saved. For example, in some embodiments, the compiler can communicate the organization of data in a memory group or a method for organizing data in a memory group to the host 2230 to facilitate access to the data during a given task. Then, the configuration manager 2212 can define which memory banks and in some cases, which ports of the memory banks can be accessed by the accelerator. This synchronization between the location of the data in the memory bank and the data access method improves the computational task by feeding the data to the accelerator with the least latency. For example, in an embodiment where the configuration manager 2212 includes RISC\CPU, the method can be implemented with offline software (SW), and then the configuration manager 2212 can be programmed to execute the method. The method can be developed in any language that can be executed by a RISC/CPU computer and can be executed on any platform. The input of this method can include the configuration of the memory behind the memory controller and the data itself, as well as the pattern of memory access. In addition, the method can be implemented in an embodiment-specific language or machine language, and can also be only a series of configuration values expressed in binary or text.

As discussed above, in some embodiments, the compiler may provide instructions to the host 2230 for organizing data in memory blocks 2202 and 2204 when preparing for pipelined memory access. The pipelined memory access usually includes the following steps: receiving a plurality of addresses of a plurality of memory banks or memory blocks 2202 and 2204; using independent data lines to access the plurality of memories according to the received addresses Group; the data from the first address is supplied to at least one of the plurality of processing units via the first communication line and the second communication line is opened to the second address, the first address is in the plurality of memories In the first memory group in the body group, the second address is in the second memory group 2204 of the plurality of memory groups; and in the second clock cycle, the second communication line The data from the second address is supplied to the at least one of the plurality of processing units and is open to the third communication line of the third address in the first memory bank in the first line. In some embodiments, the pipelined memory access can be performed with two memory blocks connected to a single port. In these embodiments, the memory controller 2210 can hide two memory blocks behind a single port, but uses a pipelined memory access method to transfer data. Input to the processing unit.

In some embodiments, the compiler can run on the host 2230 and then perform tasks. In these embodiments, the compiler may be able to determine the configuration of the data stream based on the architecture of the memory device, because the configuration will be known to the compiler.

In other embodiments, if the configuration of the memory blocks 2204 and 2202 is unknown at offline time, the pipeline method can be run on the host 2230, which can allocate the data in the memory block before starting the calculation in. For example, the host 2230 can write data directly into the memory blocks 2204 and 2202. In these embodiments, processing units such as the configuration manager 2212 and the memory controller 2210 may not have information about the required hardware before runtime. Then, it may be necessary to delay the selection of accelerator 2216 until the task starts to run. In these situations, the processing unit or memory controller 2210 can randomly select the accelerator 2216 and generate a test data access pattern, which can be modified when the task is performed.

However, when the task is known in advance, the compiler can organize the data and instructions in a memory group for the host 2230 to provide to the processing unit such as the configuration manager 2212 to set the signal connection to minimize the access latency. For example, in some situations, the accelerator 2216 may require n words at the same time. However, each memory instance supports to retrieve only m words at a time, where "m" and "n" are integers and m<n. Therefore, the compiler can place required data across different memory instances or blocks to facilitate data access. In addition, in order to avoid errors during troubleshooting, when the processing device 2200 includes multiple memory memories, the host can split data in different rows of different memory instances. The division of data allows access to the next row of data in the next example, while still using the data from the current example.

For example, accelerator 2216(a) can be configured to multiply two vectors. Each of the vectors can be stored in separate memory blocks such as memory blocks 2202 and 2204, and each vector can include multiple words. Therefore, in order to complete the task that requires the accelerator 2216(a) to perform multiplication, it may be necessary to access two memory blocks and retrieve multiple words. However, in some embodiments, the memory block is only Allows access to one word per clock cycle. For example, the memory block may have a single port. Under these conditions, in order to speed up data transmission during operation, the compiler can organize the vector zigzags in different memory blocks to allow parallel and/or simultaneous reading of the characters. In these situations, the compiler can store the words in a memory block with dedicated lines. For example, if each vector includes two words and the memory controller can directly access four memory blocks, the compiler can arrange the data in four memory blocks, each memory block Transmit one word and speed up data delivery. In addition, in an embodiment, when the memory controller 2210 can have more than a single connection to each memory block, the compiler can issue instructions to the configuration manager 2212 (or other processing unit) to access the port Specific port. In this way, the processing device 2200 can perform pipelined memory access to continuously provide data to the processing unit by simultaneously loading words in some lines and transmitting data in other lines. Therefore, this pipelined memory access avoidance can avoid latency problems.

FIG. 23 is a block diagram of an exemplary processing device 2300 in accordance with the disclosed embodiments. The block diagram shows a simplified processing device 2300, which shows a single accelerator in the form of a MAC unit 2302, a configuration manager 2304 (equivalent or similar to the configuration manager 2212), and a memory controller 2306 (equivalent or similar to The memory controller 2210) and a plurality of memory blocks 2308(a) to 2308(d).

In some embodiments, the MAC unit 2302 may be a specific accelerator for processing specific tasks. As an example, the processing device 2300 may 2D convolution as a task. Then, the configuration manager 2304 can send a message to an accelerator with appropriate hardware to perform calculations associated with the task. For example, the MAC unit 2302 may have four internal increment counters (logical adders and registers used to manage the four loops required for the convolution calculation) and a multiplication and accumulation unit. The configuration manager 2304 can send a message to the MAC unit 2302 to process the incoming data and perform tasks. The configuration manager 2304 may transmit the prompt to the MAC unit 2302 to perform the task. In these situations, the MAC unit 2302 can iterate on the calculated address, multiply the numbers, and add them to the internal register.

In some embodiments, the configuration manager 2304 can configure the accelerator, and the memory control The controller 2306 authorizes the use of the dedicated bus access block 2308 and the MAC unit 2302. However, in other embodiments, the memory controller 2306 can directly configure the accelerator based on instructions received from the configuration manager 2304 or an external interface. Alternatively or in addition, the configuration manager 2304 may pre-load several configurations and allow the accelerator to repeatedly run on different addresses with different sizes. In these embodiments, the configuration manager 2304 may include a cache memory that stores commands, which are then transmitted to at least one of a plurality of processing units such as the accelerator 2216. However, in other embodiments, the configuration manager 2304 may not include cache memory.

In some embodiments, the configuration manager 2304 or the memory controller 2306 may receive the address that needs to be accessed for the task. The configuration manager 2304 or the memory controller 2306 can check the register to determine whether the address is already in the load line to one of the memory blocks 2308. If it is in the load line, the memory controller 2306 can read the word from the memory block 2308 and transfer the word to the MAC unit 2302. If the address is not in the load line, the configuration manager 2304 can request the memory controller 2306 to load the line and send a message to the MAC unit 2302 to delay until the address is retrieved.

In some embodiments, as shown in FIG. 23, the memory controller 2306 may include two inputs forming two independent addresses. However, if more than two addresses should be accessed at the same time, and these addresses are in a single memory block (for example, the addresses are only in memory block 2308(a)), then the memory controller 2306 or Configuration manager 2304 may cause exceptions. Alternatively, when the two addresses can only be accessed via a single line, the configuration manager 2304 may return an invalid data signal. In other embodiments, the unit can delay the execution of the processing procedure until it is possible to retrieve all the required data. This can reduce overall performance. However, the compiler may be able to find the configuration and data placement that will prevent the delay.

In some embodiments, the compiler can generate a configuration or instruction set for the processing device 2300. The configuration or instruction set can configure the configuration manager 2304 and the memory controller 2306 and the accelerator 2302 to handle the need to access Multiple addresses of a single memory block but the memory block has one port. For example, the compiler can reconfigure the data in the memory block 2308 so that the processing unit Multiple rows in the memory block 2308 can be accessed.

In addition, the memory controller 2306 can also work on more than one input at the same time. For example, the memory controller 2306 may allow access to one of the memory blocks 2308 via one port and supply data when a request for a different memory block is received in the other input. Therefore, this operation can cause the accelerator 2216, which takes the exemplary 2D convolution as the task, to receive data from the dedicated communication line of the relevant memory block.

Additionally or alternatively, the memory controller 2306 or logic block may maintain a refresh counter for each memory block 2308 and handle the refresh of all rows. Having this counter allows the memory controller 2306 to be inserted into the recycle between the stalled access times of the device.

In addition, the memory controller 2306 can be configurable to perform pipelined memory access to receive addresses and open lines in the memory block, and then supply data. The pipelined memory access can provide data to the processing unit without interrupting or delaying the clock cycle. For example, although one of the memory controller 2306 or the logic block uses the right line to access data in FIG. 23, the memory controller or logic block may be transmitting data in the left line. These methods will be explained in more detail with respect to FIG. 26.

In response to the required data, the processing device 2300 can use a multiplexer and/or other switching devices to select which devices to serve to perform a given task. For example, the configuration manager 2304 can configure the multiplexer so that at least two data lines reach the MAC unit 2302. In this way, tasks that require data from multiple addresses (such as 2D convolution) can be performed faster, because vectors or words that need to be multiplied during convolution can reach the processing unit at the same time in a single clock . This data transmission method allows processing units such as accelerator 2216 to output results quickly.

In some embodiments, the configuration manager 2304 may be configurable to execute processing procedures based on task priority. For example, the configuration manager 2304 can be configured so that the running process can be completed without any interruption. In that situation, the configuration manager 2304 can provide task instructions or configuration It is provided to the accelerator 2216, so that the accelerators can run without interruption, and the multiplexer is switched only when the task is completed. However, in other embodiments, the configuration manager 2304 can interrupt the task and reconfigure the data delivery when it receives a priority task (such as a request from an external interface). However, if the memory block 2308 is sufficient, the memory controller 2306 can be configurable to use dedicated lines to send data to the processing unit or to authorize access to the processing unit. These dedicated lines are used when the task is completed. No need to change before. In addition, in some embodiments, all devices can be connected to the entity of the configuration manager 2304 through the bus, and the device can manage the access between the device itself and the bus (for example, using the same logic as the multiplexer ). Therefore, the memory controller 2306 can be directly connected to several memory instances or memory blocks.

Alternatively, the memory controller 2306 can be directly connected to the memory sub-instance. In some embodiments, each memory instance or block can be built by sub-instances (for example, DRAM can be built by pads with independent data lines arranged in multiple sub-blocks). In addition, the instance may include at least one of a DRAM pad, a DRAM, a bank, a flash memory pad, or an SRAM pad, or any other type of memory. Next, the memory controller 2306 may include dedicated lines to directly address the sub-instances, thereby minimizing the latency during pipelined memory access.

In some embodiments, the memory controller 2306 can also maintain logic (such as column\row decoder, renew logic, etc.) required by a specific memory instance, and the memory block 2308 can handle its own logic. Therefore, the memory block 2308 can obtain an address and generate a command to return\write data.

FIG. 24 depicts an exemplary memory configuration diagram consistent with the disclosed embodiment. In some embodiments, a compiler that generates code or configuration for the processing device 2200 can be executed to configure the load from the memory blocks 2202 and 2204 by pre-arranging data in each block. Method of entry. For example, the compiler can pre-configure data so that each word required by the task is related to a row of memory instances or memory blocks. But for tasks that require more memory blocks than one memory block available in the processing device 2200, the compiler can implement more than one memory location to fit the data in each memory block The method of setting. The compiler can also store data in sequence and evaluate the latency of each memory block to avoid troubleshooting and missing latency. In some embodiments, the host may be part of the processing unit, such as the configuration manager 2212, but in other embodiments, the compiler host may be connected to the processing device 2200 via an external interface. In these embodiments, the host may run a compilation function, such as the compilation function described for the compiler.

In some embodiments, the configuration manager 2212 may be a CPU or a microcontroller (uC). In these embodiments, the configuration manager 2212 may have to access the memory to retrieve the commands or instructions placed in the memory. A specific compiler can generate code and place the code in memory in a way that allows continuous commands to be stored in the same memory bank and across several memory banks, thereby allowing pipelined memory of the extracted commands Body access. In these embodiments, the configuration manager 2212 and the memory controller 2210 may be able to avoid column latency in linear execution by facilitating pipelined memory access.

The previous state of the linear execution of the program describes the method for the compiler to recognize and place instructions to allow pipelined memory execution. However, other software structures may be more complex and will require the compiler to recognize other software structures and take actions accordingly. For example, in a situation where the task requires loops and branches, the compiler can place all loop code in a single line, so that a single line can loop without the line open latent time. Then, the memory controller 2210 may not need to change lines during execution.

In some embodiments, the configuration manager 2212 may include internal cache memory or small memory. The internal cache memory can store commands executed by the configuration manager 2212 to handle branches and loops. For example, the commands in the internal cache memory can include an accelerator used to configure access to memory blocks. Instructions.

FIG. 25 is an exemplary flowchart illustrating a possible memory configuration processing procedure 2500 in accordance with the disclosed embodiment. For the convenience of describing the memory configuration processing program 2500, reference may be made to the identifiers of the components depicted in FIG. 22 and described above. In some embodiments, the processing program 2500 may be executed by a compiler, which provides instructions to a host connected via an external interface. In other embodiments Here, the processing program 2500 can be executed by a component of the processing device 2200 (such as the configuration manager 2212).

Generally speaking, the processing procedure 2500 may include: determining the number of characters required to perform the task at the same time; determining the number of characters that can be accessed from each of the plurality of memory groups at the same time; and determining the number of characters required at the same time When the number is greater than the number of characters that can be accessed at the same time, the number of characters required at the same time is divided among multiple memory groups. In addition, dividing the number of words required at the same time can include performing a cyclic organization of words and assigning a word to each memory group sequentially.

More specifically, the processing program 2500 may start in step 2502, in which the compiler may receive task specifications. The specifications include required calculations and/or priority levels.

In step 2504, the compiler may identify the accelerator or accelerator group that can perform the task. Alternatively, the compiler can generate instructions, so the processing unit (such as the configuration manager 2212) can identify the accelerator to perform the task. For example, using the required computing configuration manager 2212 can identify the accelerators in the group of accelerators 2216 that can handle tasks.

In step 2506, the compiler can determine the number of words that need to be accessed simultaneously to perform the task. For example, the multiplication of two vectors requires access to at least two vectors, and the compiler can therefore determine that the vector words must be accessed at the same time to perform operations.

In step 2508, the compiler can determine the number of loops necessary to execute the task. For example, if the task requires a convolution operation on four sub-products, the compiler can determine that at least 4 cycles will be necessary to execute the task.

In step 2510, the compiler can place the words that need to be accessed simultaneously in different memory banks. In that way, the memory controller 2210 can be configured to open to the lines of different memory instances and access the required memory blocks within the clock cycle without any need for cache memory data.

In step 2512, the compiler places the sequentially accessed words in the same memory bank. For example, in the case of four cycles of operation, the compiler can generate instructions to write the required words into a single memory block in a sequential cycle to avoid different memory blocks during execution. between Change the line.

In step 2514, the compiler generates instructions for programming processing units such as configuration manager 2212. These instructions can specify conditions for operating a switching device (such as a multiplexer) or configuring a data bus. With these commands, the configuration manager 2212 can configure the memory controller 2210 according to the task to use dedicated communication lines to send data from the memory block to the processing unit or authorize access to these memory blocks .

FIG. 26 is an exemplary flowchart illustrating a memory read processing program 2600 in accordance with the disclosed embodiment. For the convenience of describing the memory read processing program 2600, reference may be made to the identifiers of the components depicted in FIG. 22 and described above. In some embodiments, the processing procedure 2600 may be implemented by the memory controller 2210 as described below. However, in other embodiments, the processing program 2600 may be implemented by other elements in the processing device 2200 (such as the configuration manager 2212).

In step 2602, the memory controller 2210, the configuration manager 2212, or other processing unit may receive the posted data from the memory bank or a prompt to authorize access to the memory bank. The request can specify the address and memory block.

In some embodiments, the request may be received via a data bus that specifies a read command in line 2218 and a specified address in line 2220. In other embodiments, the request may be received via a demultiplexer connected to the memory controller 2210.

In step 2604, the configuration manager 2212, the host or other processing unit may query the internal register. The internal register may include information about open lines, open addresses, open memory blocks, and/or upcoming tasks to the memory bank. Based on the information in the internal register, it can be determined whether there is an open line to the memory bank and/or whether the memory block has received a request in step 2602. Alternatively or in addition, the memory controller 2210 may directly query the internal register.

If the internal register prompts that the memory bank is not loaded into the open line (step 2606: No), the processing procedure 2600 can continue to step 2616, and the line can be loaded to the memory associated with the received address Memory body group. In addition, the memory controller 2210 or a processing unit such as the configuration manager 2212 can send a delay in step 2616 to the device requesting information from the memory address. For example, if the accelerator 2216 is requesting memory information in an occupied memory block, in step 2618, the memory controller 2210 may send a delay signal to the accelerator. In step 2620, the configuration manager 2212 or the memory controller 2210 may update the internal register to indicate that the line has been opened to a new memory bank or a new memory block.

If the internal register prompts that the memory bank is loaded into the open line (step 2606: Yes), the processing procedure 2600 can continue to step 2608. In step 2608, it can be determined whether the line loaded with the memory bank is being used at a different address. If the line is being used at a different address (step 2608: Yes), this will indicate that there are two instances in a single block, and therefore, the two instances cannot be accessed at the same time. Therefore, an error or exemption signal can be sent in step 2616 to the device requesting information from the memory address. But if the line is not being used at a different address (step 2608: No), you can open the line for that address and retrieve data from the target memory bank, and continue to step 2614 to transfer the data to the request from The component of the information of the memory address.

Using the processing program 2600, the processing device 2200 can establish a direct connection between the processing unit and the memory block or memory instance containing the information required to perform the task. This organization of the data will enable information to be read from organized vectors in different memory instances, and to simultaneously retrieve information from different memory blocks when the device requests multiple such addresses.

FIG. 27 is an exemplary flowchart illustrating the execution processing program 2700 in accordance with the disclosed embodiment. To facilitate the description of the execution processing program 2700, reference may be made to the identifiers of the elements depicted in FIG. 22 and described above.

In step 2702, the compiler or a regional unit such as the configuration manager 2212 may receive prompts for tasks that need to be performed. This task may include a single operation (e.g., multiplication) or more complex operation (e.g., convolution between matrices). The task can also prompt the required calculation.

In step 2704, the compiler or configuration manager 2212 can determine the number of words required to perform the task at the same time. For example, the configuration compiler can determine that two words are needed to perform multiplication between vectors. In another example (2D convolution task), the configuration manager 2212 may determine that the convolution between matrices requires "n" by "m" words, where "n" and "m" are matrix dimensions. In addition, in step 2704, the configuration manager 2212 can also determine the number of cycles necessary to execute the task.

In step 2706, depending on the determination in step 2704, the compiler can write the words that require simultaneous access to a plurality of memory banks arranged on the substrate. For example, when the number of words that can be accessed simultaneously from one of a plurality of memory groups is less than the number of words required at the same time, the compiler can organize the data in multiple memory groups for convenience Access different required words in the clock. In addition, when the configuration manager 2212 or the compiler determines the number of cycles necessary to execute the task, the compiler can write the required words into a single memory group of the plural memory groups in a sequential cycle to Prevent line switching between memory banks.

In step 2708, the memory controller 2210 may be configured to use the first memory line to read at least one first word from the first memory group in the plurality of memory groups or blocks or to authorize the at least one Access to the first word.

In step 2170, the processing unit (for example, one of the accelerator 2216) may use at least one first word to process the task.

In step 2712, the memory controller 2210 may be configured to open the second memory line in the second memory group. For example, based on the task and using the pipelined memory access method, the memory controller 2210 can be configured to open the second memory block in the second memory block in which the information required by the task is written in step 2706 Memory cable. In some embodiments, the second memory line can be opened when the task in step 2170 is about to be completed. For example, if the task requires 100 clocks, the second memory line can be opened at the 90th clock.

In some embodiments, steps 2708 to 2712 can be performed within one line access cycle.

In step 2714, the memory controller 2210 can be configured to authorize the use of the second memory line opened in step 2710 to access the data of at least one second word from the second memory group.

In step 2176, the processing unit (e.g., one of the accelerator 2216) may use at least the second word to process the task.

In step 2718, the memory controller 2210 may be configured to open the second memory line in the first memory bank. For example, based on tasks and using a pipelined memory access method, the memory controller 2210 can be configured to open to the second memory line of the first memory block. In some embodiments, the second memory line to the first block can be opened when the task in step 2176 is about to be completed.

In some embodiments, steps 2714 to 2718 can be performed within one line access cycle.

In step 2720, the memory controller 2210 can use the second memory line in the first group or the first line in the third group and continue in different memory groups from a plurality of memory groups or blocks The first memory group reads at least one third word or authorizes access to the at least one third word.

Partially renewed

Some memory chips, such as dynamic random access memory (DRAM) chips, are reused to avoid loss of stored data (for example, using capacitors) due to voltage attenuation in capacitors or other electrical components of the chip. For example, in DRAM, each cell must be renewed from time to time (based on specific processing procedures and design) to restore the charge in the capacitor so that data will not be lost or damaged. As the memory capacity of DRAM chips increases, the amount of time required to regenerate the memory becomes significant. During a certain period of time when the memory is being renewed, the group containing the line being renewed cannot be accessed. This can result in reduced performance. In addition, the power associated with the renewal process can also be significant. Previous efforts have been made to reduce the rate of performing refresh to reduce the adverse effects associated with refreshing memory, but most of these efforts have focused on the physical layer of DRAM.

Renewing is similar to reading and writing back a row of memory. Using this principle and focusing on accessing memory patterns, the embodiments of the present invention include software and hardware technology and repairs to memory chips. Changed to use less power for renewing and reducing the amount of time during renewing memory. For example, as an overview, some embodiments may use hardware and/or software to track the line access timing and skip the most recently accessed row in the new cycle (e.g., based on timing thresholds). In another example, some embodiments may rely on software executed by the renewed controller of the memory chip to assign reads and writes so that access to the memory is non-random. Therefore, the software can more accurately control the renewal to avoid wasting the renewal cycle and/or line. These techniques can be used alone or in combination with a compiler that encodes the commands for the renewed controller and the machine code for the processor, so that the access to the memory is also non-random. Using any combination of these techniques and configurations described in detail below, the disclosed embodiments can reduce memory regeneration power requirements and/or improve system performance by reducing the amount of time during memory cell regeneration.

Figure 28 depicts an example memory chip 2800 with a refresh controller 2803 in accordance with the present invention. For example, the memory chip 2800 may include a plurality of memory groups (for example, the memory group 2801a and the like) on the substrate. In the example of FIG. 28, the substrate includes four memory banks, each of which has four lines. A line can refer to one or more of the memory chips 2800 or any other collection of memory cells within the memory chip 2800 (such as a part of a memory group or an entire row along one of the memory groups or a memory The word line in the group of the group).

In other embodiments, the substrate may include any number of memory banks, and each memory bank may include any number of lines. Some memory groups may include the same number of lines (as shown in FIG. 28), while other memory groups may include a different number of lines. As further depicted in FIG. 28, the memory chip 2800 may include a controller 2805 for receiving input to the memory chip 2800 and transmitting output from the memory chip 2800 (for example, as described above in "Code Division" description).

In some embodiments, the plurality of memory banks may include dynamic random access memory (DRAM). However, the plurality of memory banks can include any volatile memory that stores data that needs to be refreshed periodically.

As will be discussed in more detail below, the disclosed embodiments of the present invention can use counters or A resistor-capacitor circuit is used to time the renewal cycle. For example, a counter or timer can be used to count the time since the last complete recycle, and then when the counter reaches its target value, another counter can be used to repeat on all columns. The embodiment of the present invention can additionally track the access to the section of the memory chip 2800 and reduce the renewal power required. For example, although not depicted in FIG. 28, the memory chip 2800 may further include a data memory configured to store prompts for access to one or more sections of a plurality of memory banks Access information for the operation. For example, the one or more segments may include any part of the row, row, or any other grouping of memory cells in the memory chip 2800. In a specific example, the one or more segments may include at least one row of memory structures in a plurality of memory groups. The refresh controller 2803 may be configured to perform the refresh operation of the one or more sections based at least in part on the stored access information.

For example, the data storage may include one or more registers associated with a section of the memory chip 2800 (for example, the row, row, or any other grouping of memory cells in the memory chip 2800), Static random access memory (SRAM) cells, or the like. In addition, the data storage can be configured to store bits that indicate whether the associated segment has been accessed in one or more previous cycles. "Bit" can include any data structure that stores at least one bit, such as a register, SRAM cell, non-volatile memory, or the like. In addition, the bit can be set by setting the corresponding switch (or switching element, such as a transistor) of the data structure to ON (which can be equivalent to "1" or "true"). Additionally or alternatively, bits can be modified by modifying any other properties within the data structure (such as charging the floating gate of flash memory, modifying the state of one or more flip-flops in SRAM, or the like ) In order to write "1" into the data structure (or any other value of the prompt bit setting) for setting. If the bit is determined to be set as part of the renew operation of the memory controller, the renew controller 2803 may skip the renewal cycle of the associated section and clear the register associated with that part.

In another example, the data storage may include one or more sections associated with the memory chip 2800 (for example, the row, row, or any other grouping of memory cells within the memory chip 2800). Multiple non-volatile memories (for example, flash memory or the like). The non-volatile memory can be configured to store bits that indicate whether the associated segment has been accessed in one or more previous cycles.

Some embodiments may additionally or alternatively add a time stamp register to each row or group of rows (or other sections of the memory chip 2800), and the time stamp register saves the current recycle access line last moment. This means that in the case of each row access, the new controller can update the row time stamp register. Therefore, when the next renewal occurs (for example, at the end of the renewed cycle), the renewed controller can compare the stored timestamps, and if the associated section was previously within a certain period of time (for example, in the case of After accessing the stored time stamp within a certain threshold, the new controller can jump to the next section. This prevents the system from consuming new power on the recently accessed section. In addition, the new controller can continue to track the access to ensure that each section is accessed in the next cycle or renewed.

Therefore, in yet another example, the data storage may include one or more sections associated with the memory chip 2800 (for example, the row, row, or any other grouping of memory cells within the memory chip 2800). Register or non-volatile memory. These registers or non-volatile memory can be configured to store time stamps or other information that prompts the associated section of the most recently accessed, instead of using bits to prompt whether the associated section has been accessed. In this example, the refresh controller 2803 may be based on the time stamp stored in the associated register or memory and the current time (for example, from a timer, as explained below in FIGS. 29A and 29B) Whether the amount of time exceeds a predetermined threshold (for example, 8ms, 16ms, 32ms, 64ms, or the like) to determine whether to renew or access the associated section.

Therefore, the predetermined threshold may include the amount of time for the renewal cycle to ensure that the associated section is renewed (if not accessed) at least once in each renewal cycle. Alternatively, the predetermined threshold value may include an amount of time that is shorter than the amount of time required for the recycle (e.g., to ensure that any required recycle or access signals can reach the associated section before the recycle is completed). For example, the predetermined time may include 7ms for a memory chip with a renew period of 8ms, so that if the segment has not been accessed within 7ms, the renew controller will send a message that arrives at the end of the 8ms renew period. Renew or access signal of the section. In some In an embodiment, the predetermined threshold value may depend on the size of the associated section. For example, for a smaller section of the memory chip 2800, the predetermined threshold value may be smaller.

Although the memory chips are described above, the new controllers of the present invention can also be used in distributed processor architectures, such as those described in the above sections and throughout the present invention. An example of such an architecture is depicted in Figure 7A. In these embodiments, the same substrate as the memory chip 2800 may include a plurality of processing groups disposed thereon, for example, as depicted in FIG. 7A. As explained above with respect to FIG. 3A, the "processing group" can refer to two or more processor sub-units on the substrate and their corresponding memory groups. The group may represent a spatial distribution on the substrate and/or a logical grouping for the purpose of compiling program codes for execution on the memory chip 2800. Therefore, the substrate may include a memory array including a plurality of groups, such as the group 2801a shown in FIG. 28 and other groups. In addition, the substrate may include a processing array, which may include a plurality of processor sub-units (such as the sub-units 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h shown in FIG. 7A).

As further explained above with respect to FIG. 7A, each processing group may include a processor sub-unit and one or more corresponding memory groups dedicated to the processor sub-unit. In addition, in order to allow each processor sub-unit to communicate with its corresponding dedicated memory bank, the substrate may include a first plurality of buses connecting one of the processor sub-units to its corresponding dedicated memory bank.

In these embodiments, as shown in FIG. 7A, the substrate may include a substrate for connecting each processor subunit to at least another processor subunit (for example, adjacent subunits in the same column, in the same row The second plurality of bus bars adjacent to the processor sub-unit, or any other processor sub-unit on the substrate. The first plurality of buses and/or the second plurality of buses may not contain timing hardware logic components, so that the data transmission between the processor subunits and across the corresponding ones of the plurality of buses is not subject to timing Control of hardware logic components, as explained in the chapter "Synchronization using software" above.

In an embodiment where the same substrate as the memory chip 2800 may include a plurality of processing groups (for example, as depicted in FIG. 7A) disposed thereon, the processor subunit may further include bit Address generator (e.g., address generator 450 as depicted in FIG. 4). In addition, each processing group may include a processor sub-unit and one or more corresponding memory groups dedicated to the processor sub-unit. Therefore, each of the address generators can be associated with a dedicated memory group corresponding to one of the plurality of memory groups. In addition, the substrate may include a plurality of bus bars, and each bus bar connects one of the plurality of address generators to its corresponding dedicated memory bank.

Figure 29A depicts an example renewed controller 2900 in accordance with the present invention. The renewed controller 2900 can be incorporated into the memory chip of the present invention (such as the memory chip 2800 of FIG. 28). As depicted in FIG. 29A, the refresh controller 2900 may include a timer 2901, which may include an on-chip oscillator or any other timing circuit for the refresh controller 2900. In the configuration depicted in Figure 29A, the timer 2901 can trigger a recycle periodically (e.g., every 8ms, 16ms, 32ms, 64ms, or the like). Recycling can use the column counter 2903 to cycle through all columns of the corresponding memory chip, and use the adder 2901 to combine the valid bits 2905 to generate a new signal for each column. As shown in Figure 29A, bit 2905 can be fixed to 1 ("True") to ensure that each row is renewed during the cycle.

In an embodiment of the present invention, the refresh controller 2900 may include a data storage device. As described above, the data storage may include one or more temporary memories associated with a section of the memory chip 2800 (for example, the row, row, or any other grouping of memory cells within the memory chip 2800) Device or non-volatile memory. These registers or non-volatile memory can be configured to store time stamps or prompt other information of the most recently accessed associated section.

The refresh controller 2900 can use the stored information to skip the refresh of the section of the memory chip 2900. For example, if the information indicates that the section has been renewed during one or more previous renew cycles, the renew controller 2900 may skip the section in the current renew cycle. In another example, if the difference between the stored time stamp of the section and the current time is lower than the threshold value, the renew controller 2900 may skip the section in the current renew cycle. The refresh controller 2900 may further continue to track the access and refresh of the section of the memory chip 2800 through multiple refresh cycles. For example, the new controller 2900 can use timing The device 2901 updates the stored time stamp. In these embodiments, the refresh controller 2900 can be configured to use the output of the timer to clear the access information stored in the data storage after the threshold time interval. For example, in an embodiment in which the data storage stores the time stamp of the most recent access or renewal to the associated section, the controller is renewed whenever an access command or renewal signal is sent to the section 2900 can store the new time stamp in the data memory. If the data storage stores bits instead of time stamps, the timer 2901 can be configured to clear bits that have been set to last longer than the threshold time period. For example, in an embodiment where the data storage prompts that the associated section has been accessed in one or more previous cycles, whenever the timer 2901 triggers a new recycle, the new controller 2900 can clear Bits in the data storage (for example, set it to 0), the new recycling cycle is a critical number of cycles (for example, one, two) after setting the associated bit (for example, set to 1) Or the like).

The renewed controller 2900 can cooperate with other hardware of the memory chip 2800 to track the access to the section of the memory chip 2800. For example, a memory chip uses a sense amplifier to perform a read operation (e.g., as shown in FIGS. 9 and 10 above). The sense amplifiers may include a plurality of transistors configured to sense low-power signals from a section of the memory chip 2800 storing data in one or more memory cells And the sense amplifiers amplify small voltage swings to higher voltage levels, so that data can be interpreted by logic such as external CPU or GPU or integrated processor sub-units (as explained above). Although not depicted in FIG. 29A, the refresh controller 2900 can further communicate with a sense amplifier that is configured to access one or more sectors and change the state of at least one bit register. For example, when the sense amplifier accesses one or more sections, it can set the bits associated with those sections (for example, set to 1), and these bits indicate that the associated section is first Accessed in one cycle. In an embodiment where the data storage stores the time stamp of the most recent access or renewal of the associated section, when the sense amplifier accesses one or more sections, it can trigger the time stamp from the timer 2901 Write to registers, memory, or other components including data storage.

In any of the above-described embodiments, the new controller 2900 can be used with Integration of memory controllers in multiple memory banks. For example, similar to the embodiment depicted in FIG. 3A, the renewed controller 2900 can be incorporated into the logic and control sub-units associated with the memory bank or other sections of the memory chip 2800.

Figure 29B depicts another example renewed controller 2900' in accordance with the present invention. The renewed controller 2900' can be incorporated into the memory chip of the present invention (such as the memory chip 2800 of FIG. 28). Similar to the renew controller 2900, the renew controller 2900' includes a timer 2901, a column counter 2903, a valid bit 2905, and an adder 2907. In addition, the new controller 2900 may include a data storage 2909. As shown in FIG. 29B, the data storage 2909 may include one or more sections associated with the memory chip 2800 (for example, the row, row, or any other grouping of memory cells within the memory chip 2800) Register or non-volatile memory, and the state in the data storage can be configured to respond to changes in one or more sectors being accessed (for example, by sensing amplifiers and/or renewing the controller The other components of 2900' are as described above). Therefore, the refresh controller 2900' can be configured to skip the refresh of one or more sections based on the state in the data storage. For example, if the state associated with the section is activated (for example, set to 1 by turning on, changing the properties so as to store "1" or the like), then the new controller 2900' can be skipped Recycle the associated section and clear the state associated with that part. This state can be stored by at least one bit register or any other memory structure configured to store at least one data bit.

In order to ensure that the segments of the memory chip are renewed or accessed during each renewal cycle, the renewal controller 2900' can reset or clear the state in other ways to trigger the renewal signal during the next renewal cycle. In some embodiments, after the section is skipped, the renew controller 2900' may clear the associated state to ensure that the section is renewed in the next recycle. In other embodiments, the renew controller 2900' can be configured to reset the state in the data storage after the threshold time interval. For example, every time the associated state is set (for example, set to 1 by turning on, changing the properties so as to store "1" or the like), the timer 2901 exceeds the threshold time, and then re-controls Device 2900' can clear the capital The status in the material storage (for example, set it to 0). In some embodiments, the refresh controller 2900' may use a critical number of refresh cycles (e.g., one, two, or the like) or a critical number of clock cycles (e.g., two, four, or the like). Similar) instead of a threshold time.

In other embodiments, the state may include the most recently renewed or accessed time stamp of the associated section, so that if the time stamp is between the time stamp and the current time (for example, the timer 2901 from FIG. 29A and FIG. 29B) When the amount of time exceeds a predetermined threshold (for example, 8ms, 16ms, 32ms, 64ms or the like), the renew controller 2900' may send an access command or renew signal to the associated section and update the part The associated timestamp (for example, using timer 2901). Additionally or alternatively, if the renewed time indicator prompts that the last renewed time is within the predetermined time threshold, the renewed controller 2900' can be configured to skip one or more of the plurality of memory groups Renew operation of the section. In these embodiments, after skipping the renew operation with respect to one or more sections, the renew controller 2900' may be configured to change the stored renewal operations associated with one or more sections. The new time indicator, so that during the next operating cycle, the one or more sections will be renewed. For example, as described above, the refresh controller 2900' may use the timer 2901 to update the stored refresh time indicator.

Therefore, the data storage may include a time stamp register configured to store a new time indicator that prompts the time of the last new new one or more sections of the plurality of memory banks. In addition, the new controller 2900' can use the output of the timer to clear the access information stored in the data storage after the threshold time interval.

In any of the embodiments described above, access to one or more sectors may include write operations associated with one or more sectors. Additionally or alternatively, access to one or more sectors may include read operations associated with one or more sectors.

In addition, as depicted in FIG. 29B, the refresh controller 2900' may include a row counter 2903 and an adder 2907 configured to assist in updating the data storage 2909 based at least in part on the state within the data storage. The data storage 2909 may include bit tables associated with a plurality of memory banks. Lift For example, the bit table may include an array of switches (or switching elements, such as transistors) or registers (e.g., SRAM or the like) configured to store bits for the associated segment . Additionally or alternatively, the data storage 2909 may store time stamps associated with a plurality of memory banks.

In addition, the renew controller 2900' may include a renew gate 2911 configured to control whether to renew one or more sections based on the corresponding value stored in the bit table. For example, the renew gate 2911 may include a logical gate (such as "and (and) gate", which is configured to prompt the associated section when the corresponding state of the data storage 2909 is one or more previous times). In the case of renewing or accessing during the pulse cycle, the renewing signal from the column counter 2903 is invalidated. In other embodiments, the renewing gate 2911 may include a microprocessor or other circuit. It is configured to invalidate the renew signal from the column counter 2903 when the corresponding time stamp from the data storage 2909 prompts that the associated section is renewed or accessed within the predetermined threshold time value.

FIG. 30 is a flowchart of an example of a processing procedure 3000 used for partial renewal of a memory chip (for example, the memory chip 2800 of FIG. 28). The processing program 3000 can be executed by a renewed controller according to the present invention, such as the renewed controller 2900 of FIG. 29A or the renewed controller 2900' of FIG. 29B.

At step 3010, the new controller can access the information prompting the access operation to one or more sections of the plurality of memory banks. For example, as explained above with respect to FIGS. 29A and 29B, the renewed controller may include a data storage that is connected to a section of the memory chip 2800 (for example, the memory cell in the memory chip 2800). The rank, row, or any other grouping) is associated and configured to store a time stamp or prompt other information of the most recently accessed associated section.

At step 3020, the refresh controller may generate a refresh and/or access command based at least in part on the accessed information. For example, as explained above with respect to FIGS. 29A and 29B, if the accessed information prompts the last renewal or the access time is within the predetermined time threshold and/or if the accessed information prompts the last renewal or access During one or more previous clock cycles, the renew controller can skip the renew operation relative to one or more sectors of the plurality of memory banks. Additionally or alternatively, the new controller can be based on Whether the accessed information prompts the last renewal or whether the access time exceeds a predetermined threshold and/or whether the accessed information prompts the last renewal or whether the access did not occur during one or more previous clock cycles. New or access comments on related sections.

At step 3030, the renew controller may change the stored renew time indicator associated with one or more sections so that during the next operating cycle, the one or more sections will be renewed. For example, after skipping the renew operation relative to one or more sectors, the renew controller can change the information prompting the access operation of the one or more sectors so that during the next clock cycle, The one or more sections will be renewed. Therefore, after skipping the renewal cycle, the renewing controller can clear the state of the section (for example, set it to 0). Additionally or alternatively, the renew controller may set the status of the section to be renewed and/or accessed during the current cycle (for example, set to 1). In an embodiment where the information that prompts access operations to one or more sections includes a time stamp, the renew controller may update any stored information associated with the section renewed and/or accessed during the current cycle Timestamp.

The method 3000 may further include additional steps. For example, in addition to or as an alternative to step 3030, the sense amplifier can access one or more segments and can change the information associated with the one or more segments. Additionally or alternatively, the sense amplifier can send a signal to the renewed controller when the access has occurred, so that the renewed controller can update the information associated with one or more sections. As explained above, the sense amplifier may include a plurality of transistors that are configured to sense low levels from a section of a memory chip that stores data in one or more memory cells. The power signal and the sense amplifier amplify the small voltage swing to a higher voltage level, so that the data can be interpreted by logic such as an external CPU or GPU or an integrated processor sub-unit (as explained above). In this example, whenever the sense amplifier accesses one or more sections, it can set the bits associated with the section (for example, set to 1), and these bits indicate that the associated section is first Accessed in one cycle. In an embodiment where the information prompting the access operation to one or more sections includes a time stamp, each time the sense amplifier accesses one or more sections, it can trigger the timing from the new controller The time stamp of the device is written to the data storage to update the Any associated time stamps stored.

FIG. 31 is a flowchart of an example of a processing procedure 3100 for determining the renewal of a memory chip (for example, the memory chip 2800 in FIG. 28). The processing program 3100 can be implemented in a compiler conforming to the present invention. As explained above, "compiler" refers to a higher-level language (for example, a procedural language such as C, FORTRAN, BASIC or the like; an object-oriented language such as Java, C++, Pascal, Python or the like ; Etc.) Any computer program that is converted into a lower-level language (for example, assembly code, object code, machine code, or the like). The compiler may allow humans to program a series of instructions in a human-readable language, and then convert the human-readable language into a machine executable language. The compiler may include software instructions executed by one or more processors.

At step 3110, one or more processors may receive higher-level computer program codes. For example, the higher-level computer code can be encoded on one of memory (for example, non-volatile memory such as hard disk drives or the like, volatile memory such as DRAM, or the like) or Multiple files or received via a network (for example, the Internet or the like). Additionally or alternatively, the higher-level computer code can be received from the user (for example, using an input device such as a keyboard).

At step 3120, one or more processors can identify a plurality of memory segments distributed on a plurality of memory banks associated with the memory chip to be accessed by higher-level computer code. For example, one or more processors can access a data structure that defines a plurality of memory banks and a corresponding structure of the memory chip. One or more processors can access data structures from memory (for example, non-volatile memory such as hard disk drives or the like, volatile memory such as DRAM, or the like), or via a network (For example, the Internet or the like) Receive data structure. In these embodiments, the data structure is included in one or more libraries accessible by the compiler to allow the compiler to generate instructions for the specific memory chip to be accessed.

At step 3130, one or the processor may evaluate the higher-level computer code to identify a plurality of memory read commands that occur within a plurality of memory access cycles. For example, one or more A processor can identify each operation in a higher-level computer code that requires one or more read commands for the memory and/or one or more write commands for the memory. These instructions may include variable initialization, variable reassignment, logic operation on variables, input and output operations, or the like.

At step 3140, the one or more processors may cause the data associated with the plurality of memory access commands to be distributed across each of the plurality of memory segments, such that in the plurality of memory access cycles Each of them accesses each of a plurality of memory segments during each period. For example, one or more processors can customize the data structure of the memory chip to identify the memory segment, and then assign variables from higher-level code to each of the memory segments, so that Each memory segment is accessed (e.g., via writing or reading) at least once during each refresh cycle (which may include a specific number of clock cycles). In this example, one or more processors can access information indicating how many clock cycles are required for each line of higher-level code in order to assign variables from the line of higher-level code so that at a certain number of clock cycles Each memory segment is accessed (for example, by writing or reading) at least once during the cycle.

In another example, one or more processors may first generate machine code or other lower-level code from higher-level code. One or more processors can then assign variables from the lower-level code to each of the memory segments so that they can be accessed during each renewal cycle (which can include a specific number of clock cycles) (e.g., , By writing or reading) each memory section at least once. In this example, each line of lower-level code may require a single clock cycle.

In any of the examples given above, one or more processors may further assign logical operations or other commands using temporary outputs to each of the memory segments. These temporary outputs can also generate read and/or write commands so that even if the named variable has not been assigned to the assigned memory segment, the memory segment is still accessed during its recycle.

Method 3100 may further include additional steps. For example, in embodiments where variables are assigned before compilation, one or more processors can generate machine code or other relatively high-level code from higher-level code. Low-level code. In addition, one or more processors can transmit compiled code for execution by the memory chip and corresponding logic circuits. The logic circuits may include conventional circuits such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as depicted in FIG. 7A. Therefore, as described above, the substrate may include a memory array including a plurality of groups, such as the group 2801a shown in FIG. 28 and other groups. In addition, the substrate may include a processing array, which may include a plurality of processor sub-units (such as the sub-units 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h shown in FIG. 7A).

FIG. 32 is a flowchart of another example of the processing procedure 3200 for determining the renewal of a memory chip (for example, the memory chip 2800 of FIG. 28). The processing program 3200 can be implemented in a compiler conforming to the present invention. The processing program 3200 may be executed by one or more processors that execute software instructions including a compiler. The processing program 3200 can be implemented separately or in combination with the processing program 3100 of FIG. 31.

At step 3210, similar to step 3110, one or more processors may receive higher-level computer program codes. At step 3220, similar to step 3210, one or more processors can identify a plurality of memory segments distributed on a plurality of memory banks associated with the memory chip to be accessed by higher-level computer code .

At step 3230, one or more processors may evaluate higher-level computer code to identify a plurality of memory read commands each involving one or more of the plurality of memory sections. For example, one or more processors can identify each operation in higher-level computer code that requires one or more read commands for memory and/or one or more write commands for memory. These instructions may include variable initialization, variable reassignment, logic operation on variables, input and output operations, or the like.

In some embodiments, one or more processors can use logic circuits and a plurality of memory sections to simulate the execution of higher-level code. For example, the simulation may include a step-by-step pass of higher-level code, which is similar to the case of a debugger or other instruction set simulator (ISS). The simulation can further maintain the internal variables representing the addresses of multiple memory segments, which is similar to how the debugger can be maintained The protection represents the internal variables of the processor's register.

At step 3240, one or more processors can track the last memory segment from the memory segment based on the analysis of the memory access command and for each memory segment among the plurality of memory segments. Take the accumulated amount of time. For example, using the simulation described above, one or the processor can determine each access (e.g., read or write) to one or more addresses in each of a plurality of memory sections The length of time between entering). The length of time can be measured in terms of absolute time, clock cycle, or recycle (for example, determined by the known renew rate of the memory chip).

At step 3250, in response to the determination that the amount of time since the last access to any particular memory section will exceed a predetermined threshold, one or more processors may be configured to cause the At least one of the memory access command or the memory access command of the section is introduced into the higher-level computer code. For example, the one or more processors may include a refresh command for a refresh controller (for example, the refresh controller 2900 of FIG. 29A or the refresh controller 2900' of FIG. 29B) to execute. In the embodiment where the logic circuit is not embedded on the same substrate as the memory chip, one or more processors can generate a second code for sending to the memory chip separate from the lower-level code for sending to the logic circuit. New order.

Additionally or alternatively, the one or more processors may include access commands for the memory controller (which may be separate from the renewed controller or incorporated into the renewed controller) to execute. The access command may include a dummy command, which is configured to trigger a read operation to the memory section, but does not cause the logic circuit to perform any other operations on the read or write variables from the memory section .

In some embodiments, the compiler may include a combination of steps from the processing program 3100 and steps from the processing program 3200. For example, the compiler can assign variables according to step 3140 and then run the simulation described above according to step 3250 to add to any additional memory renew commands or memory access commands. This combination allows the compiler to distribute variables across as many memory sections as possible, and to generate new or access commands for any memory section that cannot be accessed within a predetermined threshold amount of time. make. In another combination example, the compiler can simulate the code according to step 3230, and assign variables according to step 3140 based on the simulation prompts any memory section that will not be accessed within a predetermined threshold amount of time. In some embodiments, the combination may further include step 3250 to allow the compiler to generate renew or access commands for any memory section that cannot be accessed within a predetermined threshold amount of time, even if the assignment according to step 3140 is completed The same is true afterwards.

The renewed controller of the present invention can allow software to be executed by logic circuits (whether it is a conventional logic circuit such as CPU and GPU or a processing group on the same substrate as a memory chip, for example, as depicted in FIG. 7A) Disable the automatic renewal performed by the renewed controller, and instead control the renewal by the executed software. Therefore, some embodiments of the present invention can provide software with known access patterns to the memory chip (for example, if the compiler can access a plurality of memory groups defining the memory chip and a data structure corresponding to the structure) . In these embodiments, the post-compilation optimizer can automatically disable renewal, and manually set renewal control only for sections of the memory chip that have not been accessed within the threshold amount of time. Therefore, similar to step 3250 described above but after compilation, the post-compilation optimizer can generate a renew command to ensure that each memory segment is accessed or renewed with a predetermined threshold amount of time.

Another example of reducing recycling cycles can include using predefined patterns of access to memory chips. For example, if the software executed by the logic circuit can control its access pattern for the memory chip, some embodiments can generate new access patterns for renewing beyond the conventional linear line. For example, if the controller determines that the software executed by the logic circuit regularly accesses every second row of memory, the renewed controller of the present invention can use an access pattern that is not renewed every second row in order to speed up Memory chips and reduce power usage.

An example of this renewed controller is shown in Figure 33. Figure 33 depicts the renewal of the controller 3300 by conforming to the example of the stored pattern configuration of the present invention. The new controller 3300 can be incorporated into the memory chip of the present invention, the memory chip having, for example, a plurality of memory banks and a plurality of memory sections included in each of the plurality of memory banks, Such as the memory chip 2800 of FIG. 28.

The new controller 3300 includes a timer 3301 (similar to the timer 2901 in FIGS. 29A and 29B), a column counter 3303 (similar to the column counter 2903 in FIGS. 29A and 29B), and an adder 3305 (similar to FIGS. 29A and 29B). Adder of 29B 2907). In addition, the renewed controller 3300 includes a data storage 3307. Different from the data storage area 2909 of FIG. 29B, the data storage 3307 can store at least one memory renew pattern, the at least one memory renew pattern to be newly included in each of the plurality of memory groups Implemented when there are multiple memory segments. For example, as depicted in FIG. 33, the data storage 3307 may include Li that defines the segments in the memory bank in rows and/or rows (for example, in the example of FIG. 33, L1, L2, L3, and L4) and Hi (for example, in the example of FIG. 33, H1, H2, H3, and H4). In addition, each section can be associated with an Inci variable (for example, in the example of FIG. 33, Inc1, Inc2, Inc3, and Inc4) that defines how the column associated with the section is incremented (for example, whether to access or Renew each column, whether to access every other column or renew, or the like). Therefore, as shown in Figure 33, the new pattern can include a table that includes a plurality of memory segment identifiers assigned by the software, and the plurality of memory segment identifiers are used to identify the specific memory group The range of a plurality of memory sections to be renewed during the recycle period, and the range of the plurality of memory sections in the specific memory group that do not need to be renewed during the recycle period.

Therefore, the data storage 3308 can define software executed by logic circuits (whether it is a conventional logic circuit such as CPU and GPU or a processing group on the same substrate as a memory chip, for example, as depicted in FIG. 7A). Select the new pattern for use. The memory refresh pattern can be configurable by software to identify which of the plurality of memory sections in the specific memory group need to be refreshed during the refresh cycle, and the plural in the specific memory group Which of the memory sections does not need to be renewed during the recirculation period. Therefore, the renew controller 3300 may renew some or all columns in the defined section that have not been accessed during the current cycle according to Inci. The new controller 3300 may skip other rows of the defined section that are set to be accessed during the current cycle.

The data storage 3308 of the renewed controller 3300 includes a plurality of memories and renewed picture In an embodiment of the case, each memory renewal pattern may represent a different renewal pattern for renewing a plurality of memory sections included in each of the plurality of memory groups. The memory renew pattern can be selectable for use in a plurality of memory segments. Therefore, the refresh controller 3300 may be configured to allow selection of which of a plurality of memory refresh patterns to be implemented during a particular refresh cycle. For example, the software executed by the logic circuit (whether it is a conventional logic circuit such as CPU and GPU or a processing group on the same substrate as the memory chip, for example, as depicted in Figure 7A) can choose different memory Renew patterns for use during one or more different renew cycles. Alternatively, the software executed by the logic circuit can select a memory regeneration pattern for use through some or all of the different regeneration cycles.

One or more variables stored in the data storage 3308 can be used to encode the memory renew pattern. For example, in an embodiment where a plurality of memory segments are arranged in rows, each memory segment identifier can be configured to identify a specific position in a row of memory where the memory should start or end. For example, in addition to Li and Hi, one or more additional variables can also define which parts of the column defined by Li and Hi are in the segment.

FIG. 34 is a flowchart of an example of a processing procedure 3400 for determining the renewal of a memory chip (for example, the memory chip 2800 in FIG. 28). The processing procedure 3100 can be implemented by software in a renewed controller (for example, the renewed controller 3300 of FIG. 33) according to the present invention.

At step 3410, the renew controller may store at least one memory renew pattern, and the at least one memory renew pattern is to be renewed in a plurality of memory areas included in each of the plurality of memory groups Implemented in a short period of time. For example, as explained above with respect to FIG. 33, the new pattern may include a table including a plurality of memory segment identifiers assigned by the software, and the plurality of memory segment identifiers are used to identify a specific memory The range of a plurality of memory segments in the body group that need to be renewed during the recycle, and the range of a plurality of memory segments in the specific memory group that do not need to be renewed during the recycle.

In some embodiments, at least one renew pattern can be coded to renew control during manufacturing. Controller (e.g., coded onto a read-only memory that is associated with the renewed controller or at least can be accessed by the renewed controller). Therefore, the refresh controller can access at least one memory refresh pattern, but does not store the at least one memory refresh pattern.

At steps 3420 and 3430, the renewal controller can use software to identify which of the plurality of memory sections in the specific memory group need to be renewed during the renewal cycle, and the plural of the specific memory group Which ones of the memory segments do not need to be renewed during the recirculation period. For example, as explained above with respect to FIG. 33, by a logic circuit (whether it is a conventional logic circuit such as CPU and GPU or a processing group on the same substrate as a memory chip, for example, as depicted in FIG. 7A) The running software can select at least one memory to renew the pattern. In addition, the refresh controller can access the selected at least one memory refresh pattern during each refresh cycle to generate a corresponding refresh signal. The renewal controller can renew some or all parts of the defined section that have not been accessed during the current cycle according to the at least one memory renewal pattern, and can skip setting to be accessed during the current cycle The other parts of the defined section.

At step 3440, the renew controller may generate a corresponding renew command. For example, as depicted in FIG. 33, the adder 3305 may include a logic circuit configured to be used in a specific area that has not been renewed according to at least one memory renew pattern in the data storage 3307 The segment renew signal is invalid. Additionally or alternatively, the microprocessor (not shown in FIG. 33) may generate a specific renewal signal based on which sections will be renewed according to the at least one memory renewal pattern in the data storage 3307.

Method 3400 may further include additional steps. For example, the renew pattern in at least one memory is configured to change with one, two or other number of renew cycles (for example, moving from L1, H1, and Inc1 to L2, H2, and Inc2, as shown in Figure 33 In the embodiment shown in), the refresh controller can access different parts of the data storage according to steps 3430 and 3440 for the next determination of the refresh signal. Similarly, if the software is executed by a logic circuit (whether it is a conventional logic circuit such as CPU and GPU or a processing group on the same substrate as a memory chip, for example, as depicted in Figure 7A), the software is self-funded The data storage device selects a new memory renew pattern for use in one or more future renew cycles, and the renew controller can access different parts of the data storage according to steps 3430 and 3440 for use in the renew signal. Next decision.

Memory chip with selectable size

When the memory chip is designed and the target is a certain capacity of the memory, changing the memory capacity to a larger or smaller size may require redesigning the product and redesigning the entire mask set. Usually, product design and market research are carried out side by side, and in some cases, product design is completed before market research is available. Therefore, there may be a disconnect between product design and actual market demand. The present invention proposes a way to flexibly provide a memory chip with a memory capacity that meets market demand. The design method may include designing the die on the wafer together with the appropriate interconnection circuit system, so that the memory chip that may contain one or more dies can be selectively cut from the wafer, so as to provide the size from a single wafer production Opportunities for memory chips with variable memory capacity.

The present invention relates to a system and method for manufacturing memory chips by cutting the memory chips from the wafer. This method can be used to produce memory chips of selectable sizes from the wafer. An example embodiment of a wafer 3501 containing die 3503 is shown in FIG. 35A. The wafer 3501 can be made of semiconductor materials (for example, silicon (Si), silicon germanium (SiGe), silicon-on-insulator (SOI), gallium nitride (GaN), aluminum nitride (AlN), aluminum gallium nitride (AlGaN), nitrogen Boron (BN), gallium arsenide (GaAs), aluminum gallium arsenide (AlGaAs), indium nitride (InN), combinations of the above and the like) are formed. The die 3503 may include any suitable circuit elements (for example, transistors, capacitors, resistors, and/or the like), and the circuit elements may include any suitable semiconductor, dielectric, or metal components. The die 3503 may be formed of a semiconductor material that may be the same as or different from the material of the wafer 3501. In addition to the die 3503, the wafer 3501 may also include other structures and/or circuit systems. In some embodiments, one or more coupling circuits may be provided and the one or more coupling circuits couple one or more of the dies together. In an example embodiment, the coupling circuit may include a bus bar shared by two or more dies 3503. In addition, the coupling The interconnection circuit may include one or more logic circuits designed to control the circuit system associated with the die 3503 and/or direct information to the die 3503/direct information from the die. In some cases, the coupling circuit may include memory access management logic. This logic can translate the logical memory address into a physical address associated with the die 3503. It should be noted that, as used herein, the term manufacturing can collectively refer to any of the steps used to build the disclosed wafers, dies, and/or chips. For example, manufacturing may refer to the simultaneous placement and formation of various dies (and any other circuit systems) included on a wafer. Manufacturing can also refer to cutting memory chips of selectable sizes from the wafer to include one die under some conditions, or multiple dies under other conditions. Of course, the term manufacturing is not intended to be limited to these examples, but may include other aspects associated with the production of any or all of the disclosed memory chips and intermediate structures.

The die 3503 or die group can be used to manufacture memory chips. The memory chip may include a distributed processor, as described in other chapters of the present invention. As shown in FIG. 35B, the die 3503 may include a substrate 3507 and a memory array disposed on the substrate. The memory array may include one or more memory cells, such as memory banks 3511A to 3511D designed to store data. In various embodiments, the memory bank may include semiconductor-based circuit elements such as transistors, capacitors, and the like. In an example embodiment, the memory bank may include multiple rows and multiple rows of storage cells. In some cases, this memory bank may have a capacity greater than one million bytes. These memory banks can include dynamic or static access memory.

The die 3503 may further include a processing array disposed on the substrate, the processing array including a plurality of processor subunits 3515A to 3515D, as shown in FIG. 35B. As described above, each memory bank may include dedicated processor subunits connected by dedicated buses. For example, the processor sub-unit 3515A is associated with the memory group 3511A via a bus or connector 3512. It should be understood that various connections between the memory groups 3511A to 3511D and the processor subunits 3515A to 3515D are possible, and only some illustrative connections are shown in FIG. 35B. In an example embodiment, the processor sub-unit can perform read/write operations on the associated memory bank, and can be further compared to storage in various memories The memory in the body group performs a renew operation or any other suitable operation.

As mentioned, the die 3503 may include a first group of buses configured to connect the processor sub-units with their corresponding memory groups. An example bus may include a set of wires or conductors that connect electrical components, and allow data and addresses to be transmitted to each memory bank and its associated processor subunits, and from each memory bank and its associated The processor subunit transmits data. In an example embodiment, the connector 3512 can serve as a dedicated bus for connecting the processor subunit 3515A to the memory bank 3511A. The die 3503 may include groups of such buses, each of which connects the processor sub-units to a corresponding dedicated memory group. In addition, die 3503 may include another group of bus bars, each bus bar connecting processor sub-units (eg, sub-units 3515A to 3515D) to each other. For example, such a bus bar may include connectors 3516A to 3516D. In various embodiments, the data for the memory banks 3511A to 3511D can be delivered via the input/output bus 3530. In an example embodiment, the input/output bus 3530 can carry data-related information and command-related information for controlling the operation of the memory cell of the die 3503. The data information may include one or more of the data stored in the memory bank, the data read from the memory bank, and the processor subunits based on operations performed relative to the data stored in the corresponding memory bank. User’s processing results, command-related information, various codes, etc.

Under various conditions, the data and commands transmitted by the input-output bus 3530 can be controlled by the input-output (IO) controller 3521. In an example embodiment, the IO controller 3521 can control the data flow from the bus 3530 to the processor sub-units 3515A to 3515D and from the processor sub-units 3515A to 3515D. The IO controller 3521 can determine which of the processor sub-units 3515A to 3515D to retrieve information. In various embodiments, the IO controller 3521 may include a fuse 3554 configured to not activate the IO controller 3521. If multiple dies are combined to form a larger memory chip (also called a multi-die memory chip, as an alternative to a single-die memory chip containing only one die), the fuse 3554 can be used . The multi-die memory chip can then use one of the IO controllers that form one of the die units of the multi-die memory chip, and at the same time by using one of the IO controllers corresponding to other die units Meta-related fuses of other IO controllers to disable other IO controllers.

As mentioned, each memory chip or pre-die or die group may include a distributed processor associated with the corresponding memory group. In some embodiments, these distributed processors may be arranged in a processing array arranged on the same substrate as a plurality of memory banks. In addition, the processing array may include one or more logic parts each including an address generator (also referred to as an address generator unit (AGU)). In some cases, the address generator may be part of at least one processor subunit. The address generator can generate memory addresses required to retrieve data from one or more memory groups associated with the memory chip. The address generation calculation may involve integer arithmetic operations, such as addition, subtraction, modulus operations, or bit shifting. The address generator can be configured to perform operations on multiple operands at once. In addition, multiple address generators can perform more than one address calculation operation at the same time. In various embodiments, the address generator may be associated with the corresponding memory bank. These address generators can be connected to their corresponding memory banks by means of corresponding bus lines.

In various embodiments, memory chips of selectable sizes can be formed from the wafer 3501 by selectively cutting different areas of the wafer. As mentioned, the wafer may include a group of dies 3503, the group including two or more than two dies included on the wafer (for example, 2, 3, 4, 5 , 10 or more than 10 dies). As will be discussed further below, in some cases, a single memory chip may be formed by cutting a portion of the wafer that includes only one die in a die group. Under these conditions, the resulting memory chip will include memory cells associated with one die. However, under other conditions, memory chips of selectable sizes may be formed to include more than one die. These memory chips can be formed by cutting regions of the wafer that include two or more of the die group included on the wafer. Under these conditions, the die and the coupling circuit that couples the die together provide a multi-die memory chip. Some additional circuit components can also be connected between the chips on the on-board ground, such as clock components, data buses or any suitable logic circuits.

In some cases, at least one controller associated with a die group can be configured to The control die group operates as a single memory chip (for example, a multi-memory cell memory chip). The controller may include one or more circuits that manage the flow of data into and from the memory chip. The memory controller may be part of the memory chip, or it may be part of a separate chip that is not directly related to the memory chip. In an example embodiment, the controller can be configured to facilitate read and write requests or other commands associated with the distributed processor of the memory chip, and can be configured to control any other suitable of the memory chip (For example, new memory chips, interaction with distributed processors, etc.). In some cases, the controller may be part of die 3503, and in other cases, the controller may be arranged adjacent to die 3503. In various embodiments, the controller may also include at least one memory controller of at least one of the memory cells included on the memory chip. In some situations, for the replication logic and memory units (for example, memory banks) that can exist on the memory chip, the protocol used to access the information on the memory chip may be unknown. The protocol can be configured to have different IDs or address ranges for full access to the data on the memory chip. Examples of chips with this protocol may include chips with a joint electronic device engineering committee (JEDEC) dual data rate (DDR) controller, where different memory banks may have different address ranges and serial peripheral interface (SPI) connections, Among them, different memory units (for example, memory groups) have different identification items (ID), and the like.

In various embodiments, multiple regions may be diced from the wafer, where each region includes one or more dies. In some cases, each partition can be used to build a multi-die memory chip. In other cases, each area to be diced from the wafer may include a single die to provide a single die memory chip. In some cases, two or more of the regions may have the same shape and have the same number of dies coupled to the coupling circuit in the same manner. Alternatively, in some example embodiments, the first group of regions can be used to form memory chips of the first type, and the second group of regions can be used to form memory chips of the second type. For example, as shown in FIG. 35C, the wafer 3501 may include a region 3505, which may include a single die, and the second region 3504 may include a group of two dies. When cutting area 3505 from wafer 3501, Single-die memory chips will be provided. When the area 3504 is cut from the wafer 3501, a multi-die memory chip will be provided. The group shown in FIG. 35C is only illustrative, and various other regions and groups of dies can be cut from the wafer 3501.

In various embodiments, the dies may be formed on the wafer 3501 such that they are arranged along one or more rows of the wafer, as shown, for example, in FIG. 35C. The dies can share the input and output bus 3530 corresponding to one or more rows. In an example embodiment, various cutting shapes can be used to cut die groups from the wafer 3501. When the die groups that can be used to form a memory chip are cut, they may not include at least the shared input and output bus 3530. A part (for example, only a part of the input/output bus 3530 may be included as a part of a memory chip formed to include a die group).

As previously discussed, when multiple dies (eg, dies 3506A and 3506B, as shown in FIG. 35C) are used to form the memory chip 3517, one IO controller corresponding to one of the dies It can be enabled and configured to control the data flow to all the processor subunits of the die 3506A and 3506B. For example, FIG. 35D shows memory dies 3506A and 3506B assembled to form a memory chip 3517, which includes memory groups 3511A to 3511H, processor subunits 3515A to 3515H, and IO controllers 3521A and 3521B , And fuses 3554A and 3554B. It should be noted that before the memory chip 3517 is removed from the wafer, the memory chip corresponds to the region 3517 of the wafer 3501. In other words, as used here and elsewhere in the present invention, once cut from wafer 3501, regions 3504, 3505, 3517, etc. of wafer 3501 will produce memory chips 3504, 3505, 3517, etc. In addition, the fuse in this article is also called a disabling element. In the example embodiment, the fuse 3554B can be used to disable the IO controller 3521B, and the IO controller 3521A can be used to control the data to all the memory banks 3511A to 3511H by transmitting data to the processor subunits 3515A to 3515H flow. In an example embodiment, the IO controller 3521A may use any suitable connection to connect to the various processor sub-units. In some embodiments, as described further below, the processor sub-units 3515A to 3515H can be interconnected, and the IO controller 3521A can be configured to control the processing logic to form the memory chip 3517 The data stream of the processor sub-units 3515A to 3515H of the series.

In an example embodiment, IO controllers such as controllers 3521A and 3521B and corresponding fuses 3554A and 3554B may be formed on wafer 3501 along with forming memory banks 3511A to 3511H and processor subunits 3515A to 3515H. In various embodiments, when the memory chip 3517 is formed, one of the fuses (eg, fuse 3554B) can be activated so that the dies 3506A and 3506B are configured to form a memory chip 3517, which serves as A single chip is controlled by a single input and output controller (eg, controller 3521A). In an example embodiment, activating the fuse may include applying current to trigger the fuse. In various embodiments, when more than one die is used to form a memory chip, all other IO controllers except one IO controller may not be activated via the corresponding fuse.

In various embodiments, as shown in FIG. 35C, multiple dies are formed on the wafer 3501 along with a set of input and output bus bars and/or control bus bars. An example input and output bus 3530 is shown in Figure 35C. In an example embodiment, one of the input and output buses (for example, the input and output bus 3530) may be connected to multiple dies. Figure 35C shows an example embodiment of an input-output bus 3530 close to which dies 3506A and 3506B pass through. The configuration of the dies 3506A and 3506B and the input-output bus 3530 as shown in FIG. 35C is only illustrative, and various other configurations can be used. For example, FIG. 35E illustrates a die 3540 formed on a wafer 3501 and configured in a hexagonal shape. A memory chip 3532 including four dies 3540 can be cut from the wafer 3501. In an example embodiment, the memory chip 3532 may include a portion of the input and output bus 3530 connected to the four dies by a suitable bus wire (eg, wire 3533, as shown in FIG. 35E). In order to deliver information to the appropriate memory cells of the memory chip 3532, the memory chip 3532 may include input/output controllers 3542A and 3542B placed at the branch points of the output bus 3530. The controllers 3542A and 3542B can receive command data via the input/output bus 3530, and select the branch of the bus 3530 to transfer the information to the appropriate memory unit. For example, if the command data includes a memory cell associated with die 3546 Read information/write information to the memory units, the controller 3542A can receive the command request and transmit the data to the branch 3531A of the bus 3530, as shown in Figure 35D, and the controller 3542B can receive the command Request and transfer data to branch 3531B. Figure 35E prompts various cuts in different areas that can be performed, where the cutting line is indicated by a dashed line.

In an example embodiment, the die group and interconnect circuitry may be designed to be included in the memory chip 3506 as shown in FIG. 36A. This embodiment may include processor sub-units (for in-memory processing) that can be configured to communicate with each other. For example, each die to be included in the memory chip 3506 may include various memory units such as memory banks 3511A to 3511D, processor sub-units 3515A to 3515D, and IO controllers 3521 and 3522. The IO controllers 3521 and 3522 can be connected to the input and output bus 3530 in parallel. The IO controller 3521 may have a fuse 3554, and the IO controller 3522 may have a fuse 3555. In an example embodiment, the processor sub-units 3515A to 3515D may be connected by means of a bus 3613, for example. In some cases, one of the IO controllers can use the corresponding fuse to disable it. For example, the fuse 3555 can be used to disable the IO controller 3522, and the IO controller 3521 can control the data flow to the memory banks 3511A to 3511D through the processor subunits 3515A to 3515D. These processor subunits They are connected to each other via a bus 3613.

The configuration of the memory cell shown in FIG. 36A is only illustrative, and various other configurations can be formed by cutting different areas of the wafer 3501. For example, FIG. 36B shows a configuration with three domains 3601 to 3603, which contain memory cells and are connected to the input/output bus 3530. In the example embodiment, the domains 3601 to 3603 are connected to the input and output bus 3530 using IO control modules 3521 to 3523 that can be disabled by the corresponding fuses 3554 to 3556. Another example of an embodiment of configuring domains containing memory cells is shown in FIG. 36C, in which bus lines 3611, 3612, and 3613 are used to connect three domains 3601, 3602, and 3603 to an input-output bus 3530. FIG. 36D shows another example embodiment of the memory chips 3506A to 3506D connected to the input and output buses 3530A and 3530B via the IO controllers 3521 to 3524. In an example embodiment, a corresponding fuse element can be used Items 3554 to 3557 do not start the IO controller, as shown in Figure 36D.

FIG. 37 shows various groups of dies 3503, such as group 3713 and group 3715, which may include one or more dies 3503. In the example embodiment, in addition to forming the die 3503 on the wafer 3501, the wafer 3501 may also contain a logic circuit 3711 called a glue logic 3711. Compared with the number of dies that may have been manufactured without the glue logic 3711, the glue logic 3711 may occupy some space on the wafer 3501, resulting in a smaller number of dies per wafer 3501. However, the presence of glue logic 3711 may allow multiple dies to be configured to act together as a single memory chip. For example, glue logic can connect multiple dies without having to change the configuration and without specifying the area within any one of the dies themselves for the circuit system used only to connect the dies together. In various embodiments, the glue logic 3711 provides an interface with other memory controllers so that the multi-die memory chip acts as a single memory chip. The glue logic 3711 may be cut along with the die group (e.g., as shown by the group 3713). Alternatively, for example, for group 3715, if the memory chip only requires one die, the glue logic may not be cut. For example, the glue logic can be selectively eliminated when it is not necessary to enable the matching of different dies. In Fig. 37, various cuts in different areas can be performed, as shown, for example, by the dotted area. In various embodiments, as shown in FIG. 37, for every two dies 3506, one glue logic element 3711 may be arranged on the wafer. In some cases, one glue logic element 3711 can be used to form any suitable number of die 3506 of the die group. The glue logic 3711 can be configured to connect to all dies from the die group. In various embodiments, the die connected to the glue logic 3711 can be configured to form a multi-die memory chip, and can be configured to form a separate single die memory when the die is not connected to the glue logic 3711 Body wafer. In various embodiments, the die connected to the glue logic 3711 and designed to work together may be cut from the wafer 3501 as a group, and may include the glue logic 3711, as suggested by the group 3713, for example. The die not connected to the glue logic 3711 can be cut from the wafer 3501 without the glue logic 3711 (as suggested by the group 3715, for example) to form a single-die memory chip.

In some embodiments, during the production of multi-die memory chips from wafer 3501, one or more cut shapes (for example, shapes forming groups 3713, 3715) can be determined to be used in the production of multi-die memory chips. What you want to gather. In some cases, as shown by group 3715, the cut shape may not include glue logic 3711.

In various embodiments, the glue logic 3711 may be a controller for controlling multiple memory cells of a multi-die memory chip. In some cases, glue logic 3711 may include parameters that can be modified by various other controllers. For example, the coupling circuit for the multi-die memory chip may include a circuit for configuring the parameters of the glue logic 3711 or the parameters of the memory controller (for example, the processor subunits 3515A to 3515D, as shown in For example, in Figure 35B). Glue logic 3711 can be configured to perform a variety of tasks. For example, the logic 3711 can be configured to determine which die may need to be addressed. In some cases, logic 3711 can be used to synchronize multiple memory cells. In various embodiments, the logic 3711 can be configured to control various memory cells so that the memory cells operate as a single chip. In some cases, an amplifier may be added between the input and output bus (for example, the bus 3530, as shown in FIG. 35C) and the processor subunits 3515A to 3515D to amplify the data signal from the bus 3530.

In various embodiments, cutting complex shapes from the wafer 3501 may be technically difficult/expensive, and a simpler cutting method may be used, and the limitation is that the die is aligned on the wafer 3501. For example, Figure 38A shows die 3506 aligned to form a rectangular grid. In an example embodiment, a vertical cut 3803 and a horizontal cut 3801 across the entire wafer 3501 can be performed to separate the cut die groups. In an example embodiment, the vertical cut 3803 and the horizontal cut 3801 can produce a group containing a selected number of dies. For example, cutting 3803 and 3801 can produce a region containing a single die (eg, region 3811A), a region containing two grains (eg, region 3811B), and a region containing four grains (eg, region 3811C) . The regions formed by cuts 3801 and 3803 are merely illustrative, and any other suitable regions may be formed. In various embodiments, depending on the die alignment, various cuts can be made. For example, if The dies are arranged in a triangular grid, as shown in FIG. 38B, and dicing lines such as lines 3802, 3804, and 3806 can be used to make a multi-die memory chip. For example, some regions may include six crystal grains, five crystal grains, four crystal grains, three crystal grains, two crystal grains, one crystal grain, and any other suitable number of crystal grains.

FIG. 38C shows the bus wires 3530 arranged in a triangular grid, in which the die 3503 is aligned at the center of the triangle formed by the bus wires 3530 intersecting. The die 3503 can be connected to all adjacent bus lines via the bus line 3820. By cutting a region containing two or more adjacent dies (e.g., region 3822, as shown in FIG. 38C), at least one bus line (e.g., line 3824) remains in region 3822, and the bus The line 3824 can be used to supply data and commands to the multi-die memory chip formed in the usage area 3822.

Figure 39 shows various connections that can be formed between processor sub-units 3515A to 3515P to allow groups of memory cells to act as a single memory chip. For example, the group 3901 of various memory units may include a connector 3905 between the processor sub-unit 3515B and the sub-unit 3515E. The connector 3905 can be used as a bus line for transmitting data and commands from the sub-unit 3515B to the sub-unit 3515E that can be used to control the respective memory group 3511E. In various embodiments, the connections between the processor sub-units may be implemented during the formation of the die on the wafer 3501. In some cases, the additional connectors may be manufactured during the packaging stage of a memory chip formed of a number of dies.

As shown in FIG. 39, the processor sub-units 3515A to 3515P may be connected to each other using various bus bars (for example, connectors 3905). The connector 3905 may not contain sequential hardware logic components, so that the data transfer between the processor sub-units and across the connector 3905 may not be controlled by the sequential hardware logic components. In various embodiments, the bus bars connecting the processor sub-units 3515A to 3515P may be arranged on the wafer 3501 before the various circuits are fabricated on the wafer 3501.

In various embodiments, processor sub-units (e.g., sub-units 3515A to 3515P) may be interconnected. For example, the sub-units 3515A to 3515P can be connected by a suitable bus (for example, the connecting member 3905). The connector 3905 can connect any one of the subunits 3515A to 3515P with the subunit 3515A Connect to any other of 3515P. In an example embodiment, the connected subunits may be on the same die (e.g., subunits 3515A and 3515B), and in other cases, the connected subunits may be on different die (e.g., subunits 3515B and 3515E). ). The connector 3905 may include a dedicated bus for connecting the sub-units and may be configured to efficiently transfer data between the sub-units 3515A to 3515P.

Various aspects of the present invention relate to methods for producing memory chips of selectable sizes from wafers. In an example embodiment, a memory chip with a selectable size may be formed of one or more dies. As mentioned above, the dies can be arranged along one or more rows, as shown in, for example, Figure 35C. In some cases, at least one common I/O bus corresponding to one or more rows may be arranged on the wafer 3501. For example, bus bar 3530 may be arranged, as shown in Figure 35C. In various embodiments, the bus bar 3530 can be electrically connected to the memory cells of at least two of the dies, and the connected dies can be used to form a multi-die memory chip. In an example embodiment, one or more controllers (for example, input and output controllers 3521 and 3522, as shown in FIG. 35B) may be configured to control at least two chips used to form a multi-die memory chip. The memory unit of the grain. In various embodiments, a die having a memory cell connected to the bus 3530 can be cut from the wafer, wherein at least one of the common input and output bus (for example, the bus 3530, as shown in FIG. 35B) corresponds to Part of the information is transmitted to at least one controller (for example, controllers 3521, 3522) to configure the controller to control the memory cells of the connected die so as to act as a single chip together.

In some cases, the memory cells located on the wafer 3501 can be tested before the memory chip is manufactured by cutting the area of the wafer 3501. At least one common input and output bus (for example, bus 3530, as shown in Figure 35C) can be used for testing. When the memory cell passes the test, the memory chip can be formed by a group of dies containing the memory cell. The memory cells that fail the test can be discarded, and the memory cells are not used for manufacturing memory chips.

FIG. 40 shows an example process 4000 of building a memory chip from a die group. in At step 4011 of the processing program 4000, dies can be arranged on the semiconductor wafer 3501. At step 4015, any suitable method may be used to fabricate dies on the wafer 3501. For example, the die can be manufactured by etching the wafer 3501, depositing various dielectric, metal or semiconductor layers, and further etching the deposited layers. For example, multiple layers can be deposited and etched. In various embodiments, any suitable doping element may be used to do n-type or p-type doping of the layer. For example, the semiconductor layer can be doped n-type with phosphorus and the semiconductor layer can be p-type doped with boron. As shown in FIG. 35A, the dies 3503 can be separated from each other by the space that can be used to cut the dies 3503 from the wafer 3501. For example, the dies 3503 can be separated from each other by spacers, wherein the width of the spacers can be selected to allow wafer dicing in the spacers.

At step 4017, any suitable method may be used to cut the die 3503 from the wafer 3501. In an example embodiment, a laser may be used to cut the die 3503. In an example embodiment, wafer 3501 may be scribed first, followed by mechanical scribing. Alternatively, a mechanical dicing saw can be used. In some cases, an invisible cutting process can be used. During dicing, once the die is cut, the wafer 3501 can be mounted on the dicing tape for holding the die. In various embodiments, large cuts may be made, as shown, for example, by cuts 3801 and 3803 in FIG. 38A or cuts 3802, 3804, or 3806 in FIG. 38B. Once the dies 3503 are cut individually or in groups, as shown by the group 3504 in FIG. 35C, for example, the dies 3503 can be packaged. The packaging of the die may include forming contacts to the die 3503, depositing a protective layer over the contacts, attaching a thermal management device (for example, a heat sink), and encapsulating the die 3503. In various embodiments, depending on how many dies are selected to form the memory chip, appropriate configurations of contacts and buses can be used. In an example embodiment, some of the contacts between the different dies forming the memory chip may be fabricated during the packaging of the memory chip.

FIG. 41A shows an example process 4100 for manufacturing a memory chip containing multiple dies. The step 4011 of the processing program 4100 may be the same as the step 4011 of the processing program 4000. At step 4111, as shown in FIG. 37, the glue logic 3711 may be placed on the wafer 3501. The glue logic 3711 may be any suitable logic for controlling the operation of the die 3506, as shown in FIG. 37. As described above, the existence of glue logic 3711 allows multiple dies to act as a single memory chip. The glue logic 3711 can provide an interface with other memory controllers, so that a memory chip formed by multiple dies can act as a single memory chip.

At step 4113 of the processing procedure 4100, bus bars (for example, input/output bus bars and control bus bars) may be arranged on the wafer 3501. The bus bar can be arranged such that it is connected to various dies and logic circuits such as glue logic 3711. In some cases, the bus can be connected to a memory unit. For example, the bus bar can be configured to connect processor sub-units of different dies. At step 4115, any suitable method can be used to fabricate the die, glue logic, and bus. For example, the logic device can be manufactured by etching the wafer 3501, depositing various dielectric, metal or semiconductor layers, and further etching the deposited layers. The bus bar can be manufactured using, for example, metal evaporation.

At step 4140, a cut shape can be used to cut a group of dies connected to a single glue logic 3711, as shown, for example, in FIG. 37. The memory requirements for a memory chip containing multiple dies 3503 can be used to determine the cutting shape. For example, FIG. 41B shows the processing program 4101, which can be a variant of the processing program 4100, where step 4140 of the processing program 4100 can be steps 4117 and 4119. At step 4117, the system for dicing the wafer 3501 may receive an instruction describing the requirements for the memory chip. For example, the requirement may include forming a memory chip including four dies 3503. In some cases, at step 4119, the programming software can determine the periodic pattern for die group and glue logic 3711. For example, the periodic pattern may include two glue logic 3711 elements and four dies 3503, wherein every two dies are connected to one glue logic 3711. Alternatively, at step 4119, the pattern may be provided by the designer of the memory chip.

In some cases, the pattern can be selected to maximize the yield of memory chips from wafer 3501. In an example embodiment, the memory cell of the die 3503 can be tested to identify the die with a faulty memory cell (such die is called a failed die), and based on the location of the faulty die, Identify the group of dies 3503 containing memory cells that passed the test and determine the appropriate cut Cut the pattern. For example, if a large number of die 3503 fail to pass at the edge of the wafer 3501, the cutting pattern can be determined to avoid the die at the edge of the wafer 3501. The other steps of the processing program 4101, such as steps 4011, 4111, 4113, 4115, and 4140, may be the same as the same numbered steps of the processing program 4100.

FIG. 41C shows an example processing program 4102 that can be a variation of the processing program 4101. Steps 4011, 4111, 4113, 4115, and 4140 of processing program 4102 can be the same as the same numbered steps of processing program 4101, step 4131 of processing program 4102 can replace step 4117 of processing program 4101, and step 4133 of processing program 4102 can replace processing Step 4119 of program 4101. At step 4131, the system for dicing the wafer 3501 may receive an instruction describing the requirements for the first memory chip set and the second memory chip set. For example, the requirements may include: forming a first memory chip set having memory chips composed of four dies 3503; and forming a second memory chip set having memory chips composed of two dies 3503 . In some cases, it may be necessary to form more than two sets of memory chips from the wafer 3501. For example, the third memory chip set may include a memory chip consisting of only one die 3503. In some cases, at step 4133, the programming software can determine the periodic patterns for die groups and glue logic 3711 to be used to form the memory chips in each memory chip set. For example, the first set of memory chips may include a memory chip including two glue logic 3711 and four dies 3503, wherein every two dies are connected to one glue logic 3711. In various embodiments, glue logic units 3711 for the same memory chip can be linked together to act as a single glue logic. For example, during the manufacture of the glue logic 3711, suitable bus lines can be formed to link the glue logic units 3711 to each other.

The second memory chip set may include a memory chip including one glue logic 3711 and two dies 3503, wherein the die 3503 is connected to the glue logic 3711. In some cases, when the third memory chip set is selected and when the third memory chip set includes memory chips composed of a single die 3503, these memory chips may not require glue logic 3711.

Dual port functionality

When designing a memory chip or memory instance within the chip, an important characteristic is the number of ZigZags that can be accessed simultaneously during a single clock cycle. For reading and/or writing, the more addresses that can be accessed at the same time (for example, addresses along a column also called a word or word line and a row also called a bit or bit line) , The faster the memory chip. Although there have been some activities in the development of memory that includes multiple ports, which allow simultaneous access to multiple addresses, such as for building register files, cache memory, or shared memory, most The example uses a memory pad that is large in size and supports multiple address access. However, a DRAM chip usually includes a single bit line and a single column line connected to each capacitor of each memory cell. Therefore, the embodiment of the present invention attempts to provide multi-port access to the existing DRAM chip without modifying the conventional single-port memory structure of the DRAM array.

Embodiments of the present invention can use memory to time control memory instances or chips at twice the speed of logic circuits. Any logic circuit that uses memory can therefore "correspond" to the memory and any of its components. Therefore, the embodiment of the present invention can retrieve or write two addresses in two memory array clock cycles, which are equivalent to a single processing time for logic circuits. Pulse circulation. The logic circuits may include circuits such as controllers, accelerators, GPUs, or CPUs, or may include processing groups on the same substrate as the memory chip, for example, as depicted in FIG. 7A. As explained above with respect to FIG. 3A, the "processing group" can refer to two or more processor sub-units on the substrate and their corresponding memory groups. The group may represent a spatial distribution on the substrate and/or a logical grouping for the purpose of compiling program codes for execution on the memory chip 2800. Therefore, as described above with respect to FIG. 7A, the substrate with memory chips may include a memory array having a plurality of groups, such as the group 2801a shown in FIG. 28 and other groups. In addition, the substrate may also include a processing array, which may include a plurality of processor sub-units (such as the sub-units 730a, 730b, 730c, 730d, 730e, 730f, 730g, and 730h shown in FIG. 7A).

Therefore, the embodiments of the present invention can be self-contained in each of two consecutive memory cycles. The array retrieves data to handle two addresses for each logical cycle and provides two results to the logic, just as if the single-port memory array is a dual-port memory chip. The additional timing allows the memory chip of the present invention to function as if the single-port memory array is a dual-port memory instance, a three-port memory instance, a four-port memory instance port, or any other multi-port memory instance.

42 depicts an example circuit system 4200 consistent with the present invention that provides dual-port access along the rows of memory chips using the circuit system 4200. The embodiment depicted in FIG. 42 can use a memory with two row multiplexers ("mux") 4205a and 4205b to access two words on the same column during the same clock cycle used for logic circuits Array 4201. For example, during the memory clock cycle, RowAddrA is used in the column decoder 4203 and ColAddrA is used in the multiplexer 4205a to buffer data from memory cells with addresses (RowAddrA, ColAddrA). During the same memory clock cycle, ColAddrB is used in the multiplexer 4205b to buffer data from memory cells with addresses (RowAddrA, ColAddrB). Therefore, the circuit system 4200 can allow dual-port access to data (for example, DataA and DataB) stored on memory cells at two different addresses along the same row or word line. Therefore, two addresses can share a column so that the column decoder 4203 activates the same word line for two retrievals. In addition, the embodiment of the example depicted in FIG. 42 can use a row multiplexer so that two addresses can be accessed during the same memory clock cycle.

Similarly, FIG. 43 depicts an example circuit system 4300 in accordance with the present invention that provides dual-port access along the rows of memory chips using the circuit system 4300. The embodiment depicted in FIG. 43 can use a memory array 4301 that has a column decoder 4303 coupled to a multiplexer ("mux") for storage during the same clock cycle used in the logic circuit. Take two words on the same line. For example, on the first memory clock cycle of the two memory clock cycles, RowAddrA is used in the column decoder 4303 and ColAddrA is used in the row multiplexer 4305 to buffer data from the address (RowAddrA, ColAddrA) memory cell data (for example, to the "buffer word" buffer in FIG. 43). On the second memory clock cycle of the two memory clock cycles, RowAddrB is used to list The decoder 4303 and ColAddrA are used in the row multiplexer 4305 to buffer the data from the memory cell with the address (RowAddrB, ColAddrA). Therefore, the circuit system 4300 can allow dual-port access to the data (for example, DataA and DataB) stored on the memory cells at two different addresses along the same row or bit line. Therefore, two addresses can share a column so that the row decoder (which can be separated or combined with one or more row multiplexers, as depicted in FIG. 43) enables the same bit line for two acquisitions. The embodiment of the example depicted in FIG. 43 may use two memory clock cycles, because the row decoder 4303 may require one memory clock cycle to activate each word line. Therefore, if the timing is at least twice the speed of the corresponding logic circuit, the memory chip using the circuit system 4300 can serve as a dual-port memory.

Therefore, as explained above, FIG. 43 can retrieve DataA and DataB during two memory clock cycles that are faster than the clock cycle used for the corresponding logic circuit. For example, a column decoder (e.g., column decoder 4303 of FIG. 43) and a row decoder (which can be separated or combined with one or more row multiplexers, as depicted in FIG. 43) can be configured as The timing is controlled at a rate that is at least twice the rate at which the corresponding logic circuit generates two addresses. For example, a clock circuit (not shown in FIG. 43) used in the circuit system 4300 can time the circuit system 4300 at a rate that is at least twice the rate at which the corresponding logic circuit generates two addresses.

The embodiment of FIG. 42 and the embodiment of FIG. 43 can be used separately or in combination. Therefore, a circuit system that provides dual-port functionality on a single-port memory array or pad (for example, the circuit system 4200 or 4300) may include a plurality of memory banks arranged along at least one row and at least one row. The plurality of memory banks are depicted as a memory array 4201 in FIG. 42 and as a memory array 4301 in FIG. 43. These embodiments may further use at least one column multiplexer (as depicted in FIG. 43) or at least one row multiplexer configured to receive two addresses for reading or writing during a single clock cycle. Worker (as depicted in Figure 42). In addition, these embodiments may use column decoders (e.g., column decoder 4203 of FIG. 42 and column decoder 4303 of FIG. 43) and row decoders (which may be separated or combined with one or more row multiplexers, 42 and 43) to read from or write to two addresses. For example, the column decoder and the row decoder may retrieve the first of the two addresses from at least one column multiplexer or at least one row multiplexer during the first cycle, and the decoding corresponds to the first The word line and bit line of the address. In addition, the column decoder and the row decoder can retrieve the second address of the two addresses from at least one column multiplexer or at least one row multiplexer during the second cycle, and the decoding corresponds to the second address Zigzag line and bit line. The captures may each include using a column decoder to activate a word line corresponding to the address and using a row decoder to activate a bit line corresponding to the address on the activated word line.

Although the capture is described above, the embodiments of FIGS. 42 and 43 (whether implemented separately or in combination) may include write commands. For example, during the first cycle, the row decoder and the row decoder can write the first data retrieved from at least one row multiplexer or at least one row multiplexer to the first of the two addresses. Address. In addition, during the second cycle, the column decoder and the row decoder can write the second data retrieved from at least one column multiplexer or at least one row multiplexer to the second address of the two addresses .

The example in FIG. 42 shows the processing when the first address and the second address share the word line address, and the example in FIG. 43 shows the processing when the first address and the second address share the row address program. As described further below with respect to FIG. 47, the same processing procedure can be implemented when the first address and the second address do not share the word line address or the row address.

Therefore, although the above examples provide dual-port access along at least one of a row or a row, additional embodiments may provide dual-port access along both the row and the row. FIG. 44 depicts an example circuit system 4400 in accordance with the present invention that provides dual-port access along both the column and row of memory chips using the circuit system 4400. Therefore, the circuit system 4700 can represent a combination of the circuit system 4200 of FIG. 42 and the circuit system 4300 of FIG. 43.

The embodiment depicted in FIG. 44 can use a memory array 4401 with a column decoder 4403 coupled to a multiplexer ("mux") for use in the same logic circuit Two columns are accessed during the clock cycle. In addition, the embodiment depicted in FIG. 44 can use a memory array 4401 that has a row decoder (or multiplexer) 4405 coupled to a multiplexer ("mux") to cycle at the same clock Access two rows during the period. For example, on the first memory clock cycle of the two memory clock cycles, RowAddrA is used in the column decoder 4403 and ColAddrA is used in the row multiplexer 4405 to buffer data from the address (RowAddrA, ColAddrA) memory cell data (for example, to the "buffer word" buffer in FIG. 44). On the second memory clock cycle of the two memory clock cycles, RowAddrB is used in the column decoder 4403 and ColAddrB is used in the row multiplexer 4405 to buffer the memory from the address (RowAddrB, ColAddrB) Somatic data. Therefore, the circuit system 4400 can allow dual-port access to data (for example, DataA and DataB) stored on memory cells at two different addresses. The embodiment of the example depicted in FIG. 44 may use additional buffers because the column decoder 4403 may require one memory clock cycle to activate each word line. Therefore, if time control is performed at least twice the speed of the corresponding logic circuit, the memory chip using the circuit system 4400 can serve as a dual-port memory.

Although not depicted in FIG. 44, circuitry 4400 may further include additional circuitry of FIG. 46 (described further below) along column or word lines and/or similar additional circuitry along row or bit lines. Therefore, the circuit system 4400 can activate the corresponding circuit system (for example, by turning off one or more switching elements, such as one or more of the switching elements 4613a, 4613b, and the like in FIG. 46) to activate the corresponding circuit system including the address The disconnected part (for example, by connecting a voltage or allowing current to flow to the disconnected part). Therefore, when an element of the circuit system (such as a wire or the like) includes the location of an identification address and/or when an element of the circuit system (such as a switching element) controls the supply to the memory cell identified by the address Or voltage and/or current, the circuit system can "correspond". The circuit system 4400 can then use the column decoder 4403 and the row multiplexer 4405 to decode the corresponding word lines and bit lines, to retrieve data from the addresses in the activated disconnected part or to write data to these Address.

As further depicted in Figure 44, the circuitry 4400 can be further configured to At least one column multiplexer (depicted as being separate from column decoder 4403, but can be incorporated into it) that receives two addresses for reading or writing during a single clock cycle and/or at least one row multiplexer (E.g., depicted as separate from the row multiplexer 4405, but could be incorporated into it). Therefore, an embodiment can use a column decoder (for example, the column decoder 4403) and a row decoder (which can be separated or combined with the row multiplexer 4405) to read from or write to two addresses . For example, the column decoder and the row decoder can retrieve the first of the two addresses from at least one column multiplexer or at least one row multiplexer during the memory clock cycle, and the decoding corresponds to The word line and bit line of the first address. In addition, the column decoder and the row decoder can retrieve the second address of the two addresses from at least one column multiplexer or at least one row multiplexer during the same memory cycle, and the decoding corresponds to the second address Zigzag line and bit line.

Figures 45A and 45B depict existing replication techniques used to provide dual-port functionality on a single-port memory array or pad. As shown in Figure 45A, dual-port reads can be provided by keeping copies of data synchronized across memory arrays or pads. Therefore, reading can be performed from two copies of the memory instance, as depicted in Figure 45A. In addition, as shown in Figure 45B, dual-port writes can be provided by copying all writes across a memory array or pad. For example, the memory chip may need to use the logic circuit of the memory chip to send a write command in the form of a copy, and send a write command for each data copy. Alternatively, in some embodiments, as shown in FIG. 45A, the additional circuitry may allow the logic circuit using the memory instance to send a single write command that is automatically copied by the additional circuitry to span the memory The array or pad generates copies of the written data in order to keep the copies in sync. The embodiments of FIGS. 42, 43, and 44 can access two bit lines in a single memory clock cycle by using a multiplexer (for example, as depicted in FIG. 42) and/or by comparing corresponding logic The circuit time-controls the memory faster (for example, as depicted in Figure 43 and Figure 44) and provides additional multiplexers to handle additional addresses instead of copying all the data in the memory to reduce the cost from these existing copy technologies. redundancy.

In addition to the faster timing and/or additional multiplexers described above, the embodiments of the present invention can also use a circuit system that disconnects bit lines and/or word lines at some points in the memory array. Such reality The embodiment may allow multiple simultaneous accesses to the array, as long as the column decoder and row decoder access are not coupled to different positions of the same part of the disconnect circuit system. For example, it is possible to access locations with different word lines and bit lines at the same time, because the open circuit system allows column decoding and row decoding to access different addresses without electrical interference. During the design of the memory chip, the granularity of the disconnected area in the memory array can be weighed against the extra area required to disconnect the circuit system.

The architecture for implementing this simultaneous access is depicted in FIG. 46. In particular, FIG. 46 depicts an example circuit system 4600 that provides dual port functionality on a single port memory array or pad. As depicted in FIG. 46, the circuit system 4600 may include a plurality of memory pads (eg, memory pads 4609a, 4609b, and the like) arranged along at least one column and at least one row. The layout of the circuit system 4600 further includes a plurality of word lines, such as word lines 4611a and 4611b corresponding to columns, and bit lines 4615a and 4615b corresponding to rows.

The example of FIG. 46 includes twelve memory pads, each of which has two lines and eight rows. In other embodiments, the substrate may include any number of memory pads, and each memory pad may include any number of lines and any number of rows. Some memory pads may include the same number of lines and rows (as shown in FIG. 46), while other memory pads may include a different number of lines and/or rows.

Although not depicted in FIG. 46, the circuitry 4600 may further be configured to receive at least one of two (or three or any plural) addresses for reading or writing during a single clock cycle A column multiplexer (separate from or combined with the column decoder 4601a and/or 4601b) or at least one row multiplexer (e.g., row multiplexer 4603a and/or 4603b). In addition, embodiments may use column decoders (for example, column decoders 4601a and/or 4601b) and row decoders (which can be separated or combined with row multiplexers 4603a and/or 4603b) to select from two (or more than one) Two) addresses read or write to two (or more than two) addresses. For example, the column decoder and the row decoder can retrieve the first of the two addresses from at least one column multiplexer or at least one row multiplexer during the memory clock cycle, and the decoding corresponds to The word line and bit line of the first address. In addition, the column decoder and row decoder can be During the same memory cycle, the second address of the two addresses is retrieved from at least one row multiplexer or at least one row multiplexer, and the word line and bit line corresponding to the second address are decoded. As explained above, as long as the two addresses are in different positions that are not coupled to the same part of the open circuit system (for example, switching elements such as 4613a, 4613b and the like), they can be at the same memory clock. Access during the cycle. In addition, the circuit system 4600 can simultaneously access the first two addresses during the first memory clock cycle, and then simultaneously access the next two addresses during the second memory clock cycle. In these embodiments, if the timing is at least twice the speed of the corresponding logic circuit, the memory chip using the circuit system 4600 can serve as a four-port memory.

Figure 46 further includes at least one column circuit and at least one row circuit configured to act as a switch. For example, corresponding switching elements such as 4613a, 4613b, and the like may include a transistor or any other electrical element that is configured to allow or stop current flow and/or connect or disconnect Voltage and word lines or bit lines connected to switching elements such as 4613a, 4613b and the like. Therefore, the corresponding switching element can divide the circuit system 4600 into disconnected parts. Although depicted as including a single column and each column including sixteen rows, the disconnection region within the circuit system 4600 may include different levels of granularity depending on the design of the circuit system 4600.

The circuit system 4600 may use a controller (for example, the column control element 4607) to activate the corresponding one of the at least one column circuit and the at least one row circuit, so as to activate the corresponding disconnection area during the address operation described above. For example, the circuit system 4600 may transmit one or more control signals to turn on the corresponding ones of the switching elements (for example, the switching elements 4613a, 4613b, and the like). In embodiments where the switching elements 4613a, 4613b, and the like include transistors, the control signal may include a voltage to turn off the transistors.

Depending on the open area including the address, more than one of the switching elements can be activated by the circuit system 4600. For example, in order to reach the address in the memory pad 4609b of FIG. 46, the switching element allowing access to the memory pad 4609a and the switching element allowing access to the memory pad 4609b must be turned off. Column The control unit 4607 can determine the switching element to be activated, so as to retrieve the specific address in the circuit system 4600 according to the specific address.

FIG. 46 shows an example of a circuit system 4600 for dividing word lines of a memory array (for example, including memory pads 4609a, 4609b, and the like). However, other embodiments may use similar circuit systems (for example, switching elements that divide the memory chip 4600 into off regions) to divide the bit lines of the memory array. Therefore, the architecture of the circuit system 4600 can be used in dual-row access (as depicted in FIG. 42 or FIG. 44) and dual-column access (as depicted in FIG. 43 or FIG. 44).

The processing procedure for multi-cycle access to the memory array or pad is depicted in FIG. 47A. Specifically, FIG. 47A is an example flowchart of a processing procedure 4700 for providing dual-port access on a single-port memory array or pad (for example, using the circuit system 4300 of FIG. 43 or the circuit system 4400 of FIG. 44). The column decoder and row decoder in accordance with the present invention can be used to execute the processing program 4700, such as the column decoder 4303 or 4403 of FIG. 43 or FIG. 44, and the row decoder (which can be separated from one or more row multiplexers) Or a combination, such as the row multiplexer 4305 or 4405 depicted in Figure 43 or Figure 44, respectively).

At step 4710, during the first memory clock cycle, the circuit system may use at least one column multiplexer and at least one row multiplexer to decode the word line corresponding to the first of the two addresses And bit line. For example, at least one column decoder can activate the word line, and at least one row multiplexer can amplify the voltage from the memory cell along the activated word line and corresponding to the first address. The amplified voltage can be provided to a logic circuit using a memory chip including a circuit system, or the amplified voltage can be buffered according to step 4720 described below. The logic circuits may include circuits such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as depicted in FIG. 7A.

Although described above as read operations, method 4700 can similarly handle write operations. For example, at least one column decoder can activate a word line, and at least one row multiplexer can apply a voltage to a memory cell along the activated word line and corresponding to the first address to write new data Into the memory cell. In some embodiments, the circuit system can provide confirmation of writing to the use of the circuit The logic circuit of the memory chip of the system, or buffer the confirmation according to the following step 4720.

At step 4720, the circuit system can buffer the retrieved data at the first address. For example, as depicted in Figure 43 and Figure 44, the buffer may allow the circuit system to retrieve the second address of the two addresses (described below in step 4730) and combine the results of the two retrievals together Pass back. The buffer may include a register, SRAM, non-volatile memory or any other data storage device.

At step 4730, during the second memory clock cycle, the circuit system may use at least one column multiplexer and at least one row multiplexer to decode the word line corresponding to the second of the two addresses And bit line. For example, at least one column decoder can activate the word line, and at least one row multiplexer can amplify the voltage from the memory cell along the activated word line and corresponding to the second address. The amplified voltage can be provided to a logic circuit using a memory chip including a circuit system, whether provided individually or in conjunction with, for example, the buffered voltage from step 4720. The logic circuits may include circuits such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as depicted in FIG. 7A.

Although described above as read operations, method 4700 can similarly handle write operations. For example, at least one column decoder can activate a word line, and at least one row multiplexer can apply a voltage to a memory cell along the activated word line and corresponding to the second address to write new data Into the memory cell. In some embodiments, the circuit system may provide confirmation of writing to the logic circuit using the memory chip including the circuit system, whether provided individually or in conjunction with, for example, the buffered voltage from step 4720.

At step 4740, the circuit system can output the retrieved data at the second address and the buffered first address. For example, as depicted in FIG. 43 and FIG. 44, the circuit system can return the results of two captures (for example, from steps 4710 and 4730) together. The circuit system can transmit the result back to the logic circuit using the memory chip including the circuit system. The logic circuits may include circuits such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as shown in FIG. 7A Portrayed in.

Although described with reference to multiple cycles, if two addresses share a word line, as depicted in FIG. 42, method 4700 may allow single cycle access to the two addresses. For example, steps 4710 and 4730 can be performed during the same memory clock cycle, because multiple row multiplexers can decode different bit lines on the same word line during the same memory clock cycle. In these embodiments, the buffering step 4720 can be skipped.

The processing procedure for simultaneous access (for example, using the circuit system 4600 described above) is depicted in FIG. 47B. Therefore, although shown sequentially, the steps in FIG. 47B can all be performed during the same memory clock cycle, and at least some steps can be performed at the same time (for example, steps 4760 and 4780 or steps 4770 and 4790). Specifically, FIG. 47B is an example flowchart of a processing procedure 4750 for providing dual-port access on a single-port memory array or pad (for example, using the circuit system 4200 of FIG. 42 or the circuit system 4600 of FIG. 46). The column decoder and row decoder in accordance with the present invention can be used to execute the processing program 4750, such as the column decoder 4203 or column decoders 4601a and 4601b of FIG. 42 or FIG. The row multiplexers are separated or combined, such as the row multiplexers 4205a and 4205b or the row multiplexers 4603a and 4306b depicted in FIG. 42 or FIG. 46, respectively).

At step 4760, during the memory clock cycle, the circuit system may activate the corresponding one of at least one column circuit and at least one row circuit based on the first address of the two addresses. For example, the circuit system may transmit one or more control signals to close a corresponding one of the switching elements including at least one column circuit and at least one row circuit. Therefore, the circuit system can access the corresponding disconnected area including the first address of the two addresses.

At step 4770, during the memory clock cycle, the circuit system may use at least one column multiplexer and at least one row multiplexer to decode the word line and the bit line corresponding to the first address. For example, at least one column decoder can activate the word line, and at least one row multiplexer can amplify the voltage from the memory cell along the activated word line and corresponding to the first address. The amplified voltage can be increased For logic circuits using memory chips including circuit systems. For example, as described above, the logic circuits may include circuits such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as depicted in FIG. 7A.

Although described above as a read operation, the method 4500 can similarly handle write operations. For example, at least one column decoder can fail the word line, and at least one row multiplexer can apply a voltage to the memory cell along the activated word line and corresponding to the first address to transfer new data Write to the memory cell. In some embodiments, the circuit system can provide confirmation of writing to a logic circuit using a memory chip that includes the circuit system.

At step 4780, during the same cycle, the circuit system may activate the corresponding one of at least one column circuit and at least one row circuit based on the second address of the two addresses. For example, the circuit system may transmit one or more control signals to close a corresponding one of the switching elements including at least one column circuit and at least one row circuit. Therefore, the circuit system can access the corresponding disconnection area including the second address of the two addresses.

At step 4790, during the same cycle, the circuit system may use at least one column multiplexer and at least one row multiplexer to decode the word line and the bit line corresponding to the second address. For example, at least one column decoder can activate the word line, and at least one row multiplexer can amplify the voltage from the memory cell along the activated word line and corresponding to the second address. The amplified voltage is provided to a logic circuit that can use a memory chip including a circuit system. For example, as described above, the logic circuits may include conventional circuits such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as depicted in FIG. 7A.

Although described above as a read operation, the method 4500 can similarly handle write operations. For example, at least one column decoder can activate a word line, and at least one row multiplexer can apply a voltage to a memory cell along the activated word line and corresponding to the second address to write new data Into the memory cell. In some embodiments, the circuit system can provide confirmation of writing to the The logic circuit of the memory chip of the system.

Although the description is made with reference to a single cycle, if the two addresses are in the disconnected region of a shared word line or bit line (or otherwise share at least one column circuit and at least one switching element in the row circuit), the method 4500 Allows multiple cyclic accesses to two addresses. For example, steps 4760 and 4770 can be performed during the first memory clock cycle. In the first memory clock cycle, the first row of decoders and the first row of multiplexers can decode corresponding to the first bit Address word lines and bit lines, and steps 4780 and 4790 can be performed during the second memory clock cycle. In the second memory clock cycle, the second row of decoders and the second row of multiplexers can be Decoding corresponds to the word line and bit line of the second address.

Another example of the architecture for dual port access along both rows and rows is depicted in FIG. 48. In particular, FIG. 48 depicts an example circuitry 4800 that uses multiple column decoders in combination with multiple row multiplexers to provide dual-port access along both columns and rows. In FIG. 48, the column decoder 4801a can access the first word line, and the row multiplexer 4803a can decode data from one or more memory cells along the first word line, and the column decoder 4801b can store Take the second word line, and the row multiplexer 4803b can decode data from one or more memory cells along the second word line.

As described with respect to FIG. 47B, this access can be performed simultaneously during one memory clock cycle. Therefore, similar to the architecture of FIG. 46, the architecture of FIG. 48 (including the memory pad described in FIG. 49 below) can allow multiple addresses to be accessed in the same clock cycle. For example, the architecture of FIG. 48 can include any number of column decoders and any number of row multiplexers, so that the number of addresses corresponding to the number of column decoders and row multiplexers can all be at a single memory clock. Access within the loop.

In other embodiments, this access can be performed sequentially along two memory clock cycles. By clocking the memory chip 4800 faster than the corresponding logic circuit, two memory clock cycles can be equivalent to one clock cycle of a logic circuit using memory. For example, as described above, the logic circuits may include conventional circuits such as GPU or CPU, or may include processing groups on the same substrate as the memory chip, for example, as depicted in FIG. 7A.

Other embodiments may allow simultaneous access. For example, as described with respect to FIG. 42, multiple row decoders (which may include row multiplexers, such as 4803a and 4803b, as shown in FIG. 48) can read along the line during a single memory clock cycle Multiple bit lines of the same word line. Additionally or alternatively, as described with respect to FIG. 46, the circuitry 4800 may incorporate additional circuitry so that this access can be simultaneous. For example, the column decoder 4801a can access the first word line, and the row multiplexer 4803a can decode data from memory cells along the first word line during the same memory clock cycle. In the clock cycle, the column decoder 4801b accesses the second word line, and the row multiplexer 4803b decodes data from memory cells along the second word line.

The architecture of FIG. 48 can be used with modified memory pads forming a memory bank, as shown in FIG. 49. In FIG. 49, each memory cell (depicted as a capacitor similar to DRAM, but can also include a memory cell similar to SRAM or any other memory cell) is accessed by two word lines and two bit lines. Several transistors configured in the same way). Therefore, the memory pad 4900 of FIG. 49 allows two different bits to be accessed simultaneously by two different logic circuits, or even the same bit. However, the embodiment of FIG. 49 uses a modification to the memory pads instead of implementing a dual-port solution on standard DRAM memory pads, which are connected via wires for single-port access, as in the above embodiment.

Although described as having two ports, any of the embodiments described above can be extended to more than two ports. For example, the embodiments of FIGS. 42, 46, 48, and 49 may include additional row multiplexers or column multiplexers, respectively, to provide access to additional rows or columns during a single clock cycle. As another example, the embodiments of FIGS. 43 and 44 may include additional column decoders and/or row multiplexers to provide access to additional columns or rows during a single clock cycle, respectively.

Variable word length access in memory

As used further above and below, the term "coupled" can include direct connection, indirect connection, electrical communication, and the like.

In addition, terms such as "first", "second" and the like are used to distinguish Elements or method steps with the same or similar names or titles, and do not necessarily indicate spatial or temporal order.

Generally, the memory chip may include a memory bank. The memory bank can be coupled to column decoders and row decoders, which are configured to select specific words (or other fixed-size data units) to be read or written. Each memory bank may include memory cells for storing data cells, amplifying voltages from memory cells selected by the row decoder and row decoder, and any other appropriate circuits.

Each memory bank usually has a specific I/O width. For example, the I/O width can include words.

Although some processing procedures executed by logic circuits using memory chips can benefit from using extremely long words, some other processing procedures may only require a portion of the word.

In fact, in-memory arithmetic units (such as processor sub-units placed on the same substrate as the memory chip, for example, as depicted and described in FIG. 7A) frequently perform memory accesses that require only a portion of the word operating.

In order to reduce the latent time associated with accessing the entire word when only a part of it is used, embodiments of the present invention may provide a method and system for extracting only one or more parts of a word, thereby reducing and transmitting the word. No part of the associated data loss is required and power saving in the memory device is allowed.

In addition, the embodiments of the present invention can also reduce the memory chip and other entities (such as logic circuits, whether separate, such as CPU and GPU, or included on the same substrate as the memory chip, such as those depicted in FIG. 7A and The power consumption of the interaction between the described processor subunits), these other entities access the memory chip, which can only receive or write part of the word.

Memory access commands (for example, from logic circuits that use memory) can include addresses in memory. For example, the address may include a column address and a row address, or may be translated into a column address and a row address by a memory controller of the memory, for example.

In many volatile memories such as DRAM, the column address is sent (for example, Directly from the logic circuit or using the memory controller) to the column decoder, the column decoder activates the entire column (also called the word line) and loads all the bit lines included in the column.

The row address identifies the bit lines on the activated column, and the bit lines are sent outside the memory group including the bit lines and sent to the next level circuit system. For example, the next level of circuitry can include I/O buses of memory chips. In embodiments using in-memory processing, the next level of circuitry may include the processor sub-units of the memory chip (for example, as depicted in FIG. 7A).

Therefore, the memory chips described below can be included in the ones shown in FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22 or 23. Any one of the memory chips described in, or otherwise includes the memory chip.

The memory chip can be manufactured by a first manufacturing process optimized for memory cells instead of logic cells. For example, the memory cell manufactured by the first manufacturing process may exhibit a critical size smaller than that of the logic circuit manufactured by the first manufacturing process (for example, more than 2 times, 3 times, 4 times, 5 times, 6 times, 7 times, 8 times, 9 times, 10 times and the like). For example, the first manufacturing process may include an analog manufacturing process, a DRAM manufacturing process, and the like.

The memory chip may include an integrated circuit, and the integrated circuit may include a memory cell. The memory unit may include a memory cell, an output port, and a reading circuit system. In some embodiments, the memory unit may further include a processing unit, such as the processor sub-unit as described above.

For example, the read circuit system may include a reduction unit and a first group of memory read paths, and the memory read paths are used to output up to a first number of bits through the output port. The output port can be connected to off-chip logic circuits (such as accelerators, CPUs, GPUs, or the like) or on-chip processor sub-units, as described above.

In some embodiments, the processing unit may include a reduction unit, may be a part of the reduction unit, may be different from the reduction unit, or may include the reduction unit in other ways.

The read path in the memory can be included in the integrated circuit (for example, it can be in the memory unit Medium), and may include any circuits and/or links configured for reading from and/or writing to memory cells. For example, the read path in the memory may include a sense amplifier, a conductor coupled to a memory cell, a multiplexer, and the like.

The processing unit can be configured to send a read request to the memory unit to read the second number of bits from the memory unit. Additionally or alternatively, the read request may originate from an off-chip logic circuit (such as an accelerator, CPU, GPU, or the like).

The reduction unit can be configured to assist in reducing the power consumption associated with the access request, for example, by using any of the partial word accesses described herein.

The reduction unit can be configured to control the memory read path based on the first number of bits and the second number of bits during the read operation triggered by the read request. For example, the control signal from the reduction unit can affect the memory consumption of the read path, so as to reduce the energy consumption of the memory read path that is not related to the second number of bits requested. For example, the reduction unit can be configured to control unrelated memory read paths when the second number is less than the first number.

As explained above, the integrated circuit can be included in any of Figure 3A, Figure 3B, Figure 4 to Figure 6, Figure 7A to Figure 7D, Figure 11 to Figure 13, Figure 16 to Figure 19, Figure 22, or Figure 23 The memory chip described in any one may include the memory chip or include the memory chip in other ways.

The unrelated in-memory read path may be related to unrelated bits in the first number of bits, such as bits in the first number of bits that are not included in the second number of bits.

FIG. 50 illustrates an example integrated circuit 5000, which includes: memory cells 5001 to 5008 in a memory cell array 5050; output port 5020, which includes bits 5021 to 5028; and a read circuit system 5040, which includes memory Reading paths 5011 to 5018; and reduction unit 5030.

When using the corresponding memory read path to read the second number of bits, irrelevant bits in the first number of bits can correspond to bits that should not be read (for example, not included in the second number) The bit of the ones).

During the read operation, the reduction unit 5030 can be configured to activate the memory read path corresponding to the second number of bits, so that the activated memory read path can be configured to convey the second number of bits yuan. In these embodiments, only the memory read path corresponding to the second number of bits can be activated.

During the read operation, the reduction unit 5030 can be configured to cut off at least a part of each unrelated memory read path. For example, the unrelated memory read path may correspond to unrelated bits in the first number of bits.

It should be noted that instead of cutting off at least a part of the unrelated memory path, the reduction unit 5030 may alternatively ensure that the unrelated memory path is not activated.

Additionally or alternatively, during a read operation, the reduction unit 5030 may be configured to maintain the unrelated memory read path in a low power mode. For example, the low-power mode may include supplying voltages or currents lower than normal operating voltages or currents to unrelated memory paths, respectively.

The reduction unit 5030 can be further configured to control the bit lines of unrelated memory read paths.

Therefore, the reduction unit 5030 can be configured to load the bit lines of the relevant memory read path and maintain the bit lines of the unrelated memory read path in the low power mode. For example, only bit lines of the relevant memory read path can be loaded.

Additionally or alternatively, the reduction unit 5030 can be configured to load the bit lines of the relevant memory read path while keeping the bit lines of the unrelated memory read path inactive.

In some embodiments, the reduction unit 5030 can be configured to utilize a portion of the related memory read path during a read operation and maintain a portion of each unrelated memory read path in a low power mode , Where this part is different from the bit line.

As explained above, the memory chip can use a sense amplifier to amplify the voltage from the memory cell included in the memory chip. Therefore, the reduction unit 5030 can be configured to During operation, part of the related memory read path is used, and the sense amplifiers associated with at least some of the unrelated memory read paths are maintained in a low power mode.

In these embodiments, the reduction unit 5030 may be configured to utilize part of the related memory read path during a read operation, and associate one or more of all unrelated memory read paths. The sense amplifier is maintained in a low power mode.

Additionally or alternatively, the reduction unit 5030 may be configured to utilize a portion of the related memory read path during a read operation, and to use one or more sensing elements associated with an unrelated memory read path. Part of the unrelated memory read path after the amplifier (e.g., spatially and/or temporally) is maintained in the low power mode.

In any of the embodiments described above, the memory unit may include a row multiplexer (not shown).

In these embodiments, the reduction unit 5030 may be coupled between the row multiplexer and the output port.

Additionally or alternatively, the reduction unit 5030 may be embedded in the row multiplexer.

Additionally or alternatively, the reduction unit 5030 may be coupled between the memory cell and the row multiplexer.

The reduction unit 5030 may include a reduction sub-unit that can be independently controlled. For example, different reduced subunits can be associated with different rows of memory cells.

Although the reading operation and the reading circuit system are described above, the above embodiments can be similarly applied to the writing operation and the writing circuit system.

For example, the integrated circuit according to the present invention may include a memory cell including a memory cell, an output port, and a writing circuit system. In some embodiments, the memory unit may further include a processing unit, such as the processor sub-unit as described above. The write circuit system may include a reduction unit and a first group of memory write paths, and the memory write paths are used for Up to the first number of bits are output from the output port. The processing unit can be configured to send a write request to the memory unit to write the second number of bits from the memory unit. Additionally or alternatively, the write request may originate from an off-chip logic circuit (such as an accelerator, CPU, GPU, or the like). The reduction unit 5030 can be configured to control the memory write paths based on the first number of bits and the second number of bits during the write operation triggered by the write request.

Figure 51 illustrates a memory bank 5100 that includes the use of column and row addresses (for example, from on-chip processor sub-units or off-chip logic circuits, such as accelerators, CPUs, GPUs, or the like) for addressing The array of memory cells 5111. As shown in Figure 51, the memory cells are fed to bit lines (vertical) and word lines (horizontal, many word lines are omitted for simplicity). In addition, the column decoder 5112 can be fed with column addresses (for example, from on-chip processor sub-units, off-chip logic circuits, or memory controllers not shown in FIG. 51), and the row multiplexer 5113 can be fed There are row addresses (for example, from on-chip processor sub-units, off-chip logic circuits, or memory controllers not shown in Figure 51), and the row multiplexer 5113 can receive up to the entire line via the output bus 5115 Line output and up to one word output. In FIG. 51, the output bus 5115 of the row multiplexer 5113 is coupled to the main I/O bus 5114. In other embodiments, the output bus 5115 may be coupled to a processor sub-unit of a memory chip (e.g., as depicted in FIG. 7A) that sends column addresses and row addresses. For the sake of simplicity, the division of memory components into memory pads is not shown.

Figure 52 illustrates the memory bank 5101. In FIG. 52, the memory bank is also illustrated as including in-memory processing (PIM) logic 5116, which has an input terminal coupled to the output bus 5115. The PIM logic 5116 can generate an address (for example, including a column address and a row address) and output the address via the PIM address bus 5118 to access the memory bank. PIM logic 5116 is an example of a reduction unit (e.g., unit 5030) that also includes a processing unit. The PIM logic 5016 can control other circuits not shown in FIG. 52 that assist in power reduction. The PIM logic 5116 can further control the memory path of the memory cells including the memory group 5101.

As explained above, in some situations, the word length (for example, the number of bit lines selected for one transmission) may be large.

Under these conditions, each word used for reading and/or writing can be associated with a memory path that can consume power at various stages of the reading and/or writing operation, such as:

a. Load the bit line. In order to prevent the bit line from loading to the required value (from the capacitor on the bit line in the read cycle, or the new value to be written to the capacitor in the write cycle), it is necessary to use The sense amplifier at the end of the memory array is disabled and the capacitors that store data are not discharged or charged (otherwise, the data stored on it will be destroyed); and

b. Move the data from the sense amplifier through the row multiplexer that selects the bit line and move it to the rest of the chip (moving to the I/O bus that transfers data to and from the chip or to the I/O bus that will use the data) Embedded logic, such as a processor subunit on the same substrate as the memory).

In order to achieve power saving, the integrated circuit of the present invention can determine that some parts of a word are irrelevant at the column start time and then send a disable signal to one or more sensors for these irrelevant parts of the word Amplifier.

FIG. 53 illustrates a memory unit 5102, which includes a memory cell array 5111, a column decoder 5112, a row multiplexer 5113 coupled to an output bus 5115, and a PIM logic 5116.

The memory unit 5102 also includes a switch 5201 for enabling or disabling the channel of the bit-to-row multiplexer 5113. The switch 5201 may include an analog switch, a transistor configured to act as a switch, or any other circuit system configured to control the supply or the flow of voltage and/or current to a portion of the memory unit 5102. The sense amplifier (not shown) may be located at the end of the memory cell array, for example, before the switch 5201 (spatially and/or temporally).

The switch 5201 can be controlled by an enabling signal sent from the PIM logic 5116 via the bus 5117. When disconnected, the switches are configured to disconnect the sense amplifier (not shown) of the memory unit 5102. (Illustration), and therefore do not discharge or charge the bit line disconnected from the sense amplifier.

The switch 5201 and the PIM logic 5116 may form a reduced unit (for example, the reduced unit 5030).

In yet another example, the PIM logic 5116 may send the enabling signal to the sense amplifier (for example, when the sense amplifier has an enabling input) instead of sending it to the switch 5201.

The bit line may additionally or alternatively be disconnected at other points, for example, not at the end of the bit line and after the sense amplifier. For example, the bit line can be disconnected before entering the array 5111.

In these embodiments, power can also be saved when data is transmitted from the sense amplifier and the forwarding hardware (such as the output bus 5115).

Other embodiments (which can save less power, but are easier to implement) focus on saving the power of the row multiplexer 5113 and transferring the loss self-multiplexer 5113 to the next level circuit system. For example, as explained above, the next level of circuitry may include I/O buses of memory chips (such as bus 5115). In embodiments using in-memory processing, the next level of circuitry may additionally or alternatively include processor sub-units of memory chips (such as PIM logic 5116).

FIG. 54A illustrates a row multiplexer 5113 that is segmented into a plurality of sections 5202. Each section 5202 of the row multiplexer 5113 can be individually enabled or disabled by the enabling and/or disabling signal sent from the PIM logic 5116 via the bus 5119. The row multiplexer 5113 can also be fed by the address row bus 5118.

The embodiment of FIG. 54A can provide better control of different parts of the output from the row multiplexer 5113.

It should be noted that the control of different memory paths may have different resolutions, for example, the range is from a one-bit resolution to a multi-bit resolution. The former may be more effective in the sense of power saving. The latter may be simpler to implement and require fewer control signals.

FIG. 54B illustrates an example method 5130. For example, the method 5130 can be implemented using any of the memory cells described above with respect to FIG. 50, FIG. 51, FIG. 52, FIG. 53, or FIG. 54A.

The method 5130 may include steps 5132 and 5134.

Step 5132 may include: sending an access request by the processing unit of the integrated circuit (for example, PIM logic 5116) and sending it to the memory cell of the integrated circuit to read the second number of bits from the memory cell. The memory unit may include a memory cell (for example, the memory cell of the array 5111), an output port (for example, the output bus 5115), and a read/write circuit system. The read/write circuit system It may include a reduction unit (for example, reduction unit 5030) and a first group of memory read/write paths, which are used to output and/or input up to a first number through output ports Bits.

The access request may include a read request and/or a write request.

The memory input/output path may include a memory read path, a memory write path, and/or a path for both reading and writing.

Step 5134 may include responding to the access request.

For example, step 5134 may include controlling the memory read/write based on the first number of bits and the second number of bits by reducing the unit (e.g., unit 5030) during the access operation triggered by the access request. Write path.

Step 5134 may further include any of the following operations and/or any combination of any of the following operations. Any of the operations listed below can be performed during the response to the access request, but can also be performed before and/or after the response to the access request.

Therefore, step 5134 may include at least one of the following operations:

a. Control unrelated memory read paths when the second number is less than the first number, where the unrelated memory read paths and the first number of bits are not included in the second number of bits Bit-associated

b. Start the relevant memory read path during the read operation, wherein the relevant memory read path is configured to transmit a second number of bits;

c. Cut off at least part of each of the unrelated memory read paths during the read operation;

d. Maintain the unrelated memory read path in the low power mode during the read operation;

e. Control the bit lines of unrelated memory read paths;

f. Load the bit lines of the relevant memory read path and maintain the bit lines of the unrelated memory read path in low power mode;

g. Load the bit lines of the relevant memory read path, while keeping the bit lines of the unrelated memory read path inactive;

h. During a read operation, use a portion of the related memory read path and maintain a portion of each unrelated memory read path in a low power mode, where the portion is different from the bit line;

i. Use part of the related memory read path during a read operation and maintain the sense amplifiers used for at least some of the unrelated memory read paths in a low power mode;

j. Use part of the relevant memory read path during a read operation and maintain the sense amplifiers of at least some of the unrelated memory read paths in a low power mode; and

k. Use part of the related memory read path during the read operation and maintain the unrelated memory read path after the sense amplifier of the unrelated memory read path in a low power mode.

The low power mode or the idle mode may include a mode in which the power consumption of the memory access path is lower than the power consumption of the memory access path when the memory access path is used for an access operation. In some embodiments, the low power mode may even involve cutting off the memory access path. The low power mode may additionally or alternatively include not activating the memory access path.

It should be noted that the power reduction that occurs during the bit line phase may require knowledge of the relevance or irrelevance of the memory access path before opening the word line. Power reduction that occurs elsewhere (for example, in a row multiplexer) may alternatively allow the relevance of the memory access path to be determined on each access Or irrelevance.

Fast and low power startup and fast memory access

DRAM and other memory types (such as SRAM, flash memory, or the like) are often built from memory, and these memory banks are usually built to allow column and row access schemes.

Figure 55 illustrates an example of a memory chip 5140 that includes a plurality of memory pads and associated logic (such as column and row decoders, depicted as RD and COL respectively in Figure 55). In the example of FIG. 55, the pads are grouped into groups and have word lines and bit lines passing through them. The memory pads and associated logic are represented as 5141, 5142, 5143, 5144, 5145, and 5146 in FIG. 55, and share at least one bus 5147.

The memory chip 5140 may be included in any of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22 or 23. The memory chip described may include the memory chip or include the memory chip in other ways.

For example, in DRAM, the consumption associated with starting a new row (for example, a new line ready for access) is high. Once the first line is activated (also known as open), the data in that row can be used for faster access. In DRAM, this access may be performed in a random manner.

The two issues associated with starting a new line are power and time:

a. Due to the sudden increase in current caused by accessing all the capacitors on the line and having to load the line together, the power will increase (for example, when the line with only a few memory banks is opened, the power can reach several amperes); and

b. The time delay problem is mainly related to the time it takes to load the column (word) line and then load the bit (row) line.

Some embodiments of the present invention may include systems and methods to reduce peak power consumption during a start-up line and reduce line start-up time. Some embodiments may sacrifice complete random access within a line at least to a certain extent to reduce these power and time costs.

For example, in one embodiment, the memory unit may include a first memory pad, a second memory pad, and an activation unit configured to activate the first group included in the first memory pad Group memory cells without activating the second group of memory cells included in the second memory pad. The one group of memory cells and the two groups of memory cells may all belong to a single row of the memory cell.

Alternatively, the activation unit may be configured to activate the second group of memory cells included in the second memory pad without activating the first group of memory cells.

In some embodiments, the activation unit may be configured to activate the second group of memory cells after activating the first group of memory cells.

For example, the activation unit may be configured to activate the second group of memory cells after a delay period that starts after the activation of the first group of memory cells has completed has expired.

Additionally or alternatively, the activation unit may be configured to activate the second group of memory cells based on the value of the signal, the signal being coupled to the first word line section of the first group of memory cells Produced on.

In any of the above-described embodiments, the activation unit may include an intermediate circuit disposed between the first word line section and the second word line section. In these embodiments, the first word line segment can be coupled to the first memory cell and the second word line segment can be coupled to the second memory cell. Non-limiting examples of intermediate circuits include switches, flip-flops, buffers, inverters, and the like, some of which are described throughout FIGS. 56 to 61.

In some embodiments, the second memory cell may be coupled to the second word line segment. In these embodiments, the second word line segment may be coupled to the bypass word line diameter passing through at least the first memory pad. An example of such a bypass path is illustrated in Figure 61.

The activation unit may include a control unit configured to control the voltage (and/or current) to the first group of memory cells based on the activation signal from the word line associated with a single row And the supply of the second group of memory cells.

In another example embodiment, the memory unit may include a first memory pad, a second memory pad, and an activation unit configured to supply activation signals to the first group of first memory pads Memory cells, and delay the supply of the activation signal to the second group of memory cells of the second memory pad at least until the activation of the first group of memory cells has been completed. The one group of memory cells and the two groups of memory cells can belong to a single row of the memory cell.

For example, the activation unit may include a delay unit that can be configured to delay the supply of the activation signal.

Additionally or alternatively, the activation unit may include a comparator that may be configured to receive the activation signal at its input and control the delay unit based on at least one characteristic of the activation signal.

In another example embodiment, the memory unit may include a first memory pad, a second memory pad, and an isolation unit, and the isolation unit may be configured to: in the first memory cell of the first memory pad Separate the first memory cell from the second memory cell of the second memory pad during the activated initial activation period; and couple the first memory cell after the initial activation period To these two memory cells. The first memory cell and the second memory cell may belong to a single row of memory cells.

In the following example, the memory pad itself may not need to be modified. In some instances, embodiments may rely on minor modifications to the memory bank.

The following diagram depicts the mechanism of shortening the word signal added to the memory bank to split the word line into several shorter parts.

In the following figures, various memory bank components are omitted for clarity.

Figure 56 to Figure 61 illustrate the parts of the memory group (represented as 5140(1), 5140(2), 5140(3), 5140(4), 5140(5) and 5149(6)), which include Column decoder 5112 and multiple memory pads grouped in different groups (such as 5150(1), 5150(2), 5150(3), 5150(4), 5150(5), 5150(6), 5151(1), 5151(2), 5151(3), 5151(4), 5151(5), 5151(6), 5152(1), 5152(2), 5152(3), 5152(4), 5152(5) and 5152(6)).

The memory pads arranged in a row may include different groups.

Figures 56 to 59 and Figure 61 illustrate nine groups of memory pads, where each group includes a pair of memory pads. Any number of groups can be used, and each group has any number of memory pads.

The memory pads 5150(1), 5150(2), 5150(3), 5150(4), 5150(5) and 5150(6) are arranged in a row, share multiple memory lines, and are divided into three groups: The first upper group, which includes memory pads 5150(1) and 5150(2); the second upper group, which includes memory pads 5150(3) and 5150(4); and the third upper group, which Including memory pads 5150(5) and 5150(6).

Similarly, the memory pads 5151(1), 5151(2), 5151(3), 5151(4), 5151(5) and 5151(6) are arranged in a row, share multiple memory lines and are divided into three groups Group: the first middle group, which includes memory pads 5151(1) and 5151(2); the second middle group, which includes memory pads 5151(3) and 5151(4); and the third middle group , Which includes memory pads 5151(5) and 5151(6).

In addition, the memory pads 5152(1), 5152(2), 5152(3), 5152(4), 5152(5) and 5152(6) are arranged in a row, share multiple memory lines and are grouped into three groups Group: the first lower group, which includes memory pads 5152(1) and 5152(2); the second lower group, which includes memory pads 5152(3) and 5152(4); and the third lower group Group, which includes memory pads 5152(5) and 5152(6). Any number of memory pads can be arranged in a row and share memory lines, and can be divided into any number of groups.

For example, the number of memory pads in each group can be one, two, or more than two.

As explained above, the activation circuit can be configured to activate a group of memory pads without activating memory pads that share the same memory line or are at least coupled to different memory line segments with the same line address Another group.

Figure 56 to Figure 61 illustrate different examples of starting circuits. In some embodiments, at least a part of the activation circuit (such as an intermediate circuit) may be located between the memory pad groups to allow activation One group of memory pads does not activate another group of memory pads in the same row.

FIG. 56 illustrates the intermediate circuits such as delay or isolation circuits 5153(1) to 5153(3) located between different lines of the first upper group of memory and different lines of the second upper group of memory pads. ).

FIG. 56 also illustrates the intermediate circuits such as delay or isolation circuits 5154(1) to 5154(3) located between different lines of the second upper group of memory and different lines of the third upper group of memory pads. ). In addition, some delay or isolation circuits are positioned between the groups formed by the memory pads of the middle group. In addition, some delay or isolation circuits are positioned between the groups formed by the memory pads of the lower group.

The delay or isolation circuits can delay or stop the propagation of the word line signal from the column decoder 5112 along one column to another group.

Figure 57 illustrates an intermediate circuit, such as a delay or isolation circuit, including flip-flops (such as 5155(1) to 5155(3) and 5156(1) to 5156(3)).

When the activation signal is injected into the word line, one of the first pad groups (depending on the word line) is activated, while the other groups along the word line remain inactive. Other groups can be activated in the next clock cycle. For example, the second group in other groups can be activated in the next clock cycle, and the third group in other groups can be activated after another clock cycle.

The flip-flop can include a D-type flip-flop or any other type of flip-flop. For simplicity, the clock fed to the D-type flip-flop is omitted from the diagram.

Therefore, access to the first group can use power to charge only part of the word line associated with the first group, which charge is faster and requires less current than charging the entire word line.

More than one flip-flop can be used between the memory pad groups, thereby increasing the delay between the open parts. Additionally or alternatively, embodiments may use a slower clock to increase the delay.

In addition, the activated group may still contain the group from the previous line value used. Lift For example, this method can allow new line segments to be activated while still accessing the data of the previous line, thereby reducing the penalty associated with activating the new line.

Therefore, some embodiments may have a first group activated and allow other groups of previously activated lines to remain active, where the signals of the bit lines do not interfere with each other.

In addition, some embodiments may include switches and control signals. These control signals can be controlled by the group controller or by adding a flip-flop between the control signals (for example, producing the same timing effect as the mechanism described above).

Figure 58 illustrates intermediate circuits such as delay or isolation circuits that are switches (such as 5157(1) to 5157(3) and 5158(1) to 5158(3)) and are located in one group and another group Between groups. A set of switches located between the groups can be controlled by dedicated control signals. In FIG. 58, the control signal can be sent by the column control unit 5160(1) and delayed by the sequence of one or more delay units (e.g., units 5160(2) and 5160(3)) between different sets of switches.

Figure 59 illustrates intermediate circuits such as delay or isolation circuits, which are sequences of inverter gates or buffers (such as 5159(1) to 5159(3) and 5159'(1) to 5159'(3)) And positioned between the memory pad groups.

Instead of switches, buffers can be used between memory pad groups. The buffer may not allow the voltage to be reduced along the word line between the switches. The voltage reduction is an effect that sometimes occurs when a single transistor structure is used.

Other embodiments may allow more random access, and still provide extremely low startup power and time by using the area added to the memory bank.

An example is shown in Figure 60, which illustrates the use of global word lines (such as 5152(1) to 5152(8)) located close to the memory pad. These word lines may or may not pass through the memory pad and are coupled to the word lines in the memory pad via intermediate circuits such as switches (such as 5157(1) to 5157(8)). These switches can control which memory pad will be activated and allow the memory controller to only activate at each point in time Related line part. Unlike the above-described embodiment that uses the sequential activation of the wire portion, the example of FIG. 60 can provide better control.

The enabling signals such as column part enabling signals 5170(1) and 7150(2) can be derived from unshown logic, such as a memory controller.

Figure 61 illustrates that the global word line 5180 passes through the memory pad and forms a bypass path for the word line signal that may not need to be projected outside the pad. Therefore, the embodiment shown in FIG. 61 can reduce the area of the memory bank at the expense of some memory density.

In FIG. 61, the global world line can pass through the memory pad without interruption and may not be connected to the memory cell. The local word line segment can be controlled by one of the switches and connected to the memory cell in the pad.

When the memory pad group provides substantial division of word lines, the memory group can actually support full random access.

Another embodiment for slowing the dispersion of the activation signal along the word line can also save some wiring and logic, using switches and/or other buffering or isolation circuits between memory pads, instead of using dedicated enable signals and dedicated Wire to convey the enabling signal.

For example, the comparator can be used to control a switch or other buffer or isolation circuit. When the level of the signal on the word line segment monitored by the comparator reaches a certain level, the comparator can activate a switch or other buffering or isolation circuits. For example, a certain level may indicate that the previous word line segment is fully loaded.

Figure 62 illustrates a method 5190 for operating a memory cell. For example, the method 5130 can be implemented using any of the memory sets described above with respect to FIGS. 56-61.

The method 5190 may include steps 5192 and 5194.

Step 5192 may include activating the first group of memory cells included in the first memory pad of the memory unit by the activation unit, and not activating the second group of memory cells included in the second memory pad of the memory unit Group memory cells. The one group of memory cells and the two groups of memory cells can be All belong to a single row of the memory cell.

Step 5194 may include activating the second group of memory cells by the activation unit, for example, after step 5192.

When the first group of memory cells are activated, after the first group of memory cells are fully activated, after the start of the delay period expires after the activation of the first group of memory cells has been completed, Step 5194 is executed after the first group of memory cells are not activated and under similar circumstances.

The delay period can be fixed or adjustable. For example, the duration of the delay period may be based on the expected access pattern of the memory cell, or may be set regardless of the expected access pattern. The range of the delay period can be between less than one millisecond and more than one second.

In some embodiments, step 5194 may be initiated based on the value of the signal generated on the first word line segment coupled to the first group of memory cells. For example, when the value of the signal exceeds the first threshold, it can prompt the first group of memory cells to be fully activated.

Any of steps 5192 and 5194 may involve the use of an intermediate circuit (e.g., the intermediate circuit of the activation cell) disposed between the first word line section and the second word line section. The first word line segment can be coupled to the first memory cell and the second word line segment can be coupled to the second memory cell.

Examples of the intermediate circuit are described throughout FIGS. 56 to 61.

Steps 5192 and 5194 may further include controlling the supply of the activation signal from the word line associated with a single row to the first group of memory cells and the second group of memory cells by the control unit.

Use memory parallelism to speed up test time and use vectors to test logic in memory

Some embodiments of the present invention may use in-chip test units to accelerate testing.

Generally speaking, memory chip testing requires a lot of testing time. Reducing testing time can reduce production costs and also allow more testing to produce more reliable products.

Figures 63 and 64 illustrate the tester 5200 and the chip (or wafer of chips) 5210. The tester 5200 may include software for managing tests. The tester 5200 can run different data sequences to all memories 5210, and then read back the sequences to identify where the faulty bit of the memory 5210 is located. Once identified, the tester 5200 can issue a repair bit command, and if the problem can be repaired, the tester 5200 can declare that the memory 5210 has passed. In other situations, it may be declared that some chips have failed.

The tester 5200 can write a test sequence and then read back the data to compare it with the expected result.

Figure 64 shows a test system with a tester 5200 and a complete wafer 5202 of wafers (such as 5210) being tested in parallel. For example, the tester 5200 can be connected to each of the chips by a wire bus.

As shown in FIG. 64, the tester 5200 must read and write all memory chips several times, and the data must be transferred through the external chip interface.

In addition, it can be beneficial to test both the logic of the integrated circuit and the memory bank, for example, using programmable configuration information, which can be provided using regular I/O operations.

This test can also benefit from the presence of test units in the integrated circuit.

These test units can belong to integrated circuits and can analyze the test results, and find, for example, logic (for example, the processor subunit as depicted and described in FIG. 7A) and/or memory (for example, across a plurality of memories) Group).

Memory testers are usually extremely simple and exchange test vectors with integrated circuits according to a simple format. For example, there may be write vectors that include a pair of addresses of the memory entry to be written and the value to be written to the memory entry. There may also be a read vector that includes the address of the memory entry to be read. At least some of the addresses of the write vector may be the same as at least some of the addresses of the read vector. At least some other addresses for writing the vector may be different from at least some other addresses for reading the vector. When programmed, the memory tester can also receive an expected result vector, which can include the address of the memory entry to be read and the expected value to be read. Memory tester can Compare the expected value with its read value.

According to the embodiment, the logic (for example, the processor subunit) of the integrated circuit (memory with or without the integrated circuit) can be tested by the memory tester using the same protocol/format. For example, some of the values written into the vector may be commands to be executed by the logic of the integrated circuit (and may involve calculations and/or memory access, for example). The read vector and the expected result vector may be used to program the memory tester. The expected result vector may include memory entry addresses, and at least some of the memory entry addresses store calculated expected values. Therefore, the memory tester can be used to test logic and memory. Memory testers are generally simpler and cheaper than logic testers, and the proposed method allows the use of simple memory testers to perform complex logic tests.

In some embodiments, the logic in the memory can be used by only using vectors (or other data structures) without using more complex mechanisms commonly used in logic testing (such as, for example, communicating with the controller via an interface to inform the logic of which one to test) Circuit) to enable the test of logic in the memory.

Instead of using a test unit, the memory controller can be configured to receive instructions to access memory items included in the configuration information, and execute the access instructions and output the results.

Any one of the integrated circuits illustrated in FIGS. 65 to 69 can perform testing, even in the absence of test units or in the presence of test units that cannot perform the test.

Embodiments of the present invention may include methods and systems that use memory parallelism and internal chip bandwidth to accelerate and improve test time.

The method and system can be based on the memory chip test itself (as opposed to the tester running the test, reading the test results, and analyzing the results), save the results and finally allow the tester to read the results (and when necessary, back to programmatically) Memory chip, for example, to activate the redundant mechanism). The test may include testing memory or testing memory groups and logic (in the case of arithmetic memory having a functional logic portion to be tested, such as the situation described above in FIG. 7A).

In one embodiment, the method may include reading and writing data in the chip so that the external bandwidth does not limit the test.

In embodiments where the memory chip includes processor sub-units, each processor sub-unit can be programmed by test code or configuration.

In embodiments where the memory chip has a processor subunit that cannot execute the test code or does not have a processor subunit but has a memory controller, the memory controller can then be configured to read and write patterns ( For example, externally program to the controller) and mark the location of the fault (for example, write a value to a memory entry, read the entry, and receive a value different from the written value) for further analysis.

It should be noted that testing memory may need to test a large number of bits, for example, testing each bit of the memory and verifying whether the tested bit works. In addition, sometimes the memory test can be repeated under different voltage and temperature conditions.

For some defects, one or more redundant mechanisms can be activated (for example, by programming a flash memory or OTP or blowing a fuse). In addition, it may also be necessary to test the logic and analog circuits of the memory chip (eg, controller, regulator, I/O).

In one embodiment, the integrated circuit may include a substrate, a memory array disposed on the substrate, a processing array disposed on the substrate, and an interface disposed on the substrate.

The integrated circuit described herein can be included in any of FIGS. 3A, 3B, 4-6, 7A-7D, 11-13, 16-19, 22 or 23. The memory chip described in one may include the memory chip or include the memory chip in other ways.

Figures 65 to 69 illustrate various integrated circuits 5210 and testers 5200.

The integrated circuit is illustrated as including a memory bank 5212, a chip interface 5211 (such as the I/O controller 5214 and bus 5213 shared by the memory banks) and a logic unit (hereinafter referred to as "logic") 5215. Figure 66 illustrates the fuse interface 5216 and its coupling to the fuse interface and the different records The bus 5217 of the memory body group.

Figure 65 to Figure 70 also illustrate various steps in the test process, such as:

a. Write test sequence 5221 (Figure 65, Figure 67, Figure 68 and Figure 69);

b. Read back the test result 5222 (Figure 67, Figure 68 and Figure 69);

c. Write the expected result sequence 5223 (Figure 65);

d. Read the faulty address to repair 5224 (Figure 66); and

e. Programmable fuse 5225 (Figure 66).

Each memory bank can be coupled to and/or controlled by its own logic unit 5215. However, as described above, any memory group allocation to the logic unit 5215 can be provided. Therefore, the number of logic units 5215 can be different from the number of memory groups, and the logic units can control more than a single memory group or a part of a memory group, and the like.

The logic unit 5215 may include one or more test units. Figure 65 illustrates the test unit (TU) 5218 in the logic 5215. The TU may be included in all or some of the logical units 5212. It should be noted that the test unit can be separated from the logic unit or integrated with the logic unit.

Figure 65 also illustrates the test pattern generator (denoted as GEN) 5219 in the TU 5218.

The test pattern generator may be included in all or some of the test units. For simplicity, the test pattern generator and the test unit are not illustrated in FIGS. 66 to 70, but may be included in these embodiments.

The memory array may include a plurality of memory banks. In addition, the processing array may include a plurality of test units. The plurality of test units can be configured to test multiple memory banks to provide test results. The interface can be configured to output information that prompts the test result to a device outside the integrated circuit.

The plurality of test units may include at least one test pattern generator configured to generate at least one test pattern for testing one or more of the plurality of memory groups. In some embodiments, as explained above, each of the plurality of test units One may include a test pattern generator configured to generate a test pattern for use by a specific test unit of the plurality of test units to test at least one of the plurality of memory groups. As indicated above, FIG. 65 illustrates the test pattern generator (GEN) 5219 in the test unit. One or more or even all logic units may include a test pattern generator.

The at least one test pattern generator can be configured to receive instructions for generating at least one test pattern from the interface. The test pattern may include memory entries that should be accessed (for example, read and/or write) during the test and/or values to be written to these entries, and the like.

The interface can be configured to receive configuration information from external units that can be external to the integrated circuit, the configuration information including instructions for generating at least one test pattern.

The at least one test pattern generator can be configured to read configuration information from the memory array, the configuration information including commands for generating at least one test pattern.

In some embodiments, the configuration information may include vectors.

The interface can be configured to receive configuration information from a device that can be external to the integrated circuit, and the configuration information can include commands that can be at least one test pattern.

For example, the at least one test pattern may include memory array strips to be accessed during testing of the memory array.

The at least one test pattern may further include input data to be written to the memory array entry accessed during the test of the memory array.

Additionally or alternatively, the at least one test pattern may further include input data to be written to the memory array entry accessed during the test of the memory array, and the memory array entry to be accessed during the test of the memory array freely The expected value of the output data read.

In some embodiments, the plurality of test units may be configured to retrieve the test instructions from the memory array that, once executed by the plurality of test units, cause the plurality of test units to test the memory array.

For example, these test commands can be included in the configuration information.

The configuration information may include the expected result of the test of the memory array.

Additionally or alternatively, the configuration information may include the value of the output data to be read from the memory array entry accessed during the test of the memory array.

Additionally or alternatively, the configuration information may include vectors.

In some embodiments, the plurality of test units can be configured to retrieve from the memory array once executed by the plurality of test units to make the plurality of test units test the memory array and test the processing array The test instruction.

For example, these test commands can be included in the configuration information.

The configuration information may include vectors.

Additionally or alternatively, the configuration information may include the expected result of the test of the memory array and the processing array.

In some embodiments, as described above, the plurality of test units may lack a test pattern generator, which is used to generate test patterns used during testing of a plurality of memory banks.

In these embodiments, at least two of the plurality of test units may be configured to test at least two of the plurality of memory groups in parallel.

Alternatively, at least two of the plurality of test units may be configured to test at least two of the plurality of memory groups in series.

In some embodiments, the information prompting the test result may include the identifier of the faulty memory array entry.

In some embodiments, the interface can be configured to capture partial test results obtained by a plurality of test circuits multiple times during the test of the memory array.

In some embodiments, the integrated circuit may include an error correction unit, and the error correction The unit is configured to correct at least one error detected during the test of the memory array. For example, the error correction unit can be configured to use any appropriate technology to repair memory errors, such as by disabling some memory words and replacing them with redundant words.

In any of the embodiments described above, the integrated circuit may be a memory chip.

For example, the integrated circuit may include a distributed processor, where the processing array may include a plurality of subunits of the distributed processor, as depicted in FIG. 7A.

In these embodiments, each of the processor sub-units may be associated with a corresponding dedicated memory group among multiple memory groups.

In any of the above-described embodiments, the information prompting the test result can prompt the state of at least one memory bank. The state of the memory group can be provided in one or more granularities: each memory word, each entry group, or each complete memory group.

Figure 65 to Figure 66 illustrate the four steps in the test phase of the tester.

In the first step, the tester writes (5221) the test sequence, and the logical unit of the group writes data to its memory. The logic may also be complex enough to receive commands from the tester and generate the sequence itself (as explained below).

In the second step, the tester writes (5223) the expected result to the memory under test, and the logic unit compares the expected result with the data read from its memory bank to save the error list. If the logic is complex enough to produce a sequence of expected results by itself (as explained below), the writing of expected results can be simplified.

In the third step, the tester reads (5224) the fault address from the logic unit.

In the fourth step, the tester takes action on the result (5225) and the error can be repaired. For example, the tester can be connected to a specific interface to program the fuses in the memory, but any other mechanism that allows programming of the error correction mechanism in the memory can also be used.

In these embodiments, the memory tester can use vectors to test the memory.

For example, each vector can be built from the input series and the output series.

The input series can include a pair of addresses and data written to the memory (in many embodiments, this series can be modeled as a formula that allows a program to be generated when needed, such as a program executed by a logic unit).

In some embodiments, the test pattern generator can generate such vectors.

It should be noted that the vector is an example data structure, but some embodiments may use other data structures. These data structures are compatible with other test data structures generated by testers located outside the integrated circuit.

The output series may include address and data pairs, which include expected data to be read back from memory (in some embodiments, the series may be generated in addition or alternatively by the program at the execution stage, such as by a logic unit).

Memory testing usually involves executing a list of vectors, each vector writes data to memory based on the input series, and then reads back the data based on the output series and compares the data with its expected data.

In the case of a mismatch, the memory can be classified as malfunctioning, or if the memory includes a mechanism for redundancy, the redundant mechanism can be activated so that the vector is tested on the activated redundant mechanism again.

In embodiments where the memory includes a processor sub-unit (as described above with respect to FIG. 7A) or contains many memory controllers, the entire test can be handled by the group of logic units. Therefore, the memory controller or processor sub-unit can perform the test.

The memory controller can be programmed from the tester, and the test results can be saved in the controller itself for later read by the tester.

In order to configure and test the operation of the logic unit, the tester can configure the logic unit for memory access and confirm that the result can be read by the memory access.

For example, the input vector may contain a stylized sequence for the logic unit, and the output vector may contain the expected result of this test. For example, if a logic unit such as a processor sub-unit includes a multiplier or an adder configured to perform operations on two addresses in memory, the input vector may include one of writing data to the memory Group commands and a group command to the adder/multiplier logic. As long as the adder/multiplier result can be read back to the output vector, the result can be sent to the tester.

The test may further include loading logic configuration from memory and sending logic output to memory.

In embodiments where the logic unit loads its configuration from memory (for example, if the logic is a memory controller), the logic unit can run code from the memory itself.

Therefore, the input vector can include a program for the logic unit, and the program itself can test various circuits in the logic unit.

Therefore, testing may not be limited to receiving vectors in a format used by an external tester.

If the command loaded into the logic unit is issued to the logic unit to write the results back to the memory bank, the tester can read their results and compare the results with the expected output series.

For example, the vector written to the memory may be or may include a test program for the logic unit (for example, the test may assume that the memory is valid, but even if the memory is invalid, the written test program will still not work and the test Will fail, which is an acceptable result because the chip is invalid anyway) and/or how the logic unit runs the code and writes the result back to memory. Since all the tests of the logic unit can be performed through the memory (for example, the logic test input is written to the memory and the test result is written back to the memory), the tester can use the input sequence and the expected output sequence to run simple Vector test.

The logical configuration and results can be accessed as read and/or write commands.

Figure 68 illustrates the tester 5200 sending a write test sequence 5221, which is a vector.

The part of the vector includes the test code 5232 split between the memory bank 5212 coupled to the logic 5215 of the processing array.

Each logic 5215 can execute code 5232 stored in its associated memory bank, and the execution may include accessing one or more memory banks, performing calculations and storing results (for example, test results 5231) in memory Body group 5212.

The test result may be sent back by the tester 5200 (eg, read back the result 5222).

This allows the logic 5215 to be controlled by commands received by the I/O controller 5214.

In Figure 68, the I/O controller 5214 is connected to the memory bank and logic. In other embodiments, logic can be connected between the I/O controller 5214 and the memory bank.

Figure 70 illustrates a method 5300 for testing a memory bank. For example, the method 5300 can be implemented using any of the memory groups described above with respect to FIGS. 65 to 69.

The method 5300 may include steps 5302, 5310, and 5320. Step 5302 may include receiving a request to test the memory bank of the integrated circuit. The integrated circuit may include a substrate, a memory array arranged on the substrate and including a memory group, a processing array arranged on the substrate, and an interface arranged on the substrate. The processing array may include a plurality of test units, as described above.

In some embodiments, the request may include configuration information, one or more vectors, commands, and the like.

In these embodiments, the configuration information may include the expected results, commands, data of the memory array test, the value of the output data read from the memory array entry to be accessed during the test of the memory array, and the test Patterns, and similar ones.

The test pattern may include at least one of the following: (i) memory array entries to be accessed during the test of the memory array, (ii) to be written to the memory array entries accessed during the test of the memory array The input data of the memory array entry, or (iii) the expected value of the output data to be read from the memory array entry accessed during the test of the memory array.

Step 5302 may include at least one of the following and/or may be followed by at least one of the following:

a. At least one test pattern generator receives an instruction for generating at least one test pattern from the interface;

b. Receive configuration information through the interface and an external unit outside the free integrated circuit, the configuration information including instructions for generating at least one test pattern;

c. Read configuration information from the memory array by at least one test pattern generator, the configuration information including instructions for generating at least one test pattern;

d. Receive configuration information through the interface and an external unit outside the free integrated circuit, the configuration information includes instructions for at least one test pattern;

e. Retrieving test commands from the memory array by a plurality of test units and once executed by the plurality of test units to make the plurality of test units test the memory array; and

f. By using a plurality of test units and receiving from the memory array, once executed by the plurality of test units, the plurality of test units will test the memory array and test the processing array.

Step 5302 can be followed by step 5310. Step 5310 may include testing a plurality of memory groups by a plurality of test units in response to requests to provide test results.

The method 5300 may further include receiving part of the test results obtained by the plurality of test circuits multiple times during the test period of the memory array through the interface.

Step 5310 may include at least one of the following and/or may be followed by at least one of the following:

a. Generate test patterns by one or more test pattern generators (for example, included in one, some or all of a plurality of test units) for one or more test units to test multiple memory groups At least one of;

b. Test a plurality of memory groups in parallel by at least two of the plurality of test units At least two

c. Test at least two of the plurality of memory groups in series by at least two of the plurality of test units;

d. Write the value to the memory entry, read the memory entry and compare the result; and

e. Correct at least one error detected during the test of the memory array by the error correction unit.

Step 5310 can be followed by step 5320. Step 5320 may include outputting information that prompts the test result through the interface and outside the integrated circuit.

The information prompting the test result may include the identifier of the faulty memory array entry. This can save time by not sending read data about each memory entry.

Additionally or alternatively, the information prompting the test result may prompt the state of at least one memory bank.

Therefore, in some embodiments, the information that prompts the test result can be much smaller than the total size of the data unit written to or read from the memory set during the test, and is comparable to that without the aid of the test unit The input data sent by the tester that can test the memory is much smaller.

The integrated circuit under test may include a memory chip and/or a distributed processor as described in any of the previous figures. For example, the integrated circuit described herein can be included in Figure 3A, Figure 3B, Figure 4 to Figure 6, Figure 7A to Figure 7D, Figure 11 to Figure 13, Figure 16 to Figure 19, Figure 22 or Figure The memory chip described in any one of 23 may include the memory chip, or include the memory chip in other ways.

Figure 71 illustrates an example of a method 5350 for testing the memory bank of an integrated circuit. For example, the method 5350 can be implemented using any of the memory groups described above with respect to FIGS. 65-69.

The method 5350 may include steps 5352, 5355, and 5358. Step 5352 may include borrowing The interface of the integrated circuit receives configuration information including commands. The integrated circuit including the interface may also include a substrate, a memory array including a memory group and arranged on the substrate, a processing array arranged on the substrate, and an interface arranged on the substrate.

The configuration information may include expected results, commands, data of the memory array test, the value of the output data read from the memory array entry to be accessed during the test of the memory array, test patterns, and the like.

Additionally or alternatively, the configuration information may include commands, addresses of memory entries used to write these commands, input data, and may also include memory entries used to receive output values calculated during command execution The address.

The test pattern may include at least one of the following: (i) memory array entries to be accessed during the test of the memory array, (ii) to be written to the memory array entries accessed during the test of the memory array The input data of the memory array entry, or (iii) the expected value of the output data to be read from the memory array entry accessed during the test of the memory array.

Step 5355 can be followed by step 5352. Step 5355 may include executing instructions by processing the array, the execution being performed by accessing the memory array, performing arithmetic operations, and providing results.

Step 5358 can be followed by step 5355. Step 5358 may include outputting prompt information through the interface and outside the integrated circuit.

Cyber security and tamper detection technology

The memory chip and/or processor can be the target of malicious actors and may be subject to various types of cyber attacks. In some cases, this type of attack may attempt to change the data and/or code stored in one or more memory resources. Compared to trained neural networks or other types of artificial intelligence (AI) models that depend on large amounts of data stored in memory, cyber attacks can be particularly problematic. If the stored data is manipulated or even obscured, this manipulation can be harmful. For example, if the data on which the data-intensive AI model depends is destroyed or obscured, it will rely on these models to identify other vehicles or Autonomous vehicle systems such as pedestrians may incorrectly evaluate the environment of the host vehicle. As a result, accidents may occur. As AI models become more common in a wide range of technologies, cyber attacks on data associated with such models can cause significant damage.

In other situations, cyber attacks may include one or more actors tampering or attempting to tamper with operating parameters associated with the processor or other types of integrated circuit-based logic components. For example, processors are often designed to operate within certain operating specifications. Cyber attacks involving tampering can attempt to change one or more of the operating parameters of the processor, memory unit, or other circuit, causing the processor, memory unit, or other circuit to exceed its design operating specifications (for example, clock speed, Bandwidth specifications, temperature limits, operating speeds, etc.). This tampering can cause the target hardware to malfunction.

Conventional technologies used to defend against cyber attacks may include computer programs that operate at the processor level (for example, anti-virus software or anti-malware software). Other technologies may include the use of software-based firewalls associated with routers or other hardware. Although these technologies can use software programs running outside the memory unit to fight against cyber attacks, they still need additional or alternative technologies to efficiently protect the data stored in the memory unit, especially in the accuracy of the data. And availability is critical to the operation of memory-intensive applications such as neural networks. The embodiments of the present invention can provide various integrated circuit designs including memory to resist cyber attacks on the memory.

Retrieve sensitive information and commands to the integrated circuit in a secure manner (for example, during the boot process when the interface to the chip/integrated circuit is not functional) and then maintain the sensitive information and commands in the integrated circuit. Do not expose it to the outside of the integrated circuit, which can increase the security of sensitive information and commands. CPUs and other types of processing units are vulnerable to network attacks, especially when their CPU/processing units operate together with external memory. The disclosed embodiment including the distributed processor subunits arranged on the memory chip in the memory array may not be vulnerable to cyber attacks and tampering (for example, this is because the processing takes place within the memory chip). The array includes a plurality of memory banks. Any combination that includes the disclosed security measures discussed in more detail below can further reduce the impact of the disclosed embodiments Susceptibility to cyber attacks and/or tampering.

FIG. 72A is a diagrammatic representation of an integrated circuit 7200 including a memory array and a processing array in accordance with an embodiment of the present invention. For example, the integrated circuit 7200 may include any of the on-memory-chip distributed processor architectures (and features) described in the above sections and throughout the present invention. The memory array and the processing array can be formed on a common substrate, and in some disclosed embodiments, the integrated circuit 7200 can constitute a memory chip. For example, as discussed above, the integrated circuit 7200 may include a memory chip including a plurality of memory banks and a plurality of processor sub-units spatially distributed on the memory chip, of which a plurality of Each of the memory groups is associated with a dedicated one or more of the plurality of processor subunits. In some cases, each processor subunit can be dedicated to one or more memory banks.

In some embodiments, the memory array may include a plurality of discrete memory groups 7210_1, 7210_2...7210_J1, 7210_Jn, as shown in FIG. 72A. According to an embodiment of the present invention, the memory array 7210 may include one or more types of memory, including, for example, volatile memory (such as RAM, DRAM, SRAM, phase change RAM (PRAM), magnetoresistive RAM (MRAM)) , Resistive RAM (ReRAM) or the like) or non-volatile memory (such as flash memory or ROM). According to some embodiments of the present invention, the memory banks 7210_1 to 7210_Jn may include a plurality of MOS memory structures.

As mentioned above, the processing array may include a plurality of processor subunits 7220_1 to 7220_K. In some embodiments, each of the processor sub-units 7220_1 to 7220_K may be associated with one or more of a plurality of discrete memory groups 7210_1 to 7210_Jn. Although the example embodiment of FIG. 72A illustrates that each processor subunit is associated with two discrete memory banks 7210, it should be understood that each processor subunit can be associated with any number of discrete dedicated memory banks. And vice versa, each memory bank can be associated with any number of processor subunits. According to the embodiment of the present invention, the number of discrete memory groups included in the memory array of the integrated circuit 7200 can be equal It is less than, less than, or greater than the number of processor subunits included in the processing array of the integrated circuit 7200.

The integrated circuit 7200 may further include a plurality of first bus bars 7260 in accordance with the embodiment of the present invention (and as described in the above section). Each bus 7260 can connect the processor subunit 7220_k to the corresponding dedicated memory bank 7210_j. According to some embodiments of the present invention, the integrated circuit 7200 may further include a plurality of second bus bars 7261. Each bus 7261 can connect the processor sub-unit 7220_k to another processor sub-unit 7220_k+1. As shown in FIG. 72A, a plurality of processor sub-units 7220_1 to 7220_K may be connected to each other via a bus 7261. Although FIG. 72A illustrates that the plurality of processor sub-units 7220_1 to 7220_K forming a loop are connected in series via the bus 7261, it should be understood that the processor unit 7220 may be connected in any other manner. For example, in some situations, a particular processor sub-unit may not be connected to other processor sub-units via the bus 7261. In other conditions, a specific processor sub-unit may be connected to only one other processor sub-unit, and in other other conditions, a specific processor sub-unit may be connected to two or more than two via one or more busbars 7261. Other processor sub-units (e.g., form a series connection, a parallel connection, a branch connection, etc.). It should be noted that the embodiment of the integrated circuit 7200 described herein is only illustrative. In some cases, the integrated circuit 7200 may have different internal components and connections, and in other cases, one or more of the internal components and the described connections may be omitted (for example, depending on the needs of a particular application).

Referring back to FIG. 72A, the integrated circuit 7200 may include one or more structures for implementing at least one safety measure with respect to the integrated circuit 7200. In some cases, these structures can be configured to detect cyber attacks that manipulate or obscure (or attempt to manipulate or obscure) data stored in one or more of the memory banks. In other situations, these structures can be configured to detect tampering with the operating parameters associated with the integrated circuit 7200, or tampering directly or indirectly affects one or more of one or more operations associated with the integrated circuit 7200 Hardware components (whether included in the integrated circuit 7200 or outside the integrated circuit 7200).

In some cases, the controller 7240 may be included in the integrated circuit 7200. The controller 7240 may be connected to, for example, the processor sub-units 7220_1...7220_k via one or more bus bars 7250 One or more of them. The controller 7240 can also be connected to one or more of the memory banks 7210_1...7210_Jn. Although the example embodiment of FIG. 72A shows one controller 7240, it should be understood that the controller 7240 may include multiple processor elements and/or logic circuits. In the disclosed embodiment, the controller 7240 may be configured to implement at least one safety measure with respect to at least one operation of the integrated circuit 7200. In addition, in the disclosed embodiment, if at least one safety measure is triggered, the controller 7240 may be configured to take (or cause) one or more remedial actions.

According to some embodiments of the present invention, the at least one security measure may include a controller implementation processing program for locking access to certain aspects of the integrated circuit 7200. Access locking involves making the controller prevent access (reading and/or writing) to certain areas of the memory from outside the chip. Access control can be applied based on address resolution, part of memory bank resolution, memory bank resolution, and the like. In some cases, one or more physical locations in the memory associated with the integrated circuit 7200 may be locked (for example, one or more memory groups or one or more of the memory groups of the integrated circuit 7200 Any part of). In some embodiments, the controller 7240 may lock access to certain parts of the integrated circuit 7200 associated with the execution of an artificial intelligence model (or other type of software-based system). For example, in some embodiments, the controller 7240 may lock access to the weights of the neural network model stored in the memory associated with the integrated circuit 7200. It should be noted that a software program (ie, a model) may include three components, including: input data of the program, code data of the program, and output data of the execution program. These components can also be applied to neural network models. During the operation of the model, input data can be generated and fed to the model, and the execution model can generate output data for reading. However, the code and data values (for example, predetermined model weights, etc.) associated with executing the model using the received input data may remain fixed.

As described herein, locking may refer to an operation in which the controller does not allow read or write operations relative to certain areas of the memory from outside the chip/integrated circuit, for example. The controller through which the chip/integrated circuit I/O can pass can not only lock all memory banks, but also lock any range of memory addresses in the memory banks, from a single memory address to all including the available memory banks Address range (Or any address range in between).

Because the memory location associated with receiving input data and storing output data is associated with changing values and interactions with components outside the integrated circuit 7200 (for example, components that supply input data or receive output data), it is locked to each other. Waiting for memory location access may be impractical in some situations. On the other hand, restricting access to memory locations associated with model code and fixed data values can effectively resist certain types of cyber attacks. Therefore, in some embodiments, as a security measure, the memory associated with the program code and data value (for example, the memory not used for writing/receiving input data and reading/providing output data) can be locked. Restricting access may include locking certain memory locations so that certain code and/or data values (for example, their code and/or data values associated with the execution model based on the received input data) cannot be changed. In addition, the memory area associated with intermediate data (for example, data generated during the execution of the model) can also be locked to resist external access. Therefore, although various arithmetic logics (whether on the integrated circuit 7200 board or located outside the integrated circuit 7200) can provide data to or from memory locations associated with receiving input data or capturing output data generated The memory location receives data, but this calculation logic will not be able to access or modify the memory location that stores the code and data values associated with program execution based on the received input data.

In addition to locking the memory location on the integrated circuit 7200 to provide security measures, it is also possible to restrict certain arithmetic logic elements (and their access to them) that are configured to execute the code associated with a specific program or model. Memory area) to implement other security measures. In some cases, it may be relative to the arithmetic logic (and its associated memory area) located on the integrated circuit 7200 (e.g., arithmetic memory (e.g., a memory that includes computing power, such as the memory disclosed herein) Distributed processors on bulk chips), etc.) implement this access constraint. It can also be locked/restricted to any execution associated with the code stored in the locked memory portion of the integrated circuit 7200 or related to any access to the data value stored in the locked memory portion of the integrated circuit 7200 The access to the associated operation logic (and the associated memory location) is independent of whether the operation logic is located on the integrated circuit 7200 board. Limited liability The access to the operational logic of the execution program/model can further ensure that the code and data values associated with the operation of the received input data are still protected from manipulation, obscuration, etc.

The security measures implemented by the controller can be implemented in any suitable manner, including locking or restricting access to hardware-based areas associated with certain portions of the memory array of the integrated circuit 7200. In some embodiments, this locking can be implemented by adding or supplying commands to the controller 7240 that is configured to cause the controller 7240 to lock certain memory portions. In some embodiments, the hardware-based memory portion to be locked may be specified by a specific memory address (for example, an address associated with any memory element of the memory group 7210_1...7210_J2, etc.). In some embodiments, the locked area of the memory can be kept fixed during the execution of the program or model. In other situations, the locked area can be configurable. That is, in some situations, a command can be supplied to the controller 7240 so that the lock area can be changed during the execution of the program or model. For example, at a specific time, certain memory locations can be added to the locked area of the memory. Or at a specific time, certain memory locations (for example, previously locked memory locations) can be excluded from the locked area of the memory.

Any suitable method can be used to lock certain memory positions. In some cases, the records of the locked memory location (for example, files, databases, data structures, etc.) that store and identify the locked memory address can be accessed by the controller 7240, so that the controller 7240 can determine a certain memory Whether the physical request is related to the locked memory location. In some cases, the controller 7240 maintains a database of locked addresses to use to control access to certain memory locations. In other situations, the controller may have a table or a collection of one or more registers that can be configured until locked, and may include identifying the location of the memory to be locked (for example, it should be restricted from outside the chip to the memory The fixed preset value of the memory access of the location. For example, when a memory access is requested, the controller 7240 may compare the memory address associated with the memory access request with the locked memory address. If it is determined that the memory address associated with the memory access request is in the list of locked memory addresses, the memory access request (for example, read or write operation) can be rejected.

As discussed above, at least one security measure may include locking access to certain memory portions of the memory array 7210 that are not used for receiving input data or for providing access to generated output data. In some cases, the memory part in the locked area can be adjusted. For example, the locked memory portion can be unlocked, and the non-locked memory portion can be locked. Any suitable method can be used to partially unlock the locked memory. For example, the implemented security measures may include complex passwords required to unlock one or more parts of the locked memory area.

After detecting any action against the implemented security measures, the implemented security measures can be triggered. For example, an attempt to access a portion of the locked memory (whether it is a read or write request) can trigger security measures. In addition, if the entered complex password (for example, an attempt to unlock part of the locked memory) does not match the predetermined complex password, a security measure can be triggered. In some situations, if the correct complex password is not provided in the allowable threshold number of complex password entry attempts (eg, 1, 2, 3, etc.), security measures can be triggered.

The memory part can be locked at any suitable time. For example, in some situations, the memory portion can be locked at various times during program execution. In other situations, the memory part can be locked after startup or before the program/model is executed. For example, it can be combined with the programming of the program/model code or determine and identify the memory address to be locked after generating and storing the data to be accessed by the program/model. In this way, the vulnerability of attacks on the memory array 7210 can be reduced or eliminated during the time period at the beginning or after the execution of the program/model, after the data to be used by the program/model has been generated and stored.

The unlocking of the locked memory can be achieved by any suitable method or at any suitable time. As described above, the locked memory can be partially unlocked after receiving the correct complex password or password. In other situations, the locked memory can be unlocked by restarting (by command or by power-off and power-on) or by deleting the entire memory array 7210. Additionally or alternatively, a release command sequence can be implemented to unlock one or more memory portions.

According to an embodiment of the present invention and as described above, the controller 7240 can be configured to Control the traffic from the integrated circuit 7200, especially the traffic from the source outside the integrated circuit 7200. For example, as shown in FIG. 72A, the controller 7240 can control the components outside the integrated circuit 7200 and the components inside the integrated circuit 7200 (for example, the memory array 7210 or the processor subunit 7220). Inter-communication. This traffic can be controlled or monitored by the controller 7240 or by the controller 7240 to control or monitor one or more buses (for example, 7250, 7260, or 7261).

According to some embodiments of the present invention, the integrated circuit 7200 can receive immutable data (e.g., fixed data; e.g. model weights, coefficients, etc.) and certain commands (e.g., code; Memory part). Here, unchangeable data may refer to data that remains fixed during the execution of the program or model and can remain unchanged until the subsequent boot process. During program execution, the integrated circuit 7200 can interact with changeable data, which can include input data to be processed and/or output data generated by processing associated with the integrated circuit 7200. As discussed above, access to the memory array 7210 or the processing array 7220 can be restricted during the execution of the program or model. For example, access can be limited to certain parts of the memory array 7210 or limited to certain processor subunits that are associated with each of the following: processing of incoming input data to be written or The interaction with the incoming input data to be written, or the processing of the generated output data to be read or the interaction with the generated output data to be read. During the execution of the program or model, the part of the memory containing the immutable data can be locked and thus made inaccessible. In some embodiments, the immutable data and/or commands associated with the portion of the memory to be locked can be included in any suitable data structure. For example, such data and/or commands can be made available to the controller 7240 through one or more configuration files that can be accessed during or after the boot sequence.

Referring back to FIG. 72A, the integrated circuit 7200 may further include a communication port 7230. As shown in FIG. 72A, the controller 7240 may be coupled between the communication port 7230 and the bus 7250, which is shared between the processing sub-units 7220_1 to 7220_K. In some embodiments, the communication port 7230 may be indirectly or directly coupled to the host computer 7270, which may include, for example, a non-volatile memory The memory 7280 is associated with the host memory. In some embodiments, the host computer 7270 can retrieve changeable data 7281 (for example, input data to be used during the execution of a program or model), unchangeable data 7282 and/or commands from its associated host memory 7280 7283. The changeable data 7181, the unchangeable data 7282, and the command 7283 can be uploaded from the host computer 7270 to the controller 7240 via the 7230 during the boot process.

FIG. 72B is a diagrammatic representation of a memory area inside an integrated circuit according to an embodiment of the present invention. As shown, FIG. 72B depicts an example of the data structure included in the host memory 7280.

Referring now to FIG. 7A3, it is another example of an integrated circuit according to the embodiment of the present invention. As shown in FIG. 73A, the controller 7240 may include a network attack detector 7241 and a response module 7242. In some embodiments of the present invention, the controller 7240 can be configured to store or access the access control rules 7243. According to some embodiments of the present invention, the access control rule 7243 may be included in a configuration file accessible by the controller 7240. In some embodiments, the access control rules 7243 can be uploaded to the controller 7240 during the boot process. The access control rules 7243 may include information that prompts the access rules associated with any of the following: changeable data 7281, unchangeable data 7282 and commands 7283 and their corresponding memory locations. As explained above, the access control rule 7243 or the configuration file may include information identifying certain memory addresses in the memory array 7210. In some embodiments, the controller 7240 can be configured to provide a locking mechanism and/or function that locks various addresses of the memory array 7210, such as a location for storing commands or unchangeable data site.

The controller 7240 can be configured to enforce access control rules 7243, for example, to prevent unauthorized entities from changing unalterable data or commands. In some embodiments, the reading of unchangeable data or commands can be prohibited according to the access control rule 7243. According to some embodiments of the present invention, the controller 7240 can be configured to determine whether an access attempt is made to certain commands or at least a part of the unchangeable data. The controller 7240 (for example, including the network attack detector 7241) can compare the memory address associated with the access request with the memory address used for immutable data and commands to detect whether the An unauthorized access attempt was made in one or more locked memory locations. In this way, for example, the network attack detector 7241 of the controller 7240 can be configured to determine whether a suspected network attack occurs, such as changing one or more commands or changing or masking and locking one or more memory parts The associated request for unchangeable information. The response module 7242 can be configured to determine how to respond to a detected network attack and/or implement a response to a detected network attack. For example, in some situations, in response to detecting an attack on data or commands in one or more locked memory locations, the response module 7242 of the controller 7240 can implement or cause the response to be implemented. The response can include For example, stop one or more operations, such as memory access operations associated with the detected attack. The response to the detected attack can also include stopping one or more operations associated with the execution of the program or model, returning a warning or other indicator of the attempted attack, confirming the prompt to the host, or deleting the entire memory Wait.

In addition to locking the memory portion, other techniques for defending against network attacks can also be implemented to provide the described security measures associated with the integrated circuit 7200. For example, in some embodiments, the controller 7240 can be configured to replicate programs or models in different memory locations and processor subunits associated with the integrated circuit 7200. In this way, the program/model and the copy of the program/model can be executed independently, and the results of the independent program/model execution can be compared. For example, the programs/models can be copied in two memory banks 7210 and executed in different processor subunits 7220 in the integrated circuit 7200. In other embodiments, the program/model can be copied in two different integrated circuits 7200. In any case, the results of the program/model execution can be compared to determine whether there is any difference between the execution of the copied program/model. The detected differences in execution results (for example, intermediate execution results, final execution results, etc.) may indicate a network attack that has changed one or more aspects of the program/model or its associated data. In some embodiments, different memory groups 7210 and processor sub-units 7220 can be assigned to execute two copy models based on the same input data. In some embodiments, the intermediate results can be compared during the execution of the two replication models based on the same input data, and if there is a mismatch between the two intermediate results at the same stage, the execution can be temporarily suspended as a potential remedial action. Execute two replicated models in the processor subunit of the same integrated circuit Under the condition of, the integrated circuit can also compare the results. This can be done without informing any entity outside the integrated circuit about the execution of the two replicated models. In other words, entities outside the chip do not know that the replicated model is running in parallel on the integrated circuit.

Figure 73B is a diagrammatic representation of a configuration for simultaneous execution of a copy model in accordance with an embodiment of the present invention.

Although a single program/model copy is described as an instance for detecting possible cyber attacks, any number of copies (for example, 1, 2, 3, or more than 3) can be used to detect possible cyber attacks. Road attack. As the number of replication and independent program/model execution increases, the level of confidence in the detection of cyber attacks can also increase. A larger number of copies can also reduce the potential success rate of a network attack, because it may be more difficult for an attacker to influence multiple program/model copies. The number of copies of the program or model can be determined during the execution phase to further increase the difficulty for network attackers to successfully influence the execution of the program or model.

In some embodiments, the replication models may be different in one or more aspects from each other. In this example, the codes associated with the two programs/models can be made different from each other, but the programs/models can be designed so that both return the same output result. At least in this way, two programs/models can be regarded as copies of each other. For example, two neural network models may have different neuron rankings relative to each other in one layer. However, despite this change in the model code, both neural network models can return the same output results. Copying the program/model in this way can make it more difficult for cyber attackers to identify these effective replicators of the program or model to be cracked, and as a result, the copied model/program not only provides redundancy to minimize cyber attacks It can also enhance the detection of network attacks (for example, by highlighting tampering or unauthorized access, in which a network attacker changes a program/model or its data, but fails to affect the copy of the program/model Make corresponding changes).

In many situations, copy programs/models (especially including copy programs/models that show code differences) can be designed so that their output does not match exactly, but constitutes a soft value (for example, approximately the same output value) instead of being accurate Fixed value. In these embodiments, it is possible to compare (e.g., use dedicated Module or by the host processor) the output results from two or more valid program/model copiers to determine whether the difference between the output results (no matter intermediate results or final results) is within a predetermined range . The difference between the output soft values does not exceed the predetermined threshold or range can be regarded as evidence of no tampering, unauthorized access, etc. On the other hand, if the difference between the output soft values exceeds a predetermined threshold or range, the difference can be regarded as evidence that a network attack has occurred in the form of tampering, unauthorized access to memory, etc. . Under these conditions, the copy program/model safety measures will be triggered and one or more remedial actions can be taken (for example, stop executing the program or model, close one or more operations of the integrated circuit 7200, in case of limited functionality Operate in safe mode, along with many other actions).

The security measures associated with the integrated circuit 7200 may also involve quantitative analysis of data associated with the executing or executed program or model. For example, in some embodiments, the controller 7240 may be configured to calculate one or more checksum/hash/cyclic redundancy checks (CRC) on the data stored in at least a portion of the memory array 7210 )/Parallel value. The calculated value can be compared with one or more predetermined values. If there is a deviation between the compared values, the deviation can be interpreted as evidence of tampering with the data stored in at least part of the memory array 7210. In some embodiments, a checksum/hash/CRC/parity value can be calculated for all memory locations associated with the memory array 7210 to identify data changes. In this example, the entire memory (or memory group) in question can be read by, for example, the host computer 7270 or the processor associated with the integrated circuit 7200 for calculating the checksum/hash/CRC /Parallel value. In other cases, the checksum/hash/CRC/parity value can be calculated for a predetermined subset of memory locations associated with the memory array 7210 to identify changes in data associated with the subset of memory locations . In some embodiments, the controller 7240 may be configured to calculate a checksum/hash/CRC/parity value associated with a predetermined data path (for example, associated with a memory access pattern), and the calculated value It can be compared with each other or with a predetermined value to determine whether tampering or another form of network attack has occurred.

By protecting one of the positions within the integrated circuit 7200 or accessible by the integrated circuit 7200 Or multiple predetermined values (for example, expected checksum/hash/CRC/parity value, expected difference of intermediate or final output results, expected difference range associated with certain values, etc.), the integrated circuit 7200 Even more securely against cyber attacks. For example, in some embodiments, one or more predetermined values can be stored in the registers of the memory array 7210, and can be used during or after each run of the model (for example, by integrated circuit The controller 7240 of the 7200) evaluates intermediate or final output results, sum check codes, etc. In some cases, the "save final result data" command can be used to update the register value to calculate a predetermined value during operation, and the calculated value can be saved in the register or another memory location. In this way, the effective output value can be used to update the predetermined value for comparison after each program or model is executed or partially executed. This technique can increase the difficulty that a network attacker may experience when trying to modify or otherwise tamper with one or more predetermined reference values designed to expose the network attacker's activities.

In operation, the CRC calculator can be used to track memory access. For example, the calculation circuit can be placed at the memory bank level, in the processor sub-unit, or at the controller, where each calculation circuit can be configured to accumulate to the CRC calculator for each memory access .

Referring now to FIG. 74A, a diagrammatic representation of another embodiment of an integrated circuit 7200 is provided. In the example embodiment represented by FIG. 74A, the controller 7240 may include a tamper detector 7245 and a response module 7246. Similar to the other disclosed embodiments, the tamper detector 7245 can be configured to detect evidence of potential tampering attempts. According to some embodiments of the present invention, the security measures associated with the integrated circuit 7200 and implemented by the controller 7240 may include, for example, comparing the actual program/model operation pattern with a predetermined/allowed operation pattern. If the actual program/model operation pattern is different from the predetermined/allowed operation pattern in one or more aspects, a safety measure can be triggered. And if a safety measure is triggered, the response module 7246 of the controller 7240 can be configured to implement one or more remedial measures in response.

FIG. 74C is a diagrammatic representation of detection elements that can be located at various points within the chip according to an illustratively disclosed embodiment. As described above, the detection components located at various points within the chip can be used to perform network attack and tampering detection, as shown in, for example, FIG. 74C. For example, a program The code can be associated with an expected number of processing events within a certain period of time. The detector shown in FIG. 74C can count the number of events (monitored by the event counter) experienced by the system during a certain period of time (monitored by the time counter). If the number of events exceeds a predetermined threshold (for example, the number of expected events during a predefined time period), tampering can be prompted. Such detectors can be included in multiple points of the system to monitor various types of events, as shown in Figure 74C.

More specifically, in some embodiments, the controller 7240 may be configured to store or access the expected program/model operation pattern 7244. For example, in some situations, the operation pattern may be represented as a curve 7247 that prompts the allowable load per time pattern and the prohibition or illegal load per time pattern. Attempts to tamper may cause the memory array 7210 or the processing array 7220 to operate outside of certain operating specifications. This can cause the memory array 7210 or the processing array 7220 to generate heat or malfunction, and can enable the data or code related to the memory array 7210 or the processing array 7220 to be changed. These changes can cause the operation pattern to exceed the allowable operation pattern as prompted by curve 7247.

According to some embodiments of the present invention, the controller 7240 may be configured to monitor the operation pattern associated with the memory array 7210 or the processing array 7220. The operation pattern can be associated with the number of access requests, the type of access requests, the timing of the access requests, and so on. The controller 7240 may be further configured to detect a tampering attack if the operation pattern is different from the allowable operation pattern.

It should be noted that the disclosed embodiments can be used not only to defend against network attacks, but also to defend against non-malicious errors in operation. For example, the disclosed embodiments can also effectively protect systems such as integrated circuit 7200 from errors caused by environmental factors such as temperature or voltage changes or levels, especially when these levels exceed those used for integrated circuits. In the case of the operating specifications of the circuit 7200.

In response to the detection of a suspected cyber attack (for example, as a response to a triggered security measure), any appropriate remedial action can be implemented. For example, the remedial action may include stopping one or more operations associated with the program/model execution, operating one or more components associated with the integrated circuit 7200 in the safe mode, and resetting one or more of the integrated circuit 7200 Multiple components are locked to additional input or access, etc.

FIG. 74B provides a flowchart representation of a method 7450 of protecting an integrated circuit from tampering according to an illustratively disclosed embodiment. For example, step 7452 may include using a controller associated with the integrated circuit to implement at least one safety measure with respect to the operation of the integrated circuit. At step 7454, if at least one safety measure is triggered, one or more remedial actions can be taken. The integrated circuit includes: a substrate; a memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; and a processing array arranged on the substrate, the processing array including a plurality of processor subunits, Each of the plurality of processor subunits is associated with one or more of the plurality of discrete memory groups.

In some embodiments, the disclosed security measures can be implemented in multiple memory chips, and at least one or more of the disclosed security mechanisms can be implemented for each memory chip/integrated circuit. In some situations, each memory chip/integrated circuit can implement the same security measures, but in some situations, different memory chips/integrated circuits can implement different security measures (for example, when different security measures May be more suitable for a certain type of operation associated with a particular integrated circuit). In some embodiments, more than one safety measure can be implemented by a specific controller of the integrated circuit. For example, a particular integrated circuit can implement any number or type of disclosed security measures. In addition, specific integrated circuit controllers can be configured to implement a number of different remedial measures in response to triggered safety measures.

It should also be noted that two or more of the above security mechanisms can be combined to improve the security against network attacks or tampering attacks. In addition, safety measures can be implemented across different integrated circuits, and these integrated circuits can coordinate the implementation of safety measures. For example, model replication can be performed within one memory chip or can be performed across different memory chips. In this example, the results from one memory chip or the results from two or more memory chips can be compared to detect potential cyber attacks or tampering attacks. In some embodiments, the copy security measures applied across multiple integrated circuits may include one or more of the following: the disclosed access locking mechanism, hash protection mechanism, model copy, program/model execution Pattern analysis, or any combination of these or other disclosed embodiments.

Multi-port processor subunit in DRAM

As described above, the disclosed embodiments of the present invention may include a distributed processor memory chip, the memory chip includes an array of processor sub-units and an array of memory banks, wherein each of the processor sub-units can be dedicated At least one of the memory bank arrays. As discussed in the following sections, distributed processor memory chips can serve as the basis for scalable systems. That is, in some cases, the distributed processor memory chip may include one or more communication ports configured to transfer data from one distributed processor memory chip to another distributed processor memory chip . In this way, any desired number of distributed processor memory chips can be linked together (for example, in series, in parallel, in a loop, or any combination thereof) to form an expandable array of distributed processor memory chips. This array can provide a flexible solution for efficiently performing memory-intensive operations and for expanding the computing resources associated with the performance of memory-intensive operations. Because the distributed processor memory chip can include clocks with different timing patterns, the embodiments disclosed in the present invention include methods to accurately control the distributed processor memory chip even when there is a difference in clock timing The characteristics of the data transfer between. Such embodiments can enable efficient data sharing among different distributed processor memory chips.

FIG. 75A is a diagrammatic representation of an expandable processor memory system including a plurality of distributed processor memory chips in accordance with an embodiment of the present invention. According to an embodiment of the present invention, the expandable processor memory system may include a plurality of distributed processor memory chips, such as a first distributed processor memory chip 7500, a second distributed processor memory chip 7500', and The third distributed processor memory chip 7500". Each of the first distributed processor memory chip 7500, the second distributed processor memory chip 7500', and the third distributed processor memory chip 7500" Those may include any of the configurations and/or features associated with any of the embodiments described in the distributed processor of the present invention.

In some embodiments, each of the first distributed processor memory chip 7500, the second distributed processor memory chip 7500', and the third distributed processor memory chip 7500" may The implementation is similar to the integrated wafer 7200 shown in FIG. 7200. As shown in FIG. 75A, the first distributed processor memory chip 7500 may include a memory array 7510, a processing array 7520, and a controller 7540. The memory array 7510, the processing array 7520, and the controller 7540 can be configured similarly to the memory array 7210, the processing array 7220, and the controller 7240 in FIG. 72A.

According to an embodiment of the present invention, the first distributed processor memory chip 7500 may include a first communication port 7530. In some embodiments, the first communication port 7530 may be configured to communicate with one or more external entities. For example, the communication port 7530 can be configured to create a distributed processor memory chip 7500 and other distributed processor memory chips (such as distributed processor memory chips 7500' and 7500") Communication connection between external entities. For example, the communication port 7530 can be indirectly or directly coupled to a host computer (for example, as illustrated in FIG. 72A) or any other computing device, communication module, etc.

According to an embodiment of the present invention, the first distributed processor memory chip 7500 may further include one or more additional communication ports configured to communicate with other distributed processor memory chips such as 7500' or 7500". In some embodiments, the one or more additional communication ports may include a second communication port 7531 and a third communication port 7532, as shown in Figure 75A. The second communication port 7531 may be configured to interact with the second distributed processing The processor memory chip 7500' communicates and establishes a communication connection between the first distributed processor memory chip 7500 and the second distributed processor memory chip 7500'. Similarly, the third communication port 7532 can be configured To communicate with the third distributed processor memory chip 7500', and establish a communication connection between the first distributed processor memory chip 7500 and the third distributed processor memory chip 7500". In some embodiments, the first distributed processor memory chip 7500 (and any of the memory chips disclosed herein) may include a plurality of communication ports, including any suitable number of communication ports (for example, 2 1, 3, 4, 5, 6, 7, 8, 9, 10, 20, 50, 100, 1000, etc.).

In some embodiments, the first communication port, the second communication port, and the third communication port correspond to The bus is associated. The corresponding bus may be a bus common to each of the first communication port, the second communication port, and the third communication port. In some embodiments, the corresponding bus associated with each of the first communication port, the second communication port, and the third communication port are connected to a plurality of discrete memory banks. In some embodiments, the first communication port is connected to at least one of a main bus inside the memory chip or at least one processor subunit included in the memory chip. In some embodiments, the second communication port is connected to at least one of a bus inside the memory chip or at least one processor subunit included in the memory chip.

Although the configuration of the disclosed distributed processor memory chip is explained relative to the first distributed processor memory chip 7500, it should be noted that the second processor memory chip 7500' and the third processor memory chip 7500" can be configured similarly to the first distributed processor memory chip 7500. For example, the second distributed processor memory chip 7500' can also include a memory array 7510', a processing array 7520', and a controller 7540' and/or a plurality of communication ports, such as ports 7530', 7531' and 7532'. Similarly, the third distributed processor memory chip 7500" may include a memory array 7510", a processing array 7520", and a controller 7540" and/or multiple communication ports, such as ports 7530", 7531" and 7532". In some embodiments, the second communication port 7531' and the third communication port 7532' of the second distributed processor memory chip 7500' can be configured to correspond to the third distributed processor memory chip 7500" and The first distributed processor memory chip 7500 communicates. Similarly, the second communication port 7531" and the third communication port 7532" of the third distributed processor memory chip 7500" can be configured to communicate with the first distributed processor respectively The distributed processor memory chip 7500 and the second distributed processor memory chip 7500' communicate. This configuration similarity between distributed processor memory chips can facilitate the expansion of computing systems based on the disclosed distributed processor memory chips. In addition, the disclosed configuration and configuration of the communication ports associated with each distributed processor memory chip can enable flexible configuration of the array of distributed processor memory chips (for example, including series connection, parallel connection, ring Connection, star connection or network connection, etc.).

According to an embodiment of the present invention, for example, the first to third distributed processor memory chips 7500, 7500' and 7500" distributed processor memory chips can communicate with each other via bus 7533. In some embodiments, bus 7533 can connect two communication ports of two different distributed processor memory chips For example, the second communication port 7531 of the first processor memory chip 7500 can be connected to the third communication port 7532' of the second processor memory chip 7500' via the bus 7533. According to an embodiment of the present invention, For example, the distributed processor memory chips of the first to third distributed processor memory chips 7500, 7500', and 7500" can also communicate with external entities (for example, a host computer) via a bus such as a bus 7534. For example, the first communication port 7530 of the first distributed processor memory chip 7500 can be connected to one or more external entities via the bus 7534. Distributed processor memory chips can be connected to each other in various ways. In some cases, distributed processor memory chips may exhibit serial connectivity, where each distributed processor memory chip is connected to a pair of adjacent distributed processor memory chips. In other situations, distributed processor memory chips can exhibit a higher degree of connectivity, where at least one distributed processor memory chip is connected to two or more other distributed processor memory chips. In some cases, all distributed processor memory chips in the plurality of memory chips can be connected to all other distributed processor memory chips in the plurality of memory chips.

As shown in FIG. 75A, bus 7533 (or any other bus associated with the embodiment of FIG. 75A) may be unidirectional. Although FIG. 75A illustrates the bus 7533 as unidirectional and has a certain data transfer flow (as indicated by the arrow shown in FIG. 75A), the bus 7533 (or any other bus in FIG. 75A) can be implemented as Two-way bus. According to some embodiments of the present invention, the bus connected between the two distributed processor memory chips can be configured to have more communication than the bus connected between the distributed processor memory chip and an external entity High speed communication speed. In some embodiments, the communication between the distributed processor memory chip and external entities can occur during a limited time period, such as during execution preparation (loading of code from the host computer, input data, weight data, etc.), Occurs during the period of time when the results generated by the execution of the neural network model are output to the host computer. In the execution of one or more programs associated with distributed processors with chips 7500, 7500' and 7500" During runtime (for example, during memory-intensive operations associated with artificial intelligence applications, etc.), the communication between the memory chips of the distributed processors can be performed via the buses 7533, 7533', etc. In some embodiments, communication between a distributed processor memory chip and an external entity may occur less frequently than the communication between two processor memory chips. According to the communication requirements and embodiments, the bus between the distributed processor memory chip and the external entity can be configured to have a communication speed equal to, greater than, or less than the communication speed of the bus between the distributed processor memory chip speed.

In some embodiments, as represented by FIG. 75A, a plurality of distributed processor memory chips such as the first to third distributed processor memory chips 7500, 7500', and 7500" may be configured to communicate with each other. As mentioned, this capability can facilitate the assembly of scalable distributed processor memory chip systems. For example, memory arrays 7510, 7500' and 7500" from the first to third processor memory chips 7500, 7500', and 7500" 7510' and 7510" and processing arrays 7520, 7520', and 7520" can be regarded as actually belonging to a single distributed processor memory chip when linked by a communication channel (such as the bus shown in FIG. 75A).

According to the embodiments of the present invention, the communication between a plurality of distributed processor memory chips and/or the communication between the distributed processor memory chips and one or more external entities can be managed in any suitable manner. In some embodiments, such communications can be managed by processing resources such as the processing array 7520 in the distributed processor memory chip 7500. In some other embodiments, for example, in order to reduce the computational load imposed by communication management on the processing resources provided by the array of distributed processors, the distributed processor memory chips such as controllers 7540, 7540', 7540" Controllers such as those can be configured to manage the communication between the distributed processor memory chips and/or the communication between the distributed processor memory chips and one or more external entities. For example, as opposed to other Distributed processor memory chips, each controller 7540, 7540' and 7540" of the first to third processor memory chips 7500, 7500' and 7500" can be configured to manage its corresponding distributed processor memory Body chip-related communications. In some embodiments, the controllers 7540, 7540', and 7540" can be configured to pass through ports such as ports 7531, 7531', 7531", Corresponding communication ports such as 7532, 7532' and 7532" control these communications.

The controllers 7540, 7540', and 7540" can also be configured to manage the communication between the distributed processor memory chips while taking into account the timing differences that may exist between the distributed processor memory chips. For example, Distributed processor memory chips (for example, 7500) can be fed in by an internal clock, which may be different from other distributed processor memory chips (for example, 7500' and 7500"). Therefore, in some embodiments, the controller 7540 can be configured to implement one or more strategies for considering the different clock timing patterns between distributed processor memory chips, and by considering the distributed processor memory The possible time deviations between bulk chips are used to manage the communication between the memory chips of distributed processors.

For example, in some embodiments, the controller 7540 of the first distributed processor memory chip 7500 can be configured to enable data to be transferred from the first distributed processor memory chip 7500 under certain conditions To the second processor memory chip 7500'. In some situations, if one or more of the processor sub-units of the first distributed processor memory chip 7500 is not ready to transmit data, the controller 7540 may inhibit the data transmission. Alternatively or in addition, if the receiving processor subunit of the second distributed processor memory chip 7500' is not ready to receive data, the controller 7540 may inhibit data transmission. In some cases, the controller 7540 may initiate data transmission from the transmitting processor sub-unit (for example, in the chip 7500) after determining that the transmitting processor sub-unit is ready to send data and the receiving processor sub-unit is ready to receive data. To the receiving processor sub-unit (for example, in the wafer 7500'). In other embodiments, the controller 7540 may initiate data transmission based only on whether the sending processor subunit is ready to send data, especially if the data can be buffered in the controller 7540 or 7540', for example, until the receiving processor subunit is ready. In the case of receiving the transmitted data.

According to an embodiment of the present invention, the controller 7540 may be configured to determine whether one or more other timing constraints are satisfied to enable data transfer. These time constraints can be related to each of the following: between the transmission time from the transmitting processor subunit and the receiving time in the receiving processor subunit Time difference, access requests from external entities (for example, host computers) to processed data, renew operations performed on memory resources (for example, memory arrays) associated with the sending or receiving processor subunits, And other.

FIG. 75E is an example timing diagram according to an embodiment of the present invention. Figure 75E illustrates the following example.

In some embodiments, the controller 7540 and other controllers associated with the distributed processor memory chip can be configured to use clock-enable signals to manage the transfer of data between the chips. For example, the processing array 7520 can be fed in by a clock. In some embodiments, for example, the controller 7540 can use a clock energizing signal (for example, shown as "to CE" in FIG. 75A) to control whether one or more processor sub-units respond to the supplied clock signal. Response. Each processor sub-unit, such as 7520_1 to 7520_K, can execute program code, and the program code may include communication commands. According to some embodiments of the present invention, the controller 7540 can control the timing of the communication command by controlling the clock enabling signals to the processor sub-units 7520_1 to 7520_K. For example, according to some embodiments, when the sending processor sub-unit (for example, in the first processor memory chip 7500) is programmed to send data in a certain cycle (for example, the 1000th clock cycle) and The receiving processor sub-unit (for example, in the second processor memory chip 7500') is programmed to receive data in a certain cycle (for example, the 1000th clock cycle), the first processor memory chip 7500 The controller 7540 of the second processor memory chip 7500' and the controller 7540' of the second processor memory chip 7500' may not allow data transfer until both the sending processor sub-unit and the receiving processor sub-unit are ready to perform data transfer. For example, the controller 7540 can "suppress" the data transmission from the transmitter processor subunit by supplying a certain clock enable signal (for example, logic low) to the transmitter processor subunit in the chip 7500. The enabling signal can prevent the transmitting processor sub-unit from transmitting data in response to the received clock signal. A certain clock energizing signal can "freeze" the entire distributed processor memory chip or any part of the distributed processor memory chip. On the other hand, the controller 7540 can make the transmitting processor sub-unit by supplying the opposite clock energizing signal (for example, logic high) to the transmitting processor sub-unit. The unit initiates data transmission, and the clock enable signal causes the sending processor subunit to respond to the received clock signal. The clock energizing signal sent by the controller 7540' can be used to control similar operations, such as receiving or not receiving by the receiving processor sub-unit of the chip 7500'.

In some embodiments, the clock energizing signal can be sent to all the processor sub-units (for example, 7520_1 to 7520_K) in the processor memory chip (for example, 7500). Generally speaking, the clock energizing signal can have the effect of causing the processor sub-units to respond to their respective clock signals or to ignore their clock signals. For example, in some situations, when the clock energizing signal is high (depending on the convention of a particular application), the processor sub-unit can respond to its clock signal and can perform one or more operations according to its clock signal timing. Instructions. On the other hand, when the clock energizing signal is low, the processor sub-unit is prevented from responding to its clock signal, so that it does not execute instructions in response to the clock timing. In other words, when the clock energizing signal is low, the processor subunit can ignore the received clock signal.

Returning to the example of FIG. 75A, any one of the controllers 7540, 7540', or 7540" can be configured to use a clock energizing signal, thereby enabling one or more processor subunits in a respective array to pair The received clock signal responds or does not respond to control the operation of the respective distributed processor memory chip. In some embodiments, the controller 7540, 7540' or 7540" can be configured to selectively advance the program Code execution, for example, when the code is related to or includes data transfer operations and their timing. In some embodiments, the controller 7540, 7540' or 7540" can be configured to use the clock energizing signal to control the communication ports 7531, 7531', 7531 between two different distributed processor memory chips. ", 7532, 7532', and 7532". In some embodiments, the controller 7540, 7540' or 7540" can be configured to use a clock enable signal to control the two The time of receiving data between two different distributed processor memory chips via any one of the communication ports 7531, 7531', 7531", 7532, 7532', and 7532".

In some embodiments, the data transfer timing between two different distributed processor memory chips can be configured based on the compilation optimization step. Compilation allows to build processing routines, which can be Efficiently assign tasks to processing sub-units without being affected by the transmission delay on the bus connected between two different processor memory chips. Compilation can be executed by the compiler in the host computer or transferred to the host computer. Generally, the transmission delay on the bus between two different processor memory chips will cause the data bottleneck of the processing sub-units that require data. The disclosed code can be used to schedule data transmission in a way that enables the processing unit to continuously receive data even when there is an unfavorable transmission delay on the bus.

Although the embodiment of FIG. 75A includes three ports for each distributed processor memory chip (7500', 7500", 7500"'), according to the disclosed embodiment, any number of ports can be included in the distributed processing For example, in some cases, distributed processor memory chips may include more or fewer ports. In the embodiment of FIG. 75B, each distributed processor memory chip (e.g., , 7500A to 7500I) can be configured with multiple ports. These ports may be substantially the same or may be different from each other. In the example shown, each distributed processor memory chip includes five ports, including a host communication Port 7570 and four chip ports 7572. The host communication port 7570 can be configured to any one of the distributed processors in the array (as shown in FIG. 75B) and, for example, relative to the distributed processor memory chip The array is located between remote host computers for communication (via bus 7534). Chip port 7572 can be configured to enable communication between distributed processor memory chips via bus 7535.

Any number of distributed processor memory chips can be connected to each other. In the example shown in FIG. 75B where each distributed processor includes four chip ports, the memory chip can energize the array in which each distributed processor memory chip is connected to two or more On two other distributed processor memory chips and in some cases, some chips can be connected to four other distributed processor memory chips. Including more chip ports in a distributed processor memory chip can achieve more interconnectivity between distributed processor memory chips.

In addition, although the distributed processor memory chips 7500A to 7500I are shown in FIG. 75B as having two different types of communication ports 7570 and 7572, in some cases, a single type of The communication port can be included in each distributed processor memory chip. In other situations, more than two different types of communication ports may be included in one or more of the distributed processor memory chips. In the example of FIG. 75C, each of the distributed processor memory chips 7500A' to 7500C' includes two (or more than two) communication ports 7570 of the same type. In this embodiment, the communication port 7570 can be configured to enable communication with an external entity such as a host computer via the bus 7534, and can also be configured to enable the distributed processor memory via the bus 7535 Communication between chips (for example, between distributed processor memory chips 7500B' and 7500C').

In some embodiments, ports provided on one or more distributed processor memory chips can be used to provide access to more than one host. For example, in the embodiment shown in FIG. 75D, the distributed processor memory chip includes two or more ports 7570. The port 7570 can constitute a host port, a chip port, or a combination of a host port and a chip port. In the example shown, the two ports 7570 and 7570' enable two different hosts (for example, host computers or computing elements or other types of logic units) to access the distributed processor memory via buses 7534 and 7534' Bulk wafer 7500A. This embodiment enables two (or more than two) different host computers to be able to access the distributed processor memory chip 7500A. However, in other embodiments, both the bus 7534 and 7534' can be connected to the same host entity, for example, where the other host entity requires additional bandwidth or the processor subunit/memory of the distributed processor memory chip 7500A Parallel access of one or more of the group.

In some cases, as shown in FIG. 75D, more than one controller 7540 and 7540' can be used to control access to the distributed processor subunit/memory bank of the distributed processor memory chip 7500A. In other situations, a single controller can be used to handle communications from one or more external host entities.

In addition, one or more buses inside the distributed processor memory chip 7500A can enable parallel access to the distributed processor subunits/memory groups of the distributed processor memory chip 7500A. For example, the distributed processor memory chip 7500A may include the first bus 7580 And a second bus 7580', which enable parallel access to, for example, distributed processor sub-units 7520_1 to 7520_6 and their corresponding dedicated memory groups 7510_1 to 7510_6. This configuration allows simultaneous access to two different locations in the distributed processor memory chip 7500A. In addition, under the condition that all ports are not used at the same time, the ports can share the hardware resources (for example, common bus and/or common controller) in the distributed processor memory chip 7500A, and can form multiple tasks. The IO of that hardware.

In some embodiments, some of the arithmetic units (for example, the processor sub-units 7520_1 to 7520_6) can be connected to an additional port (7570') or a controller, while the others are not connected to an additional port or controller. However, data from arithmetic units not connected to the extra port 7570' can pass through the internal grid of connections to the arithmetic units connected to the port 7570'. In this way, communication can be performed at the two ports 7570 and 7570' at the same time without adding an additional bus.

Although the communication ports (for example, 7530 to 7532) and the controller (for example, 7540) have been described as separate components, it should be understood that the communication ports and the controller (or any other components) can be implemented as products according to embodiments of the present invention. Body unit. Figure 76 provides a diagrammatic representation of a distributed processor memory chip 7600 with integrated controller and interface modules in accordance with an embodiment of the invention. As shown in Figure 76, the processor memory chip 7600 can be implemented as an integrated controller and interface module 7547, which is configured to execute the controller 7540 and communication ports 7530, 7531 and 7532 in Figure 75 The function. As shown in Figure 76, the controller and interface module 7547 is configured to communicate with external entities such as one or more distributed processors via interfaces 7548_1 to 7548_N similar to communication ports (for example, 7530, 7531, and 7532) Communication between multiple different entities such as memory chips. The controller and interface module 7547 can be further configured to control the communication between the distributed processor memory chip or between the distributed processor memory chip 7600 and an external entity such as a host computer. In some embodiments, the controller and interface module 7547 may include communications configured to communicate in parallel with one or more other distributed processor memory chips and with external entities such as host computers, communication modules, etc. Interfaces 7548_1 to 7548_N.

FIG. 77 provides an example of the method shown in FIG. 75 according to an embodiment of the present invention. The flow chart of the processing procedure for transferring data between the distributed processor memory chips in the extended processor memory system. For illustrative purposes, the process for transferring data will be described with reference to FIG. 75, and it is assumed that the data is transferred from the first processor memory chip 7500 to the second processor memory chip 7500'.

At step S7710, a data transmission request can be received. However, it should be noted that and as described above, in some embodiments, the data transfer request may not be necessary. For example, in some situations, the timing of data transmission may be predetermined (for example, by specific software code). Under these conditions, data transmission can continue without a separate data transmission request. Step S7710 can be executed by, for example, the controller 7540 and others. In some embodiments, the data transfer request may include transferring data from one processor subunit of the first distributed processor memory chip 7500 to another processor subunit of the second distributed processor memory chip 7500' Request.

At step S7720, the data transmission timing can be determined. As mentioned, the data transmission sequence can be predetermined and can depend on the execution order of a specific software program. Step S7720 may be executed by, for example, the controller 7540 and others. In some embodiments, the data transmission timing can be determined by considering (1) whether the transmitting processor sub-unit is ready to transmit data and/or (2) whether the receiving processor sub-unit is ready to receive data. According to the embodiment of the present invention, it may also be considered whether one or more other timing constraints are satisfied to enable this data transmission. One or more time constraints may be related to each of the following: the time difference between the transmission time of the sending processor subunit and the receiving time at the receiving processor subunit, the processing of the pair from an external entity (for example, a host computer) Data access requests, renew operations performed on memory resources (for example, memory arrays) associated with the sending or receiving processor subunits, etc. According to an embodiment of the present invention, the processing sub-unit can be fed by a clock. In some embodiments, a clock energizing signal can be used, for example, to control the clock supplied to the processing sub-unit. According to some embodiments of the present invention, the controller 7540 can control the timing of the communication command by controlling the clock enabling signals to the processor sub-units 7520_1 to 7520_K.

At step S7730, it can be determined based on the data transmission timing determined at step S7720 Perform data transfer. Step S7730 may be executed by, for example, the controller 7540 and others. For example, the transmitting processor subunit of the first processor memory chip 7500 can transmit data to the receiving processor subunit of the second processor memory chip 7500' according to the data transmission timing determined at step S7720.

The disclosed architecture is applicable to a variety of applications. For example, in some situations, the above architecture can facilitate the sharing of data between different distributed processor memory chips, such as weights or neuron values or parts of nerves associated with neural networks (especially large-scale neural networks). Meta value. In addition, certain operations such as SUM and AVG may require data from multiple different distributed processor memory chips. Under these conditions, the disclosed architecture can facilitate the sharing of this data from multiple distributed processor memory chips. In addition, for example, the disclosed architecture can facilitate the sharing of records between distributed processor memory chips to support query bonding operations.

It should also be noted that although the above embodiments have been described with respect to distributed processor memory chips, the same principles and techniques can be applied to, for example, conventional memory chips that do not include distributed processor sub-units. For example, in some situations, multiple memory chips can be combined together to form a multi-port memory chip to form an array of memory chips that does not even have an array of processor subunits. In another embodiment, multiple memory chips can be combined to form an array of connected memory, thereby actually providing the host with a larger memory containing multiple memory chips.

The internal connection of the port can be to the main bus or to one of the internal processor subunits included in the processing array.

Zero detection in memory

Some embodiments of the present invention relate to memory cells used to detect zero values stored in one or more specific addresses in a plurality of memory groups. The zero value detection feature of the disclosed memory unit can be applied to reduce the power consumption of the computing system, and additionally or alternatively, can also reduce the processing time required for retrieving zero values from the memory. This feature can be particularly relevant in the following systems: In this system, the large amount of data read is actually zero and is also used for calculation operations, such as multiplication\addition\subtraction\ and more operations For these operations, it may not be necessary to retrieve the zero value from the memory (for example, the product of the zero value and any other value is zero), and the arithmetic circuit can use the fact that one of the operands is zero. And calculate the result more efficiently in terms of energy. Under these conditions, the detection of the existence of the zero value can be used instead of memory access and retrieval of the zero value from the memory.

Throughout this section, the disclosed embodiments are described with respect to reading functions. However, it should be noted that the disclosed architecture and technology are also applicable to zero-value write operations, or other specific predetermined non-zero value operations under conditions where other values may occur more frequently.

In the disclosed embodiment, instead of retrieving the zero value from the memory, when the value is detected at a specific address, the memory unit can return the zero value indicator to one or more of the external memory units Circuits (for example, one or more processors, CPUs, etc.) located outside the memory unit. The zero value is a multi-bit zero-valued zero (for example, a zero-valued byte, a zero-valued word, a multi-bit zero value less than one bit, greater than one bit, and the like). The zero value indicator is a 1-bit signal that prompts the zero value stored in the memory. Therefore, compared to transmitting n data bits stored in the memory, it is beneficial to transmit the 1-bit zero value of the prompt signal. The transmitted zero reminder can reduce the energy consumption for transmission to 1/n, and can speed up calculations, such as calculating the input by the weight of the neuron, convolution, applying the core to the input data and interacting with the trained nerve Multiplication is involved in many other calculations associated with the Internet, artificial intelligence, and a wide range of other types of operations. To provide this functionality, the disclosed memory unit may include one or more zero-value detection logic units that can detect the presence of zero values in specific locations in the memory to prevent Retrieve the zero value (for example, via a read command) and cause the zero value indicator to be transmitted instead to a circuit system outside the memory unit (for example, using one or more control lines of the memory, associated with the memory unit) One or more bus bars, etc.). Zero detection can be performed at the memory pad level, at the group level, at the sub-group level, at the chip level, etc.

It should be noted that although the disclosed embodiments are described with respect to delivering the zero indicator to a location outside the memory chip, the disclosed embodiments and features can also be processed within the memory chip. Provides significant benefits in a partially implemented system. For example, in embodiments such as the distributed processor memory chip disclosed herein, the corresponding processor sub-units can perform processing on data in various memory groups. In many situations, such as neural network execution or data analysis where the associated data can include many zeros, the disclosed technology can speed up processing and/or reduce processing performed by the processor subunits in the distributed processor memory chip The associated power consumption.

FIG. 78A illustrates a system 7800 for detecting zero values stored in one or more specific addresses in a plurality of memory groups at the chip level according to an embodiment of the present invention, and the plurality of memory groups are implemented in the memory Bulk wafer 7810. The system 7800 may include a memory chip 7810 and a host 7820. The memory chip 7810 may include a plurality of control units and each control unit may have a dedicated memory bank. For example, the control unit can be operatively connected to a dedicated memory bank.

In some situations, such as the distributed processor memory chip disclosed herein, the processing within the memory chip may involve memory access (whether it is read or write). The distributed processor memory The chip includes processor sub-units spatially distributed in an array of memory banks. Even in the context of processing inside the memory chip, the disclosed technology of detecting the zero value associated with a read or write command can allow the internal processor unit or sub-unit to give up transmitting the actual zero value. In fact, in response to zero value detection and zero value indicator transmission (for example, to one or more internal processing sub-units), the distributed processor memory chip can be saved or would have been used to transmit the zeros in the memory chip The energy of the data value.

In another example, each of the memory chip 7810 and the host 7820 may include input/output (IO) to enable communication between the memory chip 7810 and the host 7820. Each IO can be coupled to the zero indicator line 7830A and the bus 7840A. The zero-value indicator line 7830A can transmit the zero-value indicator from the memory chip 7810 to the host 7820, where the zero-value indicator can include a zero stored in a specific address of the memory bank requested by the host 7820. The 1-bit signal generated by the memory chip 7810 after the value. After receiving the zero value indicator via the zero value indicator line 7830A, the master The machine 7820 may perform one or more predefined actions associated with the zero value indicator. For example, if the host 7820 requests the memory chip 7810 to retrieve operands for multiplication, the host 7820 can calculate the multiplication more efficiently. This is because the host 7820 will confirm from the received zero-value indicator (do not receive the actual One of the operands is zero. The host 7820 can also provide commands, data and other inputs to the memory chip 7810 via the bus 7840, and read and output from the memory chip 7810. After receiving the communication from the host 7820, the memory chip 7810 can retrieve data associated with the received communication, and transmit the retrieved data to the host 7820 via the bus 7840.

In some embodiments, the host may send a zero value indicator instead of a zero data value to the memory chip. In this way, the memory chip (for example, a controller placed on the memory chip) can store or renew the zero value in the memory without having to receive the zero data value. This update may occur based on the receipt of a zero value indicator (e.g., as part of a write command).

FIG. 78B illustrates a memory chip 7810 for detecting zero values stored in one or more specific addresses in a plurality of memory banks 7811A to 7811B at the memory bank level in accordance with an embodiment of the present invention. The memory chip 7810 may include a plurality of memory banks 7811A to 7811B and an IO bus 7812. Although FIG. 78B depicts two memory banks 7811A to 7811B implemented on the memory chip 7810, the memory chip 7810 may include any number of memory banks.

The IO bus 7812 can be configured to transmit data to/from an external chip (for example, the host 7820 in FIG. 78A) via the bus 7840B. The bus 7840B can function similarly to the bus 7840A in FIG. 78A. The IO 7812 can also transmit a zero-value indicator via the zero-value indicator line 7830B, where the zero-value indicator line 7830B can function similarly to the zero-value indicator line 7830A in FIG. 78A. The IO bus 7812 can also be configured to communicate with the memory banks 7811A to 7811B via the internal zero indicator line 7831 and the bus 7841. The IO bus 7812 can transmit the received data from the external chip to one of the memory banks 7811A to 7811B. For example, the IO bus 7812 can transmit data via the bus 7841, and the data includes data for reading and storing in the memory bank 7811A The command of the data in the specific address. The multiplexer can be included between the IO bus 7812 and the memory banks 7811A to 7811B, and can be connected by the internal zero indicator line 7831 and the bus 7841A. The multiplexer can be configured to transmit the received data from the IO bus 7812 to a specific memory group, and can be further configured to transmit the received data or the received zero-value indicator from the specific memory group To IO bus 7812.

In some situations, the host entity may only be configured to receive regular data transmissions, and may not be equipped to interpret or respond to the disclosed zero-value indicator. In this situation, the disclosed embodiments (for example, the controller/chip IO, etc.) can regenerate a zero value on the data line to the host IO to replace the zero value indicator signal, and thus can save the data transmission power inside the chip.

Each of the memory groups 7811A to 7811B may include a control unit. The control unit can detect the zero value stored in the requested address of the memory bank. After detecting the stored zero value, the control unit can generate a zero value indicator and transmit the generated zero value indicator to the IO bus 7812 via the internal zero value indicator line 7831, where the zero value indicator passes through the zero value The indicator line 7830B is further transferred to the external wafer.

FIG. 79 illustrates a memory set 7911 for detecting zero values stored in one or more of the specific addresses of a plurality of memory pads at the memory pad level in accordance with an embodiment of the present invention. In some embodiments, the memory group 7911 can be organized into memory pads 7912A to 7912B, each of which can be independently controlled and accessed independently. The memory bank 7911 may include memory pad controllers 7913A to 7913B, and the controllers may include zero value detection logic units 7914A to 7914B. Each of the memory pad controllers 7913A to 7913B can allow reading and writing of positions on the memory pads 7912A to 7912B. The memory bank 7911 may further include a read disabling element, area sense amplifiers 7915A to 7915B, and/or a global sense amplifier 7916.

Each of the memory pads 7912A to 7912B may include a plurality of memory cells. Each of the plurality of memory cells can store one binary information bit. For example, memory Any one of the cells can store zero values individually. If all memory cells in a particular memory pad store zero values, the zero value can be associated with the entire memory pad.

Each of the memory pad controllers 7913A to 7913B can be configured to access the dedicated memory pad, and read data stored in the dedicated memory pad or write data to the dedicated pad.

In some embodiments, the zero value detection logic unit 7914A or 7914B can be implemented in the memory bank 7911. One or more zero-value detection logic units 7914A to 7914B may be associated with a set of memory banks, memory sub-groups, memory pads, and one or more memory cells. The zero value detection logic unit 7914A or 7914B can detect the requested specific address (for example, the memory pad 7912A or 7912B) to store the zero value. This detection can be performed in many ways.

The first method may include using a digital comparator relative to zero. The digital comparator can be configured to take two numbers as input in binary form and determine whether the first number (the captured data) is equal to the second number (zero). If the digital comparator determines that the two numbers are equal, the zero value detection logic unit can generate a zero value indicator. The zero value indicator can be a 1-bit signal, and can enable the amplifiers (for example, area sense amplifiers 7915A to 7915B) that can send data bits to the next level (for example, the IO bus 7812 in FIG. 78B), The transmitter and buffer are disabled. The zero value indicator can be further transmitted to the global sense amplifier 7916 via the zero value indicator line 7931A or 7931B, but in some cases, the global sense amplifier can be bypassed.

The second method for zero detection may include the use of analog comparators. In addition to using the voltages of the two analog inputs for comparison, the analog comparator can also function like a digital comparator. For example, all bits can be sensed, and the comparator can act as a logical "OR" function between signals.

The third method for zero value detection can include the use of signal transmission from area sense amplifiers 7915A to 7915B to global sense amplifier 7916, where global sense amplifier 7916 is configured to sense any of the inputs Whether it is high (non-zero) and use that logic signal to control the next level of the amplifier. The area sense amplifier 7915A to 7915B and the global sense amplifier 7916 can include complex Several transistors are configured to sense low-power signals from multiple memory banks, and the amplifiers amplify small voltage swings to higher voltage levels for storage in multiple memories The data in the body group can be interpreted by at least one controller such as the memory pad controller 7913A or 7913B. For example, the memory cells can be arranged on the memory bank 7911 in columns and rows. Each line can be attached to each memory cell in the row. The lines running along the columns are called word lines, and the word lines are activated by selectively applying voltage to the word lines. The line extending along the row is called the bit line, and two of these complementary bit lines can be attached to the sense amplifier at the edge of the memory array. The number of sense amplifiers can correspond to the number of bit lines (rows) on the memory bank 7911. In order to read bits from a specific memory cell, the word line along the cell row is turned on to activate all the memory cells in the row. The stored value (0 or 1) from each cell is then available on the bit line associated with the specific cell. At the ends of the two complementary bit lines, the sense amplifier can amplify a small voltage to a normal logic level. The bit from the desired cell can then be latched into the buffer from the sense amplifier of the cell and placed on the output bus.

The fourth method for zero value detection can include: if the value is 0, use an extra bit for each word stored in the memory and stored at the write time, and use that extra bit when reading data Yuan to know whether the data is zero. This method can avoid writing all zeros to the memory, thus saving more energy.

As described above and throughout the present invention, some embodiments may include a memory unit (such as a memory unit 7800) that includes a plurality of processor sub-units. These processor sub-units may be spatially distributed on a single substrate (for example, a substrate of a memory chip such as the memory unit 7800). In addition, each of the plurality of processor subunits can be dedicated to a corresponding memory group among the plurality of memory groups of the memory unit 7800. And these memory groups dedicated to the corresponding processor sub-units can also be spatially distributed on the substrate. In some embodiments, the memory unit 7800 may be associated with a specific task (for example, performing one or more operations associated with running a neural network, etc.), and each of the processor subunits of the memory unit 7800 The person may be responsible for performing part of this task. For example In other words, each processor sub-unit can be equipped with instructions that can include data processing and memory operations, arithmetic and logical operations, and so on. In some cases, the zero value detection logic can be configured to provide a zero value indicator to one or more of the described processor subunits that are spatially distributed on the memory unit 7800.

Referring now to FIG. 80, it is a flowchart illustrating an exemplary method 8000 for detecting zero values stored in specific addresses of a plurality of memory banks in accordance with an embodiment of the present invention. The method 8000 can be performed by a memory chip (for example, the memory chip 7810 of FIG. 78B). Specifically, the controller of the memory unit (for example, the controller 7913A of FIG. 79) and the zero-value detection logic unit (for example, the zero-value detection logic unit 7914A) can perform the method 8000.

In step 8010, any suitable technique can be used to initiate a read or write operation. In some cases, the controller may receive a request to read data stored in a specific address of a plurality of discrete memory groups (for example, the memory group depicted in FIG. 78). The controller can be configured to control at least one aspect of read/write operations with respect to a plurality of discrete memory groups.

In step 8020, one or more zero value detection circuits can be used to detect the existence of zero values associated with the read or write commands. For example, a zero value detection logic unit (eg, zero value detection logic unit 7830 of FIG. 78) can detect a zero value associated with a specific address that is associated with reading or writing.

In step 8030, the controller may transmit the zero value indicator to one or more circuits outside the memory unit in response to the zero value detection performed by the zero value detection logic unit in step 8020. For example, the zero value detection logic can detect that the requested address stores the zero value, and can transmit the zero value prompt to the outside of the memory chip (or inside the memory chip, such as in the disclosed distributed processing The memory chip includes entities (for example, one or more circuits) distributed among the processor sub-units in the array of the memory bank. If the zero value associated with the read or write command is not detected, the controller may transmit the data value instead of the zero value indicator. In some embodiments, one or more circuits of the returned zero indicator may be inside the memory cell.

Although the disclosed embodiments have been described with respect to zero value detection, the same principles and techniques will be applicable to detecting other memory values (for example, 1, etc.). In some cases, in addition to the zero value indicator, the detection logic may also return one or more indicators of other values (for example, 1, etc.) associated with the read or write command, and these indicators Can be returned/transmitted when any value corresponding to the value indicator is detected. In some cases, these values can be adjusted by the user (for example, by updating one or more registers). Such updates may be especially useful in situations where you may know about the characteristics of the data set and understand (for example, as far as the user is concerned) that certain values may be more common in the data than others. Under these conditions, one, two, three, or more than three value indicators can be associated with the most common data, which is associated with the data set.

Compensation for DRAM startup penalty

In some types of memory (for example, DRAM), memory cells can be arranged in an array in a memory bank, and one row of memory cells in the array can be accessed and retrieved (read) at a time The value included in the memory cell. This reading process may involve first opening (activating) a row (or row) of memory cells to make the data values stored by the memory cells available. Next, the value of the memory cell in the open row can be sensed at the same time, and the row address can be used to cycle through individual memory cell values or groups of memory cell values (that is, words), and each A memory cell value is connected to the external data bus for reading the memory cell value. Such processing procedures are time consuming. In some cases, the memory bank open for reading may require 32 cycles of computing time, and reading the value from the open bank may require another 32 cycles. If the next row to be read is opened only after the reading operation of the currently open row is completed, significant latency can be generated. In this example, during the 32 cycles required to open the next row, no data is read, and reading each row effectively requires a total of 64 cycles instead of only 32 cycles to traverse the row data. The conventional memory system does not allow the second row in the same group to be opened while the first row is being read or written. To save latency, the next row to be opened can therefore be in a different group o in the special group for dual-row access, as discussed in further detail below. Before opening the next row, the current row can be sampled to the flip-flop or lock When the next row can be opened, all processing is done on the flip-flop\latch. If the next prediction is in the same group (and none of the above situations exist), the latency may not be avoided and the system may need to wait. These mechanisms are related to both standard memory and especially memory processing devices.

The disclosed embodiment of the present invention can reduce this latency by, for example, predicting the next memory bank to be opened before the read operation of the current open memory bank is completed. That is, if the next row can be predicted to be opened, the processing procedure for opening the next row can be started before the reading operation of the current row is completed. Depending on when the next row prediction is performed in the processing procedure, the latent time associated with opening the next row can be reduced from 32 cycles (in the specific example described above) by at least 32 cycles. In a specific example, if the next row is predicted to open 20 cycles in advance, the additional latency is only 12 cycles. In another example, if the next row is predicted to open 32 cycles in advance, there is no latent time at all. As a result, instead of requiring a total of 64 cycles to serially open and read each row, by opening the next row while reading the current row, the effective time for reading each row can be reduced.

The following mechanisms may require the current row and forecast to be in the same group, but if there is such a group that can support simultaneous activation and work on a row, these mechanisms can also be used.

In the disclosed embodiment, various techniques (discussed in more detail below) can be used to perform the next column of predictions. For example, the next row prediction can be based on pattern recognition, based on a predetermined row access schedule, based on artificial intelligence models (for example, a trained neural network used to analyze row access and make predictions for the next row to be opened) Output or based on any other suitable forecasting technique. In some embodiments, 100% success prediction can be achieved by using the delay address generator or formula described below or other methods. Prediction can include building a system that has the ability to fully predict the next row before it needs to be accessed. In some cases, the next column of predictions can be performed by the next column of predictors, which can be implemented in various ways. For example, a predictive address generator used to generate the current address for reading and/or writing the memory row. The entity that generates the address used to access the memory (read or write) can be based on any logic circuit or controller\CPU that executes software commands. Predictive address generator can include Including the pattern learning model, the pattern learning model observes the accessed rows, identifying and accessing (for example, sequential access, access to every second row, access to every third row, etc.) associated with One or more patterns and the next column to be accessed is estimated based on the observed patterns. In other examples, the predictive address generator may include applying formulas/algorithms to predict the cells in the next column to be accessed. In still other embodiments, the predictive address generator may include a trained neural network based on, for example, the current address/row being accessed, the last 2, 3, 4 Inputs of one or more addresses/columns etc. are used to output the predicted next column to be accessed (including one or more addresses associated with the predicted column). Using any of the described predictive address generators to predict the next memory bank to be accessed can significantly reduce the latent time associated with memory access. The described predictive address/row generator can be applied to any system that involves accessing memory to retrieve data. In some cases, the described predictive address/row generator and the associated technology for predicting the next bank access can be particularly suitable for systems that execute artificial intelligence models, because AI models can be compatible with Facilitate the association of repeated memory access patterns for the next row of predictions.

FIG. 81A illustrates a system 8100 for activating the next row associated with the memory bank 8180 based on the prediction of the next row in accordance with an embodiment of the present invention. The system 8100 may include a current and predicted address generator 8192, a group controller 8191, and memory groups 8180A to 8180B. The address generator can be an entity that generates addresses for accessing the memory banks 8180A to 8180B, and can be based on any logic circuit, controller, or microprocessor that executes software programs. The group controller 8191 can be configured to access the current row of the memory group 8180A (for example, using the current row identifier generated by the address generator 8192). The group controller 8191 can also be configured to activate the predicted next row to be accessed in the memory group 8180B based on the predicted row identifier generated by the address generator 8192. The following examples describe two groups. In other instances, more groups can be used. In some embodiments, there may be memory banks that allow more than one row (as discussed below) to be accessed at a time, and therefore the same processing can be performed on a single bank. As described above, the activation of the predicted next row to be accessed may start before the completion of the read operation performed with respect to the current row being accessed. Therefore, in some situations, the address generator 8192 can predict the next row to be accessed, and can perform the current row At any time before the access is completed, the predicted identifier (for example, one or more addresses) of the next column is sent to the group controller 8191. This timing may allow the group controller to start the predicted activation of the next row at any point during which the current row is being accessed and before the access to the current row is completed. In some cases, the group controller 8291 may initiate the memory group 8180 at the same time (or within several clock cycles) when the activation of the current row to be accessed is completed and/or the read operation relative to the current row has started. It predicts the start of the next column.

In some embodiments, the operation relative to the current column associated with the current address may be a read or write operation. In some embodiments, the current row and the next row may be in the same memory bank. In some embodiments, the same memory bank may allow access to the next row while the current row is being accessed. The current row and the next row can be in different memory banks. In some embodiments, the memory unit may include a processor configured to generate the current address and the predicted address. In some embodiments, the memory unit may include a distributed processor. The distributed processor may include a plurality of processor subunits of the processing array among the plurality of discrete memory groups of the memory array distributed in space. In some embodiments, the predicted address can be generated by a series of flip-flops that sample the delayed generated address. The delay can be configurable via a multiplexer that selects between flip-flops storing sampled addresses.

It should be noted that after confirming that the predicted next row is actually executing a software request to access the next row (for example, after completing a read operation relative to the current row), the predicted next row can become the current row to be accessed . In the disclosed embodiment, because the processing program for starting the predicted next row can be started before the current row read operation is completed, it may be the correct next row to be accessed after confirming that the predicted next row is the correct next row to be accessed. Fully or partially activate the next row to be accessed. This can significantly reduce the latent time associated with platoon activation. If the next column is activated so that the activation ends before or at the same time that the reading of the current column ends, the power reduction can be obtained.

The current and predicted address generator 8192 may include a row configured to identify the row to be accessed in the memory bank 8180 (for example, based on program execution) and predict the next row to be accessed (eg, based on all the rows in the row access). Observation patterns, any suitable logic components based on predetermined patterns (n+1, n+2), etc.), Operation unit, memory unit, algorithm, trained model, etc. For example, in some embodiments, the current and predicted address generator 8192 may include a counter 8192A, a current address generator 8192B, and a predicted address generator 8192C. The current address generator 8192B can be configured to generate the current address of the current row to be accessed in the memory bank 8180 based on the output of the counter 8192A, for example, based on a request from the arithmetic unit. The address associated with the current row to be accessed can be provided to the group controller 8191. The predictive address generator 8192C can be configured to determine the memory based on the output of the counter 8192A, based on a predetermined access pattern (for example, combined with the counter 8192A), or based on the output of a trained neural network or other types of pattern prediction algorithms The predicted address of the next row to be accessed in group 8180. The pattern prediction algorithm observes row accesses and predicts the next row to be accessed based on, for example, patterns associated with the observed row accesses. The address generator 8192 can provide the predicted next column address from the predicted address generator 8192C to the group controller 8191.

In some embodiments, the current address generator 8192B and the predicted address generator 8192C can be implemented inside or outside the system 8100. An external host can also be implemented outside the system 8100 and further connected to the system 8100. For example, the current address generator 8192B can be software at the external host that executes the program, and to avoid any latency, the predictive address generator 8192C can be implemented inside the system 8100 or outside the system 8100.

As mentioned, the predicted next row address can be determined using a trained neural network that predicts the next row to be accessed based on an input that can include one or more previously accessed row addresses . A trained neural network or other type of model can run within the logic associated with the predictive address generator 8192C. In some cases, the trained neural network, etc. can be executed by one or more arithmetic units external to the predictive address generator 8192C but in communication with the predictive address generator.

In some embodiments, the predicted address generator 8192C may include a copy or a substantial copy of the current address generator 8192B. In addition, the operation timings of the current address generator 8192B and the predicted address generator 8192C can be fixed or adjustable with respect to each other. For example, in some situations, The predictive address generator 8192C can be configured to output the address identifier associated with the next row to be accessed relative to the current address generator 8192B at a fixed time (for example, a fixed number of clock cycles) to output and The address identifier associated with the predicted next column. In some cases, before or after the start of the current row to be accessed, before or after the start of the read operation associated with the current row to be accessed, or after the read operation associated with the current row to be accessed At any time before the completion of the fetch operation, the predicted next list of identifiers can be generated. In some cases, the predicted next row identifier may be generated at the same time as the start of the current row to be accessed or at the same time the read operation associated with the current row to be accessed starts.

In other situations, the time between the generation of the predicted next row identifier and the start of the current row to be accessed or the start of the read operation associated with the current row may be adjustable. For example, in some situations, this time may be lengthened or shortened during the operation of the memory unit 8100 based on the value associated with one or more operating parameters. In some cases, the current temperature (or any other parameter value) associated with the memory unit or another component of the computing system may cause the current address generator 8192B and the predicted address generator 8192C to change their relative operating timing. In an embodiment, in the memory processing, the prediction mechanism may be part of that logic.

The current and predicted address generator 8192 can generate a confidence level associated with the predicted next row for access determination. This confidence level (which can be determined by the prediction address generator 8192C as part of the prediction processing program) can be used to determine, for example, whether it is during the read operation of the current row (that is, before the current row read operation has been completed and is pending Access to the next row before the identification has been confirmed) start the predicted activation of the next row. For example, in some situations, the confidence level associated with the predicted next row to be accessed may be compared with the threshold level. If the trust level drops below the threshold level, for example, the memory unit 8100 may give up starting the predicted next row. On the other hand, if the trust level exceeds the threshold level, the memory unit 8100 can initiate the activation of the predicted next row in the memory group 8180.

Any suitable way can be used to implement the mechanism of testing the confidence level of the predicted next column relative to the threshold level and the subsequent start or non-initial start of the predicted next column. In some situations Next, for example, if the confidence level associated with the predicted next column drops below the threshold, the predicted address generator 8192C may abandon outputting the predicted result of the next column to the downstream logic component. Alternatively, in this situation, the current and predicted address generator 8192 can suppress the predicted next column identifier from the group controller 8191, or the group controller (or another logic unit) can be equipped to use the predicted next The confidence level of a column is used to determine whether to start the predicted next column before the read operation associated with the current column being read is completed.

Any suitable method can be used to generate the confidence level associated with the predicted next column. In some situations, such as when the next row is predicted based on a predetermined known access pattern recognition, the predicted address generator 8192C can generate a high confidence level or in view of the predetermined pattern of row access, it can completely abandon generating a confidence level . On the other hand, the predicted address generator 8192C executes one or more algorithms to monitor column access, and outputs the predicted column based on the pattern calculated relative to the monitored column access, or one or more In the case where the training neural network or other model is configured to output the predicted next row based on the input including the most recent row access, the confidence level of the predicted next row can be determined based on any relevant parameters. For example, in some situations, the confidence level may depend on whether one or more previous predictions in the next column prove to be accurate (e.g., past performance indicators). The trust level can also be based on one or more characteristics of the input of the algorithm/model. For example, an input that includes actual column access that follows a pattern can result in a higher level of trust than actual column access that exhibits less patterning. And in some situations where randomness is detected relative to the input stream including the most recent row access, for example, the resulting reliability may be low. In addition, when randomness is detected, the next prediction process can be completely stopped, one or more of the components of the memory unit 8100 can ignore the next prediction, or any other action can be taken to abandon the activation of the prediction Next column.

In some cases, the operation with respect to the memory 8100 includes a feedback mechanism. For example, periodically or even after each next column prediction, the accuracy of the prediction address generator 8192C in predicting the actual next column to be accessed can be determined. In some cases, if the next column is predicted to be accessed If there is an error (or after a predetermined number of errors), the prediction operation of the next column of the prediction address generator 8192C can be temporarily suspended. In other cases, the predictive address generator 8192C may include a learning element so that one or more aspects of its predictive operation can be adjusted based on the received feedback regarding the accuracy of its predicting the next column to be accessed. This capability can improve the operation of the predictive address generator 8192C, so that the address generator 8192C can adapt to changing access patterns, etc.

In some embodiments, the timing of the predicted generation of the next row and/or the predicted activation of the next row may depend on the overall operation of the memory unit 8100. For example, after power-on or after resetting the memory unit 8100, the prediction of the next row to be accessed can be temporarily suspended (or the predicted next row is forwarded to the group controller 8191) (for example, for a predetermined amount of time or time). The pulse loops until the predetermined number of column access/reading has been completed, until the predicted confidence level of the next column exceeds a predetermined threshold, or based on any other suitable criteria).

FIG. 81B illustrates another configuration of the memory cell 8100 according to the illustratively disclosed embodiment. In the system 8100B of FIG. 81B, the cache memory 8193 can be associated with the group controller 8191. For example, the cache memory 8193 can be configured to store one or more data rows after the one or more data rows are accessed, and prevent the need to activate the data rows again. Therefore, the cache memory 8193 can enable the group controller 8191 to access the row data from the cache memory 8193 instead of accessing the memory group 8180. For example, the cache memory 8193 can store the last X rows of data (or any other cache memory saving strategy), and the group controller 8191 can fill the cache memory 8193 according to the predicted rows. In addition, if the predicted row is already in the cache memory 8193, there is no need to open the predicted row again, and the group controller (or the cache controller implemented in the cache memory 8193) can protect the predicted row from Was replaced. Cache 8193 can provide several benefits. First, since the cache 8193 loads the rows into the cache 8193 and the group controller can access the cache 8193 to retrieve row data, there is no need for a special group or more than one group for the next row prediction. Secondly, reading and writing to the cache memory 8193 can save energy, because the physical distance between the self-organizing controller 8191 and the cache memory 8193 is smaller than that of the self-organizing controller The physical distance between 8191 and 8180 of the memory bank. Third, compared to the memory bank 8180, the latency caused by the cache memory 8193 is usually lower, because the cache memory 8193 is smaller and closer to the controller 8191. In some cases, when the group controller 8191 activates the predicted next row in the memory group 8180, the predicted next row identifier generated by the prediction address generator can be stored in the cache memory 8193, for example . Based on program execution, etc., the current address generator 8192B can identify the actual next row to be accessed in the memory bank 8191. The identifier associated with the actual next row to be accessed can be compared with the predicted next row identifier stored in the cache memory 8193. If the actual next row to be accessed is the same as the predicted next row to be accessed, the group controller 8191 can start the read operation relative to that row after the activation of the actual next row to be accessed has been completed. Fully or partially activated due to the next column of prediction processing procedures). On the other hand, if the actual next row to be accessed (determined by the current address generator 8192B) does not match the predicted next row identifier stored in the cache memory 8193, it will not be activated relative to the complete or partial It predicts that the next row will start the read operation, but the system will start the actual next row to be accessed.

Dual boot group

As discussed, it is valuable to describe several mechanisms that allow the build to be able to activate a group of one row while the other is still being processed. Several embodiments can be provided for activating a group of additional rows while another row is being accessed. Although the embodiment only describes two column activations, it should be understood that it can be applied to more columns. In the first proposed embodiment, the memory group can be divided into memory sub-groups, and the described embodiment can be used to perform read operations relative to one row in one sub-group, and at the same time activate the prediction in the other sub-group Or the next column is required. For example, as shown in FIG. 81C, the memory bank 8180 may be configured to include multiple memory sub-groups 8181. In addition, the group controller 8191 associated with the memory group 8180 may include a plurality of sub-group controllers associated with the corresponding sub-group. The first sub-group controller among the plurality of sub-group controllers can be configured to enable access to data included in the current row of the first sub-group among the plurality of sub-groups, and the second sub-group controller among the plurality of sub-group controllers The sub-group controller can start the second of a plurality of sub-groups The next column in the subgroup. Only one row decoder can be used when only accessing the ZigZag in one subgroup at a time. Two groups can be tied to the same output bus to appear as a single group. The new single group input can also be a single address and used to open the additional column address of the next column.

Figure 81C illustrates the first and second row controllers (8183A, 8183B) of each memory subgroup 8181. The memory group 8180 may include a plurality of sub-groups 8181, as shown in FIG. 81C. In addition, the group controller 8191 may include a plurality of sub-group controllers 8183A to 8183B each associated with the corresponding sub-group 8181. The first sub-group controller 8183A of the plurality of sub-group controllers can be configured to enable access to the data in the current row of the first part included in the sub-group 8181, and the second sub-group controller 8183B can launch the The next column in the second part of group 8181.

Since activating a row directly adjacent to the row being accessed may distort the accessed row and/or damage the data being read from the accessed row, the disclosed embodiment can be configured so that the row to be activated The predicted next row can be separated from the current row of the data being accessed in the first subgroup by at least two rows (for example). In some embodiments, the column to be activated can be separated by at least one pad, so that activation can be performed in different pads. The second sub-group controller can be configured to access data included in the current row of the second sub-group, and the first sub-group controller activates the next row in the first sub-group. The activated next row of the first subgroup can be separated from the current row of the data being accessed in the second subgroup by at least two rows.

The predefined distance between the row being read/accessed and the row being activated can be determined by, for example, the hardware that couples different parts of the memory bank to different row decoders, and the software can maintain the predefined distance Distance so as not to destroy the data. The interval between the current rows can exceed two rows (for example, it can be 3 rows, 4 rows, 5 rows, and even more than 5 rows). The distance can change over time, for example based on an assessment of the distortion introduced in the stored data. Various methods can be used to evaluate the distortion, such as by calculating the signal-to-noise ratio, error rate, error code required to repair the distortion, and the like. If the two rows are far enough and the two group controllers are implemented on the same group, then two rows can actually be activated. The new architecture (implementing two controllers on the same group) prevents the opening of multiple rows in the same pad.

Figure 81D illustrates an embodiment of the next column prediction in accordance with an embodiment of the present invention. The embodiment may include an additional pipeline of flip-flops (address registers A to C). The pipeline can be implemented by any number of flip-flops (stages) to start after the address generator and delay the overall execution to use the delay required for the delayed address, and then predict the new address that can be generated (in the pipeline The beginning is below the address register C) and the current address is the end of the pipeline. In this embodiment, there is no need to duplicate the address generator. A selector (the multiplexer shown in Figure 81D) can be added to configure the delay, and the address register provides the delay.

FIG. 81E illustrates an embodiment of a memory bank in accordance with an embodiment of the present invention. The memory bank can be implemented such that if the newly activated row is far enough from the current row, the new row will not be destroyed if the new row is activated. As shown in FIG. 81E, the memory bank may include additional memory pads (black) between every two rows of pads. Therefore, a control unit (such as a column decoder) can activate multiple rows that separate one pad.

In some embodiments, the memory unit can be configured to receive the first address at a predetermined time for processing and receiving the second address for function and access.

FIG. 81F illustrates another embodiment of the memory bank in accordance with the embodiment of the present invention. The memory bank can be implemented such that if the newly activated row is far enough from the current row, the new row will not be destroyed if the new row is activated. The embodiment depicted in FIG. 81F can allow the row decoder to open rows n and n+ by ensuring that all even rows are implemented at the upper half of the memory bank and all odd rows are implemented at the lower half of the memory bank. 1. The implementation may allow access to consecutive rows that are always far enough away.

According to the disclosed embodiments, the dual-control memory bank can allow access and activation of different parts of a single memory bank, even when the dual-control memory bank is configured to output one data unit at a time. For example, as described, dual control can enable the memory bank to access the first row when the second row is activated (for example, the predicted next row or the predetermined next row to be accessed).

FIG. 82 illustrates a dual-control memory bank 8280 for reducing a memory bank activation penalty (eg, latency) according to an embodiment of the present invention. The dual control memory group 8280 can include inputs, including data input (DIN) 8290, row address (ROW) 8291, row address (COLUMN) 8292. The first command input (COMMAND_1) 8293 and the second command input (COMMAND_2) 8294. The memory bank 8280 may include a data output (Dout) 8295.

The pseudo location address can include a column address and a row address, and there are two column decoders. Other configurations of addresses can be provided, the number of column decoders can exceed two, and there can be more than a single row decoder.

The column address (ROW) 8291 can identify the column associated with a command such as a start command. Since the column can be read from or written to the column after it is started, it is then opened (after it is started), and may not need to be sent for writing to the open column or read from the open column Address.

The first command input (COMMAND_1) 8293 can be used to send a command (such as but not limited to a start command) to the column accessed by the first column decoder. The second command (COMMAND_2) input 8294 can be used to send a command (such as but not limited to a start command) to the column accessed by the second column decoder.

Data input (DIN) 8290 can be used to feed data when performing write operations.

Since the entire column cannot be read at one time, individual column segments can be read sequentially, and the row address (COLUMN) 8292 can prompt which segment (which row) of the column is to be read. For simplicity of explanation, it can be assumed that there are 2Q sectors and the line input has Q bits; Q is a positive integer exceeding one.

The dual control memory bank 8280 can operate with or without the address prediction described above with respect to FIGS. 81A to 81B. Of course, in order to reduce the operating latency, according to the disclosed embodiment, the dual control memory bank can be operated with address prediction.

83A, 83B, and 83C illustrate examples of accessing and activating the rows of the memory bank 8180. As mentioned above, assume that in one example, both the read column and the start column require 32 cycles (segments). In addition, in order to reduce the startup penalty (having a length denoted as delta), it may be beneficial to know in advance (at least the delta before the next column needs to be accessed) that the next column should be opened. In some cases, the difference can be equal to four cycles. Each memory group depicted in FIG. 83A, FIG. 83B, and FIG. 83C may include two or more than two sub-groups. Within the two or more sub-groups, in some implementations In the example, only one column can be opened at any given time. In some cases, even-numbered columns can be associated with the first sub-group, and odd-numbered columns can be associated with the second sub-group. In this example, the use of the disclosed predictive addressing embodiment can enable a memory to be started before the end of the read operation relative to another memory subgroup (the delay period before the end is reached) Start of a column of the body subgroup. In this way, sequential memory access can be performed in an efficient manner (for example, a predefined memory access sequence, in which rows 1, 2, 3, 4, 5, 6, 7, 8... are to be read, and rows 1, 3, 5... etc. are associated with the first memory sub-group and rows 2, 4, 6... etc. are associated with the second different memory sub-group).

FIG. 83A can illustrate the state for accessing memory rows included in two different memory subgroups. In the state shown in Figure 83A:

a. Column A can be accessed by the first column decoder. The first sector (leftmost sector marked in gray) can be accessed after the first row decoder activates row A.

b. Column B can be accessed by the second column decoder. In these states shown in Figure 83A, column B is closed and has not yet been activated.

The state illustrated in FIG. 83A can be preceded by sending the start command and the address of column A to the first column decoder.

FIG. 83B illustrates the state for accessing column B after accessing column A. According to this example: column A may be accessible by the first column decoder. In the state shown in FIG. 83B, the first row decoder activates row A and has accessed all sectors except the four rightmost sectors (four sectors not marked in gray). Because the difference (four white sections in column A) is equal to four cycles, the group controller can enable the second column decoder to activate column B before accessing the rightmost section in column A. In some cases, the activation row B can respond to a predetermined access pattern (for example, access in a sequence, where the odd-numbered rows are designated in the first sub-group and the even-numbered rows are designated in the second sub-group). In other situations, the activation column B can respond to any of the column prediction techniques described above. The group controller can enable the second column decoder to activate column B in advance, so that when column B is accessed, column B has been activated (opened) instead of waiting for column B to be activated to open column B.

The state illustrated in Figure 83B can be the following operations before:

a. Send the start command and the address of column A to the first column decoder.

b. Write or read the twenty-eight sectors before column A.

c. After reading or writing the twenty-eight sectors of the column, send the start command relative to the address of the column B to the decoder of the second column.

In some embodiments, the even-numbered columns are located in one half of one or more memory banks. In some embodiments, the odd-numbered columns are located in one half of one or more memory banks.

In some embodiments, an extra row of redundant pads is placed between each of the two pad rows to establish a distance for allowing activation. In some embodiments, multiple rows that are close to each other may not be activated at the same time.

FIG. 83C can illustrate the state for accessing column C (for example, the next odd-numbered column included in the first subgroup) after accessing column A. As shown in Figure 83C, column B may be accessible by the second column decoder. As shown, the second row decoder has activated row B and has accessed all sectors except the four rightmost sectors (the four remaining sectors not marked in gray). Because in this example, the difference is equal to four cycles, the group controller can enable the first column decoder to activate column C before accessing the rightmost section in column B. The group controller can enable the first column decoder to activate column C in advance, so that when column C is accessed, column C has been activated instead of waiting to be activated. Operating in this way can reduce or completely eliminate the latency associated with memory read operations.

As a memory pad for temporary storage files

In the computer architecture, the processor register constitutes a storage location that the computer processor (for example, a central processing unit (CPU)) can quickly access. The register usually includes the memory unit closest to the processor core (L0). Registers can provide the fastest way to access certain types of data. A computer may have several types of registers, each of which is classified according to the type of information it stores or based on the type of instructions operating on the information in a certain type of register. For example, the computer may include: a data register, which protects Store numerical information, operands, intermediate results and configuration; address register, which stores address information used by commands to access the main memory; general purpose register, which stores both data and address information; And status registers; and other registers. The register file includes a logical group of registers that can be used by the computer processing unit.

In many cases, the computer's register file is located in the processing unit (for example, CPU) and implemented by logic transistors. However, in the disclosed embodiment, the arithmetic processing unit may not reside in a traditional CPU. In fact, these processing elements (for example, processor sub-units) can be spatially distributed in the memory chip (as described in the above section) as a processing array. Each processor subunit can be associated with one or more corresponding and dedicated memory units (e.g., memory banks). With this structure, each processor sub-unit can be spatially located near one or more memory elements that store data for the operation of a specific processor sub-unit. As described herein, this architecture can significantly speed up operations in certain memory-intensive operations by, for example, eliminating memory access bottlenecks experienced by typical CPU and external memory architectures.

However, the distributed processor memory chip architecture described in this article can still use temporary memory files, which include various types of temporary memory used to manipulate data from memory components dedicated to the corresponding processor subunits. Device. However, since the processor subunits can be distributed among the memory elements of the memory chip, it is possible to combine one or more memory elements (compared to the logic elements in a specific manufacturing process, the one or more memory elements Can benefit from the same process) is added to the corresponding processor sub-unit to serve as a register file or cache memory for the corresponding processor sub-unit instead of acting as the main memory storage.

This architecture can provide several advantages. For example, since the register file is a part corresponding to the processor sub-unit, the processor sub-unit can be spatially located near the relevant register file. This configuration can significantly increase operating efficiency. The conventional register file is implemented by logic transistors. For example, each bit of the conventional register file is made of about 12 logic transistors, and therefore a 16-bit register The file is made of 192 logic transistors. This register file may require a large number of logic components to access the logic transistors, and therefore may occupy a large space. Compared with a register file implemented by a logic transistor, the register file of the embodiment disclosed in the present invention may require significantly less space. This size reduction can be achieved by implementing the register file of the disclosed embodiment using memory pads including memory cells that are optimized for use in manufacturing memory structures It is not manufactured by the process used to manufacture logical structures. The size reduction can also allow larger scratchpad files or cache memory.

In some embodiments, a distributed processor memory chip may be provided. The distributed processor memory chip may include: a substrate; a memory array disposed on the substrate and including a plurality of discrete memory groups; and a processing array disposed on the substrate and including a plurality of processor subunits. Each of the processor subunits can be associated with a corresponding dedicated memory group among a plurality of discrete memory groups. The distributed processor memory chip may also include a first plurality of buses and a second plurality of buses. Each of the first plurality of bus bars can connect one of the plurality of processor subunits to its corresponding dedicated memory bank. Each of the second plurality of bus bars can connect one of the plurality of processor subunits to another of the plurality of processor subunits. In some cases, the second plurality of buses may connect one or more of the plurality of processor sub-units to two or more other processor sub-units among the plurality of processor sub-units. One or more of the processor sub-units may also include at least one memory pad disposed on the substrate. The at least one memory pad may be configured to serve as at least one register for a register file for one or more of the plurality of processing subunits.

In some cases, the register file may be associated with one or more logical components so that the memory pad can serve as one or more registers of the register file. For example, these logic components may include switches, amplifiers, inverters, sense amplifiers, and others. In the case where the register file is implemented by a dynamic random access memory (DRAM) pad, logic components may be included to perform renew operations to prevent the loss of stored data. These logical components may include column and row multiplexers ("mux"). In addition, the register file implemented by the DRAM pad may include redundant mechanisms to combat yield degradation.

FIG. 84 illustrates a traditional computer architecture 8400 including a CPU 8402 and an external memory 8406. During operation, the value from the memory 8406 can be loaded into the register associated with the register file 8504 included in the CPU 8402.

FIG. 85A illustrates an exemplary distributed processor memory chip 8500a in accordance with the disclosed embodiments. Compared with the architecture of FIG. 84, the distributed processor memory chip 8500a includes memory elements and processor elements arranged on the same substrate. That is, the chip 8500a may include a memory array and a processing array, the processing array including a plurality of processor subunits each associated with one or more dedicated memory groups included in the memory array. In the architecture of FIG. 85, the register used by the processor subunit is provided by one or more memory pads disposed on the same substrate, and the memory array and the processing array are formed on the substrate.

As depicted in FIG. 85A, the distributed processor memory chip 8500a can be formed by a plurality of processing groups 8510a, 8510b, and 8510c disposed on a substrate 8502. More specifically, the distributed processor memory chip 8500a may include a memory array 8520 and a processing array 8530 disposed on a substrate 8502. The memory array 8520 may include a plurality of memory banks, such as memory banks 8520a, 8520b, and 8520c. The processing array 8530 may include a plurality of processor subunits, such as processor subunits 8530a, 8530b, and 8530c.

In addition, each of the processing groups 8510a, 8510b, and 8510c may include a processor subunit and one or more corresponding memory groups dedicated to the processor subunit. In the embodiment depicted in FIG. 85A, each of the processor subunits 8530a, 8530b, and 8530c may be associated with a corresponding dedicated memory bank 8520a, 8520b, or 8520c. That is, the processor subunit 8530a can be associated with the memory group 8520a; the processor subunit 8530b can be associated with the memory group 8520b; and the processor subunit 8530c can be associated with the memory group 8520c.

In order to allow each processor subunit to communicate with its corresponding dedicated memory bank, the distributed processor memory chip 8500a may include connecting one of the processor subunits to its corresponding dedicated memory bank. Use the first plurality of buses 8540a, 8540b and 8540c of the memory bank. In the embodiment depicted in FIG. 85A, the bus 8540a can connect the processor sub-unit 8530a to the memory bank 8520a; the bus 8540b can connect the processor sub-unit 8530b to the memory bank 8520b; and the bus 8540c can Connect the processor sub-unit 8530c to the memory bank 8520c.

In addition, in order to allow each processor sub-unit to communicate with other processor sub-units, the distributed processor memory chip 8500a may include a second plurality that connects one of the processor sub-units to at least another processor sub-unit Two busbars 8550a and 8550b. In the embodiment depicted in FIG. 85, bus 8550a can connect processor sub-unit 8530a to processor sub-unit 8530b, and bus 8550b can connect processor sub-unit 8530a to processor sub-unit 8550b, etc. .

Each of the discrete memory groups 8520a, 8520b, and 8520c may include a plurality of memory pads. In the embodiment depicted in FIG. 84, the memory set 8520a may include memory pads 8522a, 8524a, and 8526a; the memory set 8520b may include memory pads 8522b, 8524b, and 8526b; and the memory set 8520c may include memory Pads 8522c, 8524c, and 8526c. As previously disclosed with respect to FIG. 10, the memory pad may include a plurality of memory cells, and each cell may include a capacitor, a transistor, or other circuit system that stores at least one data bit. The conventional memory pad may include, for example, 512 bits×512 bits, but the embodiments disclosed herein are not limited thereto.

At least one of the processor subunits 8530a, 8530b, and 8530c may include at least one memory pad configured to serve as a register file for the corresponding processor subunits 8530a, 8530b, and 8530c, such as a memory pad 8532a , 8532b and 8532c. That is, at least one of the memory pads 8532a, 8532b, and 8532c provides at least one register of the register files used by one or more of the processor subunits 8530a, 8530b, and 8530c. The register file may include one or more registers. In the embodiment depicted in FIG. 85A, the memory pad 8532a in the processor sub-unit 8530a can serve as a memory pad for the processor sub-unit 8530a (and/or any other processing included in the distributed processor memory chip 8500a). Register file (also known as "register file 8532a"); processor subunit The memory pad 8532b in the 8530b can serve as a register file for the processor subunit 8530b; and the memory pad 8532c in the processor subunit 8530c can serve as a register file for the processor subunit 8530c.

At least one of the processor subunits 8530a, 8530b, and 8530c may also include at least one logic component, such as logic components 8534a, 8534b, and 8534c. Each logic component 8534a, 8534b, or 8534c can be configured so that the corresponding memory pad 8532a, 8532b, or 8532c can serve as a register file for the corresponding processor subunit 8530a, 8530b, or 8530c.

In some embodiments, at least one memory pad may be disposed on the substrate, and at least one memory pad may contain at least one redundant temporary configured to provide for one or more of the plurality of processor subunits. At least one redundant memory bit of the memory. In some embodiments, at least one of the processor sub-units may include a mechanism for stopping the current task and triggering a memory renew operation at a certain time to renew the memory pad.

FIG. 85B illustrates an exemplary distributed processor memory chip 8500b in accordance with the disclosed embodiment. The memory chip 8500b illustrated in FIG. 85B is substantially the same as the memory chip 8500 illustrated in FIG. 85A, except that the memory pads 8532a, 8532b, and 8532c in FIG. 85B are not included in the corresponding processor subunits 8530a, 8530b, and 8530c Chinese and foreign. In fact, the memory pads 8532a, 8532b, and 8532c in FIG. 85B are arranged outside the corresponding processor sub-units 8530a, 8530b, and 8530c but are spatially close to the processor sub-units. In this way, the memory pads 8532a, 8532b, and 8532c can still serve as register files for the corresponding processor subunits 8530a, 8530b, and 8530c.

Figure 85C illustrates a device 8500c in accordance with the disclosed embodiment. The device 8500c includes a substrate 8560, a first memory group 8570, a second memory group 8572, and a processing unit 8580. The first memory group 8570, the second memory group 8572 and the processing unit 8580 are disposed on the substrate 8560. The processing unit 8580 includes a processor 8584 and a register file 8582 implemented by a memory pad. During the operation of the processing unit 8580, the processor 8584 can access the register file 8582 to read or write data.

The distributed processor memory chip 8500a, 8500b or the device 8500c can provide multiple functions based on the processor subunit's access to the register provided by the memory pad. For example, in some embodiments, the distributed processor memory chip 8500a or 8500b may include a processor sub-unit that acts as an accelerator coupled to the memory, allowing it to use more memory frequency. width. In the embodiment depicted in FIG. 85A, the processor sub-unit 8530a may act as an accelerator (also referred to as "accelerator 8530a"). The accelerator 8530a can use the memory pad 8532a disposed in the accelerator 8530a to provide one or more registers of the register file. Alternatively, in the embodiment depicted in FIG. 85B, the accelerator 8530a may use the memory pad 8532a disposed outside the accelerator 8530a as the register file. In addition, the accelerator 8530a can use any one of the memory pads 8522b, 8524b, and 8526b in the memory set 8520b or any one of the memory pads 8522c, 8524c, and 8526c in the memory set 8520c to provide a Or multiple registers.

The disclosed embodiments are particularly suitable for certain types of image processing, neural networks, database analysis, compression and decompression, and more applications. For example, in the embodiment of FIG. 85A or FIG. 85B, the memory pad may provide one or more register files for one or more processor subunits included on the same chip as the memory pad The register serves as a memory pad. One or more registers can be used to store data frequently accessed by the processor subunits. For example, during convolutional image processing, the convolution accelerator can repeatedly use the same coefficients on the entire image stored in memory. The proposed implementation for this convolutional accelerator can save all these coefficients in a "closed" register file in one or more registers, the one or more registers including those dedicated to Within the memory pad of one or more processor sub-units, the one or more processor sub-units and the register file memory pad are located on the same chip. This architecture can place the register (and the stored coefficient value) in close proximity to the processor subunit that operates on the coefficient value. Because the register file implemented by the memory pad can serve as an efficient cache memory that is tightly spaced, a significantly lower loss of data transmission and a lower latency of access can be achieved.

In another example, the disclosed embodiment may include words that can be input to a memory pad Provide the accelerator in the register. The accelerator can treat the scratchpad as a circular buffer to multiply the vectors in a single cycle. For example, in the device 8500c illustrated in FIG. 85C, the processor 8584 in the processing unit 8580 acts as an accelerator, which uses a register file 8582 implemented by a memory pad as a circular buffer to store data A1, A2, A3……. The first memory group 8570 stores the data B1, B2, B3... to be multiplied by the data A1, A2, A3.... The second memory group 8572 stores the multiplication results C1, C2, C3.... That is, Ci=Ai×Bi. If there is no register file in the processing unit 8580, the processor 8584 will need more memory bandwidth and more cycles to read data A1, A2, A3 from an external memory bank such as the memory bank 8570 or 8572 ...And data B1, B2, B3... both, this can cause a significant delay. On the other hand, in this embodiment, the data A1, A2, A3... are stored in the register file 8582 formed in the processing unit 8580. Therefore, the processor 8584 will only need to read the data B1, B2, B3... from the external memory bank 8570. Therefore, the memory bandwidth can be significantly reduced.

In the memory processing procedure, the memory pad usually allows one-way access (ie, single access). In one-way access, there is a port to the memory. As a result, only one access operation to a specific address, such as reading or writing, can be performed at a certain time. However, if the memory pad itself allows two-way access, then two-way access may be a valid option. In bidirectional access, two different addresses can be accessed at a certain time. The method of accessing the memory pad can be determined based on the area and requirements. In some situations, if the register files implemented by the memory pad are connected to a processor that needs to read two sources and has a destination register, the register files can allow four-way access. In some situations, when the register file is implemented by a DRAM pad to store configuration or cache memory data, the register file may only allow one-way access. Standard CPUs may include multi-directional access pads, while unidirectional access pads may be better for DRAM applications.

When the controller or accelerator is designed in such a way that it only needs a single access to the register (in a few possible cases), the register implemented by the memory pad can be used instead of the traditional register file. In a single access, only one word can be accessed at a time. For example, the processing unit can access two words from two register files at a certain time. Each of the two register files can be accessed by only allowing single access The memory pad (for example, DRAM pad) is implemented.

In most technologies, the memory pad IP, which is a closed block (IP) obtained from the manufacturer, will be accompanied by wiring in place for column and row access, such as word lines and column lines. But the memory pad IP does not include surround logic components. Therefore, the register file implemented by the memory pad disclosed in the embodiment of the present invention may include logic components. The size of the memory pad can be selected based on the required size of the register file.

When a memory pad is used to provide a register for a register file, certain challenges may arise, and these challenges may depend on the specific memory technology used to form the memory pad. For example, in memory production, not all memory cells manufactured can be properly operated after production. This is a known problem, especially when there is a high density of SRAM or DRAM on the chip. To solve this problem in memory technology, one or more redundant mechanisms can be used to maintain the yield rate at a reasonable level. In the disclosed embodiment, because the number of memory instances (for example, memory banks) of the register used to provide the register file can be quite small, the redundancy mechanism may not be as important as in normal memory applications. On the other hand, the same production issues that affect the functionality of the memory can also affect whether a particular memory pad can function properly when one or more registers are provided. As a result, redundant elements can be included in the disclosed embodiments. For example, at least one redundant memory pad can be disposed on the substrate of the distributed processor memory chip. The at least one redundant memory pad can be configured to provide at least one redundant register for one or more of the plurality of processor subunits. In another example, the pad may be larger than the required size (for example, 620×620 instead of 512×512), and the redundancy mechanism may be built into the area of the memory pad outside the 512×512 area or its equivalent .

Another challenge can be related to timing. The timing of loading words and bit lines is usually determined by the size of the memory. Since the register file can be implemented by a relatively small single memory pad (for example, 512×512 bits), the time required to load words from the memory pad will be less. Compared with logic, the timing can be sufficient Runs fairly quickly.

Some memory types like DRAM need to be renewed periodically. Renew can be executed when the processor or accelerator is paused. For small memory pads, the new time can be a small part of the time. Therefore, even if the system stops in a short period of time, from the perspective of overall performance, the gain obtained by using the memory pad as a register is worth the downtime. In one embodiment, the processing unit may include a counter that counts backward from a predefined number. When the counter reaches "0", the processing unit can stop the current task executed by the processor (for example, the accelerator), and trigger the renew operation of renewing the memory pad row by row. When the new operation is completed, the processor can resume its task, and the counter can be reset to count backward by a predefined number.

Figure 86 provides a flowchart 8600 representing an exemplary method for executing at least one instruction in a distributed processor memory chip consistent with the disclosed embodiments. For example, at step 8602, at least one data value can be retrieved from the memory array on the substrate of the distributed processor memory chip. At step 8604, the retrieved data value can be stored in the register provided by the memory pad of the memory array on the substrate of the distributed processor memory chip. At step 8606, a processor element such as one or more of the distributed processor subunits on the distributed processor memory chip may operate on the stored data value from the memory pad register.

Here and throughout the text, it should be understood that all references to the register file should equally refer to the cache, because the register file can be the lowest level cache.

Deal with bottlenecks

The terms "first", "second", "third" and the like are only used to distinguish different terms. These terms may not indicate the order and/or timing and/or importance of the components. For example, the first processing procedure can be preceded by the second processing procedure, and the like.

The term "coupled" can mean direct connection and/or indirect connection.

The terms "memory/processing", "memory and processing" and "memory processing" are used interchangeably.

Multiple methods, computer-readable media, memory/processing units, and/or systems that can be memory/processing units can be provided.

The memory/processing unit is a hardware unit with memory and processing capabilities.

The memory/processing unit may be a memory processing integrated circuit, may be included in a memory processing integrated circuit, or may include one or more memory processing integrated circuits.

The memory/processing unit may be a distributed processor as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may include a distributed processor as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may belong to a distributed processor as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may be a memory chip as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit may include a memory chip as described in PCT Patent Application Publication WO2019025892.

The memory/processing unit can be a distributed processor as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may belong to a distributed processor as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may be a memory chip as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may include a memory chip as described in PCT Patent Application No. PCT/IB2019/001005.

The memory/processing unit may belong to the PCT patent application No. PCT/IB2019/001005 The memory chip described in the number.

The memory/processing unit can be an integrated circuit that uses inter-wafer bonding and multiple conductors to connect to each other.

Any reference to distributed processor memory chips, distributed memory processing integrated circuits, memory chips, and distributed processors can be implemented as a pair of integrated circuits by bonding between wafers and connecting multiple conductors to each other .

The memory/processing unit can be manufactured by a first manufacturing process that is better suited for memory cells than logic cells. Therefore, the first manufacturing process can be regarded as the manufacturing process of the memory type. The memory cell may include one of a plurality of transistors. The logic cell may include one or more transistors. The first manufacturing process can be applied to manufacture the memory bank. A logic cell can include one or more transistors that implement logic functions together, and can be used as a basic building block for larger logic circuits. A memory cell may include one or more transistors that perform memory functions together, and may be used as a basic building block for larger logic circuits. Corresponding logic cells can implement the same logic function.

The memory/processing unit may be different from any one of the processor, the processing integrated circuit and/or the processing unit, and the processor, the processing integrated circuit and/or the processing unit are better than the memory cell. It is suitable for the second manufacturing process of logic cells. Therefore, the first manufacturing process can be regarded as a logical type of manufacturing process. The second manufacturing process can be used to manufacture central processing units, graphics processing units and the like.

Compared to processors, processing integrated circuits, and/or processing units, memory/processing units may be more suitable for performing less arithmetic-intensive operations.

For example, the memory cells manufactured by the first manufacturing process can exhibit more than and even greater than (for example, more than 2 times, 3 times, 4 times, 5 times, 9 times, 7 times, 8 times, 9 times, 10 times and the like) the critical dimension of the critical dimension of the logic circuit manufactured by the first manufacturing process.

The first manufacturing process can be an analog manufacturing process, and the first manufacturing process can be a DRAM manufacturing process Manufacturing process, and the like.

The size of the logic cell manufactured by the first manufacturing process can exceed the size of the corresponding logic cell manufactured by the second manufacturing process by at least twice. The corresponding logic call may have the same functionality as the logic cell manufactured by the first manufacturing process.

The second manufacturing process may be a digital manufacturing process.

The second manufacturing process can be any of complementary metal oxide semiconductor (CMOS), bipolar, bipolar CMOS (BiCOMS), double diffused metal oxide semiconductor (DMOS), silicon-on-oxide manufacturing process, and the like .

The memory/processing unit may include multiple processor sub-units.

The processor subunits of one or more memory/processing units can operate independently of each other and/or can cooperate with each other and/or perform distributed processing. Distributed processing can be performed in various ways, such as in a planar manner or in a hierarchical manner.

The planar approach may involve causing processor sub-units to perform the same operation (and may or may not output processing results between the processor sub-units).

The hierarchical approach may involve executing a sequence of processing operations at different levels, and the processing operations at one level are performed after the processing operations at another level. Processor sub-units can be allocated (dynamically or statically) to different layers and participate in hierarchical processing.

Distributed processing may also involve other units, such as memory/processing unit controllers and/or units that are not memory/processing units.

The terms logic and processor subunit are used interchangeably.

Any processing mentioned in this application can be executed in any way (distributed and/or non-distributed and the like).

In the following applications, various references and/or references are made to PCT Patent Application Publication WO2019025892 and PCT Patent Application No. PCT/IB2019/001005 (September 9, 2019) Incorporated by reference. PCT Patent Application Publication WO2019025892 and/or PCT Patent Application No. PCT/IB2019/001005 provide non-limiting examples of various methods, systems, processors, memory chips and the like. Other methods, systems, and processors can be provided.

A processing system (system) can be provided, in which the processor is previously one or more memory/processing units, and each memory and processing unit (memory/processing unit) has processing resources and storage resources.

The processor can request or issue instructions to one or more memory/processing units to perform various processing tasks. The execution of various processing tasks can reduce the burden on the processor, reduce latency, and in some cases reduce the total information bandwidth between one or more memory/processing units and the processor, and the like.

The processor can provide instructions and/or requests with different granularities. For example, the processor can send instructions for certain processing resources or can send higher-level instructions for memory/processing units without specifying any processing resources.

The memory/processing unit can manage its processing and/or memory resources in any way (dynamic, static, distributed, centralized, offline, online, and the like). Resource management can be performed in the following situations: autonomously, under the control of the processor, after the processor is configured, and the like.

For example, the task can be divided into subtasks that may require one or more processing resources of one or more memory/processing units and/or execution of memory resources or one or more instructions. Each processing resource can be configured to execute (e.g., independently or non-independently) at least one instruction. See, for example, the execution of the instruction sub-series by the processing resources of the processor sub-unit such as PCT Patent Application Publication WO2019025892.

It is also possible to provide at least the allocation of memory resources to entities other than one or more memory/processing units, such as direct access memory (DMA) units that can be coupled to one or more memory/processing units.

The compiler can prepare groups for each type of task performed by the memory/processing unit State file. The configuration file includes memory allocation and processing resource allocation associated with task types. The configuration file can include commands that can be executed by different processing resources and/or can define memory allocation.

For example, the configuration file related to the task of matrix multiplication (multiply matrix A by matrix B, A*B=C) can prompt where to store the elements of matrix A and where to store the elements of matrix B. Where to store the elements of the matrix C, where to store the intermediate results generated during the matrix multiplication, and may include instructions for processing resources used to perform any mathematical operations related to the matrix multiplication. The configuration file is an example of the data structure, and other data structures can be provided.

The matrix multiplication can be performed in any manner by one or more memory/processing units.

One or more memory/processing units can multiply the matrix A by the vector V. This can be done in any way. For example, this may involve maintaining one column or row of the matrix per processing resource (different columns for each different processing resource maintenance row), and looping (between different processing resources) the final result of the multiplication of the column or row of the matrix and the vector (During the first iteration), and loop the final result of the previous multiplication (during the second to the last iteration).

Assume that matrix A is a 4×4 matrix, vector V is a 1×4 vector, and there are four processing resources. Under these assumptions, the first row of matrix A is stored at the first processor subunit, the second row of matrix A is stored at the second processor subunit, and the third row of matrix A is stored at the third processing resource , And the matrix A is stored at the fourth processor subunit in the fourth column. Start the multiplication by: sending the first to fourth elements of vector V to the first to fourth processing resources; and multiplying the first to fourth elements of vector V by different vectors of A to provide the first intermediate result . The multiplication is continued by looping the first intermediate result by the following operation: With each processing resource, the first intermediate result calculated by the first processing resource is sent to its neighboring processing resources. Each processing resource multiplies the first multiplication result by the vector to provide the second multiplication result. This process is repeated many times until the multiplication of the matrix A and the vector V ends.

90A is an example of a system 10900 including one or more memory/processing units (collectively designated as 10910) and a processor 10920. The processor 10920 can send a request or instruction (via Link 10931) to one or more memory/processing units 10920, which in turn completes (or selectively completes) requests and/or instructions and sends the results (via link 10932) To the processor 10920, as described above. The processor 10920 may further process the results to provide (via link 10933) one or more outputs.

One or more memory/processing units may include J (J is a positive integer) memory resources 10912 (1, 1) to 10912 (1, J) and K (K is a positive integer) processing resources 10911 (1, 1) to 10911 (1, K).

J can be equal to K or can be different from K.

The processing resources 10911(1,1) to 10911(1,K) may be, for example, processing groups or processor subunits, as described in PCT Patent Application Publication WO2019025892.

The memory resources 10912(1,1) to 10912(1,J) can be memory instances, memory pads, and memory groups, as described in PCT Patent Application Publication WO2019025892.

There may be any connectivity and/or any functional relationship between any of the resources (memory or processing) of one or more memory/processing units.

Figure 90B shows an example of the memory/processing unit 10910(1).

In FIG. 90B, K (K is a positive integer) processing resources 10911(1,1) to 10911(1,K) form a loop because these processing resources are connected in series with each other (see link 10915). Each processing resource is also coupled to its own pair of dedicated memory resources (for example, processing resource 10911(1) is coupled to memory resources 10912(1) and 10912(2), and processing resource 10911(K) is coupled Connect to memory resources 10912(J-1) and 10912(J)). Processing resources can be connected to each other in any other way. The number of memory resources allocated for each processing resource can be different from two. An example of the connectivity between different resources is described in PCT Patent Application Publication WO2019025892.

90C is an example of a system 10901 with N (N is a positive integer) memory/processing units 10910(1) to 10910(N) and a processor 10920. The processor 10920 can send requests or instructions (via links 10931(1) to 10931(N)) to the memory/processing units 10920(1) to 10910(N), and these memories The body/processing unit in turn completes the request and/or instruction and sends the result (via links 10932(1) to 3232(N)) to the processor 10920, as explained above. The processor 10920 may further process the results to provide (via link 10933) one or more outputs.

90D is an example of a system 10902 including N (N is a positive integer) memory/processing units 10910(1) to 10910(N) and a processor 10920. Figure 90D illustrates the pre-processor 10909 before the memory/processing units 10910(1) to 10910(N). The preprocessor can perform various preprocessing operations, such as frame extraction, header detection, and the like.

FIG. 90E is an example of a system 10903 including one or more memory/processing units 10910 and a processor 10920. FIG. 90E illustrates a pre-processor 10909 before one or more memory/processing units 10910 and DMA controller 10908.

Figure 90F illustrates a method 10800 for distributed processing of at least one information stream.

The method 10800 can start with the step 10810 of receiving at least one information stream via the first communication channel by one or more memory processing integrated circuits; wherein each memory processing integrated unit includes a controller and a plurality of processors Unit and multiple memory units.

Step 10810 can be followed by steps 10820 and 10830.

Step 10820 may include buffering the information stream with one or more memory processing integrated circuits.

Step 10830 may include performing a first processing operation on at least one information stream by one or more memory processing integrated circuits to provide a first processing result.

Step 10830 may involve compression or decompression.

Therefore, the total size of the information stream may exceed the total size of the first processing result. The total size of the information stream can reflect the amount of information received during a period of a given duration. The total size of the first processing result can reflect the amount of the first processing result output during any period of the same given duration.

Instead, information stream (any other information entity mentioned in this manual) The total size of is smaller than the total size of the first processing result. In this situation, compression is obtained.

Step 10830 can be followed by step 10840 of sending the first processing result to one or more processing integrated circuits.

One or more memory processing integrated circuits can be manufactured by the manufacturing process of the memory type.

One or more memory processing integrated circuits can be manufactured by a logic type manufacturing process.

In the memory processing integrated unit, each of the memory units can be coupled to a processor sub-unit.

Step 10840 can be followed by step 10850 of performing a second processing operation on the first processing result by one or more processing integrated circuits to provide a second processing result.

Step 10820 and/or step 10830 can be commanded by one or more processing integrated circuits, can be requested by one or more processing integrated circuits, and can be used by one or more processing integrated circuits. The memory processing integrated circuit is configured to be executed, or can be executed independently without the intervention of one or more processing integrated circuits.

The first processing operation may have a lower arithmetic strength than the second processing operation.

Step 10830 and/or step 10850 can be at least one of the following: (a) cellular network processing operations; (b) other network-related processing operations (different from cellular network processing) ; (C) Database processing operation; (d) Database analysis processing operation; (e) Artificial intelligence processing operation; or any other processing operation.

Decomposable system memory/processing unit and method for distributed processing

Decomposable systems, methods for distributed processing, processing/memory units, methods for operating decomposable systems, methods for operating processing/memory units, and computer-readable media can be provided. It is non-transitory and stores instructions for executing any of these methods. The decomposition system allocates different subsystems to perform different functions. For example, the storage may be mainly implemented in one or more storage subsystems, and the operation may be mainly performed in one or more storage subsystems.

The decomposition system may be a decomposition server, one or more decomposition servers, and/or may be different from one or more servers.

The decomposition system may include one or more switching subsystems, one or more computing subsystems, one or more storage subsystems, and one or more processing/memory subsystems.

One or more processing/memory subsystems, one or more computing subsystems, and one or more storage subsystems are coupled to each other via one or more switching subsystems.

One or more processing/memory subsystems may be included in one or more of the decomposed systems.

Figure 87A illustrates various examples of the exploded system.

Any number of subsystems of any type can be provided. The decomposed system may include one or more additional subsystems of the types not included in FIG. 87A, may include fewer types of subsystems, and the like.

The decomposition system 7101 includes two storage subsystems 7130, a computing subsystem 7120, a switching subsystem 7140, and a processing/memory subsystem 7110.

The decomposition system 7102 includes two storage subsystems 7130, a computing subsystem 7120, a switching subsystem 7140, a processing/memory subsystem 7110, and an accelerator subsystem 7150.

The decomposition system 7103 includes two storage subsystems 7130, a computing subsystem 7120, and a switching subsystem 7140 including a processing/memory subsystem 7110.

The decomposition system 7104 includes two storage subsystems 7130, a computing subsystem 7120, a switching subsystem 7140 including a processing/memory subsystem 7110, and an accelerator subsystem 7150.

Including the processing/memory subsystem 7110 in the switching subsystem 7140 can reduce the traffic in the decomposed systems 7101 and 7102, and can reduce the latency of handover, and the like.

The different subsystems of the decomposed system can communicate with each other using various communication protocols. It has been found that the use of Ethernet and even the Ethernet RDMA communication protocol can increase throughput, and may even Reduce the complexity of various control and/storage operations related to the exchange of information units between the components of the decomposition system.

Disaggregated systems can perform distributed processing by allowing the processing/memory subsystem to participate in calculations (especially by performing memory-intensive calculations).

For example, assuming that N arithmetic units should share information units among them (all shared), then (a) N information units can be sent to one or more processing/memory subsystems or one or more processing/memory Volume unit, (b) one or more processing/memory units can perform calculations that need to be all shared, and (c) send N updated information units to N arithmetic units. This will require about N transfer operations.

For example, Figure 87B illustrates the decentralized process of updating a model of a neural network (the model includes the weights assigned to the nodes of the neural network).

Each of the N arithmetic units PU(1) 7120(1) to PU(N) 7120(N) may belong to the arithmetic subsystem 7120 of any one of the decomposition systems 7101, 7102, 7103, and 7104.

The N arithmetic units calculate N partial model updates (updated N different parts) 7121(1) to 7121(N), and send them (via the exchange subsystem 7140) to the processing/memory subsystem 7110.

The processing/memory subsystem 7110 calculates the updated model 7122 and sends (via the exchange subsystem 7140) the updated model to N arithmetic units PU(1) 7120(1) to PU(N) 7120(N).

Figure 87C, Figure 87D, and Figure 87E illustrate examples of memory/processing units 7011, 7012, and 7013, respectively, and Figures 87F and 87G illustrate integrated circuits 7014 and 7015. The integrated circuits include memory/processing units 9010 such as Ethernet One or more communication modules of the network module and the Ethernet RDMA module 22.

The memory/processing unit includes a controller 9020, an internal bus 9021, and multiple pairs of logic 9030 and a memory bank 9040. The controller is configured to operate as a communication module or can be coupled to a communication module group.

The connectivity between the controller 9020 and the multiple pairs of logic 9030 and memory bank 9040 can be implemented in other ways. Other ways (unpaired) can be used to configure memory groups and logic.

One or more memory/processing units 9010 of the processing/memory subsystem 7110 can be processed in parallel (using different logics and fetching different parts of the model in parallel from different memory groups) model updates and benefiting from a large amount of memory The extremely high bandwidth of the connections between resources, memory banks, and logic allows these calculations to be performed in an efficient manner.

The memory/processing units 7011, 7012 and 7013 of FIGS. 87C to 87E and the integrated circuits 7014 and 7015 of FIGS. 87C to 87E include one or more communication modules, such as an Ethernet module 7023 (in FIG. 87C To Figure 87G) and Ethernet RDMA module 7022 (in Figure 87E and Figure 87G).

Having these RDMA and/or Ethernet modules (in the memory/processing unit or in the same integrated circuit as the memory/processing unit) greatly accelerates the communication between the different components of the decomposed system, and In the case of RDMA, the communication between the different components of the decomposition system is greatly simplified.

It should be noted that the memory/processing unit that includes RDMA and/or Ethernet modules can be beneficial in other environments, even when the memory/processing unit is not included in the disassembled system.

It should also be noted that, for example, for cost reduction reasons, RDMA and/or Ethernet modules can be allocated to each group of memory/processing units.

It should be noted that the memory/processing unit, the group of memory/processing units, and even the processing/memory subsystem may include other communication ports, such as PCIe communication ports.

The use of RDMA and/or Ethernet modules can be cost-effective because it eliminates the need to connect the memory/processing unit to a bridge that connects to a network product that can have an Ethernet port Body circuit (NIC).

Use RDMA and/or Ethernet modules to enable Ethernet (or Ethernet RDMA) is native to the memory/processing unit.

It should be noted that Ethernet is only an example of a local area network (LAN) protocol. PCIe is only an example of another communication protocol that can be used over longer distances than Ethernet.

Figure 87H illustrates a method 7000 for distributed processing.

Method 7000 may include one or more process iterations.

The processing repetition can be performed by one or more memory processing integrated circuits of the decomposable system.

The processing repetition can be performed by one or more processing integrated circuits of the decomposition system.

The processing iterations performed by more memory processing integrated circuits can be followed by processing iterations performed by one or more processing integrated circuits.

The process repetition performed by more memory processing integrated circuits may precede the process repetition performed by one or more processing integrated circuits.

Yet another process iteration can be performed by other circuits of the decomposition system. For example, one or more pre-processing circuits can perform any type of pre-processing, including information units that are prepared for processing iterations performed by one or more memory processing integrated circuits.

The method 7000 may include a step 7020 of processing the integrated circuit receiving information unit by one or more memories of the decomposable system.

Each memory processing integrated unit may include a controller, multiple processor sub-units, and multiple memory units.

One or more memory processing integrated circuits can be manufactured by the manufacturing process of the memory type.

The information unit can convey the part of the neural network model.

The information unit can deliver part of the results of at least one database query.

The information unit can deliver part of the results of at least one aggregate database query.

Step 7020 may include receiving information units from one or more storage subsystems of the self-decomposable system.

Step 7020 may include receiving information units from one or more arithmetic subsystems of the self-decomposable system, and the one or more arithmetic subsystems may include a plurality of processing integrated circuits manufactured by a logic type manufacturing process.

Step 7020 can be followed by step 7030 in which one or more memory processing integrated circuits perform processing operations on the information unit to provide processing results.

The total size of the information unit may exceed, may be equal to, or may be smaller than the total size of the processing result.

Step 7030 can be followed by step 7040 of outputting processing results by one or more memory processing integrated circuits.

Step 7040 may include outputting the processing result to one or more computing subsystems of the decomposition system, and the one or more computing subsystems may include multiple processing integrated circuits manufactured by the manufacturing process of the logic category.

Step 7040 may include outputting the processing result to one or more storage subsystems of the disaggregated system.

Information units can be sent from different groups of processing units of multiple processing integrated circuits, and can be different parts of intermediate results of processing procedures executed in a distributed manner by multiple processing integrated circuits. The group of processing units may include at least one processing integrated circuit.

Step 7030 may include processing the information unit to provide the result of the entire processing procedure.

Step 7040 may include sending the result of the entire processing procedure to each of the multiple processing integrated circuits.

The different parts of the intermediate result can be different parts of the updated neural network model, and the result of the entire processing procedure is the updated neural network model.

Step 7040 may include sending the updated neural network model to each of the multiple processing integrated circuits.

Step 7040 can be followed by multiple processing integrated circuits based at least in part on Step 7050 of another processing is executed to the processing result of a plurality of processing integrated circuits.

Step 7040 may include using the exchange subunit of the decomposition system to output the processing result.

Step 7020 may include receiving information units, which are preprocessed information units.

Figure 87I illustrates a method 7001 for distributed processing.

The method 7001 is different from the method 7000 in that it includes a step 7010 of preprocessing information by a plurality of processing integrated circuits to provide a preprocessed information unit.

Step 7010 can be followed by steps 7010, 7020, 7030, and 7040.

Database analysis acceleration

A device, a method, and a computer-readable medium are provided. The device, the method, and the computer-readable medium store instructions for performing at least screening by a screening unit belonging to the same integrated circuit as a memory unit, and the filter can Prompt which items are related to a certain database query. The arbiter or any other flow control manager can send related items to the processor and not send irrelevant items to the processor, thus saving almost most of the traffic to and from the processor.

See, for example, Figure 91A, which shows a processor (CPU 9240), an integrated circuit including memory and a screening system 9220. The memory and screening system 9220 may include a screening unit 9224 coupled to the memory unit entry 9222 and one or more arbiters (such as the arbiter 9229 for sending related entries to the processor). Any arbitration process can be applied. There can be any relationship between the number of entries, the number of screening units, and the number of arbiters.

The arbiter can be replaced by any unit capable of controlling the flow of information, such as a communication interface, a flow controller, and the like.

Reference screening, which is based on one or more relevance/screening criteria.

The relevance can be set for each database query, and the relevance can be displayed in any way. For example, the memory unit can store a relevance flag 9224' indicating which item is relevant. Storage also exists The storage device 9210 of K database sections 9220(k), and the range of k is between 1 and K. It should be noted that the entire database can be stored in the memory unit but not in the storage device (this solution is also known as the database of volatile memory storage).

The memory cell entry may be too small to store the entire database, and therefore can receive one section at a time.

The filtering unit can perform filtering operations, such as comparing the value of the field with the threshold value, comparing the value of the field with a predefined value, and determining whether the value of the field is within the predefined range, and the like.

Therefore, the screening unit can perform screening operations of the known database, and can be a compact and inexpensive circuit.

The final result of the screening operation (for example, the content of the relevant database entry) 9101 is sent to the CPU 9420 for processing.

The memory and screening system 9220 can be replaced by the memory and processing system as illustrated in FIG. 91B.

The memory and processing system 9229 includes a processing unit 9225 coupled to a memory unit entry 9222. The processing unit 9225 may perform screening operations, and may at least partially participate in performing one or more additional operations on related records.

The processing unit may be customized to perform specific operations and/or may be a programmable unit configured to perform multiple operations. For example, the processing unit may be a pipelined processing unit, may include ALU, may include multiple ALUs, and the like.

The processing unit 9225 may perform all one or more additional operations.

Alternatively, part of the one or more additional operations is performed by the processing unit, and the processor (CPU 9240) may perform another part of the one or more additional operations.

Send the final result of the processing operation (for example, the partial response 9102 to the database query, or the complete response 9103) to the CPU 9420.

Some responses require further processing.

Figure 92A illustrates a memory/processing system 9228 that includes a memory/processing unit 9227 configured to perform filtering and additional processing.

The memory/processing system 9228 implements the processing unit and the memory unit of FIG. 91 through the memory/processing unit 9227.

The role of the processor may include controlling the processing unit, performing at least part of one or more additional operations, and the like.

The combination of memory entries and processing units can be implemented at least in part by one or more memory/processing units.

Figure 92B illustrates an example memory/processing unit 9010.

The memory/processing unit 9010 includes a controller 9020, an internal bus 9021, and multiple pairs of logic 9030 and a memory bank 9040. The controller is configured to operate as a communication module or can be coupled to the communication module.

The connectivity between the controller 9020 and the multiple pairs of logic 9030 and memory bank 9040 can be implemented in other ways. Other ways (unpaired) can be used to configure memory groups and logic. Multiple memory banks can be coupled to and/or managed by a single logic.

The memory/processing system receives database query 9100 via interface 9211. The interface 9211 can be a bus, a port, an input/output interface, and the like.

It should be noted that the response to the database query can be generated by at least one of the following (or a combination of one or more of the following): one or more memories/processing systems, one or more memories And processing systems, one or more memory and screening systems, one or more processors located outside these systems, and the like.

It should be noted that the response to the database query can be generated by at least one of the following (or a combination of one or more of the following): one or more filtering units, one or more memories/processing Unit, one or more processing units, one or more other processors (such as one or more other CPUs), and the like.

Any processing procedure may include searching for related database entries and processing related database entries. The processing can be performed by one or more processing entities.

The processing entity can be at least one of the following: the processing unit of the memory and the processing system (for example, the processing unit 9225 of the memory and processing system 9229), the processor subunit (or logic) of the memory/processing unit , Another processor (for example, the CPU 9240 of FIG. 91A, FIG. 91B, and FIG. 74), and the like.

The processing involved in generating a response to a database query can be generated by any one of the following or a combination of the following:

a. Memory and processing unit 9225 of processing system 9229.

b. Processing unit 9225 of different memory and processing system 9229.

c. One or more of the memory/processing system 9228 processor sub-units (or logic 9030) of the memory/processing unit 9227.

d. Different memory/processing system 9228 memory/processing unit 9227 processor sub-unit (or logic 9030).

e. The controller of one or more memory/processing units 9227 of the memory/processing system 9228.

f. One of the different memory/processing systems 9228 or a controller of multiple memory/processing units 9227.

Therefore, the processing involved in the response to the database query can be generated by a combination or sub-combination of: (a) one or more memory/processing unit or one or more controllers, (b) memory One or more processing units of the processing system, (c) one or more memory/processing units or one or more processor sub-units, and (d) one or more other processors, and the like.

Processing performed by more than one processing entity can be referred to as distributed processing.

It should be noted that the screening can be performed by one or more screening units and/or one or more processing units and/ Or the screening entity in one or more processor sub-units executes. In this sense, the processing unit and/or the processor subunit that performs the screening operation can be referred to as the screening unit.

The processing entity may be a screening entity or may be different from the screening entity.

The processing entity can perform processing operations on the database entries deemed to be related by another screening entity.

The processing entity can also perform filtering operations.

One or more screening entities and one or more processing entities can be used in response to database queries.

One or more screening entities and one or more processing entities may belong to the same system (eg, memory/processing system 9228, memory and processing system 9229, memory and screening system 9220) or different systems.

The memory/processing unit may include multiple processor sub-units. The processor sub-units can operate independently of each other, can partially cooperate with each other, can participate in distributed processing, and the like.

FIG. 92C illustrates multiple memory and screening systems 9220, multiple other processors (such as CPU 9240), and storage device 9210.

Multiple memories and screening systems 9220 can participate (at the same time or at different times) in the screening of one or more database entries based on one or more screening criteria in one of the multiple database queries.

FIG. 92D illustrates multiple memory and processing systems 9229, multiple other processors (such as CPU 9240), and storage device 9210. FIG.

Multiple memories and processing systems 9229 can participate (simultaneously or at different times) in the screening and at least part of the processing involved in responding to one of the multiple database queries.

Figure 92F illustrates multiple memory/processing systems 9228, multiple other processors (such as CPU 9240), and storage device 9210.

Multiple memory/processing systems 9228 can participate (simultaneously or at different times) The screening and at least partial processing involved in the response to one of the database queries.

Figure 92G illustrates the method 9300 method for database analysis acceleration.

The method 9300 can start with the step 9310 of receiving a database query by processing the integrated circuit in the memory, the database query including at least one relevance criterion that prompts the database entries in the database related to the database query.

The database entries related to the database query in the database may not be the database entries of the database, but may be one, some or all of the database entries in the database.

The memory processing integrated circuit may include a controller, a plurality of processor sub-units, and a plurality of memory units.

Step 9310 can be followed by step 9320 of determining the group of related database entries stored in the memory processing integrated circuit based on at least one correlation criterion by the memory processing integrated circuit.

Step 9320 can be followed by sending the group of related database entries to one or more processing entities for further processing without substantially sending irrelevant data entries stored in the memory processing integrated circuit to the one or Step 9330 of multiple processing entities.

The phrase "but not to send in substance" means not to send at all (during the response to database queries) or to send a small number of irrelevant items. Not much can mean at most 1, 2, 3, 4, 5, 9, 7, 8, 9, 10 percentages, or sending any amount that has no significant effect on bandwidth.

Step 9330 can be followed by step 9340 of processing groups of related database entries to provide responses to database queries.

Figure 92H illustrates a method 9301 for database analysis acceleration.

It is assumed that the screening and the entire processing required to respond to database queries are performed by the memory processing integrated circuit.

Method 9301 can be started by processing the integrated circuit receiving database query through the memory Step 9310: The database query includes at least one relevance criterion that prompts database entries in the database that are related to the database query.

Step 9310 can be followed by step 9320 of determining the group of related database entries stored in the memory processing integrated circuit based on at least one correlation criterion by the memory processing integrated circuit.

After step 9320, the group of related database entries can be sent to one or more processing entities of the memory processing integrated circuit for complete processing without substantially not storing irrelevant data stored in the memory processing integrated circuit. The data entry is sent to step 9331 of the one or more processing entities.

Step 9331 can be followed by step 9341 of completely processing the group of related database entries to provide responses to database queries.

Step 9341 can be followed by step 9351 of outputting the response to the database query from the memory processing integrated circuit.

Figure 92I illustrates a method 9302 for database analysis acceleration.

It is assumed that only part of the screening and processing required to respond to database queries is performed by the memory processing integrated circuit. The memory processing integrated circuit will output partial results, and these partial results will be processed by one or more other processing entities located outside the memory processing integrated circuit.

The method 9301 can start with the step 9310 of receiving a database query through the memory processing integrated circuit, and the database query includes at least one correlation criterion that prompts the database entries in the database related to the database query.

Step 9310 can be followed by step 9320 of determining the group of related database entries stored in the memory processing integrated circuit based on at least one correlation criterion by the memory processing integrated circuit.

Step 9320 can be followed by sending the group of related database entries to one or more processing entities of the memory processing integrated circuit for partial processing without substantially storing them in the memory. Step 9332 of sending irrelevant data items in the integrated circuit to the one or more processing entities.

Step 9332 can be followed by step 9342 of partially processing groups of related database entries to provide intermediate responses to database queries.

Step 9342 can be followed by step 9352 of outputting an intermediate response to the database query from the memory processing integrated circuit.

Step 9352 can be followed by step 9390 of further processing the intermediate response to provide a response to the database.

Figure 92J illustrates a method 9303 for database analysis acceleration.

It is assumed that the memory processing integrated circuit performs the screening of related database entries, but does not perform the processing of related database entries. The memory processing integrated circuit will output a group of related database entries that will be completely processed by one or more other processing entities located outside the memory processing integrated circuit.

The method 9301 can start with the step 9310 of receiving a database query through the memory processing integrated circuit, and the database query includes at least one correlation criterion that prompts the database entries in the database related to the database query.

Step 9310 can be followed by step 9320 of determining the group of related database entries stored in the memory processing integrated circuit based on at least one correlation criterion by the memory processing integrated circuit.

After step 9320, the group of related database entries can be sent to one or more processing entities located outside the memory processing integrated circuit without substantially storing the irrelevant data entries in the memory processing integrated circuit. Step 9333 of sending to the one or more processing entities.

Step 9333 can be followed by step 9391 of completely processing the intermediate response to provide a response to the database.

Figure 92K illustrates a method 9304 for database analysis acceleration.

Method 9303 can start at step 9315 of receiving database query by integrated circuit, The database query includes at least one correlation criterion that prompts a database entry related to the database query in the database; wherein the integrated circuit includes a controller, a screening unit, and a plurality of memory units.

Step 9315 can be followed by step 9325 of determining the group of related database entries stored in the integrated circuit based on at least one correlation criterion by the screening unit.

After step 9325, the group of related database entries can be sent to one or more processing entities located outside the integrated circuit for further processing, and irrelevant data entries stored in the integrated circuit are not substantially sent to Step 9335 of one or more processing entities.

Step 9335 can be followed by step 9391.

Figure 92L illustrates a method 9305 for database analysis acceleration.

The method 9305 may start at step 9314 of receiving a database query by an integrated circuit, the database query including at least one correlation criterion that prompts a database entry in the database related to the database query; wherein the integrated circuit includes a controller , Screening unit and multiple memory units.

Step 9314 can be followed by step 9324 of determining the group of related database entries stored in the integrated circuit by the processing unit based on at least one correlation criterion.

Step 9324 can be followed by step 9334 of processing the group of related database entries by the processing unit instead of processing the irrelevant data entries stored in the integrated circuit by the processing unit to provide a processing result.

Step 9334 can be followed by step 9344 of outputting the processing result from the integrated circuit.

In any of methods 9300, 9301, 9302, 9304, and 9305, the memory processing integrated circuit outputs an output. The output can be a related database entry, one or more intermediate results, or a group of one or more (complete) results.

Before the output, one or more related database entries and/or one or more results (complete or intermediate) can be retrieved from the screening entity and/or the processing entity of the memory processing integrated circuit.

The capture can be controlled in one or more ways and can be processed by the memory of the integrated circuit. The cutter and/or one or more controllers control.

The output and/or capture may include controlling the capture and/or output of one or more parameters. These parameters can include acquisition timing, acquisition rate, acquisition source, bandwidth, sequence or acquisition, output timing, output rate, output source, bandwidth, sequence or output, type of acquisition method, arbitration method Types and similar ones.

The output and/or capture can execute a flow control processing program.

The output and/or retrieval (for example, application flow control processing program) may respond to the completion indicator of the processing of the group's database entry output from one or more processing entities. The indicator can prompt whether the intermediate result is ready to be retrieved by the self-processing entity.

The output may include an attempt to match the bandwidth used during the output with the maximum allowable bandwidth on the link that couples the memory processing integrated circuit to the requester unit. The link may be a link to the receiver of the output of the memory processing integrated circuit. The maximum allowable bandwidth may be specified by the capacity and/or availability of the link, the capacity and/or availability of the recipient of the output content, and the like.

Output can include trying to output the output in the best or sub-optimal way.

The output of the output content may include an attempt to maintain the fluctuation of the output traffic rate below a threshold value.

Any of methods 9300, 9301, 9302, and 9305 may include generating processing status indicators by one or more processing entities, and the processing status indicators may prompt the progress of further processing of the group of related database entries.

When the processing included in any of the methods mentioned above is performed by more than a single processing entity, then the processing can be regarded as distributed processing because the processing is performed in a distributed manner.

As indicated above, processing can be performed in a hierarchical manner or in a planar manner.

Any one of methods 9300 to 9305 can be executed by multiple systems that can respond to one or more database queries simultaneously or sequentially.

Word embedding

As mentioned above, word embedding is a general term for a collection of language modeling and feature learning techniques in natural language processing (NLP), in which words or phrases from a vocabulary are mapped to vectors of elements. Conceptually, word embedding involves mathematical embedding from a space where each word has many dimensions to a continuous vector space with much lower dimensions.

These vectors can be mathematically processed. For example, the vectors belonging to the matrix can be summed to provide a summed vector.

For another example, the covariance of the matrix (of sentences) can be calculated. This can include multiplying the matrix by its transposed matrix.

The memory/processing unit can store a glossary. In particular, parts of the vocabulary can be stored in multiple memory groups of the memory/processing unit.

Therefore, it is possible to use the access information (such as the retrieval key) to access the memory/processing unit, so that at least from the memory group of the memory/processing unit Some extract vectors representing the phrase words of the sentence.

Different memory groups of the memory/processing unit can store different parts of the vocabulary and can be accessed in parallel (depending on the distribution of the index of the sentence). Even when more memory banks need to be accessed sequentially than in a single row, prediction can reduce the penalty.

It can be optimized to allocate the words of the vocabulary between different memory groups of the memory/processing unit, or the allocation can increase the meaning of the chance of parallel access of each sentence to the different memory groups of the memory/processing unit The above is highly beneficial. The allocation can be learned for each user, for each general group, or for each group.

In addition, the memory/processing unit can also be used to perform at least some of the processing operations (by its logic), and thereby reduce the bandwidth required by the bus outside the memory/processing unit, and can be used in an efficient manner (or even Parallel) Calculate multiple operations (multiple processing using memory/processing units in parallel 器).

The memory bank can be associated with logic.

At least part of the processing operations may be performed by one or more additional processors (such as vector processors, including but not limited to vector adders).

The memory/processing unit may include one or more additional processors that can be allocated to some or all of the memory groups (logical pairs).

Therefore, a single additional processor can be allocated to all or some of the memory banks (logical pairs). For yet another example, the additional processors can be configured in a hierarchical manner, so that the additional processors of a certain level process the output from the additional processors of a lower level.

It should be noted that the processing operations can be performed without using any additional processors, but can be performed by the logic of the memory/processing unit.

FIGS. 89A, 89B, 89C, 89D, 89E, 89F, and 89G illustrate examples of memory/processing units 9010, 9011, 9012, 9013, 9014, 9015, and 9019, respectively. The memory/processing unit 9010 includes a controller 9020, an internal bus 9021, and multiple pairs of logic 9030 and a memory bank 9040.

It should be noted that the logic 9030 and the memory bank 9040 can be coupled to the controller and/or each other in other ways. For example, multiple bus bars can be provided between the controller and the logic, and the logic can be configured in multiple layers. A single logic can be shared by multiple memory banks (see, for example, FIG. 89E), and the like.

The length of the page of each memory bank in the memory/processing unit 9010 can be defined in any way, for example, it can be small enough, and the number of memory banks can be large enough to output a large number of vectors in parallel without being Many bits are wasted on irrelevant information.

The logic 9020 may include a complete ALU, a partial ALU, a memory controller, a partial memory controller, and the like. Some ALU (memory controller) units can perform only part of the functions that can be performed by a complete ALU (memory controller). Any logical or stated in this application The sub-processors may include complete ALUs, partial ALUs, memory controllers, partial memory controllers, and the like.

The connectivity between the controller 9020 and the multiple pairs of logic 9030 and memory bank 9040 can be implemented in other ways. Other ways (unpaired) can be used to configure memory groups and logic.

The memory/processing unit 9010 may not have an additional vector, and the vector (from the memory bank) is processed by logic 9030.

FIG. 89B illustrates an additional processor, such as a vector processor 9050 coupled to the internal bus 9021.

Figure 89C illustrates an additional processor, such as a vector processor 9050 coupled to the internal bus 9021. One or more additional processors perform (alone or in conjunction with logic) processing operations.

FIG. 89D also illustrates the host 9018 coupled to the memory/processing unit 9010 via the bus 9022.

FIG. 89D also illustrates the mapping of words/phrases 9072 to the vocabulary 9070 of the vector 9073. The retrieval key 9071 is used to access the memory/processing unit, and each retrieval key represents a previously recognized word or phrase. The host 9018 sends a plurality of retrieval keys 9071 representing the sentence to the memory/processing unit, and the memory/processing unit can output the vector 9070 or the final result of the processing operation applied by the vector related to the sentence. The words/phrases are usually not stored in the memory/processing unit 9010.

The functionality of the memory controller used to control the memory bank may be included (alone or partially) in the logic, may be included (alone or partially) in the controller 9020, and/or may be included (alone or partially) in the One or more memory controllers (not shown) in the memory/processing unit 9010.

The memory/processing unit can be configured to maximize the throughput of vectors/results sent to the host 9018, or can be used to control internal memory/processing unit communications and/or control the memory/processing unit and the host computer (Or any other entity outside of the memory/processing unit) any processing procedures for communications between.

The different logic 9030 is coupled to the memory bank 9040 of the memory/processing unit, and can perform (preferably in parallel) mathematical operations on the vector to generate the processed vector. One logic 9030 can send a vector to another logic (see, for example, line 38 in FIG. 89G), and the other logic can apply mathematical operations to the received vector and the vector it calculates. Logic can be configured in levels, and a certain level of logic can handle vectors or intermediate results (generated by applying mathematical operations) from the previous level of logic. Such logic can form trees (binary, ternary, and the like).

When the total size of the processed vectors exceeds the total size of the result, the output bandwidth (outside the memory/processing unit) is reduced. For example, when K vectors are summed by the memory/processing unit to provide a single output vector, a K:1 reduction in bandwidth is obtained.

The controller 9020 can be configured to open multiple memory banks in parallel by broadcasting the addresses of different vectors to be accessed.

The controller can be configured to control multiple memory groups based at least in part on the order of the words or phrases in the sentence (or from any intermediate buffers or storage circuits that store different vectors, see buffer 9033 in FIG. 89D) Capture the order of different vectors.

The controller 9020 can be configured to manage the retrieval of different vectors based on one or more parameters related to the vector output external to the memory/processing unit 9010. For example, the rate at which different vectors are retrieved from the memory bank can be set to It is substantially equal to the allowable rate of outputting different vectors from the memory/processing unit 9010.

The controller can output different vectors outside the memory/processing unit 9010 by applying any traffic shaping processing program. For example, the controller 9020 may be designed to output different vectors at a rate as close as possible to the maximum rate allowed by the host computer or the link coupling the memory/processing unit 9010 to the host computer. For yet another example, the controller can output different vectors while minimizing or at least substantially reducing the fluctuation of the traffic rate over time.

The controller 9020 belongs to the same integrated circuit as the memory group 9040 and logic 9030, And therefore, it is easy to receive feedback on the acquisition status of different vectors from different logic/memory groups (for example, whether the vector is ready, whether the vector is ready but is being retrieved from the same memory group or is about to retrieve another vector) , And the like. Feedback can be provided in any way: via dedicated control lines, via shared control lines. Use one or more status bits and the like (see status line 9039 in Figure 89F).

The controller 9020 can independently control the acquisition and output of different vectors, and therefore can reduce the participation of the host computer. Alternatively, the host computer may not know the management capabilities of the controller and may continue to send detailed instructions. In this case, the memory/processing unit 9010 can ignore the detailed instructions, hide the management capabilities of the controller, and the like . The above mentioned solutions can be used based on the agreement that can be managed by the host computer.

It has been found that performing processing operations in the memory/processing unit is extremely beneficial (in terms of energy), even when these operations consume more power than processing operations in the host and even when these operations are compared This is also true when the transfer operation between the host and the memory/processing unit consumes more power. For example, assuming that the vector is large enough, the energy consumption of the data unit is 4pJ, and the energy consumption of the processing operation of the data unit (by the host) is 0.1pJ, then the energy consumption of the data unit is processed by the memory/processing unit When it is lower than 5pJ, it is more effective to process the data unit by the memory/processing unit.

Each vector (representing a matrix of sentences) can be represented by a sequence of words (or other multi-bit segments). For simplicity of explanation, it is assumed that multiple bit segments are words.

When the vector includes zero-valued words, additional power savings can be obtained. Instead of outputting the entire zero-valued word, a zero-valued flag shorter than the word (for example, bit) can be output (or even sent via a dedicated control line) instead of the entire word. The flag can be assigned to other values (for example, the value 1 zigzag).

FIG. 88A illustrates a method 9400 for embedding, or, to be precise, a method for extracting information related to feature vectors. The feature vector related information may include the feature vector and/or the result of processing the feature vector.

Method 9400 can start with the use of memory processing integrated circuits to receive and retrieve information for use In step 9410 of retrieving multiple requested feature vectors, the feature vectors can be mapped to multiple sentence segments.

The memory processing unit may include a controller, multiple processor sub-units, and multiple memory units. Each of the memory units can be coupled to a processor sub-unit.

Step 9410 may be followed by step 9420 of retrieving multiple requested feature vectors from at least some of the multiple memory cells.

The retrieval may include simultaneously requesting two or more memory units for the requested feature vectors stored in the two or more memory units.

The request is executed based on the known mapping between the sentence section and the location of the feature vector mapped to the sentence section.

The mapping can be uploaded during the boot process of the memory processing integrated circuit.

It may be beneficial to retrieve as many requested feature vectors as possible at one time, but this depends on where the requested feature vectors are stored and the number of different memory cells.

If more than one requested feature vector is stored in the same memory set, predictive retrieval can be applied to reduce the penalty associated with retrieving information from the memory set. Various methods for reducing punishment are explained in various chapters of this application.

The retrieval may include predictive retrieval using at least some of the requested feature vectors from the set of requested feature vectors stored in a single memory unit.

The requested feature vectors can be optimally distributed among memory cells.

The requested feature vector can be distributed among the memory cells based on the expected capture pattern.

The extraction of multiple requested feature vectors can be performed according to a certain order. For example, according to the order of sentence sections in one or more sentences.

The extraction of the plurality of requested feature vectors may be performed at least partially out of order; and the extraction may further include reordering the plurality of requested feature vectors.

The extraction of multiple requested features can be included in multiple requested feature vectors that can be controlled The multiple requested feature vectors are buffered before being read by the processor.

The retrieval of the plurality of requested features may include generating buffer status indicators that prompt one or more buffers associated with the plurality of memory cells when to store the one or more requested feature vectors.

The method may include transmitting the buffer status indicator via a dedicated control line.

Each memory unit can be assigned a dedicated control line.

The buffer status indicator can be a status bit stored in one or more buffers.

The method may include transmitting the buffer status indicator via one or more common control lines.

Step 9420 can be followed by step 9430 of processing multiple requested feature vectors to provide processing results.

Additionally or alternatively, step 9420 can be followed by a step 9440 of processing the integrated circuit output from memory, which can include at least one of the following: (a) the requested feature vector; and (b) processing the requested The result of the feature vector. At least one of (a) the requested feature vector and (b) the result of processing the requested feature vector is also referred to as feature vector related information.

When step 9430 is performed, step 9440 may include outputting (at least) the result of processing the requested feature vector.

When step 9430 is skipped, step 9440 includes outputting the requested feature vector and may not include outputting the result of processing the requested feature vector.

Figure 88B illustrates the method 9401 for embedding.

It is assumed that the output includes the requested feature vector, but does not include the result of processing the requested feature vector.

The method 9401 can start with the step 9410 of receiving the retrieval information by the memory processing integrated circuit for retrieving a plurality of requested feature vectors, and the feature vectors can be mapped to a plurality of sentence segments.

Step 9410 can be followed by fetching multiple memory cells from at least some of the multiple memory cells. Step 9420 of the requested feature vector.

Step 9420 can be followed by step 9431 of outputting an output including the requested feature vector but not the result of processing the requested feature vector from the memory processing integrated circuit.

Figure 88C illustrates the method 9402 for embedding.

It is assumed that the output includes the result of processing the requested feature vector.

The method 9402 can start with the step 9410 of receiving the retrieved information by the memory processing integrated circuit for retrieving a plurality of requested feature vectors, and the feature vectors can be mapped to a plurality of sentence segments.

Step 9410 may be followed by step 9420 of retrieving multiple requested feature vectors from at least some of the multiple memory cells.

Step 9420 can be followed by step 9430 of processing multiple requested feature vectors to provide processing results.

Step 9430 can be followed by step 9442 of processing the output of the integrated circuit from the memory, which can include processing the output of the requested feature vector.

The output of the output may include applying traffic shaping to the output.

The output of the output may include an attempt to match the bandwidth used during the output with the maximum allowable bandwidth on the link that couples the memory processing integrated circuit to the requester unit.

The output of the output may include an attempt to maintain the fluctuation of the output traffic rate below a threshold value.

Any step in the capture and output can be executed by the controller under the control of the host and/or independently or in part.

The host can send capture commands with different granularities, from generally sending capture information regardless of the location of the requested feature vector in multiple memory cells, until based on the location of the requested feature vector in multiple memory cells Send detailed retrieval information.

The host can control (or try to control) the timing of different acquisition operations in the memory processing integrated circuit, but it may not be related to the timing.

The controller can be controlled by the host in various levels, and can even ignore the detailed commands of the host, and independently control at least the capture and/or output.

The processing of the requested feature vector can be performed by at least one of the following (one or a combination of more of the following): one or more memories/processing units, and one or more memories/processing One or more processors outside the unit, and the like.

It should be noted that the processing of the requested feature vector can be performed by at least one of the following (a combination of one or more of the following): one or more processor sub-units, a controller, one or more vectors The processor, and one or more memories/processing units located outside of the one or more memories/processing units.

The processing of the requested feature vector can be performed by any one of the following or a combination of the following, and can be generated by any one of the following or a combination of the following:

a. The processor sub-unit of the memory/processing unit (or logic 9030).

b. Processor sub-units of multiple memory/processing units (or logic 9030).

c. Controller of memory/processing unit.

d. Controller with multiple memory/processing units.

e. One or more vector processors of memory/processing unit.

f. One or more vector processors, multiple memory/processing units.

Therefore, the processing of the requested feature vector can be performed by any combination or sub-combination of: (a) one or more memories/processing unit or one or more controllers; (b) one or more memories/ One or more processor sub-units of the processing unit; (c) one or more vector processors of one or more memory/processing units; and (d) one of the one or more memory/processing units located outside Or multiple other processors.

Processing performed by more than one processing entity can be referred to as distributed processing.

The memory/processing unit may include multiple processor sub-units. The processor sub-units can each other Operate independently, partially cooperate with each other, participate in distributed processing, and the like.

The processing can be performed in a planar manner, in which all processor subunits perform the same operation (and may or may not output the processing result in between).

The processing can be performed in a hierarchical manner, where the processing involves a sequence of processing operations at different levels, and the processing operations of a certain level follow the processing operations of another level. Processor sub-units can be allocated (dynamically or statically) to different layers and participate in hierarchical processing.

Any processing of the requested feature vector can be performed by more than one processing entity (processor subunits, controllers, vector processors, other processors), and distributed processing can be performed in any manner (planar, hierarchical, or other methods). For example, the processor sub-unit can output its processing results to the controller, and the controller can further process the results. One or more other processors located outside the one or more memory/processing units can further process the output of the memory processing integrated circuit.

It should be noted that the retrieved information may also include information used to retrieve the requested feature vector that is not mapped to the sentence section. These feature vectors can be mapped to one or more persons, devices, or any other entities that can be related to the sentence segment. For example, the user of the device that senses the sentence section, the device that senses the section, the user identified as the source of the sentence section, the website accessed when the sentence was generated, the location of the captured sentence, and the like By.

After making necessary modifications in details, the methods 9400, 9401, and 9402 can be applied to processes that are not mapped to sentence segments and/or the requested extraction vector.

Non-limiting examples of the processing of feature vectors may include summation, weighted sum, average, subtraction, or application of any other mathematical function.

Mixing device

As both processor speed and memory size continue to increase, the significant limitation on effective processing speed is the von Neumann bottleneck. The Von Neumann bottleneck is caused by the throughput limitation caused by the conventional computer architecture. In particular, compared to the actual calculations performed by the processor, self-report Data transfer from the memory to the processor (outside the logic die such as external DRAM memory) often encounters bottlenecks. Therefore, the number of clock cycles used to read and write the memory increases significantly with memory-intensive processing procedures. These clock cycles result in a lower effective processing speed. This is because reading and writing to the memory consumes clock cycles, and these clock cycles cannot be used to perform operations on data. In addition, the computing bandwidth of the processor is generally greater than the bandwidth of the bus used by the processor to access the memory.

These bottlenecks are particularly obvious for the following: memory-intensive processing procedures, such as neural networks and other machine learning algorithms; database construction, index search and query; and include more reads and writes than data processing operations Other tasks of operation.

The present invention describes solutions for alleviating or overcoming one or more of the problems set forth above and other problems in the prior art.

A hybrid device for memory-intensive processing can be provided. The hybrid device can include a basic die, a plurality of processors, a first memory resource of at least another die, and a second memory of at least one other die Physical resources.

The base die and the at least another die are connected to each other by wafer-on-wafer bonding.

The multiple processors are configured to perform processing operations and retrieve the retrieved information stored in the first memory resource.

The second memory resource is configured to send additional information from the second memory resource to the first memory resource.

The total bandwidth of the first path between the base die and at least another die exceeds the total bandwidth of the second path between at least another die and at least one other die, and the storage of the first memory resource The capacity is a part of the storage capacity of the second memory resource.

The second memory resource is a high-bandwidth memory (HBM) resource.

At least one other die is a stack of high-bandwidth memory (HBM) chips.

At least some of the second memory resources may belong to another die that is connected to the base die by a different connectivity from the inter-wafer bonding.

At least some of the second memory resources belong to another die, and the other die is connected to the other die by a different connectivity from the inter-wafer bonding.

The first memory resource and the second memory resource are different levels of cache memory.

The first memory resource is positioned between the basic die and the second memory resource.

The first memory resource is positioned on one side of the second memory resource.

The other die is configured to perform additional processing, and the other die includes a plurality of processor sub-units and a first memory resource.

Each processor sub-unit is coupled to a unique portion of the first memory resource allocated to the processor sub-unit.

The only part of the first memory resource is at least one memory bank.

The plurality of processors is a plurality of processor subunits included in the first memory resource for the memory processing chip.

The basic die includes a plurality of processors, wherein the plurality of processors are a plurality of processor sub-units coupled to the first memory resource through a conductor formed by using inter-wafer bonding.

Each processor sub-unit is coupled to a unique portion of the first memory resource allocated to the processor sub-unit.

A hybrid integrated circuit can be provided, which can utilize wafer-on-wafer (WOW) connectivity to couple at least a part of the basic die to a second memory resource, and the second memory resource is included in one or more Other dies are connected using a connectivity different from that of WOW. An example of the second memory resource may be a high-bandwidth memory (HBM) memory resource. In the various figures, the second memory resource is included in the stack of HBM memory cells and can be coupled to the controller using via-silicon via (TSV) connectivity. The controller can be included in the base die or coupled (for example, via micro bumps) to the base die At least part of it.

The basic die can be a logic die, but can be a memory/processing unit.

WOW connectivity is used to couple one or more parts of the basic die to one or more parts of another die (WOW connected die), which can be a memory die or a memory /Processing unit. WOW connectivity is extremely high throughput connectivity.

The stack of high-bandwidth memory (HBM) chips can be coupled to the basic die (die connected directly or via WOW), and can provide high throughput connections and extremely extensive memory resources.

The WOW-connected die can be coupled between the stack of HBM chips and the base die to form an HBM memory chip stack that has TSV connectivity and has WOW-connected dies at the bottom.

HBM chip stacks with TSV connectivity and WOW connected dies at the bottom can provide multiple levels of memory, where WOW connected dies can be used as lower-level memory accessible by the basic die (for example, 3-level Cache memory), where the fetch and/or prefetch operation from the higher-level HBM memory stack fills the WOW connected die.

The HBM memory chip can be an HBM DRAM chip, but any other memory technology can be used.

Using a combination of WOW connectivity and HMB chips enables the provision of a multi-layer memory structure that can include multiple memory layers that can provide different trade-offs between bandwidth and memory density.

The proposed solution can serve as an additional brand new memory hierarchy between the traditional DRAM memory/HBM and the internal cache memory of the logic die, thereby achieving more bandwidth and better management and reuse on the DRAM side.

This can provide a new memory hierarchy on the DRAM side that better manages memory reads in a fast manner.

FIGS. 93A to 93I illustrate the hybrid integrated circuits 11011' to 11019', respectively.

Figure 93A illustrates an HBM DRAM stack with TSV connectivity and micro bumps at the lowest level (collectively denoted as 11030). The stack includes a first memory control coupled to each other and coupled to the base die using TSV (11039) The HDM DRAM memory chip 11032 of the device 11031 is stacked.

FIG. 93A also illustrates a wafer (collectively indicated as 11040) having at least memory resources and coupled using WOW technology. The wafer includes a wafer (11021) coupled to a DRAM wafer (11021) through one or more WOW interlayers (11023) The second memory controller 11022 of the base die 11019. One or more WOW intermediate layers may be made of different materials, but may be different from pad connectivity and/or may be different from TSV connectivity.

Conductors 11022' passing through one or more WOW intermediate layers electrically couple the DRAM die to the components of the base die.

The base die 11019 is coupled to the interposer 11018, which in turn is coupled to the package substrate 11017 using micro bumps. The package substrate has an array of micro bumps on its lower surface.

The micro bumps can be replaced by other connectivity. The interposer 11018 and the packaging substrate 11017 can be replaced by other layers.

The first and/or second memory controllers (11031 and 11032, respectively) can be positioned (at least partly) outside the base die 11019, for example, positioned in the DRAM wafer, between the DRAM wafer and the base die, HBM Between the stack of memory cells and the base die, and the like.

The first and/or second memory controllers (11031 and 11032, respectively) may belong to the same controller or may belong to different controllers.

One or more of the HBM memory units may include logic and memory, and may or may include memory/processing units.

The first and second memory controllers are coupled to each other through a plurality of bus bars 11016 to use Transfer information between the first memory resource and the second memory resource. FIG. 93A also illustrates the bus 11014 from the second memory controller to the components of the base die (for example, multiple processors). FIG. 93A further illustrates the bus 11015 from the first memory controller to the components of the base die (for example, multiple processors, as shown in FIG. 93C).

FIG. 93B illustrates the hybrid integrated circuit 11012, which is different from the hybrid integrated circuit 11011 of FIG. 93A in that it has a memory/processing unit 11021' instead of a DRAM die 11021.

FIG. 93C illustrates the hybrid integrated circuit 11013, which is different from the hybrid integrated circuit 11011 of FIG. 93A in that it has an HBM memory chip stack, which has TSV connectivity and has WOW connected die at its bottom (commonly indicated 11040), the die includes a DRAM die 11021 between the stack of HBM memory cells and the base die 11018.

The DRAM die 11021 is coupled to the first memory controller 11031 of the base die 11019 using WOW technology (see WOW intermediate layer 11023). One or more of the HBM memory die 11032 may include logic and memory, and may or may include a memory/processing unit.

The lowermost DRAM die (denoted as DEAM die 11021 in FIG. 93C) may be an HBM memory die or may be different from an HBM die. The lowermost DRAM die (DRAM die 11021) can be replaced by a memory/processing unit 11021', as illustrated by the hybrid integrated circuit 11014 of FIG. 93D.

Figures 93E to 93G illustrate the hybrid integrated circuits 11015, 11016, and 11016', respectively, in which the base die 11019 is coupled to the HBM DRAM stack (11020) with TSV connectivity and micro bumps at the lowest level and has at least memory resources And multiple instances of wafers (11030) coupled using WOW technology, and/or multiple instances of HBM memory chip stacks (11040) coupled to TSV connectivity and with WOW connected dies at the bottom item.

FIG. 93H illustrates the hybrid integrated circuit 11014'. The difference between the hybrid integrated circuit and the hybrid integrated circuit 11014 of FIG. 93D is to illustrate the memory unit 53, the second-level cache memory (L2 fast Take memory 52), multiple processors 11051. The multiple processors 11051 are coupled to the L2 cache memory 11052, and can be fed with coefficients and/or data stored in the memory unit 11053 and the L2 cache memory 11052.

Any of the hybrid integrated circuits mentioned above can be used for artificial intelligence (AI) processing, which is bandwidth intensive.

When coupled to a memory controller using WOW technology, the memory/processing unit 11021' of Figures 93D and 93H can perform AI calculations and can receive data from HBM DRAM stacks and/or from WOW connected dies at a very high rate And the coefficient both.

Any memory/processing unit can include distributed memory arrays and processor arrays. Distributed memory and processor arrays may include multiple memory banks and multiple processors. Multiple processors can form a processing array.

Refer to FIGS. 93C, 93D, and 93H and assume that a hybrid integrated circuit (11013, 11014, or 11014') is required to perform general matrix vector multiplication (GEMV), which includes calculating the product of a matrix and a vector. Because there is no repeated use of the extracted matrix, this type of calculation is bandwidth-intensive. Therefore, it is only necessary to retrieve and use the entire matrix once.

GEMV can be part of a sequence of mathematical operations involving (i) multiplying a first matrix (A) by a first vector (V1) to provide a first intermediate vector, and applying a first nonlinear operation (NLO1) to the first intermediate vector To provide the first intermediate result; (ii) multiply the second matrix (B) by the first intermediate result with a second intermediate vector, and apply a second non-linear operation (NLO2) to the second intermediate vector to provide the second intermediate result, And so on (until the Nth intermediate result is received, N can exceed 2).

Assuming that each matrix is large (for example, 1Gb), the calculation will require 1Tbs of computing power and 1Tbs of bandwidth/transmission. Operations and calculations can be performed in parallel.

Assume that the GEMV calculation exhibits N=4 and has the following form: result=NLO4(D*(NLO3(C*(NLO2(B*(NLO1(A*V1))))))).

It is also assumed that the DRAM die 11021 (or memory/processing unit 11021') does not have enough memory resources to store A, B, C, and D at the same time, then at least some of these matrices will be stored in the HDM DRAM die 11032 in.

It is assumed that the basic die is a logic die including a computing unit such as but not limited to a processor, arithmetic logic unit, and the like.

When the first die calculates A*V1, the first memory controller 11031 retrieves the missing parts of other matrices from one or more HBM DRAM die 11032 to use in the next calculation.

Refer to Figure 93H and assume that (a) the DRAM die 11021 has a bandwidth of 2TBs and a capacity of 512Mb, (b) the HBM DRAM die 11032 has a bandwidth of 0.2TBs and a capacity of 8Gb, and (c) the L2 cache 11052 has a capacity of 6Ts SRAM with bandwidth and 10Mb capacity.

Matrix multiplication involves reusing data, segmenting a large matrix into multiple segments (for example, 5Mb segments to fit the L2 cache that can be used in a double-buffer configuration) and extracting the first The matrix segment is multiplied by the second matrix segment (one second matrix segment followed by another second matrix segment).

When the first matrix section is multiplied by the second matrix section, another second matrix section is extracted from the DRAM die 11021 (of the memory processing unit 11021') to the L2 cache.

Assuming that the matrices are each 1Gb, the DRAM die 11021 or the memory/processing unit 11021' is fed with the matrix section from the HBM DRAM die 11032 when performing the acquisition and calculation.

The DRAM die 11021 or the memory/processing unit 11021' gathers matrix segments, and the matrix segments are then fed to the base die 11019 through the WOW intermediate layer (11023).

The memory/processing unit 11021' can reduce the amount of information sent to the base die 11019 through the WOW intermediate layer (11023) by performing calculations and sending the results instead of sending the calculated intermediate values to provide the results. When multiple (Q) intermediate values are processed to provide a result, the compression ratio may be Q to 1.

FIG. 93I illustrates an example of a memory processing unit 11019' implemented using WOW technology. The logic unit 9030 (may be a processor sub-unit), the controller 9020 and the bus 9021 are located in one chip 111061, the memory group 9040 allocated to different logic units is located in the second chip 11062, and the first and second chips are used The conductors 11012' passing through the WOW junction 11061 are connected to each other, and the WOW junction may include one or more WOW intermediate layers.

Figure 93J shows an example of a method 11100 for memory-intensive processing. Memory intensive means that processing requires high bandwidth memory consumption or is related to high bandwidth memory consumption.

Method 11100 can start at steps 11110, 11120, and 11130.

Step 11110 includes performing processing operations by a plurality of processor hybrid devices, the hybrid device including a base die, a first memory resource of at least another die, and a second memory resource of at least one other die; wherein the base die The die and at least another die are connected to each other by wafer-on-wafer bonding.

Step 11120 includes using multiple processors to retrieve the retrieved information stored in the first memory resource.

Step 11130 may include sending additional information from the second memory resource to the first memory resource, wherein the total bandwidth of the first path between the basic die and at least another die exceeds that of at least another die and at least another die. The total bandwidth of the second path between one other die, and the storage capacity of the first memory resource is a part of the storage capacity of the second memory resource.

The method 11100 may also include a step 11140 of performing additional processing by another die including a plurality of processor sub-units and a first memory resource.

Each processor sub-unit can be coupled to a unique portion of the first memory resource allocated to the processor sub-unit.

The only part of the first memory resource is at least one memory bank.

Steps 11110, 11120, 11130, and 11140 can be performed simultaneously, in a partially overlapping manner, and similar manners.

The second memory resource may be a high-bandwidth memory (HBM) memory resource or may be different from the HBM memory resource.

At least one other die is a stack of high-bandwidth memory (HBM) memory chips.

Communication chip

The database includes many entries, and these entries include multiple fields. Database processing usually includes executing one or more queries that include one or more filter parameters (for example, identifying one or more related fields and one or more related field values) and also includes a Or multiple operating parameters, the one or more operating parameters can determine the type of operation to be performed, the variable or constant to be used in the application operation, and the like. Data processing may include database analysis or other database processing procedures.

For example, a database query can request a statistical operation (operation parameter) to be performed on all records in the database, and a certain field has a value within a predefined range (filter parameter). For another example, the database query may request deletion of (operation parameter) records with a certain field less than the threshold value (screening parameter).

Large databases are usually stored in storage devices. In order to respond to queries, the database is sent to the memory unit, usually one database section followed by another database section.

The entries of the database section are sent from the memory unit to the processor that does not belong to the same integrated circuit as the memory unit. These entries are then processed by the processor.

For each database section of the database stored in the memory unit, the processing includes the following steps: (i) select the record of the database section; (ii) send the record from the memory unit to the processor; (iii) ) Filter the records by the processor to determine whether the records are relevant; and (iv) Perform one or more additional operations on the related records (summing, applying any other mathematical operations and/or statistical operations).

The screening process ends after all records are sent to the processor and the processor determines which records are relevant.

In the case that the relevant entries in the database section are not stored in the processor, you need to After the screening stage, these related records are sent to the processor for further processing (operations applied after processing).

When multiple processing operations follow a single screening, the result of each operation can be sent to the memory unit and then sent to the processor again.

This processing procedure is bandwidth-consuming and time-consuming.

There is an increasing need to provide efficient ways to perform database processing.

It is possible to provide a device that can include a database to accelerate an integrated circuit.

It is possible to provide a device that can include one or more groups of database accelerated integrated circuits, and the database accelerated integrated circuits can be configured to perform data in one or more groups of database accelerated integrated circuits. The database accelerates the exchange of information and/or acceleration results between integrated circuits (the final result of the processing performed by the database to accelerate the integrated circuits).

The database acceleration integrated circuit of the group can be connected to the same printed circuit board.

The group's database acceleration integrated circuit can be a modular unit of a computerized system.

Different groups of database accelerated integrated circuits can be connected to different printed circuit boards.

Different groups of database accelerated integrated circuits can belong to different modular units of the computerized system.

The device can be configured to use one or more groups of databases to accelerate integrated circuits to execute distributed processing procedures.

The device can be configured to use at least one switch for the exchange of (a) information and (b) database acceleration results between database acceleration integrated circuits of different groups in one or more groups At least one of them.

The device can be configured to speed up some of the integrated circuits to execute distributed processing procedures through the database of some of one or more groups.

The device can be configured to perform distributed processing of the first and second data structures Sequence, where the total size of the first and second data structures exceeds the storage capacity of multiple memory processing integrated circuits.

The device can be configured to perform distributed processing by performing multiple iterations of the following steps: (a) Perform new allocation of different pairs of the first data structure part and the second data structure part to different database acceleration products Body circuit; and (b) deal with different pairs.

94A and 9B illustrate examples of the storage system 11560, the computer system 11150, and one or more devices 11520 for database acceleration. One or more devices 11520 for database acceleration can monitor the communication between the storage system 11560 and the computer system 11150 in various ways (by monitoring or by positioning between the computer system 11150 and the storage system 11560).

The storage system 11560 may include many (for example, more than 20, 50, 100, 100, and the like) storage units (such as disks or disk raids), and may store more than 100 trillion, for example Byte information. The computing system 11510 may be a large-scale computer system and may include tens, hundreds, or even thousands of processing units.

The computing system 11510 may include a plurality of computing nodes 11512 controlled by the manager 11511.

The computing node can control or interact with one or more devices 11520 for database acceleration.

The one or more devices 11520 for database acceleration may include one or more database acceleration integrated circuits (see, for example, the database acceleration integrated circuit 11530 in FIGS. 94A and 94B) and a memory resource 11550. The memory resource can belong to one or more chips dedicated to the memory but can belong to the memory/processing unit.

94C and 94D illustrate an example of the computer system 11150 and one or more devices 11520 for database acceleration.

One or more database acceleration products of one or more devices 11520 for database acceleration The body circuit can be controlled by the management unit 11513, which can be located in the computer system (see FIG. 94C) or located in one or more devices 11520 for database acceleration (FIG. 94D).

FIG. 94E illustrates a device 11520 for database acceleration. The device includes a database acceleration integrated circuit 11530 and a plurality of memory processing integrated circuits 1151. Each memory processing integrated circuit may include a controller, multiple processor sub-units, and multiple memory units.

The database acceleration integrated circuit 11530 is illustrated as including a network communication interface 11531, a first processing unit 11532, a memory controller 11533, a database acceleration unit 11535, an interconnect 11536, and a management unit 11513.

The network communication interface (11531) can be configured to receive a large amount of information from a mass storage unit (for example, via the first port 11531(1) of the network communication interface). Each storage unit can output information at a rate exceeding tens or even hundreds of megabytes per second, and the data transfer rate is expected to increase over time (for example, doubling every 2 to 3 years). The number of data storage units (large number) can exceed 10, 50, 100, 200 and even more. A large amount of information can exceed tens or tens of billions of bytes per second, and can even be in the range of trillion bytes per second and gigabytes per second.

The first processing unit 11532 can be configured to perform first processing (preprocessing) on a large amount of information to provide first processed information.

The memory controller 11533 can be configured to send the first processed information to a plurality of memory processing integrated circuits via the high throughput interface 11534.

The plurality of memory processing integrated circuits 11551 may be configured to perform second processing (processing) on at least a part of the first processed information through the plurality of memory processing integrated circuits to provide second processed information.

The memory controller 11533 can be configured to retrieve the captured information from multiple memory processing integrated circuits. The retrieved information may include at least one of the following: (a) at least a part of the first processed information; and (b) at least a part of the second processed information.

The database acceleration unit 11535 can be configured to perform database processing operations on the retrieved information to provide database acceleration results.

The database acceleration integrated circuit can be configured to output the database acceleration result, for example, via one or more second ports 11531(2) of the network communication interface.

FIG. 94E also illustrates the management unit 11513, which is configured to manage at least one of the following: the extraction of the captured information, the first processing (preprocessing), the second processing (processing), and the third Processing (database processing). The management unit 11513 may be located outside the database acceleration integrated circuit.

The management unit may be configured to perform the management based on the execution plan. The execution plan can be generated by the management unit, or by an entity located outside the accelerated integrated circuit of the database. The execution plan may include at least one of the following: (a) instructions to be executed by various components of the integrated circuit accelerated by the database, (b) data and/or coefficients required to implement the execution plan, (c) instructions And/or memory allocation of data.

The management unit can be configured to perform management by allocating at least some of the following: (a) network communication network interface resources, (b) decompression unit resources, (c) memory controller resources, ( d) Multiple memory processing integrated circuit resources, and (e) database acceleration unit resources.

As illustrated in FIG. 94E and FIG. 94G, the network communication network interface may include different types of network communication ports.

Different types of network communication ports can include storage interface protocol ports (for example, SATA port, ATA port, ISCSI port, network file system, Fibre Channel port) and general network storage interface protocol ports (for example, Ethernet ATA , Ethernet Fibre Channel, NVME, Roce and others).

Different types of network communication ports can include storage interface protocol ports and PCIe ports.

FIG. 94F includes dotted lines that illustrate the flow of large amounts of information, first processed information, retrieved information, and database acceleration results. FIG. 94F illustrates the database acceleration integrated circuit 11530 as being coupled to a plurality of memory resources 11550. Multiple memory resources 11550 may not belong to the memory processing complex Circuit.

The device 11520 for database acceleration can be configured to perform multiple tasks at the same time by the database acceleration integrated circuit 11530. This is because the network communication interface 11531 can receive multiple information streams (simultaneously), the first processing The unit 11532 can perform the first processing on multiple information units at the same time, the memory controller 11533 can send multiple first processed information units to multiple memory processing integrated circuits 11551 at the same time, and the database acceleration unit 11535 can simultaneously process multiple information units. Units of information retrieved.

The device 11520 for database acceleration can be configured to perform at least one of acquisition, first processing, sending, and third processing based on the execution plan sent to the database acceleration integrated circuit by the operation node of the large-scale computing system One.

The device 11520 for database acceleration can be configured to manage at least one of acquisition, first processing, sending, and third processing in a manner that substantially optimizes the use of database acceleration integrated circuits. The optimization takes into account latency, throughput, and any other timing or storage or processing considerations, and tries to keep all components along the flow path busy and without bottlenecks.

The database acceleration integrated circuit can be configured to output the database acceleration result, for example, via one or more second ports 11531(2) of the network communication interface.

The device 11520 for database acceleration can be configured to substantially optimize the bandwidth of the traffic exchanged through the network communication network interface.

The device 11520 for database acceleration can be configured to substantially prevent at least one of capture, first processing, sending, and third processing by substantially optimizing the use of the database to accelerate the use of integrated circuits Form a bottleneck.

The device 11520 for database acceleration can be configured to allocate the resources of the database acceleration integrated circuit according to the time I/O bandwidth.

FIG. 94G illustrates a device 11520 for database acceleration. The device includes a database acceleration integrated circuit 11530 and a plurality of memory processing integrated circuits 1151. Figure 94G also illustrates coupling to data The various units of the library acceleration integrated circuit 11530: remote RAM 11546, Ethernet memory DIMM 11547, storage system 11560, local storage unit 11561 and non-volatile memory (NVM) 11563 (the non-volatile memory can be It is a fast NVM unit (NVME)).

The database acceleration integrated circuit 11530 is illustrated as including an Ethernet port 11531 (1), an RDMA unit 11545, a serial expansion port 11531 (15), a SATA controller 11540, a PCIe port 11531 (9), and a first processing unit 11532, memory controller 11533, database acceleration unit 11535, interconnect 11536, management unit 11513, cryptographic engine 11537 for performing cryptographic operations, and second-level static random access memory (L2 SRAM) 11538.

The database acceleration unit is illustrated as including a DMA engine 11549, a third-level (L3) memory 11548, and a database acceleration sub-unit 11547. The database acceleration subunit 11547 can be a configurable unit.

The Ethernet port 11531 (1), RDMA unit 11545, serial expansion port 11531 (15), SATA controller 11540, PCIe port 11531 (9) can be regarded as part of the network communication interface 11531.

The remote RAM 11546, the Ethernet memory DIMM 11547, and the storage system 11560 are coupled to the Ethernet port 11531(1), which is in turn coupled to the RDMA unit 11545.

The local storage unit 11561 is coupled to the SATA controller 11540.

The PCIe port 11531(9) is coupled to the NVM 11563. The PCIe port can also be used to exchange commands, for example for management purposes.

FIG. 94H is an example of the database acceleration unit 11535.

The database acceleration unit 11535 can be configured to simultaneously execute database processing commands by the database processing subunit 11573, wherein the database acceleration unit can include a group of database accelerator subunits sharing a common memory unit 11575.

The different combinations of the database acceleration subunit 11535 can be dynamically linked to each other (via a configurable link or interconnection 11576) to provide the necessary database processing operations that can include multiple commands Execution pipeline.

Each database processing subunit can be configured to execute specific types of database processing commands (for example, filtering, merging, accumulation, and the like).

FIG. 94H also illustrates the independent database processing unit 11572 coupled to the cache memory 11571. Instead of the reconfigurable array 11574 of the DB accelerator, or in addition to the reconfigurable array 11574 of the DB accelerator, a database processing unit 11572 and a cache memory 11571 can also be provided.

The device can facilitate inward expansion and/or outward expansion, thus enabling multiple databases to accelerate the integrated circuit 11530 (and its associated memory resource 11550 or its associated multiple memory processing integrated circuits 11551) It can cooperate with each other, for example, by participating in distributed processing of database operations.

FIG. 94I illustrates a modular unit, such as the blade 11580, that includes two database acceleration integrated circuits 11530 (and its associated memory resource 11550). The blade may include one, two, or more than two memory processing integrated circuits 11551 and their associated memory resources 11550.

The blade can also include one or more non-volatile memory units, Ethernet switches, PCIe switches and Ethernet switches.

Multiple blades can communicate with each other using any communication method, communication protocol, and connectivity.

FIG. 94I illustrates four database acceleration integrated circuits 11530 (and its associated memory resource 11550) that are fully connected to each other. Each database acceleration integrated circuit 11530 is connected to all three other database acceleration integrated circuits 11530 . Connectivity can use any communication protocol, for example, by using the Ethernet RDMA protocol.

94I also illustrates the database acceleration integrated circuit 11530, which is connected to its associated memory resource 11550 and the unit 11531 including RAM memory and Ethernet port.

Fig. 94J, Fig. 94K, Fig. 94L, and Fig. 94M illustrate four groups 11580 of the database accelerated integrated circuit, each group includes four database accelerated integrated circuits 11530 (fully connected to each other) 11550 and its associated memory resources. Different groups are connected to each other via the switch 11590.

The number of groups can be two, three or more than four. The number of database accelerated integrated circuits in each group can be two, three, or more than four. The number of groups can be the same as (or can be different from) the number of database acceleration integrated circuits per group.

FIG. 94K illustrates two tables A and B, which are too large (for example, 1 trillion bytes) to be efficiently joined at one time.

The table is actually segmented into pieces and the splicing operation is applied to the pair including the piece of table A and the piece of table B.

The group of the database accelerated integrated circuit can handle the slices in various ways.

For example, the device can be configured to perform distributed processing by:

g. Allocate different first data structure parts (slices of Table A, such as the first to sixteenth slices A0 to A15) to one or more groups of different database acceleration integrated circuits.

h. Perform multiple iterations of each of the following: (i) Newly assign different second data structure parts (slices of table B, such as the first to sixteenth slices B0 to B15) to one or more groups Groups of different databases accelerate the integrated circuit; and (ii) use the database to accelerate the integrated circuit to process the first and second data structure parts.

The device can be configured to perform a new allocation for the next iteration in a manner that overlaps at least part of the time with the current iteration.

The device can be configured to perform a new allocation by exchanging the second data structure part between different database accelerated integrated circuits.

The exchange can be performed in a manner that overlaps at least part of the time with the processing procedure.

The device can be configured to perform a new allocation by: exchanging the second data structure part between the different database acceleration integrated circuits of the group; and once the exchange has been completed, then accelerating the integrated circuit in the database The second data structure part is exchanged between different groups.

In Fig. 94K, four cycles of some of the joining operations are shown. For example, referring to the upper left database acceleration integrated circuit 11530 in the upper left group, the four cycles include calculating Join(A0, B0), Join(A0, B3). ), Join (A0, B2) and Join (A0, B1). During these four cycles, A0 remains at the same database acceleration integrated circuit 11530, and the slices of matrix B (B0, B1, B2, and B3) are members of the same group in the database acceleration integrated circuit 11530 Rotate between.

In Figure 94L, the slices of the second matrix are rotated between different groups. (a) The slices B0, B1, B2, and B3 (previously processed by the upper left group) are sent from the upper left group to the lower left Group, (b) send the slices B4, B5, B6 and B7 (previously processed by the lower left group) from the lower left group to the upper right group, (c) send the slices B8, B9, B10 and B11 (Previously processed by the upper right group) sent from the upper right group to the lower right group, and (d) segments B12, B13, B14, and B15 (previously processed by the lower right group) were sent from the lower right group To the upper left group.

Figure 94N is an example of a system that includes multiple blades 11580, SATA controller 11540, local storage unit 11561, NVME 11563, PCIe switch 11601, Ethernet memory DIMM 11547, and Ethernet port 11531 ( 4).

The blade 11580 can be coupled to each of the PCIE switch 11601, the Ethernet port 11531, and the SATA controller 11540.

Figure 94O illustrates two systems 11621 and 11622.

The system 11621 may include one or more devices 11520 for database acceleration, an exchange system 11611, a storage system 11612, and a computing system 11613. The exchange system 11611 provides connectivity between one or more of the devices 11520, the storage system 11612, and the computing system 11613 for database acceleration.

The system 11622 may include a storage system and one or more devices 11615 for database acceleration, an exchange system 11611, and a computing system 11613. The exchange system 11611 provides a storage system and a connection between one or more devices 11615 and a computing system 11613 for database acceleration.

Figure 95A illustrates a method 11200 for database acceleration.

The method 11200 can start at step 11210 of retrieving a large amount of information from a large number of storage units through a network communication network interface of a database accelerated integrated circuit.

Connecting to a large number of storage units (for example, using multiple different buses) enables the network communication network interface to receive a large amount of information, even when a single storage unit has a limited throughput.

Step 11210 can be followed by performing first processing on a large amount of information to provide first processed information. The first processing may include buffering, extracting information from the payload, removing headers, decompressing, compressing, decrypting, filtering database queries, or performing any other processing operations. The first processing may also be limited to buffering.

Step 11210 can be followed by step 11220 of accelerating the memory controller of the integrated circuit through the database and sending the first processed information to a plurality of memory processing integrated circuits via a large throughput interface, where each memory processing The integrated circuit may include a controller, a plurality of processor sub-units, and a plurality of memory units. The memory processing integrated circuit can be a memory/processing unit or a distributed processor or a memory chip, as described in any other part of this patent application.

Step 11220 can be followed by step 11230 of performing a second processing on at least part of the first processed information by a plurality of memory processing integrated circuits to provide second processed information.

Step 11230 may include accelerating the integrated circuit to perform multiple tasks at the same time through the database.

Step 11230 may include simultaneous execution of database processing commands by the database processing subunits, where the database acceleration unit may include a group of database accelerator subunits sharing a common memory unit.

Step 11230 can be followed by step 11240 of retrieving the retrieved information from multiple memory processing integrated circuits by the memory controller of the database accelerating the integrated circuit, wherein the retrieved information may include one of the following At least one: (a) at least a part of the first processed information; and (b) at least a part of the second processed information.

After step 11240, the database acceleration of the integrated circuit by the database acceleration can be followed The unit performs a database processing operation on the retrieved information to provide a database acceleration result at step 11250.

Step 11250 may include allocating the resources of the database to accelerate the integrated circuit according to the time I/O bandwidth.

Step 11250 can be followed by step 11260 of outputting the database acceleration result.

Step 11260 may include dynamically linking the database processing subunits to provide an execution pipeline required to perform database processing operations that may include multiple instructions.

Step 11260 may include outputting the database acceleration result to the local storage and retrieving the database acceleration result from the local storage.

It should be noted that steps 11210, 11220, 11230, 11240, 11250, and 11260 or any other steps of method 11100 can be executed in a pipelined manner. These steps can be performed simultaneously or in an order different from the order mentioned above.

For example, step 1120 can be followed by step 11250, so that the first processed information is further processed by the database acceleration unit.

For yet another example, the first processed information can be sent to multiple memory processing integrated circuits, and then sent (not processed by multiple memory processing integrated circuits) to the database acceleration unit.

For another example, the first processed information and/or the second processed information can be output from the database acceleration integrated circuit, instead of database processing by the database acceleration unit.

The method may include performing at least one of the following operations based on the execution plan sent to the database acceleration integrated circuit by the computing node of the large computing system: capturing, first processing, sending, and third processing.

The method may include managing at least one of capture, first processing, sending, and third processing in a manner that substantially optimizes the database to accelerate the utilization of the integrated circuit.

The method may include substantially optimizing the bandwidth of the traffic exchanged through the network communication network interface.

The method may include substantially preventing the formation of a bottleneck in at least one of the acquisition, the first processing, the sending, and the third processing by substantially optimizing the database to accelerate the utilization of the integrated circuit.

The method 11200 may also include at least one of the following steps:

Step 11270 may include managing at least one of retrieval, first processing, sending, and third processing by the management unit of the database accelerated integrated circuit.

The management can be performed based on the execution plan generated by the management unit of the database accelerated integrated circuit.

The management can be performed based on the execution plan received by the management unit of the database accelerated integrated circuit, but not generated by the management unit.

The management may include allocating at least some of the following: (a) network communication network interface resources, (b) decompression unit resources, (c) memory controller resources, (d) multiple memory processing products Body circuit resources, and (e) database acceleration unit resources.

Step 11271 may include controlling at least one of retrieval, first processing, sending, and third processing by the computing node of the large computing system.

Step 11272 may include managing at least one of capture, first processing, sending, and third processing by a management unit located outside the database acceleration integrated circuit.

FIG. 95B illustrates a method 11300 for operating a database to accelerate a group of integrated circuits.

The method 11300 can start at step 11310 of performing a database acceleration operation by the database acceleration integrated circuit. Step 11310 may include performing one or more steps of method 11200.

The method 11300 may also include a step 11320 of exchanging at least one of (a) information and (b) database acceleration results between one or more groups of database acceleration integrated circuits.

The combination of steps 11310 and 11320 can be equivalent to using one or more groups of databases to accelerate integrated circuits to perform distributed processing.

One or more groups of databases can be used to accelerate the exchange of the network communication network interface of the integrated circuit.

The exchange can be performed through multiple groups, and the groups can be connected to each other by a star connection.

Step 11320 may include using at least one switch for accelerating the exchange of at least one of the following between the database of different groups in one or more groups: (a) information; and ( b) The result of database acceleration.

Step 11310 may include step 11311 of accelerating some of the integrated circuits to perform distributed processing by the database of some of one or more groups.

Step 11311 may include executing distributed processing of the first and second data structures, where the total size of the first and second data structures exceeds the storage capacity of multiple memory processing integrated circuits.

The execution of distributed processing may include the execution of multiple iterations of the following: (a) execution of newly allocating different pairs of the first data structure part and the second data structure part to different database accelerated integrated circuits; and (b) Deal with different pairs.

The execution of distributed processing may include the execution of database joining operations.

Step 11310 may include (a) allocating different first data structure parts to one or more groups of different database accelerated integrated circuits 11312; and (b) performing multiple iterations of each of the following: The second data structure part is newly allocated to one or more groups of different database acceleration integrated circuits in step 11314; and the database acceleration integrated circuit is used to process the first and second data structure parts in step 11316.

Step 11314 may be performed in a manner that overlaps at least part of the time with the current repeated processing.

Step 11314 may include exchanging the second data structure part between different database accelerated integrated circuits.

Step 11320 can be performed in a manner that overlaps with step 11310 at least partially in time.

Step 11314 may include exchanging the second data structure part between the different database acceleration integrated circuits of the group; and once the exchange is completed, exchanging the second data structure part between the different groups of the database acceleration integrated circuit .

Figure 95C illustrates method 11350 for database acceleration.

The method 11350 may include a step 11352 of retrieving a large amount of information from a large amount of storage unit through the network communication network interface of the database accelerated integrated circuit.

Step 11352 can be followed by step 11354 of performing first processing on a large amount of information to provide first processed information.

Step 11352 can be followed by step 11354 of speeding up the memory controller of the integrated circuit through the database and sending the first processed information to multiple memory resources via a large throughput interface.

Step 11354 can be followed by step 11356 of retrieving the retrieved information from multiple memory resources.

Step 11356 can be followed by step 11358 in which the database acceleration unit of the database acceleration integrated circuit performs a database processing operation on the retrieved information to provide a database acceleration result.

Step 11358 can be followed by step 11359 of outputting the database acceleration result.

The method may also include a step 11355 of performing second processing on the first processed information to provide second processed information. The second processing is executed by a plurality of processors, and the plurality of processors are located in an integrated circuit that further includes one or more of a plurality of memory resources. Step 11355 is after step 11354 and before step 11356.

The total size of the second processed information may be smaller than the total size of the first processed information.

The total size of the first processed information may be smaller than the total size of the large amount of information.

The first process may include screening database entries. Therefore, before performing any other processing and/or even before storing irrelevant database entries in multiple memory resources, filter out and check Consult irrelevant database entries to save bandwidth, storage resources, and other processing resources.

The second process may include filtering database entries. Filtering can be applied when the filtering conditions can be complex (including multiple conditions), and it may be necessary to receive multiple database entry fields before the filtering can proceed. For example, when searching for (a) people over a certain age who like bananas and (b) people over another age who like apples.

database

The following examples can refer to the database. The database may be a data center, may be part of a data center, or may not belong to a data center.

The database can be coupled to multiple users via one or more networks. The database can be a cloud database.

A database including one or more management units and multiple database accelerator boards can be provided. The accelerator boards include one or more memory/processing units.

FIG. 96B illustrates the database 12020. The database includes a management unit 12021 and a plurality of DB accelerator boards 12022, each of which includes a communication/management processor (processor 12024) and a plurality of memory/processing units 12026.

The processor 12024 can support various communication protocols, such as but not limited to PCIe, ROCE-like protocols, and the like.

The database commands can be executed by the memory/processing unit 12026, and the processor can be between the memory/processing unit 12026, between different DB accelerator boards 12022, and send traffic with the management unit 12021.

Especially when a large internal memory bank is included, the use of multiple memory/processing units 12026 can significantly speed up the execution of database commands and avoid communication bottlenecks.

FIG. 96C illustrates a DB accelerator board 12022 including a processor 12024 and multiple memory/processing units 12026. The processor 12024 includes a number of dedicated communication components, such as for communication with memory/ The processing unit 12026, the RDMA engine 12031, the DB query database engine 12034 and the like communicate with the DDR controller 12033. The DDR controller is an example of a communication controller, and the RDMA engine is an example of any communication engine.

A method for operating the system (or any part of the operating system) of any one of FIG. 96B, FIG. 96C, and FIG. 96D can be provided.

It should be noted that the database acceleration integrated circuit 11530 may be associated with multiple memory resources, which are not included in the multiple memory processing integrated circuits or otherwise not associated with the processing unit. In this situation, the processing is mainly and even only performed by the database accelerated integrated circuit.

Figure 94P illustrates a method 11700 for database acceleration.

The method 11700 may include the step 11710 of retrieving information from the storage unit through the network communication interface of the accelerated integrated circuit through the database.

Step 11710 can be followed by step 11720 of performing a first processing on the amount of information to provide first processed information.

Step 11720 can be followed by step 11730 of speeding up the memory controller of the integrated circuit through the database and sending the first processed information to multiple memory resources via the throughput interface.

Step 11730 can be followed by step 11740 of retrieving information from multiple memory resources.

Step 11740 can be followed by step 11750 in which the database acceleration unit of the database acceleration integrated circuit performs a database processing operation on the retrieved information to provide a database acceleration result.

Step 11750 can be followed by step 11760 of outputting the database acceleration result.

The first processing and/or the second processing may include screening database entries to determine which database entries should be further processed.

The second process involves screening database entries.

Hybrid system

The memory/processing unit can be efficient in performing calculations that can be memory-intensive and/or bottlenecks related to retrieval operations. When the bottleneck is related to arithmetic operations, processing-oriented (and less memory-oriented) processor units (such as but not limited to graphics processing units, central processing units) can be more effective.

The hybrid system may include both one or more processor units and one or more memory/processing units that can be fully or partially connected to each other.

The memory/processing unit (MPU) can be manufactured by the first manufacturing process that is better suited for memory cells than logic cells. For example, the memory cell manufactured by the first manufacturing process can exhibit a smaller and even much smaller critical size than the logic circuit manufactured by the first manufacturing process (for example, more than 2 times, 3 times, 4 times smaller). , 5 times, 6 times, 7 times, 8 times, 9 times, 10 times and the like) critical size. For example, the first manufacturing process may be an analog manufacturing process, and the first manufacturing process may be a DRAM manufacturing process, and the like.

The processor can be manufactured by a second manufacturing process that is better suited for logic. For example, the critical dimension of the logic circuit manufactured by the second manufacturing process may be smaller or even much smaller than the critical dimension of the logic circuit manufactured by the first manufacturing process. For yet another example, the critical size of the logic circuit manufactured by the second manufacturing process may be smaller or even much smaller than the critical size of the memory cell manufactured by the first manufacturing process. For example, the second manufacturing process may be an analog manufacturing process, and the second manufacturing process may be a CMOS manufacturing process, or the like.

Tasks can be allocated between different units in a static or dynamic manner by considering the benefits of each unit and any penalties associated with transferring data between the units.

For example, memory-intensive processing programs can be allocated to memory/processing units, and processing-intensive memory light processing can be allocated to processing units.

The processor can request or issue instructions to one or more memory/processing units to perform various processing tasks. The execution of various processing tasks can reduce the burden on the processor, reduce latency, and in some situations In this case, reduce the total information bandwidth between one or more memory/processing units and the processor, and the like.

The processor can provide instructions and/or requests with different granularities. For example, the processor can send instructions for certain processing resources or can send higher-level instructions for memory/processing units without specifying any processing resources.

FIG. 96D is an example of a hybrid system 12040 including one or more memory/processing units (MPU) 12043 and a processor 12042. The processor 12042 may send a request or instruction to one or more MPUs 12043, which in turn completes (or selectively completes) the request and/or instruction and sends the result to the processor 12042, as explained above .

The processor 12042 may further process the results to provide one or more outputs.

Each MPU includes memory resources, processing resources (such as compact microcontroller 12044), and cache memory 12049. The microcontroller may have limited computing capabilities (for example, it may mainly include a multiplication and accumulation unit).

The microcontroller 12044 can apply processing programs for the purpose of in-memory acceleration, and can also be a CPU or the entire DB processing engine or a subset thereof.

The MPU 12043 may include a microprocessor and a packet processing unit that can be connected in a mesh/ring/or other topology for fast inter-group communication.

There may be more than one DDR controller for fast inter-DIMM communication.

The goal of the in-memory packet processor is to reduce BW, data movement, power consumption, and increase performance. Compared to the standard solution, the use of an in-memory packet processor will significantly increase the performance/TCO.

It should be noted that the management unit is optional.

Each MPU can be operated as an artificial intelligence (AI) memory/processing unit. This is because it can perform AI calculations and only return the results to the processor, thereby reducing the amount of traffic, especially when the MPU is receiving and storing for use In multiple calculations of neural network coefficients, and one part of the neural network is used each time It is not necessary to receive coefficients from an external chip when processing new data.

The MPU can determine when the coefficient is zero, and inform the processor that it is not necessary to perform multiplication including zero-valued coefficients.

It should be noted that the first processing and the second processing may include filtering database entries.

The MPU may be any memory processing unit described in any one of the PCT patent application WO2019025862 and the PCT patent application No. PCT/IB2019/001005 in this specification.

It can provide an AI computing system (and a system that can be executed by the system), in which the network interface card has AI processing capabilities and is configured to perform some AI processing tasks, so as to reduce the need to be sent through the network coupled to multiple AI acceleration servers The amount of communications.

For example, in some inference systems, the input is a network (for example, multiple streams of IP cameras connected to an AI server). Under these conditions, using RDMA+AI on the processing and network connection unit can reduce the load of the CPU and PCIe bus and provide processing for the processing and network connection unit, instead of being excluded from the processing and network connection The GPU in the unit provides processing.

For example, instead of calculating the initial result and sending the initial result to the target AI acceleration server (applying one or more AI processing operations), the processing and network connection unit can reduce the amount of value sent to the target AI acceleration server Pretreatment. The target AI calculation server is an AI calculation server that is allocated to perform calculations on values provided by other AI acceleration servers. This reduces the bandwidth of the traffic exchanged between the AI acceleration servers and also reduces the load of the target AI acceleration server.

The target AI acceleration server can be allocated dynamically or statically by using load balancing or other allocation algorithms. There may be more than a single target AI acceleration server.

For example, if the target AI acceleration server adds multiple losses, the processing and network connection unit can add the losses generated by its AI acceleration server and send the total loss to the target AI acceleration server, thereby reducing the frequency. width. When performing other such as derivative calculation and aggregation and the like The same benefits can be obtained during pretreatment operations.

FIG. 97B illustrates a system 12060 including sub-systems, each sub-system including a switch 12061 for connecting the AI processing and network connection unit 12063 with the server motherboard 12064 to each other. The server motherboard includes one or more AI processing and network connection units 12063 with network capabilities and AI processing capabilities. The AI processing and network connection unit 12063 may include one or more NICs and ALUs or other calculation circuits for performing preprocessing.

The AI processing and network connection unit 12063 may be a chip, or may include more than a single chip. It may be beneficial to have the AI processing and network connection unit 12063 as a single chip.

The AI processing and network connection unit 12063 may include (only or main) processing resources. The AI processing and network connection unit 12063 may include in-memory arithmetic circuits, or may not include in-memory arithmetic circuits, or may not include a large number of in-memory arithmetic circuits.

The AI processing and network connection unit 12063 may be an integrated circuit, may include more than a single integrated circuit, may be a part of an integrated circuit, and the like.

The AI processing and network connection unit 12063 can transmit traffic between the AI acceleration server including the AI processing and network connection unit 12063 and other AI acceleration servers (see, for example, Figure 97C) (for example, by using channels such as DDR , Network channel and/or PCIe channel communication port). The AI processing and network connection unit 12063 can also be coupled to an external memory such as DDR memory. The processing and network connection unit may include memory and/or may include a memory/processing unit.

In FIG. 97C, the AI processing and network connection unit 12063 is illustrated as including local DDR connection, DDR channel, AI accelerator, RAM memory, encryption/decryption engine, PCIe switch, PCIe interface, multiple core processing arrays, Fast internet connection and the like.

A method for operating the system (or any part of the operating system) of any one of FIG. 97B and FIG. 97C can be provided.

Any combination of any steps of any method mentioned in this application can be provided.

Any combination of any units, integrated circuits, memory resources, logic, processing subunits, controllers, and components mentioned in this application can be provided.

Any reference to "include" and/or "include" may be applied to "composition" and "substantial composition" after making necessary modifications in details.

The foregoing description has been presented for illustrative purposes. The previous description is not exhaustive and is not limited to the precise form or embodiment disclosed. From the consideration of this specification and the practice of the disclosed embodiments, modifications and adaptations will be obvious to those familiar with the technology. In addition, although the aspects of the disclosed embodiments are described as being stored in memory, those skilled in the art will understand that these aspects can also be stored on other types of computer-readable media, such as secondary storage devices. Such as hard disk or CD ROM, or other forms of RAM or ROM, USB media, DVD, Blu-ray, 4K Ultra HD Blu-ray, or other optical drive media.

Computer programs based on written descriptions and disclosed methods are within the skills of experienced developers. Various programs or program modules can be generated using any of the technologies known to those skilled in the art or can be designed in combination with existing software. For example, program sections or program modules can be used or combined with .Net Framework, .Net Compact Framework (and related languages, such as Visual Basic, C, etc.), Java, C++, Objective-C, HTML, HTML/AJAX , XML or HTML including Java applets.

In addition, although illustrative embodiments have been described herein, those skilled in the art based on the present invention will understand that there are equivalent elements, modifications, omissions, combinations (for example, combinations across various embodiments), adaptations and/ Or modify the scope of any and all embodiments. The restrictions in the scope of the patent application should be widely interpreted based on the language used in the scope of the patent application, and are not limited to the examples described in this specification or during the examination period of this application. Examples should be interpreted as non-exclusive. In addition, the steps of the disclosed method can be modified in any manner including by reordering the steps and/or inserting or deleting steps. Therefore, this specification and examples are intended to be regarded as illustrative only, wherein the true scope and spirit are suggested by the following application scope and the full scope of its equivalents.

300: hardware chip

310a: Processing group

310b: Processing group

310c: Processing group

310d: Processing group

320a: logic and control subunit

320b: logic and control subunit

320c: logic and control subunit

320d: logic and control subunit

320e: logic and control subunit

320f: logic and control subunit

320g: logic and control subunit

320h: logic and control subunit

330a: Dedicated memory instance

330b: Dedicated memory example

330c: Dedicated memory instance

330d: Dedicated memory instance

330e: Dedicated memory instance

330f: Dedicated memory example

330g: Dedicated memory example

330h: Dedicated memory instance

340a: control

340b: control

340c: control

340d: control part

350: host

Claims (368)

An integrated circuit comprising: A substrate; A memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; A processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory groups Discrete memory group association; and A controller, which is configured to: At least one safety measure is implemented with respect to an operation of the integrated circuit. The integrated circuit of claim 1, wherein the controller is configured to take one or more remedial actions if the at least one safety measure is triggered. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure in at least one memory location. The integrated circuit according to claim 2, wherein the data includes weight data of a neural network model. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, and the at least one security measure includes locking one or more of the memory arrays that are not used for data input or data output operations Access to a portion of memory. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, the at least one security measure including locking only a subset of the memory array. The integrated circuit of claim 6, wherein the subset of the array is specified by certain memory addresses. The integrated circuit of claim 6, wherein the subset of the memory array is configurable. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, and the at least one security measure includes controlling traffic to or from the integrated circuit. The integrated circuit according to claim 1, wherein the controller is configured to implement at least one security measure, and the at least one security measure includes uploading a changeable data, a program code, or a fixed data. The integrated circuit as described in claim 1, wherein the uploading of changeable data, code or fixed data occurs during a boot process. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, and the at least one security measure includes uploading a configuration file during a boot process, the configuration file identifying the waiting Some memory addresses of at least part of the memory array are locked after the boot process is completed. The integrated circuit of claim 1, wherein the controller is further configured to require a complex password in order to store a memory portion of the memory array associated with one or more memory addresses Take to unlock. The integrated circuit according to claim 1, wherein the at least one security measure is triggered after an attempt to access one of the at least one locked memory address is detected. The integrated circuit of claim 1, wherein the controller is configured to implement at least one security measure, and the at least one security measure includes: Calculate a checksum, a hash, cyclic redundancy check (CRC) or parity calculated relative to at least part of the memory array; and Compare the calculated sum check code, hash, CRC or parity with a predetermined value. The integrated circuit of claim 15, wherein the controller is configured as part of the at least one security measure to determine whether the calculated checksum, hash, CRC, or parity matches the predetermined value. The integrated circuit according to claim 1, wherein the at least one security measure includes copying a program code in at least two different memory portions. The integrated circuit according to claim 17, wherein the at least one safety measure includes determining whether the output results of executing the code in the at least two different memory portions are different. The integrated circuit according to claim 18, wherein the output results include intermediate or final output results. The integrated circuit according to claim 17, wherein the at least two different memory portions are included in the integrated circuit. The integrated circuit according to claim 1, wherein the at least one safety measure includes determining whether an operation pattern is different from one or more predetermined operation patterns. The integrated circuit according to claim 2, wherein the one or more remedial actions include stopping execution of operations. A method of protecting an integrated circuit against tampering, the method comprising: Using a controller associated with the integrated circuit to implement at least one safety measure relative to an operation of the integrated circuit; wherein the integrated circuit includes: A substrate; A memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; and A processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory groups Discrete memory groups are associated. The method according to claim 23, further comprising taking one or more remedial actions when the at least one safety measure is triggered. An integrated circuit comprising: A substrate; A memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; A processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory groups Discrete memory group association; and A controller, which is configured to: At least one security measure is implemented relative to an operation of the integrated circuit; wherein the at least one security measure includes copying a program code in at least two different memory portions. An integrated circuit comprising: A substrate; A memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; A processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory groups Discrete memory group association; and A controller configured to implement at least one safety measure with respect to an operation of the integrated circuit. The integrated circuit of claim 26, wherein the controller is further configured to take one or more remedial actions if the at least one safety measure is triggered. A distributed processor memory chip, which includes: A substrate; A memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; A processing array disposed on the substrate, the processing array including a plurality of processor subunits, each of the plurality of processor subunits and one or more of the plurality of discrete memory groups Discrete memory group association; and A first communication port configured to establish a communication connection between the distributed processor memory chip and an external entity other than another distributed processor memory chip; and A second communication port configured to establish a communication connection between the distributed processor memory chip and a first additional distributed processor memory chip. The distributed processor memory chip of claim 28, which further includes a third communication port configured to connect the distributed processor memory chip and a second additional distributed processing A communication connection is established between the memory chips of the devices. The distributed processor memory chip of claim 29, further comprising a controller configured to pass through at least one of the first communication port, the second communication port, and the third communication port One controls the communication. The distributed processor memory chip according to claim 29, wherein each of the first communication port, the second communication port, and the third communication port is associated with a corresponding bus. The distributed processing memory chip according to claim 31, wherein the corresponding bus is a bus common to each of the first communication port, the second communication port, and the third communication port. The distributed processing memory chip described in claim 31, wherein The corresponding buses associated with each of the first communication port, the second communication port, and the third communication port are connected to the plurality of discrete memory groups. The distributed processor memory chip according to claim 31, wherein at least one bus associated with the first communication port, the second communication port and the third communication port is unidirectional. The distributed processor memory chip according to claim 31, wherein at least one bus associated with the first communication port, the second communication port and the third communication port is bidirectional. The distributed processor memory chip of claim 30, wherein the controller is configured to schedule a data between the distributed processor memory chip and the first additional distributed processor memory chip Transmitting so that one of the receiving processor subunits of the first additional distributed processor memory chip is based on the data transmission and executes its associated program code during a period of time when the data transmission is received. The distributed processor memory chip of claim 30, wherein the controller is configured to send a clock energizing signal to one of the plurality of processor subunits of the distributed processor memory chip At least one is used to control one or more operation modes of the at least one of the plurality of processor subunits. The distributed processor memory chip of claim 37, wherein the controller is configured to control the clock enable signal sent to the at least one of the plurality of processor subunits Control a timing of one or more communication commands associated with the at least one of the plurality of processor subunits. The distributed processor memory chip of claim 30, wherein the controller is configured to selectively start by one of the plurality of processor subunits on the distributed processor memory chip Or more than one code can be executed. The distributed processor memory chip of claim 30, wherein the controller is configured to use a clock energizing signal to control one or more of the plurality of processor subunits to the second The data transmission sequence of at least one of the communication port and the third communication port. The distributed processor memory chip of claim 28, wherein a communication speed associated with the first communication port is lower than a communication speed associated with the second communication port. The distributed processor memory chip of claim 30, wherein the controller is configured to determine whether one of the plurality of processor subunits is ready to transmit data to the first processor subunit included in One of the second processor sub-units in the first additional distributed processor memory chip, and after determining that the first processor sub-unit is ready to transmit the data to the second processor sub-unit, a clock is used The enable signal initiates the transfer of the data from the first processor sub-unit to one of the second processor sub-units. The distributed processor memory chip according to claim 42, wherein the controller is further configured to determine whether the second processor sub-unit is ready to receive the data, and when it is determined that the second processor sub-unit is ready After receiving the data, the clock energizing signal is used to initiate the transmission of the data from the first processor sub-unit to the second processor sub-unit. The distributed processor memory chip of claim 42, wherein the controller is further configured to determine whether the second processor subunit is ready to receive the data and buffer the data included in the transmission until After a determination that the second processor subunit of the first additional distributed processor memory chip is ready to receive the data. A method for transferring data between a first distributed processor memory chip and a second distributed processor memory chip, the method comprising: Use a controller associated with at least one of the first distributed processor memory chip and the second distributed processor memory chip to determine that it is placed on the first distributed processor memory chip Whether a first processor subunit among the plurality of processor subunits on the chip is ready to transmit data to a second processor subunit included in the second distributed processor memory chip; and determining the The first processor sub-unit is ready to transmit the data to the second processor sub-unit and then uses a clock energizing signal controlled by the controller to initiate the data from the first processor sub-unit to the second One of the processor subunits transmits. The method according to claim 45, which further includes: Use the controller to determine whether the second processor subunit is ready to receive the data; and After determining that the second processor sub-unit is ready to receive the data, the clock enabling signal is used to initiate the transfer of the data from the first processor sub-unit to the second processor sub-unit. The method according to claim 45, which further includes: Use the controller to determine whether the second processor subunit is ready to receive the data, and buffer the data included in the transmission until the second processor subunit of the first additional distributed processor memory chip is ready After receiving a judgment of the data. A memory chip including: A substrate; A memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; and A first communication port configured to establish a communication connection between the memory chip and an external entity other than another memory chip; and A second communication port is configured to establish a communication connection between the memory chip and a first additional memory chip. The memory chip according to claim 48, wherein the first communication port is connected to at least one of a main bus inside the memory chip or at least one processor subunit included in the memory chip. The memory chip according to claim 48, wherein the second communication port is connected to at least one of a main bus inside the memory chip or at least one processor subunit included in the memory chip. A memory unit, which includes: A memory array, which includes a plurality of memory banks; At least one controller configured to control at least one aspect of the read operation with respect to the plurality of memory banks; At least one zero-value detection logic unit configured to detect a multi-bit zero value associated with data stored in a specific address of one of the plurality of memory groups; and The at least one controller is configured to return a zero value indicator to one or more circuits in response to a zero value detection performed by the at least one zero value detection logic unit. The memory unit according to claim 51, wherein the one or more circuits that return the zero value indicator are outside the memory unit. The memory unit according to claim 51, wherein the one or more circuits that return the zero value indicator are inside the memory unit. The memory unit according to claim 51, wherein the memory unit further comprises at least one read disabling element, and the at least one read disabling element is configured to detect in the at least one zero value detection logic unit When a zero value associated with the specific address is reached, a read command associated with the specific address is interrupted. The memory unit of claim 51, wherein the at least one controller is configured to send the zero-value indicator to the one or more circuits instead of sending the zero-value data stored in the specific address. The memory unit according to claim 51, wherein a size of the zero value indicator is smaller than a size of zero data. The memory unit according to claim 51, wherein an energy consumed by the first processing procedure including one of the following operations is less than an energy consumed by sending zero-value data to the one or more circuits: (a) Detecting the zero value; (b) generating the zero value indicator; and (c) sending the zero value indicator to the one or more circuits. The memory unit according to claim 57, wherein the energy consumed by the first processing procedure is less than half of the energy consumed by sending the zero-value data to the one or more circuits. The memory unit according to claim 51, wherein the memory unit further includes at least one sense amplifier configured to perform a zero value detection after the at least one zero value detection unit performs a zero value detection Prevent at least one of the plurality of memory groups from being activated. The memory unit according to claim 59, wherein the at least one sense amplifier includes a plurality of transistors, and the plurality of transistors are configured to sense low-power signals from the plurality of memory banks, and The at least one sense amplifier amplifies a small voltage swing to a higher voltage level so that the data stored in the plurality of memory groups can be interpreted by the at least one controller. The memory unit according to claim 51, wherein each of the plurality of memory groups is further organized into sub-groups, the at least one controller includes a sub-group controller, and wherein the at least one zero-value detection The detection logic unit includes a zero value detection logic associated with the subgroups. The memory unit according to claim 61, wherein the memory unit further comprises at least one read disabling element, and the at least one read disabling element includes a sensor associated with each of the subgroups Test amplifier. The memory unit according to claim 51, further comprising a plurality of processor sub-units spatially distributed in the memory unit, wherein each of the plurality of processor sub-units and the plurality of processor sub-units At least one dedicated to one of the memory groups is associated, and each of the plurality of processor subunits is configured to store data stored in the corresponding memory group Take and operate. The memory unit according to claim 63, wherein the one or more circuits include one or more of the processor subunits. The memory unit according to claim 63, wherein each of the plurality of processor sub-units is connected to two or more of the plurality of processor sub-units by one or more buses Two other processor subunits. The memory unit according to claim 51, which further includes a plurality of buses. The memory unit of claim 66, wherein the plurality of buses are configured to transfer data between the plurality of memory groups. The memory unit of claim 67, wherein at least one of the plurality of buses is configured to transmit the zero value indicator to the one or more circuits. A method for detecting a zero value in a specific address of a plurality of memory groups, which includes: Receiving a request to read data stored in an address of a plurality of memory groups from a circuit outside a memory unit; In response to the received request, a zero value detection logic unit is activated to detect a zero value in the received address by a controller; and In response to the zero value detection performed by the zero value detection logic unit, a zero value indicator is transmitted to the circuit by the controller. The method of claim 69, further comprising configuring a read disable element by the controller to interrupt when the zero value detection logic unit detects a zero value associated with the requested address A read command associated with the requested address. The method according to claim 69, further comprising configuring a sense amplifier by the controller to prevent at least one of the plurality of memory groups when the zero value detection unit detects a zero value The first start of the person. A non-transitory computer-readable medium storing an instruction set that can be executed by a controller of a memory unit so that the memory unit detects a zero in a specific address of a plurality of memory groups Value, methods include: Receiving a request to read data stored in an address of a plurality of memory groups from a circuit outside a memory unit; In response to the received request, a zero value detection logic unit is activated to detect a zero value in the received address by a controller; and In response to the zero value detection performed by the zero value detection logic unit, a zero value indicator is transmitted to the circuit by the controller. The non-transitory computer-readable medium according to claim 72, wherein the method further comprises configuring a read disable element by the controller to detect and the requested address in the zero value detection logic unit When a zero value is associated, a read command associated with the requested address is interrupted. The non-transitory computer-readable medium of claim 72, wherein the method further comprises configuring a sense amplifier by the controller to prevent the plurality of values when the zero value detection unit detects a zero value An activation of at least one of the memory groups. An integrated circuit comprising: A memory unit including a plurality of memory groups, at least one controller configured to control at least one aspect of read operations relative to the plurality of memory groups, and configured to detect and store At least one zero-value detection logic unit with a multi-bit zero value associated with data in a specific address of the plurality of memory groups; A processing unit configured to send a read request to the memory unit for reading data from the memory unit; and The at least one controller and the at least one zero value detection logic are configured to return a zero value indicator to one or more in response to a zero value detection performed by the at least one zero value detection logic A circuit. A memory unit, which includes: A memory array, which includes a plurality of memory banks; At least one controller configured to control at least one aspect of the read operation with respect to the plurality of memory banks; At least one detection logic unit configured to detect a predetermined multi-bit value associated with data stored in a specific address of one of the plurality of memory groups; and The at least one controller is configured to return a value indicator to one or more circuits in response to a detection of the predetermined multi-bit value by the at least one detection logic. The memory unit according to claim 76, wherein the predetermined multi-bit value can be selected by a user. A memory unit, which includes: A memory array, which includes a plurality of memory banks; At least one controller configured to control at least one aspect of the write operation with respect to the plurality of memory banks; At least one detection logic unit configured to detect a predetermined multi-bit value associated with data to be written to a specific address of the plurality of memory groups; and The at least one controller is configured to provide a value indicator to one or more circuits in response to a detection of the predetermined multi-bit value by the at least one detection logic. A distributed processor memory chip, which includes: A substrate; A memory array including a plurality of memory banks arranged on the substrate; A plurality of processor sub-units, which are arranged on the substrate; At least one controller configured to control at least one aspect of the read operation with respect to the plurality of memory banks; At least one detection logic unit configured to detect a predetermined multi-bit value associated with data stored in a specific address of one of the plurality of memory groups; and The at least one controller is configured to return a value indicator to one of the plurality of processor subunits in response to the at least one detection logic performing a detection on the predetermined multi-bit value Or more. A memory unit, which includes: One or more memory banks; A set of controllers; and One-bit address generator; The address generator is configured to: Providing a current address of a current row to be accessed in the memory group associated with one of the one or more memory groups to the group of controllers; Determine a predicted address of the next row to be accessed in the associated memory group; and The predicted address is provided to the group of controllers before an operation relative to the current column associated with the current address is completed. The memory unit according to claim 80, wherein the operation relative to the current row associated with the current address is a read operation or a write operation. The memory unit according to claim 80, wherein the current row and the next row In the same memory group. The memory unit of claim 82, wherein the same memory group allows access to the next row while the current row is being accessed. The memory unit according to claim 80, wherein the current row and the next row are in a different memory group. The memory unit described in claim 80, a distributed processor, wherein the distributed processor includes a plurality of processors in a processing array among a plurality of discrete memory groups spatially distributed in the memory array unit. The memory unit of claim 80, wherein the set of controllers is configured to access the current row and activate the next row before a completion of the operation relative to the current row. The memory unit according to claim 80, wherein each of the one or more memory groups includes at least a first sub-group and a second sub-group, and is associated with the one or more memory groups The group of controllers associated with each of them includes a first subgroup of controllers associated with the first subgroup and a second group of controllers associated with the second subgroup. The memory unit of claim 87, wherein the first sub-group controller is configured to enable access to data included in a current row of the first sub-group, and the second sub-group controller Start the next column in this second subgroup. The memory unit according to claim 88, wherein the activated next row of the second sub-group is separated from the current row of the data being accessed in the first sub-group by at least two rows. The memory unit of claim 87, wherein the second sub-group controller is configured to access data included in a current row of the second sub-group, and the first sub-group controller activates The next column in this first subgroup. The memory unit according to claim 90, wherein the activated next row of the first subgroup is separated from the current row of the data being accessed in the second subgroup by at least two rows. The memory unit according to claim 80, wherein a trained neural network is used to determine the predicted address. The memory unit according to claim 80, wherein the predicted address is determined based on a determined line access pattern. The memory unit according to claim 80, wherein the address generator includes a first address generator configured to generate the current address and a second address configured to generate the predicted address Address generator. The memory unit of claim 94, wherein the second address generator is configured to calculate the predicted address within a predetermined period of time after the current address generator has generated the current address. The memory unit according to claim 95, wherein the predetermined time period is adjustable. The memory unit according to claim 96, wherein the predetermined time period is adjusted based on a value of at least one operating parameter associated with the memory unit. The memory unit according to claim 97, wherein the at least one operating parameter includes a temperature of the memory unit. The memory unit according to claim 80, wherein the address generator is further configured to generate a trust level associated with the predicted address, and the trust level drops below a predetermined threshold In this case, make the group of controllers give up access to the next row at the predicted address. The memory unit of claim 80, wherein the predicted address is generated by a series of flip-flops that sample the address generated in a delay. The memory unit of claim 100, wherein the delay can be configured via a multiplexer that selects between flip-flops storing sampled addresses. The memory unit according to claim 80, wherein the group of controllers is configured to The predicted address received from the address generator is ignored during a predetermined period of time after one of the memory cells is reset. The memory unit of claim 80, wherein the address generator is configured to abandon providing the predicted address after detecting a random pattern in the row access relative to the associated memory bank To the group of controllers. A memory unit, which includes: One or more memory groups, wherein each of the one or more memory groups includes: Multiple columns A first row controller configured to control a first subset of the plurality of rows; A second row controller configured to control a second subset of the plurality of rows; A single data input terminal for receiving data to be stored in the plurality of rows; and A single data output terminal, which is used to provide data retrieved from the plurality of rows. The memory unit of claim 104, wherein the memory unit is configured to receive a first address at a predetermined time for processing and receiving a second address for function and access. The memory unit according to claim 104, wherein the first subset of the plurality of rows is composed of even-numbered rows. The memory unit according to claim 106, wherein the even-numbered columns are located in one half of the one or more memory groups. The memory unit according to claim 106, wherein the odd-numbered rows are located in one half of the one or more memory groups. The memory unit according to claim 104, wherein the second subset of the plurality of rows is composed of odd-numbered rows. The memory unit according to claim 104, wherein among the plurality of rows The first subset is contained in a first subset of a memory group that is adjacent to a second subset of the memory group that contains the second subset of the plurality of rows. The memory unit of claim 104, wherein the first row controller is configured to cause an access to data included in a row in the first subset of the plurality of rows, and the The second row controller activates one row in the second subset of the plurality of rows. The memory unit according to claim 111, wherein the activated row in the second subset of the plurality of rows is separated from the row in which data is being accessed in the first subset of the plurality of rows Open at least two columns. The memory unit of claim 104, wherein the second row controller is configured to cause access to data included in one row of the second subset of the plurality of rows, and the first row A row controller activates a row in the second subset of the plurality of rows. The memory unit according to claim 113, wherein the activated row in the first subset of the plurality of rows is separated from the row in which data is being accessed in the second subset of the plurality of rows Open at least two columns. The memory unit according to claim 104, wherein each of the one or more memory groups includes a row input terminal, and the row input terminal is used to receive a row identifier that prompts a portion of a column to be accessed. In the memory unit described in claim 104, an extra row of redundant pads is placed between each of the two rows of pads to generate a distance for allowing activation. In the memory unit described in claim 104, rows close to each other may not be activated at the same time. A distributed processor on a memory chip, which includes: A substrate; A memory array arranged on the substrate, the memory array including a plurality of discrete memory groups; A processing array arranged on the substrate, the processing array including a plurality of processor sub-units, each of the processor sub-units is a dedicated memory corresponding to one of the plurality of discrete memory groups Group association; and At least one memory pad disposed on the substrate, wherein the at least one memory pad is configured to serve as at least one of a register file for one or more of the plurality of processor subunits Scratchpad. The memory chip according to claim 118, wherein the at least one memory pad is included in at least one of the plurality of processor subunits of the processing array. The memory chip according to claim 118, wherein the register file is configured as a data register file. The memory chip according to claim 118, wherein the register file is configured as a one-bit address register file. The memory chip of claim 118, wherein the at least one memory pad is configured to provide at least one temporary storage of a temporary storage file for one or more of the plurality of processor subunits A device to store data to be accessed by one or more of the plurality of processor subunits. The memory chip of claim 118, wherein the at least one memory pad is configured to provide at least one register for a register file of one or more of the plurality of processing subunits , Wherein the at least one register of the register file is configured to store coefficients, and the coefficients are executed by the plurality of processor subunits on one of the convolution accelerator operations in the plurality of processor subunits Used during the period. The memory chip according to claim 118, wherein the at least one memory pad is a DRAM memory pad. The memory chip of claim 118, wherein the at least one memory pad is configured to communicate via one-way access. The memory chip according to claim 118, wherein the at least one memory pad allows bidirectional access. The memory chip according to claim 1, further comprising at least one redundant memory pad disposed on the substrate, wherein the at least one redundant memory pad is configured to provide for the plurality of processors At least one redundant register of one or more of the subunits. The memory chip according to claim 118, further comprising at least one memory pad disposed on the substrate, wherein the at least one memory pad includes configuration to provide for use in the plurality of processor subunits At least one redundant memory bit of at least one redundant register of one or more of them. The memory chip according to claim 118, which further includes: A first plurality of buses, each of the first plurality of buses connects one of the plurality of processor subunits to a corresponding dedicated memory bank; and A second plurality of buses, and each of the second plurality of buses connects one of the plurality of processor subunits to the other of the plurality of processor subunits. The memory chip of claim 118, wherein at least one of the processor subunits includes a counter configured to count backward from a predefined number, and after the counter reaches a zero value, the Wait for the at least one of the processor subunits to be configured to stop a current task and trigger a new operation of the memory. The memory chip according to claim 118, wherein at least one of the processor subunits includes a mechanism for stopping a current task and triggering a memory renew operation at some time to renew the memory pad . The memory chip of claim 118, wherein the register file is configured to be used as a cache memory. A method for executing at least one instruction in a distributed processor memory chip, the method comprising: Retrieve one or more data values from a memory array of the distributed processor memory chip; Storing the one or more data values in a register formed in a memory pad of the distributed processor memory chip; and Access the one or more data values stored in the register according to at least one instruction executed by a processor element; The memory array includes a plurality of discrete memory groups arranged on a substrate; The processor element is one of a plurality of processor sub-units included in a processing array arranged on the substrate, wherein each of the processor sub-units and the plurality of processor sub-units One of the discrete memory groups is associated with a dedicated memory group; and The register is provided by a memory pad arranged on the substrate. The method of claim 133, wherein the processor element is configured to act as an accelerator, and the method further includes: Access the first data stored in the register; Access the second data from the memory array; Perform an operation on the first data and the second data. The method according to claim 133, wherein at least one memory pad includes a plurality of word lines and bit lines, and the method further includes: A timing of loading the word lines and bit lines is determined, and the timing is determined by a size of the memory pad. The method according to claim 133, which further comprises: Periodically renew the register. The method of claim 12, wherein the memory pad includes a DRAM memory pad. The method of claim 133, wherein the memory pad is included in at least one of the plurality of discrete memory groups in the memory array. A device comprising: A substrate; A processing unit arranged on the substrate; and A memory unit arranged on the substrate, wherein the memory unit is configured to store data to be accessed by the processing unit, and The processing unit includes a memory pad configured to act as a cache for one of the processing units. A method for distributed processing of at least one information stream, the method comprising: The at least one information stream is received by one or more memory processing integrated circuits via the first communication channel; wherein each memory processing integrated unit includes a controller, a plurality of processor sub-units and a plurality of memories unit; Buffering the at least one information stream by the one or more memory processing integrated circuits; Performing a first processing operation on the at least one information stream by the one or more memory processing integrated circuits to provide a first processing result; Sending the first processing results to a processing integrated circuit; and Performing a second processing operation on the first processing results by the one or more memory processing integrated circuits to provide a second processing result; The size of the logic cell of the one or more memory processing integrated circuits is smaller than the size of the logic cell of the processing integrated circuit. The method of claim 140, wherein each of the plurality of memory units is coupled to at least one of the plurality of processor subunits. The method according to claim 140, wherein a total size of the information unit of the at least one information stream received during a certain duration exceeds a total size of the first processing result output during the certain duration. The method according to claim 140, wherein a total size of the at least one information stream is smaller than a total size of the first processing results. The method according to claim 140, wherein the manufacturing process of the memory type is a DRAM manufacturing process. The method according to claim 140, wherein the processing integrated circuit is manufactured by a memory type manufacturing process; and The processing integrated circuit is manufactured by a logic type manufacturing process. The method according to claim 140, wherein a size of a logic cell of the one or more memory processing integrated circuits is at least twice a size of a corresponding logic cell of the processing integrated circuit. The method according to claim 140, wherein a critical size of a logic cell of the one or more memory processing integrated circuits is at least twice a critical size of a corresponding logic cell of the processing integrated circuit . The method of claim 140, wherein a critical size of a memory cell of the one or more memory processing integrated circuits is at least two of a critical size of a logic cell corresponding to one of the processing integrated circuits Times. The method according to claim 140, which includes requesting the one or more memory processing integrated circuits through the processing integrated circuit to perform the first processing operations. The method according to claim 140, which comprises: sending instructions to the one or more memory processing integrated circuits through the processing integrated circuit to perform the first processing operations. The method according to claim 140, which includes processing the integrated circuit configuration by The one or more memory processing integrated circuits perform the first processing operations. The method according to claim 140, which includes executing the first processing operations by the one or more memory processing integrated circuits without the processing integrated circuit intervening. The method according to claim 140, wherein the computational complexity of the first processing operations is lower than that of the second processing operations. The method according to claim 140, wherein the total throughput of one of the first processing operations exceeds the total throughput of one of the second processing operations. The method of claim 140, wherein the at least one information stream includes one or more preprocessed information streams. The method according to claim 157, wherein the one or more preprocessed information streams are data extracted from a network transmission unit. The method according to claim 140, wherein a part of the first processing operations is performed by one of the plurality of processor sub-units, and the other part of the first processing operations is performed by the plurality of processor sub-units The other one is executed. The method according to claim 140, wherein the first processing operations and the second processing operations include cellular network processing operations. The method according to claim 140, wherein the first processing operations and the second processing operations include database processing operations. The method according to claim 140, wherein the first processing operations and the second processing operations include database analysis processing operations. The method according to claim 140, wherein the first processing operations and the second processing operations include artificial intelligence processing operations. A method for distributed processing, the method includes: The information unit is received by one or more memory processing integrated circuits in a decomposition system, the decomposition type The system includes one or more computing subsystems separated from one or more storage subsystems; wherein each of the one or more memory processing integrated circuits includes a controller, multiple processor sub-units, and multiple Memory unit; The one or more computing subsystems include multiple processing integrated circuits; A size of a logic cell of the one or more memory processing integrated circuits is at least twice a size of a logic cell corresponding to one of the multiple processing integrated circuits; Perform processing operations on the information units by the one or more memory processing integrated circuits to provide processing results; and The processing results are output from the one or more memory processing integrated circuits. The method according to claim 162, which includes outputting the processing results to the one or more arithmetic subsystems of the decomposition system. The method of claim 162, which includes receiving the information units from the one or more storage subsystems of the decomposition system. The method according to claim 162, which includes outputting the processing results to the one or more storage subsystems of the decomposition system. The method of claim 162, which includes receiving the information units from the one or more computing subsystems of the decomposition system. The method of claim 166, wherein the information units sent from different groups of the processing units of the plurality of processing integrated circuits include different parts of an intermediate result of a processing procedure executed by the plurality of processing integrated circuits , Wherein a group of processing units includes at least one processing integrated circuit. The method according to claim 167, which includes outputting a result of the entire processing procedure by the one or more memory processing integrated circuits. The method of claim 168, which includes sending the result of the entire processing procedure to each of the plurality of processing integrated circuits. The method of claim 168, wherein the different parts of the intermediate result are different parts of an updated neural network model, and wherein the result of the entire processing procedure is the updated neural network model. The method of claim 168, which includes sending the updated neural network model to each of the plurality of processing integrated circuits. The method according to claim 162, which includes using the exchange subunit of the decomposition system to output the processing results. The method of claim 162, wherein the one or more memory processing integrated circuits are included in a memory processing subsystem of the decomposition system. The method of claim 162, wherein at least one of the one or more memory processing integrated circuits is included in one or more arithmetic subsystems of the decomposition system. The method of claim 162, wherein at least one of the one or more memory processing integrated circuits is included in one or more memory subsystems of the decomposable system. The method of claim 162, wherein at least one of the following is true: (a) receiving the information units from at least one of the plurality of processing integrated circuits; and (b) these The processing result is sent to one of the plurality of processing integrated circuits or a plurality of memory processing integrated circuits. The method according to claim 176, wherein a critical size of a logic cell of the one or more memory processing integrated circuits exceeds at least a critical size of a logic cell corresponding to one of the multiple processing integrated circuits double. The method of claim 176, wherein a critical size of a memory cell of the one or more memory processing integrated circuits exceeds a critical size of a logic cell corresponding to one of the multiple processing integrated circuits At least twice. The method according to claim 162, wherein the information units include preprocessed information units. The method according to claim 179, which includes preprocessing the information units by the plurality of processing integrated circuits to provide the preprocessed information units. The method according to claim 162, wherein the information units are delivered as part of a model of a neural network. The method according to claim 162, wherein the information units deliver partial results of at least one database query. The method according to claim 162, wherein the information units deliver partial results of at least one aggregate database query. A method for accelerating database analysis, the method includes: Receiving a database query through a memory processing integrated circuit, the database query including at least one relevance criterion that prompts a database entry in a database related to the database query; The memory processing integrated circuit includes a controller, a plurality of processor sub-units and a plurality of memory units; Determining a group of related database entries stored in the memory processing integrated circuit by the memory processing integrated circuit and based on the at least one correlation criterion; and The group of related database entries is sent to one or more processing entities for further processing, and irrelevant database entries stored in the memory processing integrated circuit are not sent to the one or more processing entities. Processing entity Among them, these irrelevant database entries are different from these related database entries. The method of claim 184, wherein the one or more processing entities are included in the plurality of processor subunits of the memory processing integrated circuit. The method according to request item 185, which includes further processing the group of related database entries by the memory processing integrated circuit to complete a response to the database query. The method of claim 186, which includes outputting the response to the database query from the memory processing integrated circuit. The method according to claim 187, wherein the output includes applying a first-class control processing program. The method of claim 188, wherein the application of the flow control processing program responds to indicators output from the one or more processing entities, and the indicators relate to one or more databases of the group One of the processing of the entry is completed. The method of claim 185, which includes further processing the group of related database entries by the memory processing integrated circuit to provide an intermediate response to the database query. The method of claim 190, which includes outputting the intermediate response to the database query from the memory processing integrated circuit. The method according to claim 191, wherein the output includes applying a first-class control processing program. The method of claim 192, wherein the application of the flow control processing program is in response to indicators output from the one or more processing entities, and the indicators are processing part of the database entries of the group One is complete. The method according to claim 185, which includes generating processing status indicators by the one or more processing entities, the processing status indicators prompting a progress of the further processing of the group of related database entries. The method of claim 185, which includes using the memory processing integrated circuit to further process the group of related database entries. The method according to claim 195, wherein the processing is performed by the plurality of processor subunits. The method of claim 195, wherein the processing includes calculating an intermediate result by one of the plurality of processing subunits, sending the intermediate result to another of the plurality of processing subunits, and borrowing An additional calculation is performed by the other processing sub-unit. The method according to claim 195, wherein the processing is executed by the controller. The method according to claim 195, wherein the processing is performed by the plurality of processor subunits and the controller. The method of claim 184, wherein the one or more processing entities are located outside the memory processing integrated circuit. The method according to claim 200, which includes outputting the group of related database entities from the memory processing integrated circuit. The method according to claim 201, wherein the output includes applying a first-class control processing program. The method of claim 202, wherein the application of the flow control processing program responds to indicators output from the one or more processing entities, and the indicators are related to being associated with the one or more processing entities A relevance of the database entries. The method according to claim 184, wherein the plurality of processor sub-units includes a complete arithmetic logic unit. The method according to claim 184, wherein the plurality of processor subunits include part of arithmetic logic units. The method of claim 184, wherein the plurality of processor sub-units includes a memory controller. The method according to claim 184, wherein the plurality of processor sub-units include part of the memory controller. The method of claim 184, which includes outputting at least one of the following: (i) the group of related database entries, (ii) a response to one of the database queries, and (iii) the Intermediate response to one of the database queries. The method of claim 212, wherein the output includes application traffic shaping. The method of claim 212, wherein the output includes an attempt to match a bandwidth used during the output with a maximum allowable bandwidth on a link, the link coupling the memory processing integrated circuit to One requester unit. The method of claim 212, wherein the output of the output includes maintaining the fluctuation of the output traffic rate below a threshold value. The method according to claim 184, wherein the one or more processing entities include multiple processing entities, wherein at least one of the multiple processing entities belongs to the memory processing integrated circuit, and the multiple processing entities At least the other one does not belong to the memory processing integrated circuit. The method according to claim 184, wherein the one or more processing entities belong to another memory processing integrated circuit. A method for accelerating database analysis, the method includes: A database query is received by a plurality of memory processing integrated circuits, the database query including prompting at least one correlation criterion of a database entry related to the database query in a database; wherein the multiple memory processing Each of the integrated circuits includes a controller, multiple processor sub-units and multiple memory units; Determining a group of related database entries stored in the memory processing integrated circuit by each of the plurality of memory processing integrated circuits and based on the at least one correlation criterion; and By each of the plurality of memory processing integrated circuits, the group of related database entries stored in the memory processing integrated circuit is sent to one or more processing entities for further processing, and Substantially, the irrelevant database entries stored in the memory processing integrated circuit are not sent to the one or more processing entities; wherein the irrelevant database entries are different from the related database entries. A method for accelerating database analysis, the method includes: A database query is received by an integrated circuit, and the database query includes at least one correlation criterion that prompts a database entry related to the database query in a database; wherein the integrated circuit includes a controller, filtering Unit and multiple memory units; Determine a group of related database entries stored in the integrated circuit by the screening units and based on the at least one correlation criterion; and Send the group of related database entries to one or more processing entities located outside the integrated circuit for further processing, and essentially not send irrelevant database entries stored in the integrated circuit to the One or more processing entities. A method for accelerating database analysis, the method includes: Receiving a database query through an integrated circuit, the database query including at least one relevance criterion that prompts a database entry in a database related to the database query; Wherein the integrated circuit includes a controller, a processing unit and a plurality of memory units; Determining a group of related database entries stored in the integrated circuit by the processing units and based on the at least one correlation criterion; Processing the group of related database entries by the processing units instead of processing irrelevant data entries stored in the integrated circuit by the processing units to provide processing results; wherein the irrelevant database entries are different Entries in these relevant databases; and The processing results are output from the integrated circuit. A method for extracting information related to feature vectors, the method comprising: A memory processing integrated circuit receives captured information for use in capturing a plurality of requested feature vectors mapped to a plurality of sentence segments; wherein the memory processing integrated circuit includes a controller and a plurality of processors A sub-unit and a plurality of memory units, each of the memory units being coupled to a processor sub-unit; Retrieving the multiple requested feature vectors from at least some of the multiple memory units; wherein the retrieving includes simultaneously requesting two or more memory units to be stored in the two or more memories The requested feature vector in the unit; and The output from the memory processing integrated circuit includes an output of at least one of the following: (a) the requested feature vectors; and (b) a result of processing one of the requested feature vectors. The method of claim 217, wherein the output includes the requested feature vectors. The method of claim 217, wherein the output includes the result of the processing of the requested feature vectors. The method according to claim 219, wherein the processing is performed by the plurality of processor subunits. The method of claim 220, wherein the processing includes sending a requested feature vector from one processing sub-unit to another processing sub-unit. The method according to claim 220, wherein the processing includes calculating an intermediate result by one processing subunit, sending the intermediate result to another processing subunit, and calculating another intermediate result by the other processing subunit or One processing result. The method according to claim 219, wherein the processing is executed by the controller. The method according to claim 219, wherein the processing is performed by the plurality of processor subunits and the controller. The method according to claim 219, wherein the processing is performed by a vector processor of the memory processing integrated circuit. The method of claim 217, wherein the controller is configured to simultaneously request the requested feature vectors based on a known mapping between sentence segments and the locations of feature vectors mapped to the sentence segments . The method according to claim 11, wherein the mapping is uploaded during a boot process of one of the memory processing integrated circuits. The method of claim 217, wherein the controller is configured to manage the extraction of the plurality of requested feature vectors. The method of claim 217, wherein the plurality of sentence sections have a certain order, and wherein the output of the requested feature vectors is performed according to the certain order. The method according to claim 229, wherein the extraction of the plurality of requested feature vectors is performed according to the certain order. The method of claim 229, wherein the retrieval of the plurality of requested feature vectors is performed at least partially out of order; and wherein the retrieval further includes reordering the plurality of requested feature vectors. The method of claim 217, wherein the extraction of the plurality of requested features includes buffering the plurality of requested feature vectors before the plurality of requested feature vectors are read by the controller. The method of claim 232, wherein the retrieval of the plurality of requested features includes generating buffer status indicators that indicate when one or more buffers are associated with the plurality of memory units Store one or more requested feature vectors. The method according to claim 233, which includes transmitting the buffer status indicators via a dedicated control line. The method described in claim 234, wherein each memory cell is allocated a dedicated control line. The method of claim 234, wherein the buffer status indicator includes one or more status bits stored in one or more of the buffers. The method of claim 234, which includes transmitting the buffer status indicators via one or more common control lines. The method according to claim 217, wherein the captured information is included in one or more capturing commands of a first resolution, and the first resolution represents a certain number of bits. The method of claim 238, which includes managing the capture via the controller at a higher resolution, the higher resolution representing a number of bits lower than the number of bits. The method of claim 238, wherein the controller is configured to manage the capture according to a feature vector resolution. The method of claim 238, which includes independently managing the capture by the controller. The method according to claim 217, wherein the plurality of processor sub-units includes a complete arithmetic logic unit. The method according to claim 217, wherein the plurality of processor subunits include part of arithmetic logic units. The method of claim 217, wherein the plurality of processor sub-units includes a memory controller. The method according to claim 217, wherein the plurality of processor sub-units include part of the memory controller. The method of claim 217, wherein the output of the output includes applying traffic shaping to the output. The method of claim 217, wherein the output of the output includes matching a bandwidth used during the output and a maximum allowable bandwidth on a link, the link coupling the memory processing integrated circuit Connect to a requester unit. The method of claim 217, wherein the output of the output includes maintaining the fluctuation of an output traffic rate below a predetermined threshold. The method of claim 217, wherein the retrieval includes applying a predictive retrieval to at least some of the requested feature vectors from a set of requested feature vectors stored in a single memory unit . The method of claim 217, wherein the requested feature vectors are distributed among the memory cells. The method of claim 217, wherein the requested feature vectors are distributed among the memory cells based on the expected extraction pattern. A method for memory-intensive processing, the method includes: Processing operations are performed by a plurality of processors included in a hybrid device, the hybrid device including a basic die, a first memory resource associated with at least one second die, and at least one third die Connected second memory resources; wherein the base die and the at least one second die are connected to each other by an inter-wafer bonding; Use the plurality of processors to retrieve information stored in the first memory resources; and Sending additional information from the second memory resources to the first memory resources, wherein a total bandwidth of a first path between the base die and the at least one second die exceeds the at least one first path A total bandwidth of the second path between the two dies and the at least one third die, and wherein the storage capacity of one of the first memory resources is smaller than the storage capacity of one of the second memory resources. The method according to claim 252, wherein the second memory resources include high-bandwidth memory (HBM) resources. The method of claim 252, wherein the at least one third die includes a stack of high-bandwidth memory (HBM) chips. The method of claim 252, wherein at least some of the second memory resources belong to a third die in at least one third die, and the third die does not use an inter-wafer bonding Connect to the base die in case. The method of claim 252, wherein at least some of the second memory resources belong to a third die in at least one third die, and the third die does not use an inter-wafer bonding In this case, it is connected to one of the at least one second die. The method according to claim 252, wherein the first memory resources and the second memory resources include different levels of cache memory. The method according to claim 252, wherein the first memory resources are positioned between the basic die and the second memory resources. The method of claim 252, wherein the first memory resources are not located on top of the second memory resources. The method according to claim 252, which includes performing additional processing by one second die in at least one second die, the second die including a plurality of processor subunits and the first memory resources . The method of claim 260, wherein at least one processor sub-unit is coupled to a dedicated portion of the first memory resources allocated to the processor sub-unit. The method according to claim 261, wherein the dedicated portion of the first memory resources includes at least one memory bank. The method according to claim 252, wherein the plurality of processors belong to a memory processing chip that also includes the first memory resources. The method of claim 252, wherein the basic die includes the plurality of processors, wherein the plurality of processors includes a plurality of first memory resources coupled to the first memory resources through conductors formed using the inter-wafer bonding A processor subunit. The method of claim 264, wherein each processor sub-unit is coupled to a dedicated portion of the first memory resources allocated to the processor sub-unit. A hybrid device for memory-intensive processing, the hybrid device comprising: A basic die; Multiple processors At least one first memory resource of the second die; At least one second memory resource of the third die; The base die and the at least one second die are connected to each other through an inter-wafer bonding; The multiple processors are configured to perform processing operations and retrieve information stored in the first memory resources; and Wherein the second memory resources are configured to send additional information from the second memory resources to the first memory resources; One of the total frequency bandwidths of the first path between the basic crystal grain and the at least one second crystal grain exceeds the total frequency of the second path between the at least one second crystal grain and the at least one third crystal grain Wide and The storage capacity of one of the first memory resources is smaller than the storage capacity of one of the second memory resources. The hybrid device according to claim 266, wherein the second memory resources include high-bandwidth memory (HBM) resources. The hybrid device according to claim 266, wherein the at least one third die includes a stack of high-bandwidth memory (HBM) memory chips. The hybrid device according to claim 266, wherein at least some of the second memory resources belong to a third die in the at least one third die, and the third die does not use a wafer directly Connect to the base die when combined. The hybrid device according to claim 266, wherein at least some of the second memory resources belong to a third die in the at least one third die, and the third die does not use a wafer directly When combined, it is connected to one of the at least one second die. The hybrid device according to claim 266, wherein the first memory resources and the second memory resources include different levels of cache memory. The hybrid device according to claim 266, wherein the first memory resources are positioned between the basic die and the second memory resources. The hybrid device according to claim 266, wherein the first memory resources are located on one side of the second memory resources. The hybrid device according to claim 266, wherein one second die in the at least one second die is configured to perform additional processing, wherein the second die includes a plurality of processor subunits and the second die A memory resource. The hybrid device of claim 274, wherein each processor sub-unit is coupled to a dedicated portion of the first memory resources allocated to the processor sub-unit. The hybrid device according to claim 275, wherein the dedicated portion of the first memory resources includes at least one memory bank. The hybrid device according to claim 266, wherein the plurality of processors includes a plurality of processor subunits that also include a memory processing chip of one of the first memory resources. The hybrid device according to claim 266, wherein the basic die includes the plurality of processors, and wherein the plurality of processors include those coupled to the first memory resources through conductors formed using the inter-wafer bonding Multiple processor subunits. The hybrid device of claim 278, wherein each processor sub-unit is coupled to a dedicated portion of the first memory resources allocated to the processor sub-unit. A method for database acceleration, the method includes: A network communication interface of the integrated circuit is accelerated by a database to retrieve a certain amount of information from the storage unit; Perform first processing on the amount of information to provide first processed information; Use the database to accelerate the memory controller of the integrated circuit and send the first processed information to a plurality of memory processing integrated circuits through an interface, wherein each of the memory processing integrated circuits includes a controller, a plurality of Processor sub-units and multiple memory units, Using the plurality of memory processing integrated circuits to perform second processing on at least part of the first processed information to provide second processed information; The memory controllers that accelerate the integrated circuit through the database retrieve information from the multiple memory processing integrated circuits, wherein the retrieved information includes at least one of the following: (a) the first At least a part of the once processed information; and (b) at least a part of the second processed information; Using the database acceleration unit of the integrated circuit to perform database processing operations on the retrieved information to provide database acceleration results; and Output the acceleration results of these databases. The method according to claim 280, which includes using a management unit of the database acceleration integrated circuit to manage at least one of the retrieval, the first processing, the sending, and the processing. The method of claim 281, wherein the management is performed based on an execution plan generated by the management unit of the database accelerated integrated circuit. The method according to claim 281, wherein the management is performed based on an execution plan received by the management unit of the database accelerated integrated circuit but not generated by the management unit. The method of claim 281, wherein the management includes allocating at least one of the following: (a) network communication network interface resources; (b) decompression unit resources; (c) memory controller resources ; (D) Multiple memory processing integrated circuit resources; and (e) Database acceleration unit resources. The method according to claim 280, wherein the network communication interface includes two or more different types of network communication ports. The method according to claim 285, wherein the two or more different types of network communication ports include a storage interface protocol port and a common network protocol storage interface port. The method according to claim 285, wherein the two or more different types of network communication ports include a storage interface protocol port and an Ethernet protocol storage interface port. The method according to claim 285, wherein the two or more network communication ports of different types include a storage interface protocol port and a PCIe port. The method described in claim 280 includes a management unit including a computing node of a computing system and controlled by a manager of the computing system. The method according to claim 280, which includes controlling the at least one of the retrieval, the first processing, the sending, and the third processing by a computing node of a computing system. The method described in claim 280 includes using the database to accelerate the integrated circuit to perform multiple tasks at the same time. The method according to claim 280, which includes using a management unit located outside the database acceleration integrated circuit to manage at least one of the retrieval, the first processing, the sending, and the third processing. The method according to claim 280, wherein the database accelerated integrated circuit belongs to a computing system. The method according to claim 280, wherein the database accelerated integrated circuit does not belong to a computing system. The method according to claim 280, which includes executing the acquisition, the first processing, the sending, and the third processing based on an execution plan sent to the database acceleration integrated circuit by an arithmetic node of a computing system At least one of them. The method according to claim 280, wherein the execution of the database processing operations includes simultaneous execution of database processing commands by the database processing subunit, wherein the database acceleration unit includes a database sharing a common memory unit A group of accelerator subunits. The method described in claim 296, wherein each database processing subunit is configured to execute a specific type of database processing command. The method according to claim 297, which includes dynamically linking the database processing subunit to provide an execution pipeline for executing a database processing operation including a plurality of instructions. The method of claim 280, wherein the execution of the database processing operations includes allocating the resources of the database acceleration integrated circuit according to the time I/O bandwidth. The method described in claim 280 includes outputting the database acceleration results to a local storage and retrieving the database acceleration results from the local storage. The method according to claim 280, wherein the network communication interface includes an RDMA unit. The method described in claim 280 includes exchanging information between one or more groups of database accelerated integrated circuits. The method described in claim 280 includes exchanging database acceleration results among one or more groups of database acceleration integrated circuits. The method according to claim 280, which includes exchanging at least one of the following among one or more groups of database accelerated integrated circuits: (a) information; and (b) The result of database acceleration. In the method described in claim 304, the database accelerated integrated circuit of a group is connected to a common printed circuit board. In the method described in claim 304, the database accelerated integrated circuit of a group belongs to a modular unit of a computerized system. The method described in claim 304, wherein different groups of databases accelerate integrated circuits to connect to different printed circuit boards. The method described in claim 304, wherein the database accelerated integrated circuits of different groups belong to different modular units of a computerized system. The method described in claim 304 includes accelerating integrated circuits to perform distributed processing by the one or more groups of the databases. The method according to claim 304, wherein one or more groups of the databases are used to accelerate the network communication interface of the integrated circuit to perform the exchange. The method of claim 304, wherein the exchange is performed on multiple groups connected to each other by a star connection. The method according to claim 304, which includes using at least one switch for exchanging at least one of the following between the database accelerated integrated circuits of different groups of the one or more groups: (a) Information and (b) database acceleration results. The method of claim 304, which includes accelerating at least some of the integrated circuits to perform distributed processing by the one or more groups of the databases. The method according to claim 304, which includes performing distributed processing of one of the first and second data structures, wherein the total size of one of the first and second data structures exceeds the size of the plurality of memory processing integrated circuits 1. Storage capacity. The method of claim 314, wherein the execution of the distributed processing includes executing multiple iterations of: (a) executing different pairs of a first data structure part and a second data structure part to different data The library accelerates a new allocation of integrated circuits; and (b) handles the different pairs. The method of claim 315, wherein the execution of the distributed processing includes a database joining operation. The method of claim 315, wherein the execution of the distributed processing includes: Allocating different first data structure parts to different database acceleration integrated circuits of the one or more groups; and Perform multiple iterations of each of the following: Newly assign different second data structure parts to different database acceleration integrated circuits of the one or more groups, and The first and second data structure parts are processed by the integrated circuit to speed up the integrated circuit by the database. The method of claim 317, wherein the new allocation of the next iteration is performed in a manner that overlaps at least part of the time with the processing of a current iteration. The method of claim 317, wherein the new allocation includes exchanging the second data structure part between the different database accelerated integrated circuits. The method according to claim 319, wherein the exchange is performed in a manner that overlaps at least a part of the processing time. The method of claim 317, wherein the new allocation includes exchanging the second data structure part between the different database accelerated integrated circuits in a group; and once the exchange has been completed, accelerating the accumulation in the database The second data structure part is exchanged between different groups of body circuits. The method according to claim 280, wherein the database accelerated integrated circuit is included in a blade, and the blade includes a plurality of database accelerated integrated circuits, one or more non-volatile memory units, and an Ethernet Circuit switch, a PCIe switch and an Ethernet switch, and the plurality of memory processing integrated circuits. A device for database acceleration, the device includes: A database to accelerate the integrated circuit; and A plurality of memory processing integrated circuits; wherein each memory processing integrated circuit includes a controller, a plurality of processor sub-units and a plurality of memory units; Wherein, a network communication interface of the database acceleration integrated circuit is configured to receive information from the storage unit; The database acceleration integrated circuit is configured to perform first processing on a certain amount of information to provide first processed information; The memory controller of the database acceleration integrated circuit is configured to send the first processed information to the plurality of memory processing integrated circuits through an interface; The plurality of memory processing integrated circuits are configured to perform second processing on at least part of the first processed information through the plurality of memory processing integrated circuits to provide second processed information; The memory controllers of the database accelerated integrated circuit are configured to retrieve information from the multiple memory processing integrated circuits, wherein the retrieved information includes at least one of the following: (a ) At least a part of the first processed information; and (b) at least a part of the second processed information; A database acceleration unit of the database acceleration integrated circuit of the database acceleration integrated circuit is configured to perform a database processing operation on the retrieved information to provide a database acceleration result; and wherein the database acceleration unit The acceleration integrated circuit is configured to output the acceleration results of the database. The device described in claim 323, which is configured to use a management unit of the database to accelerate the integrated circuit to manage at least one of the retrieval, a first process, and the second process of the retrieved information By. The device of claim 324, wherein the management unit is configured to perform management based on an execution plan generated by the management unit of the database acceleration integrated circuit. The device of claim 324, wherein the management unit is configured to perform management based on an execution plan received by the management unit of the database accelerated integrated circuit but not generated by the management unit. The device of claim 324, wherein the management unit is configured to be managed by allocating one or more of the following: (a) network communication network interface resources; (b) decompression unit resources ; (C) Memory controller resources; (d) Multiple memory processing integrated circuit resources; and (e) Database acceleration unit resources. The device according to claim 323, wherein the network communication interface includes different types of network communication ports. The device according to claim 328, wherein the different types of network communication ports include storage interface protocol ports and general network protocol storage interface ports. The device according to claim 328, wherein the different types of network communication ports include storage interface protocol ports and Ethernet protocol storage interface ports. The device according to claim 328, wherein the different types of network communication ports include storage interface protocol ports and PCIe ports. The device according to claim 323, wherein the device is coupled to a management unit, and the management unit includes a computing node of a computing system and is controlled by a manager of the computing system. The device described in claim 323 is configured to be controlled by a computing node of a computing system. The device described in claim 323 is configured to accelerate the integrated circuit to perform multiple tasks at the same time through the database. The device according to claim 323, wherein the database accelerated integrated circuit belongs to a computing system. The device according to claim 323, wherein the database acceleration integrated circuit does not belong to a computing system. The device according to claim 323, which is configured to execute the acquisition, first processing, sending, and second execution based on an execution plan sent to the database accelerated integrated circuit by an arithmetic node of an arithmetic system At least one of the three treatments. The device according to claim 323, wherein the database acceleration unit is configured to simultaneously execute database processing commands by the database processing subunit, wherein the database acceleration unit includes a database accelerator sharing a common memory unit A group of subunits. The device of claim 338, wherein each database processing subunit is configured to execute a specific type of database processing command. The device according to claim 339, wherein the device is configured to dynamically link the database processing subunit to provide an execution pipeline for executing a database processing operation including a plurality of instructions. The device of claim 323, wherein the device is configured to allocate the resources of the database acceleration integrated circuit based on the time I/O bandwidth. The device according to claim 323, wherein the device includes a local storage that can be accessed by the database accelerated integrated circuit. The device according to claim 323, wherein the network communication interface includes an RDMA unit. The device according to claim 323, wherein the device includes one or more groups of database accelerated integrated circuits, and the database accelerated integrated circuits are configured to accelerate one or more of the integrated circuits in the database Multiple groups of databases accelerate the exchange of information between integrated circuits. The device according to claim 323, wherein the device includes one or more groups of database accelerated integrated circuits, and the database accelerated integrated circuits are configured to accelerate one or more of the integrated circuits in the database Multiple groups of databases accelerate the exchange of acceleration results between integrated circuits. The device according to claim 323, wherein the device includes one or more groups of database accelerated integrated circuits, and the database accelerated integrated circuits are configured to accelerate one or more of the integrated circuits in the database The database acceleration integrated circuits of multiple groups execute at least one of the following: (a) information; and (b) database acceleration result. In the device described in claim 346, the database accelerated integrated circuits of one group are connected to the same printed circuit board. In the device described in claim 346, the database accelerated integrated circuit of a group belongs to a modular unit of a computerized system. The device according to claim 346, wherein the database accelerating integrated circuits of different groups are connected to different printed circuit boards. The device described in claim 346, wherein the database accelerated integrated circuits of different groups belong to different modular units of a computerized system. The device of claim 346, wherein one or more groups of the databases are used to accelerate the network communication interface of the integrated circuit to perform the exchange. The device of claim 346, wherein the exchange is performed on a plurality of groups connected to each other by a star connection. The device of claim 346, wherein the device is configured to use at least one switch for exchanging one of the following between database accelerated integrated circuits of different groups in the one or more groups At least one: (a) information; and (b) database acceleration results. The device of claim 346, wherein the device is configured to accelerate some of the integrated circuits to perform distributed processing by the databases of some of the one or more groups. The device of claim 346, wherein the device is configured to perform a distributed process using first and second data structures, wherein the total size of one of the first and second data structures exceeds the plurality of memories Deal with the storage capacity of one of the integrated circuits. The device of claim 355, wherein the device is configured to perform the distributed processing by performing multiple iterations of: (a) performing a first data structure part and a second data structure part The different pairs are assigned to different databases to speed up a new allocation of integrated circuits; and (b) processing these different pairs. The device according to claim 355, wherein the distributed processing includes a database joining operation. The device of claim 355, wherein the device is configured to perform the distributed processing by: Allocating different first data structure parts to different database acceleration integrated circuits of the one or more groups; and Perform multiple iterations of each of the following: Newly assign different second data structure parts to different database acceleration integrated circuits of the one or more groups, and The first and second data structure parts are processed by the integrated circuit to speed up the integrated circuit by the database. The device of claim 358, wherein the device is configured to perform the new allocation of the next iteration in a manner that overlaps at least a portion of the time of a current iteration of the process. The device of claim 358, wherein the device is configured to perform the new allocation by exchanging the second data structure portion between the different database accelerated integrated circuits. The device according to claim 360, wherein the database accelerates the integrated circuit to perform the exchange in a manner that overlaps at least a part of the processing time. The device of claim 358, wherein the device is configured to perform the new allocation by: exchanging the second data structure part between the different database accelerated integrated circuits in a group; and once The exchange is completed, and the second data structure part is exchanged between different groups of the database accelerated integrated circuit. The device according to claim 323, wherein the database accelerated integrated circuit is included in a blade, and the blade includes a plurality of database accelerated integrated circuits, one or more non-volatile memory units, and an Ethernet Circuit switch, a PCIe switch and an Ethernet switch, and the plurality of memory processing integrated circuits. A method for database acceleration, the method includes: A network communication interface of the integrated circuit is accelerated by a database to retrieve information from the storage unit; Perform first processing on a certain amount of information to provide first processed information; Accelerating the memory controller of the integrated circuit by the database and sending the first processed information to a plurality of memory resources through an interface; Retrieve information from the multiple memory resources; A database acceleration unit of the database acceleration integrated circuit performs database processing operations on the retrieved information to provide database acceleration results; and Output the acceleration results of these databases. The method of claim 364, further comprising processing the first processed information to provide second processed information, wherein the processing of the first processed information is performed by a plurality of processors OK, the plurality of processors are located in an integrated circuit that further includes one or more of the plurality of memory resources. The method of claim 364, wherein the first process includes screening database entries. The method of claim 364, wherein the second process includes screening database entries. The method of claim 364, wherein the first processing and the second processing include filtering database entries.

Images