CN116670652A

CN116670652A - Providing host-based error detection capability in a remote execution device

Info

Publication number: CN116670652A
Application number: CN202180085623.1A
Authority: CN
Inventors: 施里坎特·甘纳帕西; 罗丝·V·拉费特拉; 约翰·卡拉马丁纳斯; 苏丹瓦·古鲁穆提; 沙泽恩·阿加; 维拉斯·斯里达兰; 迈克尔·伊格纳托斯基; 努万·贾亚塞纳
Original assignee: Advanced Micro Devices Inc
Current assignee: Advanced Micro Devices Inc
Priority date: 2020-12-29
Filing date: 2021-12-21
Publication date: 2023-08-29
Also published as: JP2024500785A; US11409608B2; KR20230121154A; EP4272079A1; US20220206901A1; WO2022146790A1

Abstract

Providing host-based error detection capability in a remote execution device is disclosed. The remote execution device performs a host offload operation that modifies the data block stored in the memory. Metadata for the modified data block is locally generated such that the local metadata generation emulation is based on metadata generation of the host. The stored metadata of the data block is updated with the locally generated metadata of the modified portion of the data block. When the host performs an integrity check on the modified data block using the updated metadata, the host does not distinguish between metadata generated by the host and metadata generated in the remote execution device.

Description

Providing host-based error detection capability in a remote execution device

Background

Computing systems typically include multiple processing resources (e.g., one or more processors) that can retrieve and execute instructions and store the results of the executed instructions in appropriate locations. The processing resources (e.g., central Processing Unit (CPU)) may include a plurality of functional units, such as, for example, arithmetic Logic Unit (ALU) circuitry, floating Point Unit (FPU) circuitry, and/or combinational logic blocks, that may be used to execute instructions by performing logical operations on data (e.g., one or more operands). For example, functional unit circuitry may be used to perform arithmetic operations on operands, such as addition, subtraction, multiplication, and/or division. In general, processing resources (e.g., processors and/or associated functional unit circuitry) may be external to a memory array and access data via a bus between the processing resources and the memory array to execute a set of instructions. To reduce the amount of access to acquire or store data in a memory array, a computing system may employ a cache hierarchy that temporarily stores recently accessed or modified data for use by a processing resource or group of processing resources. However, processing performance may be further improved by offloading certain operations to an in-memory Processor (PIM) device, wherein processing resources may be implemented within and/or near memory such that data processing is performed closer to the memory location where the data is stored for computing purposes than to the processing resources. The PIM device may save time by reducing and/or eliminating external communication and may also save power.

Some applications have low or no temporal data reuse phases during which they frequently miss in the cache hierarchy and fetch data from memory. Furthermore, these phases may also exhibit low computational intensity (flip-flop/byte ratio). During those phases, energy efficiency and performance decline because data movement is high and the phases are memory limited. These phases are thus particularly suitable for unloading to a PIM device or an accelerator. For example, a programmer may provide an indication in the application source code that a particular code sequence should be uninstalled, or a compiler may make such a determination. The interpretation and orchestration of the offload instructions must still be performed by the processing resources of the host application.

Drawings

FIG. 1 sets forth a block diagram of an exemplary system for providing host-based error detection capability in a remote execution device according to embodiments of the present disclosure.

FIG. 2 sets forth a flow chart illustrating an exemplary method for providing host-based error detection capability in a remote execution device according to embodiments of the present disclosure.

FIG. 3 sets forth a flow chart illustrating a further exemplary method for providing host-based error detection capability in a remote execution device according to embodiments of the present disclosure.

FIG. 4 sets forth a flow chart illustrating a further exemplary method for providing host-based error detection capability in a remote execution device according to embodiments of the present disclosure.

FIG. 5 sets forth a flow chart illustrating a further exemplary method for providing host-based error detection capability in a remote execution device according to embodiments of the present disclosure.

Detailed Description

In-memory Processing (PIM) architecture supports offloading instructions for execution in or near memory such that bandwidth on a data link between a processor and memory is saved and power consumption of the processor may be reduced. Execution of the offload instructions by e.g. the PIM device does not require loading of data into a local Central Processing Unit (CPU) register and writing of data from a local CPU storage back to memory. The PIM architecture typically includes a PIM enabled host (e.g., a CPU or graphics processing unit GPU) configured to execute PIM instructions for offloading operations to the PIM device. Typically, the PIM device is a memory device that implements computing power (e.g., PIM logic) within the memory device, but the PIM device may also be closely coupled to the memory device. For ease of explanation, it is assumed in this disclosure that the PIM device is a PIM enabled memory device having PIM logic embedded within the memory device that implements computing capabilities.

To support error detection of data written to memory, a host may generate reliability, availability, and serviceability (RAS) metadata for the data written to memory. This metadata is typically stored with the data in the memory device and is used to perform error detection checks when the data is later retrieved from the memory device. For example, the host may apply an error detection function to data read from memory to determine if the result of the function matches metadata. A failure to meet this error detection check may indicate a failure to correctly retrieve data from memory or corruption of the data itself. Problems arise when this data is modified by PIM operations because modifying the data may cause RAS metadata to become invalid. That is, when PIM modified data is retrieved by the host, the results of the error detection function applied to the PIM modified data will likely not match the RAS metadata stored with the data.

One solution to this problem is to mirror host-based error protection techniques in the memory device. However, this may require that all PIM enabled memory vendors and all PIM enabled host vendors agree to use the same error protection technique, or may require that the host specifically configure the PIM enabled memory device using proprietary error detection techniques employed by the host. Often, host vendors do not wish to disclose these proprietary techniques. Thus, mirroring or precisely replicating host-based error detection techniques in PIM-enabled memory devices may not be a viable solution.

Embodiments according to the present disclosure relate to: host-based error detection capabilities are provided in the PIM device without duplicating host-based error detection techniques. Rather, these embodiments update host generated metadata within the PIM device itself for subsequent modification of data by PIM operations. In some embodiments, PIM generated metadata emulates host computer metadata generation techniques based on hints, suggestions, or cues received from the host computer. Host generated metadata updated with PIM generated metadata will match, bit by bit, RAS metadata that would otherwise be generated by the host. Thus, these embodiments provide error detection capability for PIM computed data as if the host itself had generated RAS metadata for PIM computed data.

Embodiments according to the present disclosure relate to a method of providing host-based error detection capability in a remote execution device, the method comprising: the data blocks stored in the memory device are modified. The method further comprises the steps of: metadata for the modified data block is locally generated. The method further comprises the steps of: host-generated metadata of the storage of the data block is updated with locally generated metadata of the modified data block.

In some implementations, the method further includes: the modified data block and the updated metadata are stored. These implementations also include: the modified data block and the updated metadata are transmitted in response to receiving a request from the host to read the modified data block.

In some implementations, the method further includes: the data blocks and host-generated metadata for the data blocks are stored. These implementations also include: a look-up table (LUT) is generated storing metadata received from a host. Locally generating metadata for the modified portion of the data block may include emulating host metadata generation for the modified data block using the LUT.

In some implementations, the method further includes: the data block and the stored host-generated metadata for the data block are read in response to receiving a request from the host. These implementations also include: host-generated metadata of a storage of a data block is verified by emulating host metadata generation of the data block. Verifying the stored host-generated metadata for the data block by emulating host metadata generation for the data block may include generating metadata for the data block using a look-up table (LUT). The modification of the data block may be performed without waiting for verification of the stored metadata of the data block to complete.

In some implementations, the method further includes: the method includes reading locally generated Error Correction Code (ECC) information for the stored data block and metadata from an ECC storage device, and verifying the locally generated ECC information before modifying the data block. In some implementations, the method further includes: ECC information for the modified data block and the updated metadata is generated and stored in an ECC storage device.

Embodiments in accordance with the present disclosure relate to an apparatus for providing host-based error detection capability in a remote execution device that includes a memory device and a process-in-memory (PIM) device coupled to the memory device. The PIM device is configured to: the data block is modified. The PIM device is further configured to: metadata for the modified data block is locally generated. The PIM device is further configured to: host-generated metadata of the storage of the data block is updated with locally generated metadata of the modified data block.

In some implementations, the PIM device is further configured to: the data block and host-generated metadata for the data block are stored, and metadata received from the host is stored to generate a look-up table (LUT). Locally generating metadata for the modified portion of the data block may include emulating host metadata generation for the modified data block using the LUT.

In some implementations, the PIM device is further configured to: the data block and host generated metadata of the storage of the data block are read in response to receiving a PIM request from the host. In these implementations, the PIM device is further configured to: host-generated metadata of a storage of a data block is verified by simulating host metadata generation of the data block. Verifying the stored host-generated metadata for the data block by emulating host metadata generation for the data block may include generating metadata for the data block using a look-up table (LUT).

In some implementations, the PIM device is configured to: the method includes reading locally generated Error Correction Code (ECC) information for the stored data block and metadata from an ECC storage device, and verifying the locally generated ECC information before modifying the data block. In some implementations, the memory device is further configured to: ECC information for the modified data block and the updated metadata is generated and stored in an ECC storage device.

Embodiments in accordance with the present disclosure relate to a system for providing host-based error detection capability in a remote execution device that includes a memory device and a host processor coupled to the memory device. The memory device is configured to: the stored data block is modified. The memory device is further configured to: metadata for the modified data block is locally generated. The memory device is further configured to: host-generated metadata of the storage of the data block is updated with locally generated metadata of the modified data block.

In some implementations, the memory device is further configured to: the data block and host-generated metadata for the data block are stored, and metadata received from the host is stored to generate a look-up table (LUT). Locally generating metadata for the modified portion of the data block may include simulating host metadata generation for the modified data block using the LUT.

In some implementations, the memory device is further configured to: the data block and host generated metadata of the storage of the data block are read in response to receiving a PIM request from the host. In these implementations, the memory device is further configured to: host-generated metadata of a storage of a data block is verified by emulating host metadata generation of the data block. Verifying the stored host-generated metadata for the data block by emulating host metadata generation for the data block may include generating metadata for the data block using a look-up table (LUT).

In some implementations, the memory device is further configured to: the method includes reading locally generated Error Correction Code (ECC) information for the stored data block and metadata from an ECC storage device, and verifying the locally generated ECC information before modifying the data block. In some implementations, the memory device is further configured to: ECC information for the modified data block and the updated metadata is generated and stored in an ECC storage device.

Embodiments according to the present disclosure will be described in more detail starting with fig. 1. FIG. 1 sets forth a block diagram of an exemplary system 100 for providing host-based error detection capability in a remote execution device according to the present disclosure. The exemplary system 100 of fig. 1 includes a host processor 102 coupled to a remote execution device, such as a PIM enabled memory device 104, although the reader should appreciate that some embodiments may employ other types of remote execution devices that function on behalf of the host processor 102. The host processor 102 includes at least one CPU 116 that executes instructions, including instructions for reading and writing data to the memory device 104. The memory controller 112 of the host processor 102 receives memory requests generated from the executed instructions and interprets the requests as memory commands transmitted to the memory device 104 through the memory channel. Memory interface logic 118 in memory device 104 receives the command (and associated data) and satisfies the request by reading or writing data to memory bank 106. In some examples, the memory device may be a stacked memory device including one or more Dynamic Random Access (DRAM) dies stacked on top of the memory interface die and electrically and communicatively coupled to the memory interface die through a plurality of Through Silicon Vias (TSVs). For example, the memory device 104 may be a High Bandwidth Memory (HBM) device. In various examples, the memory device 104 is a remote execution device in that the execution logic for executing the offload instructions (e.g., PIM instructions) is not part of the CPU 116. For example, the memory device 104 may be implemented on the same chip or in the same package as the host processor 102 while remaining "remote" from the host processor 102 with respect to execution locality. In some examples, the host processor 102 and the memory device 104 of the system 100 are integrated on the same chip (e.g., a system on a chip (SoC)). In other examples, the system 100 implements a System In Package (SiP) architecture in which the processor 102 and its components (e.g., as a SoC) are placed on an interposer die in the same semiconductor package along with the memory device 104.

To meet the RAS target, the host processor 102 may employ error detection techniques when reading and writing data to the memory device 104. One such technique is to generate a linear error correction/detection code for data written to memory. The code may be stored as metadata with the data written to the memory device 104. When data is subsequently read from the memory device 104, the same code may be generated for the retrieved data to determine whether the code matches the code stored in the metadata with the data. If there is a mismatch, errors may occur in reading data from the memory device. For the example of fig. 1, assume that the linear error correction/detection code is a Cyclic Redundancy Check (CRC) code.

Consider the example depicted in fig. 1, wherein host processor 102 performs an operation of writing data 108 to memory bank 106 (via memory controller 112 and memory interface logic 118). Prior to issuing a write request for data 108, host error correction module 114 applies a function to generate a CRC for data 108 that is included in metadata 110. As part of the write operation, data 108 and metadata 110 are stored in the memory bank 106. For example, the host may generate CRC16 for every M bytes of data. The CRC is generated by dividing the input data by a polynomial number (also referred to as a generator polynomial). In one example, the generator polynomial is N bytes wide as required by the CRC code, and the remainder of the division is the generated CRC. Metadata including the CRC and the M-byte data are transferred and stored in the memory device.

Continuing with the above example, in response to a read request from host processor 102, data 108 and metadata 110 are read from memory bank 106 and transferred (via memory controller 112 and memory interface logic 118) to host processor 102. Error correction module 114 verifies the integrity of received data 108 by applying the same function to generate a CRC for data 108 and comparing the CRC to a CRC in metadata 110. If there is a mismatch, a read error or corruption of data 108 may have occurred. The reader will appreciate that error correction module 114 may employ other suitable error detection techniques not limited to CRC codes. Rather, any suitable linear error correction/detection code may be employed.

The memory device 104 may also include local error detection and correction capabilities for determining whether data has been correctly read or written to the memory bank 106. In some examples, the memory device 104 includes an Error Correction Code (ECC) module 128. For example, the ECC module 128 may be an on-die ECC in that the ECC module 128 is implemented on each die of a stacked-die memory device (e.g., HBM). ECC may be designed by a (proprietary or standard driven) memory vendor to provide error detection/correction capability to all data (including metadata) sent to and from the host. ECC encoding is performed before new data is written into memory. The check bits are encoded and stored in an ECC-specified storage area, such as ECC storage 130. Similarly, after reading data from memory, ECC decoding is performed by reading both data and metadata from memory bank 106 and reading check bits from the check bit store (e.g., from ECC store 130). The check bits calculated for the read data and metadata are compared with stored check bits from the check bit storage. Because ECC decoding is performed within the memory device 104, the memory device may also provide transparency information to the host processor 102 if errors are detected during decoding of the check bits.

In some examples, the memory device 104 includes a PIM device 150 that includes PIM logic 120 and PIM registers 122 for PIM operations (e.g., arithmetic operations) on data stored in the memory bank 106. In some implementations, the PIM device 150 may be implemented for each memory bank, while in other implementations, the PIM device 150 may be implemented for each memory channel. By way of example and not limitation, the memory device 104 may receive a set of commands to load data from a location in the memory bank into register r0 of the register file (i.e., PIM register 122) and to load data received from the host processor 102 calculated by the host CPU 116 into register r1 of the register file. In this example, PIM logic 120 performs an operation (e.g., a fixed function or static operation stored in a command buffer (not shown) of the memory device) that adds the value in register r1 to the value in register r0 and stores the result in register r 0. The result from register r0 (referred to as PIM data) is then written back to the location in memory bank 106. In this example, consider a memory location in memory bank 106 as an address of an M byte data block containing data 108. Since a portion 132 of the data 108 has been modified, the CRC or other linear code in the metadata 110 will likely fail the integrity check when read back to the host processor 102. In the remainder of this disclosure, the portion 132 of the PIM modified data may be referred to as "PIM data" while the unmodified portion of the data 108 may be referred to as "non-PIM data.

As can be seen above, after PIM operation modifies the original data for which the CRC was calculated, the host generated CRC (or other linear error detection code) may become invalid. To avoid fully replicating host error detection techniques in memory devices (which may require all PIM enabled memory vendors to coordinate error detection techniques with all PIM enabled host vendors), embodiments in accordance with the present disclosure iteratively construct CRCs for larger data blocks using caches of CRCs for smaller data blocks.

In some embodiments the PIM device 150 comprises a metadata generator unit 124 for updating the metadata 110 of the data blocks 108. In some examples, where only a portion of the data is modified, the CRC need only be calculated for the modified portion 132 of the data 108. For example, generating a CRC for only the modified portion 132 of the data 108 is faster than generating a CRC for the entire M bytes of the data 108. To generate the CRC for only the modified portion, some embodiments utilize hints received from host processor 102 regarding error detection techniques of the host. In some examples, the metadata 110 is updated and reconstructed with a host-filled look-up table (LUT) of CRC values of the most commonly used 16-bit remainder. In these examples, the LUT may be implemented as LUT storage 126 storing a dictionary of bit positions representing keys and CRC remainders representing values thereof. The LUT is then used to calculate the CRC without copying the entire CRC calculation logic of host processor 102. In some examples, the contents of LUT store 126 may be protected with parity or ECC. In some implementations, the host processor 102 populates the LUT storage 126 with memory mapped instructions or at boot-up as part of a power-on self test (POST) sequence. Metadata for locally generating data within the memory device 104 for modification as part of PIM operation will be described in more detail below.

For further explanation, FIG. 2 sets forth a flow chart illustrating an exemplary method for providing host-based error detection capability in a remote execution device according to the present disclosure. The example depicted in fig. 2 includes a remote execution device 214, such as the PIM enabled memory device 104 depicted in fig. 1, an accelerator, or other device that performs operations on behalf of a host system. The example depicted in fig. 2 also includes a host processor 212 similar in configuration to the host processor 102 depicted in fig. 1. The exemplary method of fig. 2 includes verifying 200 the integrity of a stored data block in response to a host request for the data block stored in a memory device. In some examples, verifying 200 the integrity of the stored data block is performed by: the remote execution device 214 receives a command to modify data stored in the memory device, compares stored error detection metadata associated with the data with error detection metadata generated for the data in response to the request, and verifies that the stored metadata matches the generated metadata.

The method of fig. 2 further comprises modifying 202 the data block. In some examples, modifying 202 the data block is performed by: remote execution device 214 loads data from a memory bank (e.g., memory bank 106 of FIG. 1) into a register (e.g., register 122 of FIG. 1) of remote execution device 214. In these examples, modifying 202 the data block is further performed by: one or more operations to modify at least a portion of the data in the registers are performed in computing logic (e.g., PIM logic 120 in fig. 1). For example, the operation may be an arithmetic or logic operation performed in an Arithmetic Logic Unit (ALU) in PIM logic.

The method of fig. 2 further includes locally generating 204 metadata for the modified data block. In some examples, locally generating 204 the metadata of the modified data block is performed by: a metadata generation unit (e.g., metadata generation unit 124 in fig. 1) generates metadata for the manipulated modified portion of the data block such that the metadata for the modified portion of the data generated within remote execution device 214 may be identified as if it were metadata generated by host processor 212. In these examples, the metadata of the data block includes a linear error detection code value (e.g., a CRC value). As previously discussed, the particular function of the linear error detection code used by the host processor 212 to generate the check value is unknown to the memory device 214, and thus the memory device 214 cannot accurately replicate the metadata generation process used by the host processor 212. Rather, the metadata generation unit in the memory device 214 emulates metadata generation of the host for the modified data block (or just for the portion of data modified by the operation). In some examples, the metadata generation unit emulates the metadata generation technique of the host processor by reconstructing the check values of the modified data using partial information about the linear error detection function used by the host processor 212. The partial information may include, for example, a function type (e.g., CRC), word encoding width, and a mapping of bit strings to check values. The host metadata generation simulation by the metadata generation unit will be described in more detail below.

The method of fig. 2 further comprises: the stored host-generated metadata for the data block is updated 206 with the locally generated metadata for the modified data block. In some examples, updating 206 the stored host-generated metadata of the data block with the locally-generated metadata of the modified portion occurs by: the remote execution device 214 replaces the check value in the host-generated metadata with the locally-generated check value for the data word. For example, the remote execution device may be a PIM enabled memory device that performs PIM operations on the data blocks. In this example, the data word includes data modified by PIM operations (i.e., PIM data). Due to the inherent nature of the linear code, the resulting updated metadata includes a combination of updated check values for PIM data and original check values for non-PIM data that bitwise match the check values that would have been generated by the host processor itself if the processor generated the metadata of the modified data block.

The method of fig. 2 further comprises: the modified data block and the updated metadata are stored 208. In some examples, storing 208 the modified data blocks and updated metadata proceeds by: the data blocks including PIM data and non-PIM data and updated metadata including host-generated metadata and locally-generated metadata are stored in a memory bank at a particular location from which the data blocks and metadata were loaded.

The method of fig. 2 further comprises: the modified data block and the updated metadata are transmitted 210 in response to receiving a request from the host to read the modified data block. In some examples, transmitting 210 the modified data block and the updated metadata in response to receiving a request from the host to read the modified data block occurs by: a read request is received by the memory device from the host processor 212 to read data in the particular memory location in which the modified data block and updated metadata are stored and the modified data block and updated metadata are transferred to the host processor 212. The data integrity verification performed by the host processor 212 may include regenerating metadata and comparing the regenerated metadata with metadata received from the memory device (i.e., updated metadata). Because the metadata of the modified portion of the data is generated by simulating host-based metadata generation, the host processor 212 will recognize the integrity of the updated metadata as valid.

For further explanation, FIG. 3 sets forth a flow chart illustrating an exemplary method for providing host-based error detection capability in a remote execution device according to the present disclosure. Similar to the exemplary method depicted in fig. 2, the exemplary method of fig. 3 further includes: modifying 202 the data block in response to verifying 200 the integrity of the stored data block in response to a host request for the data block stored in the memory device; locally generating 204 metadata for the modified data block; updating 206 the stored host-generated metadata of the data block with the locally-generated metadata of the modified data block; storing 208 the modified data block and the updated metadata; and transmitting 210 the modified data block and the updated metadata in response to receiving a request from the host to read the modified data block.

The method of fig. 3 further comprises: the data block and host generated metadata for the data block are stored 302. The data block is initially stored in the memory device prior to any modification to the data block by operation (e.g., PIM operation). In some examples, host-generated metadata that stores 302 the data blocks and the data blocks proceeds by: a request to store a data block at a particular memory location is received and the data block is stored at the particular memory location in the memory bank. In these examples, the data blocks and corresponding metadata are stored together. In one example, metadata is interleaved in a data block. In another example, metadata for a data block is appended to the data block.

The method of fig. 3 further comprises: the store 304 generates a look-up table (LUT) from metadata received by the host. In some examples, storing 304 the metadata generation LUT received from the host proceeds by: a LUT describing limited information about host-based metadata generation computations is received by the remote execution device 214. For example, the LUT may contain the most common bit remainder of CRC computation. The LUT may be implemented in a reserved memory location (e.g., LUT storage 126 of fig. 1) that stores a dictionary of bit positions representing keys and CRC remainder representing values thereof. In these examples, the LUT may be used to calculate the CRC without copying the entire CRC calculation logic in remote execution device 214.

Because host processor 212 utilizes linear code such as CRC, metadata generation for PIM data may take advantage of the following properties:the addition is performed over a galois field of 2 elements indicating an equivalent exclusive or operation. In some embodiments, the LUT storage is implemented with a Static RAM (SRAM) bank, where the exclusive-or operation may be performed within the SRAM bank. In other embodiments, the LUT storage is implemented as a flip-flop or a set of latches, with the exclusive-or operation implemented in separate logic. In some examples, the host processor 212 populates the LUT in the LUT storage by memory mapped instructions or at boot time as part of a POST sequence.

In the method of fig. 3, locally generating 204 the metadata of the modified data block includes generating the metadata using 306 the LUT. In some examples, a metadata generation unit local to remote execution device 214 iteratively generates error detection metadata (e.g., CRC values) by traversing the input data word and calculating the error detection metadata using the LUT. Consider an example in which an M-byte block of data is stored with CRC error detection metadata. To simulate host metadata generation for a block of data, for each N-byte data word starting at the most significant byte, (a) set the CRC register to '0', (b) exclusive-or the N-byte data with the CRC register, (c) read the LUT to find the bitwise remainder key and exclusive-or the key with the CRC register for each bit set in the N-byte data, and (d) set the CRC register to a value from (c). Upon completion, the contents of the CRC register represent locally generated metadata for the entire data block. The reader will appreciate that this calculation may be used to update stored metadata for only the modified portion of the data block (e.g., as a result of PIM operations). For example, in the case where only the two most significant bytes of a data block are modified, only the metadata of those N-byte data need be updated in the stored metadata. The reader will further appreciate that although a CRC code is used in the above examples, the same principles are applicable to any linear error detection code.

In the above example, the LUT has a maximum of m×n entries to store all possible checksum for CRC lookup. For 32 byte data words using CRC16 the overhead of this approach is 0.5KB of storage per PIM device (where PIM devices can be implemented per memory channel or per memory bank). In one implementation, the LUT may support multiple CRC lookups per cycle via multiple memory banks or read ports in order to maintain peak execution bandwidth for PIM computations. In another implementation, a hierarchy of LUT storage may be utilized, where each memory device channel has a global LUT storage with a specific number of entries, and each memory group (employing PIM) includes a local LUT storage with a smaller number of entries. For example, it may be assumed that the local LUT storage operates with an alternative strategy governed by the frequency of data encountered for PIM data within the memory bank.

For further explanation, FIG. 4 sets forth a flow chart illustrating an exemplary method for providing host-based error detection capability in a remote execution device according to the present disclosure. Similar to the exemplary method depicted in fig. 2, the exemplary method of fig. 4 further includes: modifying 202 the data block in response to verifying 200 the integrity of the stored data block in response to a host request for the data block stored in the memory device; locally generating 204 metadata for the modified data block; updating 206 the stored host-generated metadata of the data block with the locally-generated metadata of the modified data block; storing 208 the modified data block and the updated metadata; and transmitting 210 the modified data block and the updated metadata in response to receiving a request from the host to read the modified data block.

In the method of fig. 4, verifying 200 the integrity of the stored data block includes: the data block and stored host-generated metadata for the data block are read 402 in response to receiving a request from the host. In some examples, request 410 is a PIM request. In these examples, request 410 requires that data be read from the memory bank prior to performing PIM operations on the data. In these examples, the stored data blocks and metadata are read from the memory bank and loaded into a buffer or register prior to performing PIM operations. The stored metadata includes host-generated metadata of the data blocks. In some cases, the stored metadata may also include updated metadata generated locally (i.e., in the remote execution device 214) if the previous PIM operation has modified the data block.

In the method of fig. 4, verifying 200 the integrity of the stored data block further comprises: the stored host-generated metadata of the data block is verified 404 by emulating host metadata generation of the data block. In some examples, verifying 404 the stored host-generated metadata for a data block by emulating host metadata generation for the data block occurs by: metadata for the data block is regenerated in the remote execution device and the regenerated metadata is compared with stored metadata. This ensures that the data is not corrupted before the operation is performed. In some implementations, verifying 404 the stored metadata of the data block is performed by a metadata generation unit, while in other implementations, before performing the operation, "front-end" metadata regeneration and verification may be implemented in a component separate from metadata generation performed on the "back-end" after the operation has been completed. The simulated host metadata generation of the data block may be performed as described above with respect to the LUT. For example, verifying 404 the stored host-generated metadata of the data block is performed by the metadata generation unit by locally generating metadata of the data block using 406 the LUT. In some implementations, generating metadata for a data block using 406 a LUT is performed using LUT storage as described above.

Consider an example in which the metadata stored therein includes a CRC code for an M-byte data block. To simulate host metadata generation for a data block, (a) for each N-byte data word starting at the most significant byte, setting the CRC register to "0", (b) xoring the N-byte data with the CRC register, (c) for each bit set in the N-byte data, reading the LUT in the LUT storage to find a bitwise remainder key and xoring the key with the CRC register, and (d) setting the CRC register to a value from (c). Upon completion, the contents of the CRC register represent locally generated metadata for the entire data block. The locally generated metadata of the data block is then compared with the stored metadata of the data block to determine if the data has been corrupted before the operation is performed.

One major problem with performing metadata generation and verification within a remote execution device (e.g., PIM enabled memory device) is: memory bandwidth is potentially reduced due to the need for additional memory access cycles to perform metadata generation and validation. The CRC needs to be generated and checked once before PIM calculation (i.e., front end) and needs to be regenerated again after PIM calculation (i.e., back end). This may add several additional loops to the PIM computation, thereby reducing the overall throughput of the PIM computation, as it is bottleneck to metadata generation and validation. Furthermore, memory access latency will be higher if the host is waiting to access data operated on by the PIM. To address this problem, some implementations may utilize two layers of metadata verification on the front end. For example, a fast but low coverage error detection code may be employed to speed up overall front-end verification. Slower but high coverage codes for metadata verification (e.g., CRC verification) then provide more robust metadata verification. Initial verification using the fast code will allow PIM operation to proceed while the slower code is actively working to verify the data. For example, the fast code may be a simple interleaved parity check code capable of performing a minimum check on input data. In such an example, the memory device need not wait for metadata generation and verification to complete before performing PIM operations.

In some examples, if the results of metadata generation and verification are inconsistent with the results of the faster code error detection code, then the current PIM operation is squeezed and the data marked as corrupt. The host system may implement reactive measures. In these examples, to support rollback and recovery, the results of PIM computation are not stored back to memory banks or local registers until metadata generation and verification has verified the integrity of the data. In these examples, the results of PIM calculations are saved in a temporary buffer until front-end verification is complete. The temporary buffer may also act as a forwarding layer, where subsequent instructions that depend on these flight instructions obtain their operands directly from the buffer.

In some implementations, in the event that metadata cannot be recreated, even if the data is clean, the metadata along with its data may be explicitly marked as corrupt. For example, when marking the data as corrupt, a different signature may be used so that the host processor can distinguish between a real error and a false error due to the inability to generate metadata. In the event that metadata cannot be generated and marked as corrupt, the data may still be valid. The host processor may take reactive measures against data that did not successfully generate or recreate the metadata within the PIM device. In one example, if the host processor determines that the corrupted metadata is marked as PIM data, it may reactively populate the LUT with a checksum of each unique N-byte data in the cache line. The host processor may then attempt to replay the read request to try and see if the memory device can calculate the correct metadata with the most recently updated LUT. In another example, a threshold for the number of retries may be allowed before the host processor signals to allow the memory device to perform further actions. Data in a memory device that cannot generate metadata using available entries in the LUT storage should remain present in the memory device (not swapped) for a limited amount of time to allow the host processor to fill the LUT storage so that the memory device can attempt to calculate metadata.

For further explanation, FIG. 5 sets forth a flow chart illustrating an exemplary method for providing host-based error detection capability in a remote execution device according to the present disclosure. Similar to the exemplary method depicted in fig. 4, the exemplary method of fig. 5 further includes verifying 200 the integrity of the stored data block, including reading 402 the data block and the stored metadata of the data block, and verifying 404 the stored metadata of the data block by simulating host metadata generation of the data block; modifying 202 the data block; locally generating 204 metadata for the modified data block; updating 206 the stored host-generated metadata of the data block with the locally-generated metadata of the modified data block; storing 208 the modified data block and the updated metadata; and transmitting 210 the modified data block and the updated metadata in response to receiving a request from the host to read the modified data block.

The method of fig. 5 further comprises: the locally generated Error Correction Code (ECC) information for the stored data block and metadata is read 502 from an ECC storage device. In some implementations, the memory device 214 may implement ECC that is used by the memory device to verify data read from and written to its own memory bank. The ECC may be generated by an ECC generator/verifier (e.g., ECC module 128 in fig. 1). For example, ECC information such as check bits may be stored in an ECC storage device (e.g., 130). In some examples, reading 502 locally generated ECC information for the stored data block and metadata from the ECC storage device occurs by: ECC information for the stored data blocks and metadata is read from the check bit storage device.

The method of fig. 5 further comprises: the locally generated ECC is verified 504 before the data block is modified. In some examples, verifying 504 the locally generated ECC before modifying the data block is performed by: an ECC for the combined data block and metadata is calculated and the calculated ECC is compared with the ECC information read from the check bit storage. If the calculated ECC and the stored ECC match, then the local integrity of the stored data block and the stored metadata is intact.

In one embodiment, ECC may be used as a quick code by relying on the coverage provided by the ECC to allow operations to be performed without waiting for metadata verification to complete. Once the ECC verification signals that the data is clean, operation may continue. In another embodiment, the ECC verification and metadata verification are performed in parallel. Since the access delay includes ECC encoding/decoding, both ECC and metadata verification may be performed in parallel.

The method of fig. 5 further comprises: ECC information for the modified data block and the updated metadata is generated 506. In some examples, generating 506 ECC information for the modified data block and the updated metadata is performed by: the ECC generator/verifier calculates an ECC for the modified data block and the updated metadata.

The method of fig. 5 further comprises: the ECC information is stored 508 in an ECC storage device. In some examples, storing 508 the ECC information in the ECC storage is done by: ECC check bits for the modified data block and the updated metadata are stored in an ECC storage device.

In some cases, the ECC generator/verifier cannot proceed and cannot service subsequent requests for data in the corresponding address until generation of updated metadata for the modified data block has been completed. In some implementations, the back-end metadata generation of the modified data block may be omitted for sequences of operations that do not write data to memory but store intermediate results to a local register (e.g., register 122 in fig. 1). The forwarding network temporary buffer may provide data if a subsequent instruction requires a read access to the same address of the current instruction. In this case, ECC verification and metadata verification in the front end may be bypassed when this data forwarding occurs. If a subsequent instruction requires a write access to the same address or any other address, the memory device waits for the computation and backend metadata generation process to complete. Subsequent instructions may proceed if they only provide data from the local registers. Once metadata generation in the back-end is complete, the data is removed from the temporary buffer to make room for new data.

In some implementations, back-end metadata generation is fully pipelined and peak operation execution may be maintained provided that there is sufficient buffering to hold the modified data and its partially generated metadata. Throughput will be reduced if related operations are scheduled on the device while the metadata is being calculated.

In some implementations, metadata generation is expedited at the back end by generating a mask of bytes that change due to the operation and generating only metadata for the data. If only a portion of the data is modified, metadata for the modified portion of the data is generated by the proposed LUT algorithm. Metadata for PIM calculation data may be generated by xoring the output with the original data. The overhead of this approach may be between O1 and O (n), where "n" is the number of bytes modified.

In view of the foregoing disclosure, the reader will appreciate that a remote execution device such as a PIM device may, by using hints provided by the host for error detection metadata generation: 1) Verifying metadata generated by the host prior to performing the remote execution operation, and 2) generating error detection metadata for data modified by the remote execution operation in a manner emulating a host-based metadata generation function without copying host-based metadata generation logic to the remote execution device. The reader will also appreciate that a quick code such as in-memory ECC computation may be utilized to initially verify the integrity of the stored data to allow remote execution operations to be performed while the metadata of the stored data is regenerated and verified against the stored metadata. ECC verification and metadata regeneration/verification may be performed in parallel to improve performance. The reader will also appreciate that the results of the remote execution operations may be stored in a temporary buffer when verifying stored metadata and may be used by other remote execution operations that use the results in generating metadata for modified data. The error detection metadata for the modified data may be generated piece by piece, as the updated metadata is created only for the modified portion of the data, and not for the unmodified portion of the data. Thus, when the host reads data updated by the remote execution operation and generates metadata prompting for an update in the memory device using the metadata, the host will not distinguish between the metadata created by the host and the metadata generated in the memory.

Embodiments may be systems, apparatus, methods, and/or logic circuits. Computer readable program instructions in the present disclosure may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, c++, and the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. In some embodiments, electronic circuitry, including, for example, programmable logic circuitry, field Programmable Gate Array (FPGA), or Programmable Logic Array (PLA), can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and logic circuits according to some embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by logic circuits.

Logic circuitry may be implemented in a processor, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the processor, other programmable apparatus, or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and logic circuits according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

While the present disclosure has been particularly shown and described with reference to embodiments thereof, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the following claims. Accordingly, the embodiments described herein should be considered in descriptive sense only and not for purposes of limitation. The present disclosure is defined not by the detailed description but by the appended claims, and all differences within the scope will be construed as being included in the present disclosure.

Claims

1. A method of providing host-based error detection capability in a remote execution device, the method comprising:

modifying the data block;

locally generating metadata for the modified data block; and

host-generated metadata of the storage of the data block is updated with the locally generated metadata of the modified data block.

2. The method of claim 1, further comprising:

storing the modified data block and the updated metadata; and

the modified data block and the updated metadata are transmitted in response to receiving a request from the host to read the modified data block.

3. The method of claim 1, further comprising:

storing the data block and host-generated metadata for the data block; and

storing metadata received from a host to generate a look-up table (LUT);

wherein locally generating metadata for the modified data block includes generating metadata for the data block using the LUT.

4. The method of claim 1, further comprising:

reading the data block and the stored host-generated metadata of the data block in response to receiving a request from a host; and

verifying host-generated metadata of the storage of the data block by emulating host-metadata generation of the data block.

5. The method of claim 4, wherein validating the stored host-generated metadata of the data block by emulating host metadata generation of the data block comprises: metadata for the data block is generated locally using a look-up table (LUT).

6. The method of claim 5, wherein the data block is modified without waiting for verification of the stored host-generated metadata of the data block to complete.

7. The method of claim 4, further comprising:

reading locally generated Error Correction Code (ECC) information for the data block and the metadata from an ECC storage device; and

the locally generated ECC information is verified before the data block is modified.

8. The method of claim 7, further comprising:

generating ECC information for the modified data block and the updated metadata; and

the ECC information is stored in the ECC storage device.

9. The method of claim 1, wherein modifying the data block comprises modifying the data block by a in-memory Processing (PIM) device.

10. An apparatus, comprising:

a memory device; and

a PIM device coupled to the memory device, wherein the PIM device is configured to:

Modifying the data block;

locally generating metadata for the modified data block; and

11. The apparatus of claim 10, wherein the PIM device is further configured to:

storing the data block and host-generated metadata for the data block; and

storing metadata received from a host to generate a look-up table (LUT);

wherein locally generating metadata for the modified data block includes generating metadata using the LUT.

12. The apparatus of claim 10, wherein the PIM device is further configured to:

reading the data block and the stored host-generated metadata for the data block in response to receiving a PIM request from a host; and

13. The apparatus of claim 12, wherein validating the stored host-generated metadata of the data block by emulating host metadata generation of the data block comprises: metadata for the data block is generated locally using a look-up table (LUT).

14. The apparatus of claim 12, wherein the PIM device is further configured to:

15. The apparatus of claim 14, wherein the PIM device is further configured to:

the ECC information is stored in the ECC storage device.

16. A system, comprising:

a host processor device; and

a memory device coupled to the host processor device, wherein the memory device is configured to:

modifying the data block;

locally generating metadata for the modified data block; and

17. The system of claim 16, wherein the memory device is further configured to:

storing the data block and host-generated metadata for the data block; and

Storing metadata received from the host processor device to generate a look-up table (LUT);

wherein locally generating metadata for the modified data block includes generating the metadata using the LUT.

18. The system of claim 16, wherein the memory device is further configured to:

reading the data block and the stored host generated metadata for the data block in response to receiving a PIM request from the host processor device; and

19. The system of claim 18, wherein validating the stored host-generated metadata of the data block by emulating host metadata generation of the data block comprises: metadata for the data block is generated locally using a look-up table (LUT).

20. The system of claim 18, wherein the memory device is further configured to: