WO2014047382A1

WO2014047382A1 - Hardware integrity verification

Info

Publication number: WO2014047382A1
Application number: PCT/US2013/060816
Authority: WO
Inventors: David J. PIGNATELLI
Original assignee: Violin Memory Inc.
Priority date: 2012-09-24
Filing date: 2013-09-20
Publication date: 2014-03-27
Also published as: US20140089567A1; US10216625B2

Abstract

A flash memory management method and apparatus provides for the separation of the command and data paths so that communication paths may be used more efficiently. A unique sequence identifier is assigned to a write command and the associated data and association of the data and commands are validated prior to writing to the memory by comparing the unique sequence numbers of the data and command prior to executing the command. This comparison is performed after the data and command have traversed the communication paths.

Description

HARDWARE INTEGRITY VERIFICATION

FIELD OF THE INVENTION

[0001 ] This application may relate to the storage of data in a computer memory system.

BACKGROUND

[0002] A high performance Flash-based memory system may be designed with a very high degree of parallelism. By employing more parallel execution threads or paths, the overall performance can be very high even if the low-level devices are relatively slow.

[0003] An example of a memory system using these concepts is described in US 61/650,604, filed on May 23, 2012, entitled "FLASH MEMORY

CONTROLLER", which is commonly owned, and which is incorporated herein by reference.

[0004] In such a syst ;m, there may be numerous hazards to data integrity such as coherency corruption. Data path activity may be comprised of hundreds of data segments being dispatched to numerous low-level storage devices, each data segment with a distinct storage command. A small error in such a system can lead to user data loss: a catastrophic failure.

SUMMARY

[0005] A data transmission integrity checking scheme is described where each storage command and data block are tagged with a same, but otherwise unique, signature field allowing any subsequent processing function to validate the data storage or retrieval process. The data and the associated command may be transmitted at different times and over different data paths, and the correct association of the command with the data may be validated using the tag prior to execution of the command or prior to use of the data. [0006] An apparatus for storing digital data is described, having, a controller; a flash memory controller in communication with the controller; and a plurality of flash memory circuits in communication with the flash memory controller.

[0007] The controller is configured to associate a unique identifier with a write command and with an associated data chunk and the write command and the data chunk may be communicated to the flash memory controller by different transmission paths.

[0008] An error in association between a write command and a data chunk at the flash memory controller may be indicated by a mismatch between an unique identifier associated with the write command and an unique identifier associated with a data chunk. The error in association may be communicated to the processor, and the processor would retry the operation.

[0009] In an aspect, a write command and a data chunk are considered to be associated at the flash memory controller when the same unique identifier is found in the write command and the data chunk. The write command and the data chunk may be received at different times.

[0010] A method of managing a flash memory device is disclosed, comprising the steps of: providing a processor operable to manage a queue of read requests, write requests and data associated with the write requests; associating an unique sequence identifier with each write request and the corresponding data to be written; and transmitting the write request and the associated data to a flash memory interface using differing communication paths

[0011 ] Prior to executing a write request the unique sequence identifier of the write request and the unique sequence identifier of the associated data are compared to determine if the unique sequence identifier are the same unique sequence identifier; and, writing the data to the flash memory circuit if the unique sequence numbers are the same unique sequence number, or reporting an error condition to the processor if the unique sequence numbers are not the same unique sequence number.

[0012] In an aspect, the write data may be transmitted to the flash memory interface prior to transmission of the corresponding write command. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 is a diagram of a distributed data storage system where the data and commands travel over different paths in a system;

[0014] FIG. 2 is an illustration of an example the write command protocol timing; and ,

[0015] FIG. 3 is an illustration of an example of the read interface command protocol timing.

DESCRIPTION

[0016] Exemplary embodiments may be better understood with reference to the drawings, but these embodiments are not intended to be of a limiting nature. Like numbered elements in the same or different drawings perform equivalent functions. Elements may be either numbered or designated by acronyms, or both, and the choice between the representation is made merely for clarity, so that an element designated by a numeral, and the same element designated by an acronym or alphanumeric indicator should not be distinguished on that basis.

[0017] It will be appreciated that the methods described and the apparatus shown in the figures may be configured or embodied in machine-executable instructions, e.g., software, or in hardware, or in a combination of both. The machine-executable instructions can be used to cause a general-purpose computer, a special-purpose processor, such as a DSP or array processor, or the like, that acts on the instructions to perform functions described herein. Alternatively, the operations might be performed by specific hardware components that may have hardwired logic or firmware instructions for performing the operations described, or by any combination of programmed computer components and custom hardware components, which may include analog circuits.

[0018] The methods may be provided, at least in part, as a computer program product that may include a non-transient machine-readable medium having stored thereon instructions which may be used to program a computer (or other electronic devices) to perform the methods. For the purposes of this specification, the terms "machine-readable medium" shall be taken to include any medium that is capable of storing or encoding a sequence of instructions or data for execution by a computing machine or special-purpose hardware and that may cause the machine or special purpose hardware to perform any one of the methodologies or functions of the present invention. The term "machine-readable medium" shall, accordingly, be taken include, but not be limited to, solid-state memories, optical and magnetic disks, magnetic memories, and optical memories, as well as any equivalent device that may be developed and used for such purpose.

[0019] For example, but not by way of limitation, a machine readable medium may include read-only memory (ROM); random access memory (RAM) of all types (e.g., S-RAM, D-RAM. P-RAM); programmable read only memory (PROM); electronically alterable read only memory (EPROM); magnetic random access memory; magnetic disk storage media; FLASH memory, which may be NAND or NOR configured; memory resistors; or electrical, optical, acoustical data storage medium, or the like. A volatile memory device such as DRAM may be used to store the computer program product provided that the volatile memory device is part of a system having a power supply, and the power supply or a battery provides power to the circuit for the time period during which the computer program product is stored on the volatile memory device.

[0020] Furthermore, it is common in the art to speak of software, in one form or another (e.g., program, procedure, process, application, module, algorithm or logic), as taking an action or causing a result. Such expressions are merely a convenient way of saying that execution of the instructions of the software by a computer or equivalent device causes the processor of the computer or the equivalent device to be operable to perform an action or a produce a result, as is well known by persons skilled in the art.

[0021] To mitigate the effect of a data transmission or sequencing error, low- level integrity checking may be used, whereby a storage data chunk and a corresponding command for that storage data chunk includes an integrity checking component. For convenience this component is termed a "header" herein;

however, the "header" may be positioned in any predetermined or determinable location with respect to the storage data chunk or the command. In the case of writing data to a memory unit, the commands and the data may be transmitted from a memory controller which may be, for example, a RAID controller, to memory modules comprising the storage aspect of the memory system. A similar arrangement may be used for the return path when a READ command is issued, for ensuring the integrity of merging sequences of data being read from the memory modules by the RAID controller, although this may be omitted.

[0022] Where the data is stored in a memory module and the command is generated at a memory controller 5, the data and the command may traverse different physical paths (10, 20) between the memory controller 5 and the memory module 30. So, depending on the properties of the path that the data takes between the memory controller 5 and the memory module ("the data path") 10 and the path that the corresponding command takes between the memory controller 5 and the memory module ("the command path") 20, the correspondence between the particular command, which may have an associated physical or logical address for storing the data associated with the command and the data itself may lose coherence. This may occur for a number of reasons, including hardware failure, timing tolerances, or the like. But, such loss of coherence may also be a consequence of the execution of other functions along the data path or the command path that may interrupt the orderly sequential transmission of the data and the commands.

[0023] FIG. 1 shows the separate transmission of data and the corresponding commands associated with subsequent processing of the data from a controller to a FLASH interface (FI) of a memory module 30 which may store data or retrieve data from a plurality of FLASH memory circuits 50, which may be, for example, memory circuits (chips) or memory modules. Each of the commands and the associated data may be identified with a same specific sequence identifier (e.g., seq(y)), but transmitted over different paths. This transmission approach may be useful in subsuming various operations of the data transmission path (a data bus, for example) and routing functions which may be performed at intermediate points on the data bus, or to permit reordering of the data at the receiving memory module 30 so as to optimize the operation of the memory module 30 when considering the detailed design of the module.

[0024] Such optimization may include, for example, accommodating erase operations in one or more of a plurality of FLASH memory circuits 50 comprising the memory module, the interleaving of read and write operations, suppression of multiple consecutive writes to a fixed physical or logical address, or the like. Transmitting the commands and the data by separate paths (10, 20) may result, for example, in the command arriving before the associated data so that preprocessing operations may be completed prior to the arrival of the data. This may result in a reduced need for temporary storing (buffering) of the data. Such actions may also result in increased efficiency in the use of the internal bandwidth of the memory module.

[0025] A command generation module (a controller) 5 may generate commands, each command being tagged with a unique sequence number (seq ( )) 60-62 and a PHY 45 channel to which the command is being directed and the commands may be interleaved in time and space to meet up with the

corresponding data of the same sequence number prior to the data being written to the storage media. This process may also be used in returning data to the controller from the memory device. If there is a mismatch between the data and an associated command (that is, the sequence numbers seq( )) do not match), either the wrong data or corrupted data would being written or, alternatively, the correct data may be written to the wrong device or location. If such a mismatch is encountered, the memory module may be configured to abort the transfer and alert the controller system. FIG. 1 shows an example of a checking operation with a bad match 70. Queued data and commands may be flushed from the buffers, and transactions may be repeated to ensure correctness.

[0026] The apparatus may be comprised of separate elements such as a processor (controller), a bus, a FLI, which may comprise another processor and buffer memory (35, 40) and a PHY interface 45 to the FLASH memory circuits 50. Two or more of the elements may be combined in an ASIC, FPGA or other circuit where a portion of the control of such a circuit is by using a microcode. The PHY 45 may be realized as a state machine or as a general purpose processor, for example.

[0027] Write operations to each PHY 45 may occur substantially in parallel and one chunk of data, which may be, for example, a 1KB portion of a larger data extent such as a user data page, may be deposited into each PHY Tx FIFO (first-in first-out buffer) in a round- robin fashion. FIG. 2 illustrates the timing of operation of four PHYs. The chunk transfers may be interleaved over a bus where each PHY stream may be addressed to a different area of the FI (FLASH interface) or a different PHY. The first chunk of each stream may be marked with a SOP (start of page) indication.

[0028] In this example, three chunk transfers per channel are shown in FIG. 2. The last chunk of a sequence transmitted on each channel is marked with an EOP. Following the last chunk transfer of each channel, the Flash write command is transmitted over the Command interface (c data). A FLI Tx Command Ready indicator may be momentarily de-asserted while the command is being processed. The PHY may remain busy while a new command can be transferred, allowing some degree of data and command pipelining. With N channels (PHY) it may be possible to have each channel actively transferring data to a FLASH device, while a new command (with its data) is being prepared for execution.

[0029] As shown in FIG. 1, there are data caching operations and flash command operations for each FLASH interface, creating a task buffer of activity for the next write event (write window). The task buffer may be in two parts: data, and commands. When the write window starts, a data movement engine may transfer data for all PHYs in parallel; interleaved at approximately the chunk level. When a sufficient number of chunks have been transferred, operational commands are dispatched to each PHY and the cycle repeats. In the FLASH interface (FLI), the data and commands that have been received over separate paths are joined prior interacting with a physical FLASH device. The integrity checking is intended to prevent a mishap where the data and command streams for any sequence become time skewed or otherwise corrupted. If the data were sent to the wrong PHY at the wrong address, the result would be a loss of data. When mismatches are detected, any data or command issuance to the affected PHY ceases, and a management event is required to flush the pipeline and recover.

[0030] FLASH reads may be considered as divided into two parts; the actual FLASH Read, and the Read Data Transfer. This may be done so that the PHY may access a plurality of die or chips or other memory circuits connected to it and deposit Read commands substantially in parallel. If each Read command were to include performing a return data transfer, then the PHY may not be able to arrange many read access times in parallel, in this example, however, once all of the FLASH die have been activated, the transfer of the read data can commence after a polling operation indicates that requested read data has become available.

[0031] Referring to FIG. 3, four PHY channels are shown. The command ready signals may be asserted and the FI may load the FLASH Read commands. There can be more than one FLASH Read command issued by a PHY interface (for example, two die, a and b, may be accessed by a PHY). There can be a plurality of polling cycles to determine when the read data is available to transfer from the die. Then Read Transfer (Xfer) commands may be issued by the PHY and the data moved across the PHY DQ bus. As the data is read into the controller, and chunks are collected, a BCH context from each Read Command may be inserted into the data stream and the complete chunk is processed through a BCH decoder.

[0032] In a read operation, the function of integrity checking may be relaxed but some degree of validity checking may be performed. Each command context may be matched to the physical channel into which it was directed. Further, the incoming FLASH read chunk may be marked with the associated command Sequence number (seq( )) and BCH status results. The header can be used to route the data to the correct read context.

[0033] Although only a few exemplary embodiments of this invention have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of the invention. Accordingly, all such modifications are intended to be included within the scope of this invention.

Claims

WHAT IS CLAIMED IS:

1. An apparatus for storing digital data, comprising,

a controller;

a flash memory controller in communication with the controller; and a plurality of flash memory circuits in communication with the flash memory controller;

wherein the controller is configured to associate a unique identifier with each of a write command and an associated data chunk and the write command and the data chunk are communicated to the flash memory controller by different transmission paths.

2. The apparatus of claim 1 , wherein an error in association between the write command the data chunk at the flash memory controller is indicated by a mismatch between the unique identifier associated with the write command and the unique identifier associated with the data chunk.

3. The apparatus of claim 2, wherein the error in association is communicated to the processor and the processor retries the operation.

4. The apparatus of claim 1 , wherein a write command and a data chunk are considered to be associated at the flash memory controller when the same unique identifier is found.

5. The apparatus of claim 4, wherein the write command and the data chunk are received at different times.

6. The apparatus of claim 4, wherein data associated with a command at the flash memory controller is processed in accordance with the command.

7. A method of managing a flash memory device, comprising:

providing a processor operable to manage a queue of read requests, write requests and data associated with write requests; associating a unique sequence identifier with each write request and the corresponding data to be written;

transmitting the write request and the corresponding data to a flash memory interface using differing communication paths; and

prior to executing a write request:

comparing the unique sequence identifier of the write request with the unique sequence identifier of the corresponding data to determine if the unique sequence identifier are the same unique sequence identifier, and:

writing the corresponding data to the flash memory circuit if the unique sequence numbers are the same unique sequence number, or

reporting an error condition to the processor if the unique sequence numbers are not the same unique sequence number.

8. The method of claim 7, wherein the corresponding write data is transmitted to the flash memory interface prior to transmission of the write request.

9. A computer program product, stored on a non-transient computer readable medium, comprising:

executable instructions configuring a first processor to: manage a queue of read requests, write requests and data associated with write requests;

associate a unique sequence identifier with each write request and the corresponding data to be written;

transmit the write request and the corresponding data to a flash memory interface using differing communication paths; and executable instructions configuring a second processor to:

compare the unique sequence identifier of the write request with the unique sequence identifier of the corresponding data to determine if the unique sequence identifier are the same unique sequence identifier, and:

write the corresponding data to the flash memory circuit if the unique sequence numbers are the same unique sequence number, or

report an error condition to the first processor if the unique sequence numbers are not the same unique sequence number.