CN117369718A - System, method and apparatus for forming and selecting recovery units in storage devices - Google Patents

Abstract

Systems, methods, and apparatus for forming and selecting recovery units in storage devices are disclosed. The storage device may include: at least one storage medium; and a controller configured to: receiving a write command, wherein the write command specifies data and a reclamation unit handle; receiving information about the data; and storing data in the reclamation unit of the at least one storage medium based on the reclamation unit handle and the information. The information may include access information. The information may include error tolerance information. The information may include data attribute information. The information may include data type information. The controller may be configured to determine the information based at least in part on the reclamation unit handle. The controller may be configured to receive an indicator separate from the reclamation unit handle; and determining the information based at least in part on the indicator.

Description

System, method and apparatus for forming and selecting recovery units in storage devices

The present application claims priority and benefit from U.S. provisional patent application Ser. No. 63/421,994, filed on Ser. No. 11/2/2022, U.S. provisional patent application Ser. No. 63/358,861, filed on Ser. No. 7/6/2022, and U.S. provisional patent application Ser. No. 63/419,699, filed on Ser. No. 10/2022, the entire disclosures of which are incorporated herein by reference.

Technical Field

The present disclosure relates generally to storage devices and, more particularly, to systems, methods, and apparatus for forming and selecting reclamation units in storage devices.

Background

A storage device, such as a Solid State Drive (SSD), may store data in a storage medium that may be implemented using non-volatile memory (NVM). In some non-volatile memories, data may be updated by erasing the memory in which the data is stored and rewriting new data in the erased memory. Some nonvolatile memories may be written and/or read in units of pages, but erased in units of blocks that may include a plurality of pages. Accordingly, in order to update data stored in a page of a nonvolatile memory, valid data stored in other pages in the same block may be copied to a different block to prevent loss of valid data when the block is erased.

The above information disclosed in this background section is only for enhancement of understanding of the background of the invention principles and therefore it may contain information that does not form the prior art.

Disclosure of Invention

A storage device may include: at least one storage medium; and a controller configured to: receiving a write command, wherein the write command specifies data and a reclamation unit handle; receiving information about the data; and storing data in the reclamation unit of the at least one storage medium based on the reclamation unit handle and the information. The information may include access information. The information may include error tolerance information. The information may include data attribute information. The information may include data type information. The controller may be configured to determine the information based at least in part on the reclamation unit handle. The controller may be configured to receive an indicator separate from the reclamation unit handle; and determining the information based at least in part on the indicator. The controller may be configured to select the recycling unit based on the information. The controller may be configured to select the recovery unit based on a characteristic of at least a portion of the recovery unit. The characteristics may include a number of programming cycles. The characteristics may include error accumulation characteristics. The controller may be configured to compose a reclamation unit based on the information. The controller may be configured to compose the recovery unit based on characteristics of at least a portion of the recovery unit. The characteristics may include a number of programming cycles. The characteristics may include error accumulation characteristics.

A device may include placement logic configured to: transmitting a write command to the storage device, wherein the write command specifies data and a reclamation unit handle, wherein the reclamation unit handle is usable to reference a reclamation unit in at least one storage medium of the storage device; and transmits information about the data to the storage device. The placement logic may be configured to determine the information based on the operation of the data source. The operation may include a write operation. The placement logic may be configured to determine the information based on one or more compression operations. The placement logic may be configured to determine the information based on one or more data management schemes. The information may include access information. The information may include error tolerance information. The placement logic may be configured to send the information based at least in part on the reclamation unit handle. The placement logic may be configured to send the information at least in part using an indicator that may be separate from the reclamation unit handle.

A method may include receiving a write command at a storage device, wherein the write command specifies data and a reclamation unit handle; receiving information about the data at the storage device; and storing the data in a reclamation unit of at least one storage medium of the storage device based on the reclamation unit handle and the information store. The information may include access information. The information may include error tolerance information. The method may also include selecting a reclamation unit based on the information. The selection may be based on characteristics of at least a portion of the recovery unit. The method may also include composing a reclamation unit based on the information. The composition may be based on characteristics of at least a portion of the recovery unit.

A method may include: receiving a data operation request at a storage device, wherein the data operation request specifies data and a reclamation unit handle; receiving information about the data at the storage device; and performing a data operation associated with a reclamation unit of at least one storage medium of the storage device based on the data operation request, the reclamation unit handle, and the information. The data operation request may include a write operation request, and the data operation may include a write operation. The write operation request may include a write command, and the write operation may include storing data in the reclamation unit based on the reclamation unit handle and the information.

Drawings

The figures are not necessarily to scale and elements of similar structure or function are generally represented by like reference numerals or parts thereof throughout the figures for illustrative purposes. The drawings are only intended to facilitate the description of the various embodiments described herein. The drawings do not depict every aspect of the teachings disclosed herein and do not limit the scope of the claims. To prevent the figures from becoming unclear, not all components, connections, etc. may be shown and not all components may have reference numerals. However, the mode of the component configuration can be readily made clear from the drawings. The accompanying drawings illustrate example embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure.

Fig. 1A illustrates an embodiment of a data placement scheme for a storage device in a first data placement state according to a disclosed example embodiment.

Fig. 1B illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a second data placement state according to a disclosed example embodiment.

Fig. 1C illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a third data placement state according to a disclosed example embodiment.

Fig. 1D illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a fourth data placement state according to a disclosed example embodiment.

Fig. 1E illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a fifth data placement state according to a disclosed example embodiment.

Fig. 2A illustrates an embodiment of a flexible data placement scheme for a storage device in a first data placement state according to an example embodiment of the disclosure.

Fig. 2B illustrates an embodiment of the data placement scheme illustrated in fig. 2A in a second data placement state according to a disclosed example embodiment.

Fig. 2C illustrates an embodiment of the data placement scheme illustrated in fig. 2A in a third data placement state according to a disclosed example embodiment.

Fig. 2D illustrates an embodiment of the data placement scheme illustrated in fig. 2A in a fourth data placement state according to a disclosed example embodiment.

FIG. 3A illustrates an embodiment of a flexible data placement scheme with reference modification in a first data placement state, according to an example embodiment of the disclosure.

Fig. 3B illustrates an embodiment of the data placement scheme illustrated in fig. 3A in a second data placement state according to a disclosed example embodiment.

Fig. 3C illustrates an embodiment of the data placement scheme illustrated in fig. 3A in a third data placement state according to a disclosed example embodiment.

Fig. 3D illustrates an embodiment of the data placement scheme illustrated in fig. 3A in a fourth data placement state according to a disclosed example embodiment.

Fig. 4A illustrates an embodiment of an update operation for a flexible data placement scheme in a first state, according to a disclosed example embodiment.

Fig. 4B illustrates an embodiment of the update operation illustrated in fig. 4A in a second state according to a disclosed example embodiment.

FIG. 5 illustrates an embodiment of a flexible data placement scheme with reclamation unit handles according to an example embodiment of the disclosure.

Fig. 6 illustrates another embodiment of a flexible data placement scheme according to an example embodiment of the disclosure.

FIG. 7 illustrates a first embodiment of a technique for selecting erase blocks for one or more reclamation for a unit flexible data placement scheme in accordance with a disclosed example embodiment.

FIG. 8 illustrates a second embodiment of a technique for selecting erase blocks for one or more reclamation for a unit flexible data placement scheme in accordance with a disclosed example embodiment.

FIG. 9 illustrates an embodiment of a storage system implementing a flexible data placement scheme in accordance with a disclosed example embodiment.

FIG. 10 illustrates an embodiment of an initial isolation scheme for a flexible data placement scheme for a storage device according to an example embodiment of the disclosure.

FIG. 11 illustrates an embodiment of a persistence isolation scheme for a flexible data placement scheme for a storage device in accordance with a disclosed example embodiment.

FIG. 12 illustrates an example embodiment of a host device that can be used to implement any of the host functions disclosed herein, in accordance with the example embodiments disclosed.

FIG. 13 illustrates an example embodiment of a storage device that may be used to implement any of the storage device functions disclosed herein, in accordance with the example embodiments disclosed.

FIG. 14 illustrates an embodiment of a method for flexible data placement of a storage device according to a disclosed example embodiment.

Detailed Description

The storage device may implement a Flexible Data Placement (FDP) scheme that may enable a host to arrange data into one or more physical Reclamation Units (RUs) in the storage device. Reclamation of cells may be implemented using a portion of the physical storage medium (e.g., one or more erase blocks) that may be erased as a cell. This may reduce write amplification, for example, by enabling the host to place data that may be deallocated simultaneously in the same reclamation unit.

In a flexible data placement scheme, the reclamation unit may appear to the host as a logical representation of the storage medium. For example, the host may use the reclamation unit handle to reference one or more reclamation units to which the storage may write data. The storage device may select the actual physical reclamation unit referenced by the reclamation unit handle (reclaim unit handle) and/or the storage medium used to implement the physical reclamation unit. For example, the storage device may select the physical storage medium using a round robin technique, a random selection technique, or a technique that always selects one or more erase blocks with the lowest number of program and/or erase (P/E) cycles for the next reclamation unit. However, these techniques may not be able to utilize information about the data that may help the storage device select a more beneficial physical storage medium for the reclamation unit in which the data is stored.

In a flexible data placement scheme according to disclosed example embodiments, a storage device may receive (e.g., from a host) information about data to be stored in a reclamation unit referenced by a reclamation unit handle. The storage device may use this information to select and/or compose a physical reclamation unit to store the data. Depending on implementation details, this may improve or optimize the usage of different reclamation units, erase blocks, etc. in the storage device.

The inventive principles encompass various types of information that may be provided to a storage device. For example, the storage device may be provided with access information regarding data to be stored in the device. The access information may include, for example: how fast data has been written, read, etc.; how long the data is expected to be overwritten, read, etc.; how often data has been or is expected to be written, read, etc.; and/or other access information. As another example, the storage device may be provided with information regarding one or more acceptable characteristics of a storage medium used to implement the reclamation unit for data. Acceptable characteristics may include: error tolerance) information (such as an acceptable Bit Error Rate (BER) that may be specified based on, for example, a length of time data is stored and a number of read operations, re-write operations (e.g., refresh operations), data movement operations (e.g., garbage Collection (GC) operations), etc., a number of remaining P/E cycles, and/or other acceptable characteristics of the storage medium). As another example, information regarding one or more attributes of data to be stored in the device may be provided to the storage device. The attribute information may include: service Level Agreement (SLA) information; quality of service (QoS) information; access delay information; access bandwidth information, etc. As another example, information regarding the type and/or usage of data to be stored, such as operating system metadata, file system tables, and the like, may be provided to the storage device.

The inventive principles contemplate a variety of techniques for providing information to a storage device regarding data to be stored at the device. For example, the information may be provided implicitly (e.g., data stored in reclamation units referenced by different predefined reclamation unit handle numbers may have different access frequencies based on the reclamation unit handle numbers) and/or explicitly (e.g., the storage may be provided with fields, extensions, etc. that may provide explicit information about the data). As another example, information about the data may be provided persistently (e.g., the information of the data in the sequence of write commands may be the same until the information is updated) and/or on a separate basis (e.g., each write command may be used to provide information for the data). As another example, information about the data to be stored at the storage device may be provided to the storage device using any communication technique including: an extension that is a reclamation unit handle (e.g., an indication of reclamation unit handle plus access information); implicitly in the reclamation unit handle number; setting a storage device controller; a namespace attribute; using a request for an update operation to the reclamation unit handle; configuration settings as a flexible data placement scheme; in the field of input and/or output (I/O or IO) commands; in the field of the management command; as attributes, settings, etc. of a non-volatile memory (NVM) subsystem, controller, namespace, etc.

The inventive principles contemplate a variety of techniques for a storage device to use information about data to select and/or compose one or more reclamation units to store the data. For example, if the information indicates that data may be written and/or rewritten frequently (which may be referred to as hot data), the storage device may select and/or compose a reclamation unit having a storage medium that may have a relatively large number of P/E cycles remaining in its expected lifetime. As another example, if the information indicates that the data has a relatively high margin of error, for example, because the data may be stored in the storage device as part of a redundant storage scheme (e.g., a redundant array of independent drives RAID scheme), the storage device may select and/or compose a reclamation unit having a storage medium that may have a relatively small number of P/E cycles remaining over its expected lifetime. As another example, the storage device may use information about the data to be stored at the device to select and/or compose a reclamation unit based on one or more of the following properties of the storage medium at the drive: wafer production information, erase block locations on a storage medium die (die), voltage and/or speed characteristics of read, write (program) and/or erase operations, bit error accumulation rate, current and/or historical temperature exposure of the storage medium and/or adjacent medium, etc., access activity, etc.

The inventive principles encompass a variety of techniques for identifying, collecting, processing, etc. (e.g., by a host) information to be sent to a storage device (with respect to which data is to be stored at the storage device). For example, the host may observe that data written by a user (e.g., an operating system, applications, processes, etc.) may be, or may be, continuously used and/or may have been stored, possibly stored for a relatively long time, and may send this information to the storage device, e.g., along with a write request. As another example, the host may observe that data written by the user may be compressed (e.g., garbage collected) at the file system level, database level, etc., and send this information to the storage device, e.g., along with the write request. As another example, the host may determine the number of times data has been compressed (e.g., garbage collected) and send this information to the storage device along with the data to be stored at the device. As another example, the host may characterize data usage based on one or more data management schemes (such as one or more cache replacement policies, a tiered storage scheme, a multi-tiered storage architecture, etc.), and send this information to the storage device along with the data to be stored at the device.

In some embodiments, based on the information about the data, the storage device may select one or more existing reclamation units for the reclamation unit handle, compose one or more reclamation units for future write requests, compose one or more reclamation units for current write requests, and so forth.

The disclosure encompasses various inventive principles relating to flexible data placement. The principles disclosed herein may have independent utility and may be embodied separately and not every embodiment may utilize every principle. Furthermore, these principles may be embodied in various combinations, some of which may amplify some of the benefits of the various principles in a synergistic manner.

For purposes of illustration, some embodiments may be described in the context of particular implementation details, such as storage devices implemented with Solid State Drives (SSDs) using non-and (NAND) flash memory, nonvolatile memory express (NVMe) protocols, and the like. However, the inventive principles are not limited to these or any other implementation details. For example, some embodiments may implement a storage medium having flash memory, magnetic media, storage Class Memory (SCM), etc., or any combination thereof.

In some embodiments where the storage medium may be implemented at least in part using flash memory, the reclamation unit may represent one or more erase blocks, NVM devices (e.g., NVM dies), etc., or any combination thereof, and the reclamation group may represent one or more reclamation units, one or more NVM device partitions (e.g., planes), one or more NVM devices (e.g., NVM dies), one or more storage devices (e.g., storage drives), etc., or any combination thereof.

In some embodiments, where the storage medium may be implemented at least in part using a magnetic medium (e.g., a Shingled Magnetic Recording (SMR) medium), the recovery unit may represent one or more shingled portions, areas, sectors, tracks, etc., or any combination thereof, and the recovery group may represent one or more disks (e.g., drives), platters, tracks, areas, sectors, shingled portions, etc., or any combination thereof.

In some embodiments, where the storage medium may be at least partially implemented with a storage class memory (e.g., magnetoresistive Random Access Memory (MRAM), resistive random access memory (ReRAM), phase Change Memory (PCM), cross-grid nonvolatile memory, memory with bulk resistance change, etc.), the reclamation unit may represent one or more memory banks, programming groups, etc., or any combination thereof, and the reclamation group may represent one or more dies, memory banks, programming groups, etc., or any combination thereof.

As another example, the inventive principles are not limited to use with write commands, but rather may be applied to any type of request for data operations involving data locality, e.g., in any type of scheme involving data that may be placed based on a logical-to-physical mapping (e.g., a logical representation of a physical storage medium such as a logical block address to physical block address (LBA to PBA or L2P) mapping). For example, the storage device may receive any type of data operation request (such as a request for a write operation (e.g., write command, write zero, write one, write uncorrectable, etc.), a copy operation, a deallocation operation (e.g., demap, trim, etc.), a cleanup operation, an erase operation, a format operation, a compare and write operation (which may be referred to as a fuse command, e.g., a pair of commands that may include a read command to read data for the compare operation and a write command that may be performed based on the result of the compare operation), an update operation, etc. In some embodiments, the data operation request may include an indication (e.g., reclamation unit handle, reclamation group, etc.) related to the locality of data in the storage medium. Based on receiving a data operation request having an indication related to data locality in a storage medium, a storage device may perform a corresponding data operation based on information related to data locality in the storage medium and/or information related to data to be used in association with the data operation request (e.g., data to be stored based on a write command).

Fig. 1A illustrates an embodiment of a data placement scheme for a storage device in a first data placement state according to a disclosed example embodiment. Fig. 1B illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a second data placement state according to a disclosed example embodiment. Fig. 1C illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a third data placement state according to a disclosed example embodiment. Fig. 1D illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a fourth data placement state according to a disclosed example embodiment. Fig. 1E illustrates an embodiment of the data placement scheme illustrated in fig. 1A in a fifth data placement state according to a disclosed example embodiment.

The embodiments shown in fig. 1A-1E may include a host 102 and a storage 104 that communicate using a communication connection 105. The storage 104 may include a storage medium in an NVM subsystem 106, the NVM subsystem 106 having memory blocks 108 arranged in super blocks 110. The memory block 108 may be implemented using a nonvolatile memory that may be erased in units of blocks illustrated in fig. 1A through 1E, and thus may also be referred to as an erase block. The memory block 108 may be programmed (e.g., written) and/or read in units of pages and/or word lines that may be smaller than the memory block. Memory block 108 may be erased prior to writing pages and/or word lines of data to memory block 108. For example, the memory blocks 108 may be arranged in super blocks 110 to simplify management of the blocks 108. Accordingly, the nonvolatile memory shown in fig. 1A to 1E may be erased in units of the super block 110.

Storage 104 may receive input and/or output (I/O or IO) requests (which may also be referred to as commands) from host 102 to enable access to NVM subsystems (e.g., writing data to and/or reading data from storage media). The host may divide the data into namespaces indicated by different types of shading shown in fig. 1A-1E. In particular, data belonging to a first namespace (data that may be referenced by namespace identifier 1 (NSID 1)) may be indicated by a shadow having a diagonal from top right to bottom left, data belonging to a second namespace (data that may be referenced by NSID 2) may be indicated by a shadow having a diagonal from top left to bottom right, and data belonging to a third namespace (data that may be referenced by NSID 3) may be indicated by a diagonal cross-hatching. In some embodiments, in addition to and/or in lieu of a namespace, host 102 can divide and/or arrange data into groups based on Logical Block Addresses (LBAs), applications that can use data, host write traffic (traffic) threads, and the like.

Memory block 108 may initially be in an erased state, as indicated by the absence of shading. Prior to receiving a write command, NVM subsystem 106 may select an erased super block (e.g., super block 0 indicated by solid shading) in which write data may be placed. The erased superblock may be selected randomly or using a round robin technique. Thus, memory block 108a may be initially empty before NVM subsystem 106 receives a write command.

Referring to fig. 1A, the storage device 104 may receive a first write command with write data belonging to NSID 1. NVM subsystem 106 may place data belonging to NSID 1 (indicated by numeral 1 in the rounded rectangle) in a first memory block 108a of superblock 0 as shown in fig. 1A. (a number 1 may indicate a sequence of data that may be placed instead of NSID).

Referring to FIG. 1B, the storage device 104 may continue to receive additional write commands from the host 102 with write data belonging to various namespaces. NVM subsystem 106 may fill memory blocks 108a-108d in superblock 0 by placing write data in a sequence indicated by numbers in rounded rectangles. In some cases, the amount of write data may be greater than the remaining space in the memory block. Thus, a first portion of the write data may be placed so as to fill the remaining space in the first memory block, and a second portion may be placed in an empty memory block. For example, as shown in fig. 1B, based on receiving the second write command, the first portion 2a of data belonging to NSID 2 may be used to fill the remaining space in memory block 108a, and the second portion 2B of data belonging to NSID 2 may be placed in memory block 108B. As shown in fig. 1B, NVM subsystem 106 may continue to place write data in superblock 0 until superblock 0 is full or nearly full.

Referring to FIG. 1C, storage 104 may receive a sixth write command with data belonging to NSID 3 that is too large to fit in the remaining space in memory block 108d in superblock 0. Thus, NVM subsystem 106 may select a new empty superblock (e.g., superblock 2 indicated by solid shading) in which write data may be placed. NVM subsystem 106 may fill the remaining space in memory block 108d with a first portion 6a of data belonging to NSID 3 and a second portion 6b of data belonging to NSID 3 may be placed in memory block 108e in superblock 2. NVM subsystem 106 may continue to place write data in superblock 2 and then superblock 1, with data belonging to different namespaces mixed within one or more superblocks as shown in fig. 1D. Data belonging to different namespaces may be mixed within block 108 and/or superblock 110, for example, because host 102 may not be aware of the manner in which NVM subsystem 106 places data within block 108 and/or superblock 110 and/or have no control over the manner in which NVM subsystem 106 places data within block 108 and/or superblock 110.

The host 102 may partition data into namespaces, for example, to provide isolation between data sources such as applications, processes, logical Block Address (LBA) ranges, and the like. Thus, for example, when an application terminates, host 102 may concurrently deallocate some or all of the data belonging to the namespace.

Fig. 1E illustrates an example where host 102 has deallocated data belonging to a namespace indicated as NSID 3, which is shown using solid shading after deallocation. Reusing the deallocated memory may involve erasing the deallocated memory. However, because superblock 110 may be erased as a unit, NVM subsystem 106 may move the remaining valid data in the superblock to a different superblock to prevent losing valid data when the superblock is erased. Depending on implementation details, this may lead to write amplification that may reduce the lifetime of the storage medium.

Fig. 2A illustrates an embodiment of a flexible data placement scheme for a storage device in a first data placement state according to an example embodiment of the disclosure. Fig. 2B illustrates an embodiment of the data placement scheme illustrated in fig. 2A in a second data placement state according to a disclosed example embodiment. Fig. 2C illustrates an embodiment of the data placement scheme illustrated in fig. 2A in a third data placement state according to a disclosed example embodiment. Fig. 2D illustrates an embodiment of the data placement scheme illustrated in fig. 2A in a fourth data placement state according to a disclosed example embodiment.

The embodiments shown in fig. 2A-2D may include some components (such as host 202, storage 204, communication connection 205, and/or memory block 208) that may operate in some aspects in a manner similar to corresponding components in the embodiments shown in fig. 1A-1E. However, in the embodiment shown in fig. 2A-2D, the memory blocks 208 in NVM subsystem 212 may be arranged in one or more reclamation units 214, and reclamation units 214 may be erased as a unit and may include one or more memory blocks 208. The reclamation unit 214 may be implemented, for example, using one or more erase blocks, superblocks, or the like. Further, the reclamation units 214 may be identified by corresponding reclamation unit handles 236, which reclamation unit handles 236 may enable the host 202 to specify (e.g., in a field of a write command) the particular reclamation unit 214 used to store the write data associated with the write command. In some embodiments, some or all of the reclamation units 214 may be arranged in one or more reclamation groups 218 that the host may also specify, for example, using one or more corresponding reclamation group identifiers (e.g., in a field of a write command). In some embodiments, the reclamation unit handle 236 may identify more than one reclamation unit 216. For example, the reclamation unit handle 236 may identify (e.g., map to) one or more reclamation units 216 in one or more (e.g., each) reclamation group 218 (e.g., in a manner similar to that shown in fig. 5).

Thus, by designating a particular reclamation unit 214 and/or reclamation group 218 for storing data associated with a write command, host 202 can cause NVM subsystem 212 to store only data belonging to one or more particular namespaces in one or more reclamation units 214 and/or reclamation groups 218.

For example, referring to fig. 2A, host 202 may instruct NVM subsystem 212 to store data belonging to namespaces indicated by NSID 1, NSID 2, and NSID 3 in reclamation unit 0, reclamation unit 1, and reclamation unit 2, respectively, in the order indicated by the numbers in the rounded rectangles in fig. 2A-2D and by the numbers in brackets below. For example, data [1] of the first write command associated with NSID 1 may be written to a first portion of the top memory block 208 of reclamation unit 0 of reclamation group 0. The first portion of data [2a ] of the second write command associated with NSID 1 may be written into the remaining portion of the top memory block 208 of reclamation unit 0 of reclamation group 0, and the second portion of data [2b ] of the second write command associated with NSID 1 may be written into the first portion of the second memory block 208 following the top of reclamation unit 0 of reclamation group 0. Data [3] of the third write command associated with NSID 2 may be written to the first portion of the top memory block 208 of the reclamation unit 1 of reclamation group 0. Data [4] of the fourth write command associated with NSID 3 may be written to the first portion of the top memory block 208 of reclamation unit 0 of reclamation group 1. Data [5] of the fifth write command associated with NSID 3 may be written to the next portion of the top memory block 208 of reclamation unit 0 of reclamation group 1. Data [6] of the sixth write command associated with NSID 1 may be written to the next portion of the second memory block 208 after the top of reclamation unit 0 of reclamation group 0. The first portion of data [7a ] of the seventh write command associated with NSID 1 may be from the remaining portion of the second memory block 208 after being written to the top of reclamation unit 0 of reclamation group 0, and the second portion of data [7b ] of the seventh write command associated with NSID 1 may be written to the first portion of the third memory block 208 after being written to the top of reclamation unit 0 of reclamation group 0. The first portion of data [8a ] of the eighth write command associated with NSID 2 may be written into the remaining portion of the top memory block 208 of the reclamation unit 1 of reclamation group 0 and the second portion of data [8b ] of the eighth write command associated with NSID 2 may be written into the first portion of the second memory block 208 following the top of the reclamation unit 1 of reclamation group 0. A first portion of the data of the ninth write command associated with NSID 3 may be written into the remaining portion of the top memory block 208 of the reclamation unit 0 of reclamation group 1 and a second portion of the data of the ninth write command associated with NSID 3 [9b ] may be written into the first portion of the second memory block 208 following the top of the reclamation unit 0 of reclamation group 1.

Referring to fig. 2B, host 202 may continue to send write commands and associated write data, which may instruct NVM subsystem 212 to store write data in reclamation units corresponding to respective namespaces using write commands in a sequence indicated by numbers in rounded rectangles, which may correspond to the data of the associated write commands.

Referring to fig. 2C, as shown by the solid shading, the host may deallocate some or all of the data belonging to NSID 1. For example, data associated with any of the first, second, sixth, seventh, thirteenth, fourteenth, and/or seventeenth write commands may be deallocated at different times (with various amounts of time therebetween), in various combinations, or all at once at different times. Depending on implementation details, this may enable NVM subsystem 212 to erase reclamation unit 0 (as shown without shading in fig. 2D) without moving data belonging to other namespaces, thereby reducing or eliminating write amplification. Although FIG. 2D may illustrate reclamation unit handle 236 referencing reclamation unit 214 having one or more memory blocks 208 that may be erased, in some embodiments, one or more deallocated memory blocks 208 may be returned to the available memory pool before being erased. Thus, in some embodiments, the memory block 208 indicated as erased in fig. 2D may not be the same memory block indicated as deallocated in fig. 2C, but may be another memory block that has been previously erased (e.g., when the memory block is in a free block pool and/or is not referenced by the reclamation unit handle 236) and arranged into a new (e.g., empty) reclamation unit 214. In some embodiments, data may be partitioned and/or arranged into groups based on Logical Block Addresses (LBAs), applications that may use the data, host write traffic threads, and the like, in addition to and/or in lieu of namespaces.

FIG. 3A illustrates an embodiment of a flexible data placement scheme with reference modification in a first data placement state, according to an example embodiment of the disclosure. Fig. 3B illustrates an embodiment of the data placement scheme illustrated in fig. 3A in a second data placement state according to a disclosed example embodiment. Fig. 3C illustrates an embodiment of the data placement scheme illustrated in fig. 3A in a third data placement state according to a disclosed example embodiment. Fig. 3D illustrates an embodiment of the data placement scheme illustrated in fig. 3A in a fourth data placement state according to a disclosed example embodiment.

The embodiment shown in fig. 3A-3D may include a host 302 and a storage device 304 that communicate using a communication connection 305. The storage device 304 may include a controller 326 (the controller 326 may control the overall operation of the storage device 304), and a storage medium including reclamation units 314 arranged in one or more reclamation groups 318. One recovery group 318 referred to as recovery group 0 is shown in fig. 3A to 3D, but the recovery units 314 may be arranged in any number of recovery groups 318. For purposes of illustration, in the embodiment shown in fig. 3A-3D, reclamation group 0 may include five reclamation units 314 identified as RU 0, RU 1, RU 2, RU 3, and RU 4, each reclamation unit 314 may have a storage capacity of five pages of data, but any number and/or size of reclamation units may be used. For example, in a practical implementation, the reclamation unit may include one, two, four, or more erase blocks, each of which may have a storage capacity measured in MB or GB and a page size measured in KB.

One or more reclamation unit handles 336 may reference corresponding reclamation units 314. For example, as shown in FIG. 3A, first reclamation unit handle 336 identified as reclamation unit handle 0 (or RUH 0) may reference RU 1 as indicated by arrow 317. Second reclamation unit handle 336, identified as reclamation unit handle 1 (RUH 1), may reference RU 4 as indicated by arrow 319 a.

Referring to fig. 3A, as illustrated by diagonally shaded portions of RU 1 and RU 4, the storage 304 may be illustrated in an initial state in which two pages of data may be stored in the reclamation unit 314 identified as RU 1 and four pages of data may be stored in the reclamation unit 314 identified as RU 4.

The host 302 may send the write command 330 and the page data 321 (or an address, pointer, or other indicator of the location of the page data 321) to the storage device 304 using the communication connection 305. Command 330 may include a placement identifier 334, where placement identifier 334 may include reclamation group identifier 315 and/or reclamation unit handle 336. In the example shown in fig. 3A-3D, reclamation group identifier 315 and reclamation unit handle 336 may instruct storage 304 to store page data 321 at reclamation unit 314 referenced by reclamation unit handle 1 in reclamation group 0.

In some embodiments, command 330 and/or placement identifier 334 may use different techniques to specify the reclamation unit into which data 321 is to be written. For example, as shown in the embodiment described with respect to FIG. 4, command 330 and/or placement identifier 334 may include a placement handle that may specify a reclamation unit handle, rather than directly providing reclamation unit handle 336.

Referring to fig. 3B, the controller 326 may receive the write command 330 and/or the page data 321. Based on reclamation group identifier 315 and reclamation unit handle 336 included in placement identifier 334 in write command 330, controller 326 may determine that page data 321 should be stored in reclamation unit 314 referenced by reclamation unit handle 1 in reclamation group 0. In the example shown in fig. 3B, reclamation unit handle 1 may reference a reclamation unit 314 identified as RU 4, and thus, as shown in fig. 3C, controller 326 may store page data 321 in RU 4.

Referring to fig. 3D, page data 321 may populate reclamation unit 314 indicated as RU 4 as shown using diagonal shading. Accordingly, the controller 326 may modify the reclamation unit handle 1 to reference a different (e.g., empty) reclamation unit 314 (such as RU 2) within reclamation group 0 as indicated by arrow 319 b. Thereafter, the controller 326 may continue populating RU 2 with data received from the host using the write command 330 specifying the recovery unit handle 1 in the placement identifier 334.

Fig. 4A illustrates an embodiment of an update operation for a flexible data placement scheme in a first state, according to a disclosed example embodiment. Fig. 4B illustrates an embodiment of the update operation illustrated in fig. 4A in a second state according to a disclosed example embodiment.

The embodiments shown in fig. 4A-4B may include a host 402 and a storage 404 that communicate using a communication connection 405. The storage device 404 may include a controller 426 and a storage medium, the controller 426 may control the overall operation of the storage device 404, the storage medium including reclamation units 414 arranged in one or more reclamation groups 418.

One recovery group 418, referred to as recovery group 0, is shown in fig. 4A-4B, but recovery units 414 may be arranged in any number of recovery groups 418. For purposes of illustration, in the embodiment shown in fig. 4A-4B, while reclamation group 0 may include five reclamation units 414 identified as RU 0, RU 1, RU 2, RU 3, and RU 4, each reclamation unit 414 may have a storage capacity of five pages of data, any number and/or size of reclamation units may be used. For example, in a practical implementation, the reclamation unit may include one, two, four, or more erase blocks, each of which may have a storage capacity measured in MB and a page size measured in KB.

One or more reclamation unit handles 436 may reference the corresponding reclamation units 414. For example, reclamation unit handle 0 (RUH 0) may reference reclamation unit RU 1 as indicated by arrow 417a, and reclamation unit handle 1 (RUH 1) may reference reclamation unit RU 4 as indicated by arrow 419 a. The reclamation unit handle 436 may reference the reclamation unit 414 that the controller may store the next page of data using the reclamation unit handle 436.

Referring to fig. 4A, as illustrated by diagonally shaded portions of RU 1 and RU 4, the storage 404 may be illustrated in an initial state in which two pages of data may be stored in the reclamation unit 414 identified as RU 1 and four pages of data may be stored in the reclamation unit 414 identified as RU 4. Accordingly, storage 404 may be shown in an initial state that may be similar to the initial state of storage 304 in fig. 3A.

However, instead of receiving a write command, the controller 426 may receive an update request 423 (e.g., use command, indication, etc.). The update request 423 may request that the controller 426 may modify one or more reclamation unit handles 436 to reference the update operations of the new (e.g., empty) reclamation unit 414. The update request 423 may include an update list 425, and the update list 425 may specify one or more reclamation unit handles 436 that may be modified by the update operation. In the example shown in fig. 4A, update list 425 may include reclamation unit handle 0 and reclamation unit handle 1. In some embodiments, the update request 423 may use different techniques to specify one or more reclamation handles 436 that may be updated. For example, as shown in the embodiment described with respect to FIG. 5, update request 423 may include a placement handle that may specify a reclamation unit handle, rather than directly providing reclamation unit handle 436.

Referring to FIG. 4B, the controller 426 may perform the requested update operation by modifying the reclamation unit handle 0 to reference the reclamation unit RU 0 as indicated by arrow 417B and by modifying the reclamation unit handle 1 to reference the reclamation unit RU 3 as indicated by arrow 419B. Thus, data sent with a subsequent write command specifying reclamation unit handle 0 or reclamation unit handle 1 may be written to RU 0 or RU 3, respectively.

As shown in fig. 4B, the recovery units RU 1 and RU 4 may be only partially full after the update operation. However, depending on the physical storage medium used to implement the recovery unit 414, partially filling the recovery unit may reduce the reliability, durability, performance, etc. of the recovery unit and/or the storage medium implementing the recovery unit. Thus, in the embodiment shown in fig. 4B, as shown in solid shading, controller 426 may fill some or all of the blank portions of RU 1 and/or RU 4 with fill data (such as zero, random data, and/or metadata).

However, the use of padding data may result in wasted or underutilized storage space. For example, the portion of RU 1 and/or RU 4 shown in fig. 4B having padding data may not be used to store the data until they are reclaimed (e.g., by a garbage collection operation).

FIG. 5 illustrates an embodiment of a flexible data placement scheme with reclamation unit handles according to an example embodiment of the disclosure. The embodiment shown in fig. 5 may include a controller 526 and NVM subsystem 512, which may be located, for example, in a storage device. NVM subsystem 512 may include P reclamation groups 518 identified as reclamation group 0 through reclamation group P-1. Recovery group 518 may include one or more recovery units 514.

The controller 526 may receive I/O commands 530 from the host via the communication interface 528. The I/O command 530 (which may be a write command in the example shown in fig. 5) may include a namespace identifier (NSID) 532 and/or a placement identifier 534. Placement identifier 534 may include a reclamation group identifier 515 and/or a placement handle 516 that may enable a host to specify a reclamation group 518 and/or reclamation units 514 within reclamation group 518, respectively, for storing data associated with write command 530. Thus, in some embodiments, the scheme shown in FIG. 5 may enable a host to align data that may be deallocated together (e.g., data belonging to a namespace) with one or more reclamation units 514 that may be erased as units.

In some embodiments, the placement handle 516 may be mapped to a Reclamation Unit Handle (RUH) 536 that may reference one or more reclamation units 514 in one or more reclamation groups 518, rather than directly indicating one reclamation unit 514. For example, the reclamation unit handle may map to one reclamation unit 514 in each reclamation group 518. FIG. 5 illustrates an example mapping from each reclamation unit handle 536 to one reclamation unit in each reclamation group 518. In the embodiment shown in FIG. 5, the controller 526 may use N reclamation unit handles 536 identified as reclamation unit handle 0 through reclamation unit handle N-1. In some embodiments, reclamation unit handle 536 may be characterized as a controller resource.

The controller 526 may use the placement handle list 538 to map one or more placement handles 516 to one or more RUH identifiers (RUH IDs) 540, which in turn may identify corresponding reclamation unit handles 536. In the embodiment illustrated in FIG. 5, placement handle list 538 may include M placement handles identified as placement handle 0 through placement handle M-1.

In some embodiments, the placement handle 516 may be limited to a namespace 524 (in this example, the namespace is identified as namespace a). The namespace, in turn, may include one or more (e.g., all) reclamation units 514 referenced by one or more reclamation unit handles 536 identified in the placement handle list 538 (e.g., by the RUH ID 540). In some embodiments, the placement handle list 538 may be created, populated, modified, maintained, etc. by a host, a storage device (e.g., controller 526), or any other entity or combination thereof. In some embodiments, data may be partitioned and/or arranged into groups based on Logical Block Addresses (LBAs), applications that may use the data, host write traffic threads, and the like, in addition to and/or in lieu of namespaces.

In some embodiments, the use of the placement handle 516 and/or reclamation unit handle 536 may enable the flexible data placement scheme shown in FIG. 5 to present the reclamation units to the host as logical representations of the physical non-volatile storage devices within the reclamation group that may be physically erased, e.g., by the controller 526, without disturbing the other reclamation units 514. The controller 526 can implement a logical reclamation unit specified by the host (e.g., using the placement identifier 534) using a physical storage medium (e.g., one or more erase blocks on the NVM die) that can be selected by the controller 526 and erased as a unit.

In some embodiments, selection of the reclamation unit 514 and/or reclamation group 518 may be performed at least in part by the controller 526. For example, if the controller 526 receives a write command 530 that does not include a placement identifier 534, the controller 526 may use the reclamation unit handle 536 mapped by the default placement handle 516 (e.g., placement handle 0) and select the reclamation group 518, thereby selecting the reclamation units 514 that are within the selected reclamation group 518 and referenced by the reclamation unit handle 536 mapped by the default placement handle 516. As another example, if the controller 526 receives a write command 530 having a placement identifier that includes a placement handle 516 but does not include a reclamation group identifier 515, the controller 526 may select the reclamation unit 514 by selecting the reclamation group 518 and using the reclamation units 514 that are within the selected reclamation group 518 and referenced by a reclamation unit handle 536 (where the reclamation unit handle 536 is mapped by the placement handle 516 provided by the write command 530).

In some embodiments, the flexible data placement scheme illustrated in fig. 5 may be implemented at least in part using NVMe protocol. In such embodiments, the placement identifier 534 may be implemented as part of the command 530 (e.g., the instruction portion of the NVMe command); m, N and/or P may be parameters that may be configured, for example, using NVMe namespace management commands; the controller 526 can be implemented using one or more NVMe controllers; and/or NVM subsystem 512 may be implemented as an NVMe subsystem, a endurance set (which may be coextensive with the storage device in some embodiments), an NVMe domain, etc., or any combination thereof. In embodiments implemented at least in part using the NVMe protocol, an instruction specific (DSPEC) field may be used to provide a placement identifier (e.g., a reclamation group (which may be indicated by a reclamation group identifier) and a placement handle). The DSPEC field may be used, for example, in embodiments in which an instruction type (DTYPE) field may be used to indicate that a flexible data placement feature is used. In such embodiments, an invalid and/or default (e.g., all zero) value of the DSPEC field may indicate a miss of the reclamation group indicator and/or the placement handle.

In the embodiments of the flexible data placement schemes shown in fig. 2A-2D, 3A-3D, 4A-4B, and/or 5, the reclamation unit may be implemented (at least from the perspective of the host) as a logical representation of the underlying portion of the physical storage medium. Thus, the host may be masked from and/or unaware of one or more ongoing operations and/or conditions of the storage device that may adversely affect one or more operations involving the physical implementation of the logical reclamation unit selected by the host.

Fig. 6 illustrates another embodiment of a flexible data placement scheme according to an example embodiment of the disclosure. In the embodiment shown in fig. 6, the storage 604 may comprise a storage medium implemented using a non-volatile memory device (e.g., NVM die) 620 that is controllable by a controller 626. Storage 604 may communicate with host 602 using communication connection 605.

The embodiment shown in fig. 6 may include eight NVM devices 620 implemented as NVM dies designated 620-0, … …, 620-7, which may be collectively and/or individually referred to as 620. Each NVM die can include a first plane shown on the left side of the die and a second plane shown on the right side of the die. Each plane may include 20 erase blocks 660, which are shown as rectangles having numbers therein. In some embodiments, the numbers in erase block 660 may represent a percentage of life P/E cycles that have been performed on erase block 660. For example, an erase block having a number of 30 may have used 30% of its life P/E cycle. For convenience, individual erase blocks 660 in a plane may be represented by numbers within them. In some other embodiments, the numbers in the erase block may represent another aspect of the erase block (such as a location on the NVM die, a characterized bit error accumulation rate, historical temperature exposure, reading and/or writing to an interference log (e.g., the log may represent an accumulated count), etc.

NVM dies 620 may be arranged in channels 622, with two dies per channel. Thus, NVM dies 620_0 and 620_1 are connected to channel 622_0, NVM dies 620_2 and 620_3 are connected to channel 622_1, NVM dies 620_4 and 620_5 are connected to channel 622_2, and NVM dies 620_6 and 620_7 are connected to channel 622_3. The number and/or arrangement of dies, planes, channels, erase blocks, etc. shown in fig. 6 is for purposes of illustrating the inventive principles only, and other embodiments may include any number and/or arrangement of dies, planes, channels, erase blocks, etc. In some embodiments, RUH may be used to temporarily address one or more RUs 760 while one or more RUs 760 receive write data from host 702.

FIG. 7 illustrates a first embodiment of a technique for selecting an erase block of one or more reclamation units for a flexible data placement scheme in accordance with an exemplary embodiment of the disclosure. The technique illustrated in fig. 7 may be implemented, for example, using the storage device illustrated in fig. 6, and components similar to those illustrated in fig. 6 may be identified using reference numerals ending with the same numerals.

Referring to fig. 7, nvm dies 720 can be arranged in a single reclamation group 718. The reclamation unit may be composed by selecting (e.g., by controller 726) one or more erase blocks 760 based on one or more rules related to the layout of NVM dies and/or planes 720, channels 722, etc. For example, the reclamation unit may be composed by selecting one erase block 760 per channel 722 from each plane of one NVM die 720.

However, within a plane, one or more erase blocks may be randomly selected. For example, a first reclamation unit (which may be referred to as RU 0) may consist of erase blocks indicated using bold outlines. Using this technique, RU 0 may include erase block 64 in the left plane of NVM die 720-0, erase block 30 in the right plane of NVM die 720-0, erase block 14 in the left plane of NVM die 720-2, erase block 92 in the right plane of NVM die 720-2, erase block 53 in the left plane of NVM die 720-4, erase block 24 in the right plane of NVM die 720-4, erase block 47 in the left plane of NVM die 720-6, and erase block 79 in the right plane of NVM die 720-6.

As described above, individual erase blocks 760 in a plane may be represented by numbers within them for convenience, which may also represent other characteristics (such as the percentage of P/E cycles used).

Thus, erase block 760 (thick solid outline) within reclamation unit RU 0 may have a wide variety of program cycle counts.

Alternatively, the cells may be organized by selecting one or more erase blocks 760 in each plane that may have the lowest program loop count. For example, a second reclamation unit (which may be referred to as RU 1) may be composed of the erase block (indicated using the bold dashed outline) with the lowest program loop count. Using this technique, RU 1 may include erase block 0 in the left plane of NVM die 720-1, erase block 5 in the right plane of NVM die 720-1, erase block 0 in the left plane of NVM die 720-3, erase block 3 in the right plane of NVM die 720-3, erase block 1 in the left plane of NVM die 720-4, erase block 0 in the right plane of NVM die 720-4, erase block 2 in the left plane of NVM die 720-7, and erase block 6 in the right plane of NVM die 720-7.

Thus, erase block 760 (thick dashed outline) within reclamation unit RU 1 may have a relatively low program loop count.

Depending on implementation details, any of these techniques may result in poor performance of the recovery unit. For example, if storage 704 receives a write command with data that can be accessed frequently (e.g., hot data), controller 726 may store the data in reclamation unit RU 0, which is composed using randomly selected erase blocks, and thus may include erase blocks with various program cycle counts, including erase blocks with relatively high program cycle counts that may wear out when used to store hot data. As another example, if storage 704 receives a write command with data that may not be accessed again for a relatively long period of time (e.g., cold data), controller 726 may store the data in reclamation unit RU 1 that uses an erase block composition with the lowest program cycle count, which may be wasted on cold data that may be tolerable to be stored in erase blocks with relatively high program cycle counts (e.g., more efficiently, at lower cost, etc.).

FIG. 8 illustrates a second embodiment of a technique for selecting erase blocks for one or more reclamation for a unit flexible data placement scheme in accordance with an exemplary embodiment of the disclosure. The technique illustrated in fig. 8 may be implemented, for example, using the storage device illustrated in fig. 6, and components similar to those illustrated in fig. 6 may be identified using reference numerals ending with the same numerals.

Referring to fig. 8, nvm dies 820 may be arranged in a single reclamation group 818. The reclamation unit may be composed by selecting (e.g., by the controller 826) one or more erase blocks 860 based on one or more rules related to the layout of the NVM dies and/or planes 820, channels 822, etc. For example, the reclamation unit may be composed by selecting one erase block 860 from each plane of one NVM die 820 through each channel 822.

However, within one plane, one or more erase blocks 860 may be selected based on information 861 about the data to be stored in the reclamation unit. For example, information 861 may be received from host 802 using communication connection 805. Host 802 may include placement logic 863 that may determine (e.g., identify, collect, process, etc.) information 861 to be sent to storage 804. The controller 826 may include reclamation unit logic 865 that may use the information 861 to select reclamation units and/or erase blocks 860 for inclusion within the reclamation units.

For example, the information 861 may indicate that the data to be stored in the first reclamation unit (which may be referred to as RU 2) may be relatively cool data that is not expected to be accessed frequently. Thus, the reclamation unit logic 865 may compose RU 2 using erase blocks 860 having relatively high P/E cycle counts as indicated by the heavy solid outline. Using this technique, RU 2 can include erase block 87 in the left plane of NVM die 820-0, erase block 91 in the right plane of NVM die 820-0, erase block 92 in the left plane of NVM die 820-2, erase block 92 in the right plane of NVM die 820-2, erase block 86 in the left plane of NVM die 820-4, erase block 93 in the right plane of NVM die 820-4, erase block 92 in the left plane of NVM die 820-6, and erase block 90 in the right plane of NVM die 820-6.

The controller 826 and/or the reclamation unit logic 865 uses the constituent reclamation unit RU 2 to store data received for the current write command (e.g., the controller 826 may reference RU 2 using a reclamation unit handle provided by the current write command), or the constituent reclamation unit RU 2 may be saved for use with future write commands, reclamation unit handles, and the like.

Depending on implementation details, the reclamation unit RU 2, which is composed as described above, may be used to efficiently store data indicated by the host as relatively cold data, thereby saving other erase blocks with relatively low P/E cycle counts for relatively hot data.

Alternatively or additionally, the information 861 may indicate that data to be stored in a second reclamation unit (which may be referred to as RU 3) may be relatively sensitive to stresses generated from being located near the periphery of the NVM die 820. Thus, the reclamation unit logic 865 may compose RU 3 using erase blocks 860 located near the center of the NVM die 820 shown in bold dashed outline. Using this technique, RU 3 may include erase block 52 in the left plane of NVM die 820-1, erase block 51 in the right plane of NVM die 820-1, erase block 40 in the left plane of NVM die 820-3, erase block 39 in the right plane of NVM die 820-3, erase block 37 in the left plane of NVM die 820-5, erase block 52 in the right plane of NVM die 820-5, erase block 43 in the left plane of NVM die 820-7, and erase block 40 in the right plane of NVM die 820-7.

The controller 826 and/or the reclamation unit logic 865 uses the constituent reclamation unit RU 3 to store data received for the current write command (e.g., the controller 826 may use the reclamation unit handle provided by the current write command to reference RU 3), or may save the constituent reclamation unit RU 3 for use with future write commands, reclamation unit handles, and the like.

Depending on implementation details, reclamation unit RU 3, composed as described above, may be used to efficiently store data that is indicated to be relatively sensitive to stresses arising from being located near the periphery of NVM die 820, while other erase units are composed using other erase blocks 860 located near the periphery of NVM die 820 to store data that may be less sensitive near the periphery of NVM die 820.

Although the embodiments shown in fig. 7 and 8 are implemented using a single reclamation set that may include all eight NVM dies, in other embodiments, NVM dies may be arranged in any other configuration of reclamation set. For example, in some embodiments, each NVM die can be configured as a separate reclamation group. In such embodiments, the reclamation unit may be composed, for example, by selecting one erase block from each plane of the NVM die. Within one plane, one or more erase blocks may be selected based on information about data to be stored in the reclamation unit (e.g., received from a host). In such embodiments having multiple recovery groups, the recovery groups may be selected based on information about data to be stored in the recovery unit (e.g., to be optimally aligned with the recovery groups). Further, in NVMe implementations with flexible data placement, if the write command does not specify a reclamation group, the controller 826 may select the reclamation group, for example, based on the information 861.

FIG. 9 illustrates an embodiment of a storage system implementing a flexible data placement scheme in accordance with a disclosed example embodiment. The embodiment shown in fig. 9 may include a host 902 and a storage device 904 that communicate using a communication connection 905. The storage device 904 may include a controller 926 (which may control the overall operation of the storage device 904) and a storage medium 913 including one or more reclamation units 914 that may be selected, composed, etc. as described herein.

The controller 926 may include reclamation unit logic 965 that may implement any of the flexible data placement schemes disclosed herein, and the reclamation unit logic 965 may select and/or compose one or more reclamation units 914 based on the information 961 about the data 921 to be stored in the one or more reclamation units 914. Information 961 may be provided, for example, by host 902. The data 921 may be specified, for example, by a write command 930 sent from the host 902 to the storage device 904. The write command 930 may, for example, include the data 921 with the write command 930 and/or within the write command 930, specify the data 921 by sending pointers, addresses, and/or other indications of locations in which the data is located (e.g., in memory, in a storage device, on a network, etc.), and/or in any other manner.

Host 902 may include one or more processors 962, which may implement placement logic 963, where placement logic 963 provides any information 961 regarding data to be stored in one or more reclamation units 914 of storage 904. The one or more processors 962 may also execute one or more operating systems 972, which operating systems 972 in turn may execute applications 974, file systems 976, and the like. Any of the operating system 972, applications 974, file system 976, etc. may be data of a user that may be stored in one or more reclamation units 914 of the storage 904. In some embodiments, placement logic 963 may be implemented partially or completely separately from one or more processors 962, e.g., using dedicated hardware, such as Complex Programmable Logic Devices (CPLDs), field Programmable Gate Arrays (FPGAs), application specific circuits (ASICs), and the like. For example, in some embodiments, placement logic 963 may be implemented at least in part using hardware that may perform a lookup using one or more keys to obtain one or more values.

Placement logic 963 in host 902 and reclamation unit logic 965 in storage 904 may include information logic 963-1 and 965-1, respectively, and information logic 963-1 and 965-1 may implement host-side and device-side functions, respectively, related to the type of information 961 that may be provided to storage 904 and/or used by storage 904 to select and/or compose one or more reclamation units 914 for storing data based on, for example, write requests (e.g., write commands) received from host 902.

In some embodiments, information logic 963-1 and/or 965-1 may enable host 902 to provide and/or storage 904 to use access information regarding data to be stored in the reclamation unit. The access information may include, for example, historical access information (such as how fast and/or how frequently data has been written, read, rewritten, etc.). For example, if data has been written and is not read or rewritten for a relatively long period of time thereafter, the data may be considered relatively cool data. Historical access information may be sent from the host 902 to the storage device 904, and the storage device 904 may use the historical access information to determine whether the data 921 to be stored in the one or more reclamation units 914 may be considered hot, cold, etc. Thus, the device-side information logic 965-1 may predict future accesses to the data 921 based on the historical access information.

Additionally or alternatively, the access information sent from host 902 to storage 904 may include a determination made by host-side information logic 963-1. For example, host 902 can store a set of data in storage 904 using a first set of LBAs. The host 902 can read a portion of the data set from the storage device 904, e.g., as part of a file system compression operation (e.g., a garbage collection operation to remove one or more gaps between remaining active portions of the data set), and write the portion of the data set to the storage device 904 using the second set of LBAs. The host-side information logic 963-1 may send information 961 to the device-side information logic 965-1 indicating that the data 921 may be relatively cold (e.g., the data 921 was not recently rewritten and therefore was compressed as part of a garbage collection operation). Thus, the host-side information logic 963-1 may predict future accesses to the data 921 based on the historical access information.

In addition to predictions based on historical access information, information logic 963-1 and/or 965-1 may enable host 902 to provide and/or storage 904 to use predictions based on any other basis. For example, the host may recognize a new thread initiated by application 974 as being relatively active, important, etc., and assume that any data associated with the application may be hot data. Thus, the host-side information logic 963-1 may predict future accesses of the data 921 based on one or more characteristics of the thread, application 974, etc., and send information 961 to the device-side information logic 965-1 indicating that any write data 921 associated with the thread, application, etc., may be considered relatively hot data.

Additionally or alternatively, the access information may include information such as how often the data is expected (e.g., predicted) to be rewritten, read, etc. For example, even if some data has not been accessed recently, there may be one or more indicators that may indicate that the data may be accessed soon, for example, for upcoming events, recent accesses of related data, etc., and thus, the data may be considered relatively hot data.

Additionally or alternatively, the information logic 963-1 and/or 965-1 may enable the host 902 to provide and/or the storage 904 to use information regarding one or more acceptable characteristics of a storage medium that may be used to store the data 921 to which the information 961 is provided. Acceptable characteristics may include error tolerance information such as an acceptable Bit Error Rate (BER), which in turn may be based on bit error accumulation. For example, BER and/or bit error accumulation may be specified based on a length of time data is stored, a number of read operations performed on the data, a number of re-write operations performed on the data (e.g., refresh operations), one or more data movement operations performed on the data (e.g., garbage Collection (GC) operations), and so forth. As another example, BER and/or bit error accumulation may be related to the number of remaining P/E cycles in the expected and/or estimated life of the reclamation unit and/or the erase block used in the reclamation unit. As with the information regarding access described above, the information regarding the acceptable characteristics of the storage medium may be based on historical information, predictions based on any other basis, and so forth.

Additionally or alternatively, information logic 963-1 and/or 965-1 may enable host 902 to provide and/or store 904 to use information regarding one or more attributes of data 921 for storage in the device. For example, the attribute information may include quality of service (QoS) information, which may specify a required availability of data to be stored in the reclamation unit. As another example, the attribute information may include access delay information of data to be stored in the reclamation unit. This may be affected by wear leveling of the storage medium, for example (e.g., the lifetime of the NVM cells may affect read time, write (program) time, erase time, etc.). As another example, the attribute information may include access bandwidth information that may also be affected by wear leveling of the storage medium, for example. As another example, the attribute information may include Service Level Agreement (SLA) information that may specify, for example, a minimum number of P/E cycles required for the storage medium remaining for the data to be stored, regardless of whether the data may actually be accessed frequently (e.g., even if the data to be stored is relatively cold data). As with the information regarding access described above, the information regarding the one or more attributes of the data 921 may be based on historical information, predictions based on any other basis, and so forth.

Additionally or alternatively, the information logic 963-1 and/or 965-1 may enable the host 902 to provide and/or enable the storage 904 to use information regarding the type and/or usage of the data 921 to be stored in the reclamation unit. For example, data such as operating system metadata, file system tables, and the like may be accessed frequently, and thus, reclamation units where such data may be stored may be selected and/or composed to include one or more erase blocks that may have a relatively large number of P/E cycles remaining. As with the information regarding access described above, the information regarding the type and/or usage of the data 921 may be based on historical information, predictions based on any other basis, and so forth.

In some embodiments, placement logic 963 in host 902 and reclamation unit logic 965 in storage 904 may include communication logic 963-2 and 965-2, respectively, and communication logic 963-2 and 965-2 may implement host-side and device-side functions, respectively, related to the manner in which information 961 may be communicated from host 902 to storage 904.

Communication logic 963-2 and/or 965-2 may enable host 902 to implicitly and/or explicitly receive information 961 and/or enable storage 904 to implicitly and/or explicitly transmit information 961. For example, information 961 may be implicitly conveyed by defining one or more reclamation unit handle identifiers (e.g., reclamation unit numbers) that may be recognized and/or expected to correspond to different access information for data to be stored in the corresponding reclamation unit. For example, the reclamation unit handle RUH 0 may be recognized as implicitly indicating that the data to be stored using the reclamation unit handle is relatively hot data, and the reclamation unit handle RUH 1 may be recognized as implicitly indicating that the data to be stored using the reclamation unit handle is relatively cold data. Further, a range of reclamation unit handle identifiers (e.g., a range of numbers) may be used to indicate a range of access information (e.g., a range of access frequencies, one end of which indicates most frequently accessed data and the other end of which indicates least frequently accessed data). As another example, the reclamation unit handle RUH 3 may be recognized as implicitly indicating that data to be stored using the reclamation unit handle may be stored in reclamation units having relatively low performance characteristics (e.g., relatively high P/E cycle counts).

Additionally or alternatively, communication logic 963-2 and/or 965-2 may enable host 902 to explicitly receive information 961, e.g., using one or more fields, extensions, etc., of commands, handles, etc., that may provide explicit information regarding data, and/or storage 904 may be capable of explicitly sending information 961, e.g., using one or more fields, extensions, etc., of commands, handles, etc., that may provide explicit information regarding data. For example, the host may send a reclamation unit handle that includes an extension. One type of extension may indicate that the data to be stored using the reclamation unit handle may be relatively hot and another type of extension may indicate that the data to be stored using the reclamation unit handle may be relatively cold. One or more additional extensions may indicate that the data is somewhere between hot and cold. Depending on implementation details, such an embodiment may use one or more reclamation unit resources that may be sized based on each indication multiplied by the number of reclamation unit handles used.

Additionally or alternatively, communication logic 963-2 and/or 965-2 may enable host 902 to receive information 961 in a persistent manner and/or storage 904 to send information 961 in a persistent manner (e.g., information for writing a command sequence and/or reclaiming data of a unit handle may be the same until the information is updated). For example, the host may send an indication that may be a setting to indicate that the reclamation unit for one or more subsequent write commands may be processed, with particular types of information about the data, e.g., until the setting is changed until a particular time has elapsed until a particular number of commands have been processed using the setting, a particular number of reclamation units have been filled for reclamation unit handles, and so forth.

Additionally or alternatively, communication logic 963-2 and/or 965-2 may enable host 902 to receive information 961 on an individual basis and/or enable storage 904 to transmit information 961 on an individual basis. For example, information regarding data to be stored in one or more reclamation units may be provided for each write command, each reclamation unit handle update, and so forth.

Additionally or alternatively, communication logic 963-2 and/or 965-2 may enable host 902 to receive information 961 using any other communication technique (such as a setting of a storage device controller, a namespace attribute, a request to use an update operation to a reclamation unit handle, a configuration setting as a flexible data placement scheme, in a field of an input and/or output (I/O or IO) command, in a field of a management command, an attribute, setting, etc. as a non-volatile memory (NVM) subsystem, a controller, a namespace, etc.), and/or enable storage 904 to send information 961 using any other communication technique (such as a setting of a storage device controller, a namespace attribute, a request to use an update operation to a reclamation unit, a configuration setting as a flexible data placement scheme, in a field of an input and/or output (I/O or IO) command, in a field of a management command, an attribute, a setting, etc. as a non-volatile memory (NVM) subsystem, a controller, a namespace, etc.). For example, some or all of the information 961 may be provided with a write command (such as write command 330 shown in fig. 3A-3D). As another example, some or all of the information 961 may be provided with an update request (such as update request 423 shown in fig. 4A-4B). As another example, some or all of the information 961 may be provided along with namespace management operations, such as establishing, configuring, and/or updating the namespaces 525 shown in fig. 5. As another example, some or all of the information 961 can be provided with NVM subsystem management operations (such as establishing, configuring, and/or updating NVM subsystems 106 and/or 206 shown in fig. 1A-1E and/or fig. 2A-2D, respectively). As another example, in implementations using NVMe, the information 961 may be communicated using a Data Set Management (DSM) command (e.g., using a context attribute), using one or more bits in an instruction specific field (e.g., a DSPEC field), and/or using an instruction type field (type field) that communicates with commands in a commit queue entry (SQE). As another example, vendor specific techniques may be used to communicate information 961.

In some embodiments, reclamation unit logic 965 in storage 904 may include usage logic 965-3, and usage logic 965-3 may implement device-side functions related to the manner in which information 961 may be used to compose and/or select one or more reclamation units 914. For example, if the information 961 indicates that data may be written, read, re-read, and/or re-written frequently (e.g., is hot data), then the use logic 965-3 may enable the storage device 904 to select and/or compose a reclamation unit having a storage medium that may have a relatively large number of P/E cycles remaining in its expected lifetime. Additionally, if the information 961 indicates that data is unlikely to be written, read, re-read, and/or re-written (e.g., cold data) quickly or frequently, the use of the logic 965-3 may enable the storage device 904 to select and/or compose a reclamation unit having a storage medium that may have a relatively small number of P/E cycles remaining in its expected lifetime.

As another example, if the information 961 indicates that the data has a relatively high margin of error, for example, because the data may be stored in the storage device as part of a redundant storage scheme, the storage device may select and/or compose a reclamation unit having a storage medium that may have a relatively small number of P/E cycles remaining in its expected lifetime (e.g., a storage medium that may have relatively high wear leveling). For example, if the data is stored in a Redundant Array of Independent Drives (RAID) and/or a redundancy scheme having a relatively large number of mirrors (e.g., three mirrors instead of two mirrors), this may indicate that the host may provide a relatively high data protection rate (e.g., a relatively high data loss rate may be acceptable) and may store the data using a storage medium that may have a relatively small number of effectively estimated P/E cycles remaining over its expected lifetime. As another example, if the host does not allow the data to stay in the storage device for a long time (e.g., for a minimum amount of BER accumulation), the data may be stored using a storage medium that may have a relatively large number of effectively estimated P/E cycles remaining in its expected lifetime.

As another example, the usage logic 965-3 may identify and/or use one or more attributes of one or more portions of the storage medium 913 to compose and/or select one or more reclamation units 914 based on correlation of the attributes with an increased capability to tolerate a relatively large number of P/E cycles and/or accumulate a relatively small number of bit errors. For example, one or more erase blocks may be selected for the reclamation unit, and/or the reclamation unit may be selected based on it including a storage medium having one or more attributes, such as wafer production attributes (e.g., locations of dies on a wafer), erase block locations on a storage medium die, voltage and/or speed characteristics of read, write (program) and/or erase operations, bit error rate accumulation of the storage medium, current and/or historical temperature exposure of the storage medium, access activity (e.g., read, write (program) and/or erase operations) of the storage medium and/or adjacent medium, and the like.

In some embodiments, the manner in which the information 961 may be used to compose and/or select one or more reclamation units 914 may be based on one or more differentiation methods (such as binary threshold, binning (bin), analog threshold, etc.). For example, in the case of a binary threshold, an erase block may be considered to have a high BER if the BER is above the threshold, and an erase block may be considered to have a low BER if the BER is below the threshold. However, in the case of binning techniques, an erase block may be partitioned into bins based on two or more thresholds (e.g., behavioral expectations). For example, four different thresholds may be used to divide an erase block into five different bins, each of which may be used to compose a reclamation unit of data having different tolerance levels of bit errors. As another example, data that has been garbage collected once may be indicated as belonging to a first bin of cold data, while data that has been garbage collected twice may be indicated as belonging to a second bin of even colder data.

In some embodiments with analog or proportional-derivative schemes, the erase block may be characterized as having particular values of characteristics, properties, etc. For example, a floating point number or number of digits, which may use relatively high-quantization digit mathematics, may be used to characterize the speed and/or voltage of the programming, reading, and/or erasing operations of the erase block, and the floating point number or number of digits may be used to select the erase block of the reclamation unit 914 based on the information 961 provided to the storage 904.

In some embodiments, placement logic 963 of host 902 may include determination logic 963-4, where determination logic 963-4 may implement host-side functionality related to the manner in which host 902 may determine information 961 to send to storage device 904. The determination logic 963-4 may enable the host 902 to identify, collect, process, etc., information 961 to be sent to the storage device. For example, the host may observe that data written by a user (e.g., operating system 972, applications 974, file system 976, etc.) may be, or may be, continuously used and/or already stored, possibly stored for a relatively long period of time, and may send this information to the storage device, e.g., along with a write request. Additionally or alternatively, the determination logic 963-4 may convert this type of information into another type of information that may be acted upon more directly by the storage 904. For example, if the determination logic 963-4 determines that the data written by the user has been stored, or may be stored for a relatively long time, the determination logic 963-4 may send information 961 indicating that the data is relatively cold data.

As another example, the determination logic 963-4 may observe that data written by a user may be compressed (e.g., garbage collected) at the file system level, database level, etc., and send this information to the storage device along with a write request for the data. For example, the determination logic 963-4 may characterize user data that may or may not be compressed into relatively hot data. As another example, the host may determine the number of times data has been compressed (e.g., garbage collected) and send this information to the storage device along with the data to be stored at the device. Depending on implementation details, multiple compression operations on the same data may increase the correlation between the data and the likelihood that the data will be accessed frequently.

As another example, the determination logic 963-4 may characterize data usage based on one or more data management schemes (such as one or more cache replacement policies, a tiered storage scheme, a multi-tiered storage architecture, etc.), and send this information to the storage device along with the data to be stored at the device. For example, where data is stored as part of a cache replacement policy, the cache level at which the data is stored may indicate a relative likelihood that the data will be accessed frequently (e.g., data stored at the L1 cache level may be accessed more frequently than data stored at the L3 cache level). Similarly, data stored at a higher level of the tiered storage scheme may be accessed more frequently than data stored at a lower level of the tiered storage scheme. Thus, the determination logic 963-4 may report such usage information directly to the storage 904 as information 961 and/or convert such usage into information 961 indicating that the data is relatively hot or cold.

In some embodiments, reclamation unit logic 965 may compose and/or select one or more reclamation units 914 based on a combination of information 961 received externally from storage 904 and information obtained locally at storage 904 (which may be referred to as local information and/or locally derived information). For example, reclamation unit logic 965 may collect one or more statistics regarding data stored at storage 904. Examples of statistical information may include write counts and/or frequencies, rewrite counts and/or frequencies, read counts and/or frequencies, and the like. In some embodiments, such statistics may be thresholds and/or ranges mapped to different reclamation groups, reclamation units, etc. in the storage medium 913. Such statistics may be combined with the received information 961 to compose and/or select one or more reclamation units 914 (e.g., reclamation unit handles to reclamation unit classifications and/or mappings). For example, host 902 may provide mirrored and/or non-mirrored bit error classifications in information 961. The reclamation unit logic 965 may combine this classification with one or more locally derived statistics to compose and/or select one or more reclamation units 914 for use with the data operation request (e.g., write command 930).

FIG. 10 illustrates an embodiment of an initial isolation scheme for a flexible data placement scheme for a storage device according to an example embodiment of the disclosure. The embodiment illustrated in FIG. 10 may include three reclamation unit handles 1036 identified as RUH X, RUH Y, and RUH Z.

The reclamation unit handle RUH X may currently reference the reclamation unit 1014 identified as RU a. The reclamation unit RU a may be partially filled with data as shown by the single diagonal hatching with lines extending from top right to bottom left. The reclamation unit handle RUH X may have previously referenced reclamation units 1014 'identified as RU a' _0, RU a '_1, and RU a' _2. For example, when RU a '_0, RU a' _1, and RU a '_2 are referenced by RUH X in a manner similar to RU 4 identified as being populated with data when the reclamation unit 314 is referenced by reclamation unit handle 1 (RUH 1) as shown in fig. 3B and 3C, the previously referenced reclamation unit 1014' may have been populated with data (as shown by the single diagonal hatching with lines extending from top right to bottom left). Although not currently referenced by RUH X, the previously referenced reclamation units RUA ' _0, RUA ' _1, and RUA ' _2 may remain associated with RUH X (e.g., by using a data structure such as a reclamation unit association table).

The reclamation unit handle RUH Y may currently reference the reclamation unit 1014 identified as RU B. The reclamation unit ru_b may be partially filled with data as shown by diagonal cross hatching. The reclamation unit handle RUH Y may have previously referenced reclamation units 1014 'identified as RU B' _0, RU B '_1, and RU B' _2. For example, when RU B '_0, RU B' _1, and RU B '_2 are referenced by RUH Y, the previously referenced reclamation unit 1014' may have been populated with data (as shown by the diagonal cross hatching).

Likewise, the reclamation unit handle RUH Z may currently reference the reclamation unit 1014 identified as RU C. The reclamation unit RU C may be partially filled with data as shown by the single diagonal hatching with lines extending from top left to bottom right. The reclamation unit handle RUH Z may have previously referenced reclamation units 1014 'identified as RU C' _0, RU C '_1, and RU C' _2. For example, when RU C '_0, RU C' _1, and RU C '_2 are referenced by RUH Z, the previously referenced reclamation unit 1014' may have been populated with data (as shown by the single diagonal hatching with lines extending from top left to bottom right).

In some embodiments, a controller within the storage device may perform one or more operations (e.g., maintenance operations) on data stored in the previously referenced reclamation unit 1014'. For example, some or all of the data stored in the previously referenced reclamation unit 1014 'may be deallocated (e.g., by the host), resulting in unused storage capacity in the previously referenced reclamation unit 1014'. This is illustrated in fig. 10, where the portion of the previously referenced reclamation unit 1014' containing deallocated (e.g., invalid) data is shown using shading with relatively thin lines.

In some embodiments, the controller may perform one or more maintenance operations to enable unused storage capacity in the previously referenced reclamation unit 1014' to be erased, reused, etc. For example, the controller may perform garbage collection operations in which valid data (e.g., data that has not been deallocated) in one or more of the previously referenced reclamation units 1014 'may be copied to a different reclamation unit, such that one or more of the previously referenced reclamation units 1014' may be erased and reused.

The embodiment illustrated in FIG. 10 may implement an initial isolation scheme in which data written to reclamation units currently or previously referenced by different reclamation unit handles 1036 may be initially isolated from each other. In some embodiments, data may be considered isolated if the reclamation unit storing the data includes only data written to the reclamation unit using the same reclamation unit handle. Thus, the recovery units RUA, RUA ' _0, RUA ' _1, and RUA ' _2 may include only the data written when these recovery units are referenced by RUH X. Similarly, the recovery units RU B, RU B '_0, RU B' _1, and RU B '_2 may include only data written when the recovery units are referenced by RUH Y, and the recovery units RU C, RU C' _0, RU C '_1, and RU C' _2 may include only data written when the recovery units are referenced by RUH Z.

However, as part of the controller operation, data from reclamation units written using different reclamation unit handles may be combined in a single reclamation unit. This is illustrated in fig. 10, where the controller may read valid data from the previously referenced reclamation units RU a ' _0, RU B ' _0, and RU C ' _0 and write them to reclamation unit 1082 identified as RU alpha. Because the valid data copied from the previously referenced reclamation units RU A '_0, RU B' _0, and RU C '_0 may be the last remaining valid data in one or more of these reclamation units, one or more of reclamation units RU A' _0, RU B '_0, and/or RU C' _0 may be erased (e.g., as part of a garbage collection operation) for reuse in storing other data. Recovery units that have been erased (e.g., garbage collected) may be removed from the recovery unit association table that they may have been listed.

In some embodiments, because data written using different reclamation unit handles 1036 may be initially isolated in different reclamation units, the isolation scheme shown in fig. 10 may be referred to as an initial isolation scheme, but may ultimately be combined, for example, by subsequent operations such as garbage collection operations, media management operations (e.g., refresh programming, read disturbances), and the like. In some embodiments, because the host may determine (e.g., using the reclamation unit handle) the placement of data in the insulated reclamation units, the isolation scheme illustrated in FIG. 10 may be referred to as a host isolation scheme.

TABLE 1

Although one recovery group 1018 is shown in fig. 10, the recovery units 1014 may be arranged in any number of recovery groups. In some embodiments, data from reclamation units in different reclamation groups (e.g., valid data from previously referenced reclamation units in different reclamation groups) may be combined in the same reclamation unit.

FIG. 11 illustrates an embodiment of a persistence isolation scheme for a flexible data placement scheme for a storage device in accordance with a disclosed example embodiment. The embodiment shown in FIG. 11 may include three reclamation unit handles 1136 identified as RUH X, RUH Y, and RUH Z, which may be used to write data to the currently referenced reclamation unit 1114 and/or the previously referenced reclamation unit 1114' in a manner similar to that described above with respect to FIG. 10.

However, the isolation scheme shown in FIG. 11 may involve more isolation of data written using a different reclamation unit handle than the embodiment described above with respect to FIG. 10. For example, in the embodiment shown in FIG. 11, data written using different reclamation unit handles may not be combined in a single reclamation unit in a controller operation that may move the data from a previously referenced reclamation unit 1114'. This is illustrated in fig. 11, where the controller may read valid data from (e.g., only from) the previously referenced reclamation units RU a ' _0, RU a ' _1, and/or RU a ' _2 written using the same reclamation unit handle RUH X, and write valid data to the reclamation unit 1182 identified as ruα. However, in some embodiments, the reclamation unit RU may not receive data from reclamation units written using other reclamation handles (such as RUH Y and/or RUH Z).

Similarly, the controller may read valid data from (e.g., only from) the previously referenced reclamation units RU B ' _0, RU B ' _1, and/or RU B ' _2 written using the same reclamation unit handle RUH Y, and write valid data to the reclamation unit 1182 identified as RU β. The controller may also read valid data from (e.g., only from) the previously referenced reclamation units RU C ' _0, RU C ' _1, and/or RU C ' _2 written using the same reclamation unit handle RUH Z, and write valid data to the reclamation unit 1182 identified as RU gamma. Thus, in some embodiments, data written to one or more reclamation units 1182 may be read (e.g., read only) from one or more reclamation units written using the same reclamation unit handle.

If the valid data read from any previously referenced reclamation unit 1114' is the last remaining valid data in the reclamation unit, the reclamation unit may be erased (e.g., as part of a garbage collection operation) for reuse in storing other data.

In some embodiments, the isolation scheme illustrated in FIG. 11 may be referred to as a persistent isolation scheme because the isolation between data written using different reclamation unit handles may continue beyond write and/or deallocation operations (e.g., including one or more garbage collection and/or other controller operations). In some embodiments, the isolation scheme shown in FIG. 11 may be referred to as a full or full isolation scheme because isolation between data written using different reclamation unit handles may continue throughout the life cycle of the data in the storage device.

Although one recovery group 1118 may be shown in fig. 11, recovery units 1114 may be arranged in any number of recovery groups. In some embodiments, data from reclamation units in different reclamation groups (e.g., valid data from previously referenced reclamation units in different reclamation groups) may be combined in the same reclamation unit.

Any of the storage devices, storage media, etc. disclosed herein may be implemented using any type of non-volatile storage media based on solid state media, magnetic media, optical media, etc. For example, in some embodiments, the computing storage may be implemented as a NAND (NAND) flash based SSD, a persistent memory such as a cross-grid non-volatile memory, a memory with a change in bulk resistance, a Phase Change Memory (PCM), or the like, or any combination thereof.

Any of the storage devices disclosed herein may be implemented in any form factor (such as 3.5 inches, 2.5 inches, 1.8 inches, m.2, enterprise, and data center SSD form factor (EDSFF), NF1, etc.) using any connector configuration (such as serial Advanced Technology Attachment (ATA) (SATA), small Computer System Interface (SCSI), serial Attached SCSI (SAS), U.2, etc.).

Any of the storage devices disclosed herein may be implemented in whole or in part using and/or in combination with a server chassis, a server rack, a data room (datacenter), a data center, an edge data center, a mobile edge data center, and/or any combination thereof.

Any of the hosts disclosed herein may be implemented using any component (such as a computing server, storage server, web server, cloud server, etc., node (such as a storage node), computer (such as a workstation, personal computer, tablet), smart phone, etc., or a combination of multiple and/or combinations of components thereof).

Any of the communication connections and/or communication interfaces disclosed herein may be implemented using any type of interface and/or protocol using one or more interconnects, one or more networks, a network of networks (e.g., the internet), etc., or a combination thereof. Examples may include peripheral component interconnect express (PCIe), NVMe, fabric-based NVMe (NVMe-oh), ethernet, transmission control protocol/internet protocol (TCP/IP), direct Memory Access (DMA), remote DMA (RDMA), converged ethernet-based RDMA (ROCE), fibre channel, infiniband, serial ATA (SATA), small Computer System Interface (SCSI), serial Attached SCSI (SAS), iWARP, computing fast link (CXL) and/or coherence protocols (such as cxl.mem, cxl.cache, cxl.io, etc.), gen-Z, open coherence accelerator processor interface (opencaps), cache coherence interconnect for accelerators (CCIX), etc., advanced extensible interface (a), any generation wireless network (including 2G, 3G, 4G, 5G, 6G, etc.), wi-Fi, bluetooth, access communication (NFC), etc., or any combination thereof.

Any of the functions described herein, including host functions, storage device functions, etc. (e.g., any storage device controller, logic, etc.), may be implemented using hardware, software, firmware, or any combination thereof, including, for example, hardware and/or software combinational logic, sequential logic, timers, counters, registers, state machines, volatile memory (such as DRAM and/or SRAM), non-volatile memory (including flash memory), persistent memory (such as cross-grid non-volatile memory), memory with body resistance variations, PCM, etc., and/or any combination thereof, complex Programmable Logic Device (CPLD) Field Programmable Gate Array (FPGA), application specific circuitry (ASIC), central Processing Unit (CPU) (including Complex Instruction Set Computer (CISC) processors (such as x86 processors) and/or Reduced Instruction Set Computer (RISC) processors (such as processors), graphics processor (ARM), neural processor (GPU), tensor Processor (TPU), etc., executing instructions stored in any type of memory. In some embodiments, one or more components may be implemented as a system on a chip (SOC).

In embodiments implemented at least in part using a storage device having a Flash Translation Layer (FTL), any of the functions described herein (e.g., any storage device controller, logic, etc.) may be implemented at least in part using the FTL.

In some embodiments, the reclamation unit may include a physical non-volatile storage device that may be reclaimed (e.g., erased, reused, etc.) as a unit. Depending on implementation details, the recovery unit may be recovered without disturbing one or more other recovery units. In some embodiments, the reclamation unit may be implemented as a physical construct only (e.g., may be independent of a logical address or Logical Block Address (LBA)).

In some embodiments, the namespaces can include capacity allocated in one or more reclamation units, for example. The reclamation group may include one or more reclamation units, and the one or more placement handles may reference the one or more reclamation units (e.g., may be targeted by one or more I/O commands). In some embodiments, I/O commands executing on one reclamation group may not interfere with the performance, reliability, etc. of commands executing on another reclamation group.

In some embodiments, the placement identifier may specify a reclamation group paired with a placement handle, a reclamation unit handle, and the like. The placement identifier may reference a reclamation unit that may be used, for example, to write random LBAs (e.g., to write user data to a nonvolatile storage device assigned to the reclamation unit). The write capacity of the reclamation unit referenced by the placement identifier may be incremented (e.g., on each write command) in relation to one or more write commands specifying the placement identifier, placement handle, reclamation unit handle, etc., which may in turn be modified to reference another reclamation unit once the capacity of the reclamation unit is partially or fully written.

In some embodiments, the host may keep track of user data (e.g., one or more LBAs of user data) written to one or more reclamation units. Depending on implementation details, this may enable the host to deallocate some or all of the user data (e.g., all LBAs) associated with a particular reclamation unit together (e.g., simultaneously). Depending on implementation details, this may reduce or minimize controller garbage collection, thereby reducing write amplification. In some embodiments, the host may be responsible for managing the placement identifier, placement handle, reclamation unit handle, and/or other related device resources.

In some embodiments, the reclamation unit handle may include a reference to the reclamation unit (e.g., in each reclamation group) that may place the user data for the write command. In some embodiments, the reclamation units referenced by the reclamation unit handle may only be permitted to be referenced by at most one reclamation unit handle. However, in some embodiments, when a reclamation unit is cycled from erase and back to use, the particular reclamation unit may be referenced by the same or a different reclamation unit handle. When a reclamation unit is written to capacity, the controller may update the associated reclamation unit handle to reference a different reclamation unit that is available to write user data (e.g., a non-volatile storage medium that may have been erased prior to writing) and to have written little or no user data (e.g., an empty reclamation unit).

As described above, the storage device according to the disclosed example embodiments may perform a data operation based on a request other than a write command. For example, in some embodiments, the storage device may receive a request for an update operation (e.g., as described with respect to fig. 3A-3D) and an indication of data locality (such as one or more reclamation unit handles to be updated). In some such embodiments, the storage device may use information about data to be stored in one or more reclamation units referenced by one or more updated reclamation unit handles to compose and/or select one or more reclamation units for use in the updated reclamation unit handles. For example, the storage device may receive an update request from the host along with the reclamation unit handle to be updated. The host may also send information indicating that the data to be stored in the reclamation unit referenced by the reclamation unit handle may be relatively hot data. Thus, the storage device may compose and/or select a reclamation unit having a relatively large number of remaining P/E cycles based on information received from the host, and update the reclamation unit handle to point to the composed and/or selected reclamation unit. In some such embodiments, the storage device may hold a data isolation scheme (e.g., an initial isolation scheme and/or a persistent isolation scheme as described herein).

FIG. 12 illustrates an example embodiment of a host device that can be used to implement any of the host functions disclosed herein, in accordance with the example embodiments disclosed. The host device (or host) 1200 shown in fig. 12 may include a processor 1202, and the processor 1202 may include a memory controller 1204, a system memory 1206, host control logic 1208 (the host control logic 1208 may be used to implement placement logic 963 shown in fig. 9, for example), and/or a communication interface 1210. Any or all of the components shown in fig. 12 may communicate over one or more system buses 1212. In some embodiments, one or more of the components shown in fig. 12 may be implemented using other components. For example, in some embodiments, host control logic 1208 may be implemented by processor 1202 executing instructions stored in system memory 1206 or other memory.

FIG. 13 illustrates an example embodiment of a storage device that may be used to implement any of the storage device functions disclosed herein, in accordance with the example embodiments disclosed. The storage device 1300 may include a device controller 1302 (the device controller 1302 may be used to implement reclamation unit logic 965, such as shown in fig. 9), a media conversion layer 1304 (e.g., FTL), a storage medium 1306, and/or a communication interface 1310. The components shown in fig. 13 may communicate via one or more device buses 1312. In some embodiments, where flash memory may be used for some or all of storage media 1306, media conversion layer 1304 may be implemented, in part or in whole, as a flash conversion layer (FTL).

FIG. 14 illustrates an embodiment of a method for flexible data placement of a storage device according to a disclosed example embodiment. The method may begin at operation 1402.

At operation 1404, the method may receive a data operation request at a storage device, wherein the data operation request specifies data and a reclamation unit handle. For example, the data operation request may include a request for a write operation, a copy operation, a deallocation operation, a clean operation, an erase operation, a format operation, a compare and write operation, and the like. In some example embodiments, the data operation request may include a write command (such as write commands 330, 530, and/or 930 as shown in fig. 3A-3D, 5, and/or 9).

At operation 1406, the method may receive information about the data at the storage device. The information about the data may include one or more of access information, information about one or more acceptable characteristics of a storage medium used to implement a reclamation unit of the data, information about one or more attributes of the data to be stored in the device, information about the type and/or usage of the data to be stored, and the like. In some example embodiments, information about the data (e.g., information 861 and/or 961) may be received by storage 804 and/or 904 as shown in fig. 8 and/or 9.

At operation 1408, the method may perform a data operation associated with a reclamation unit of at least one storage medium of the storage device based on the data operation request, the reclamation unit handle, and the information. For example, as shown in fig. 8, information 861 may indicate that the data to be stored may be relatively cool data that may be stored in a reclamation unit that includes an erase block having a relatively high P/E cycle count. The method may end at operation 1410.

The embodiment illustrated in fig. 14, as well as all other embodiments described herein, are example operations and/or components. In some embodiments, some operations and/or components may be omitted and/or other operations and/or components may be included. Moreover, in some embodiments, the temporal order and/or the spatial order of operations and/or components may be changed. Although some components and/or operations may be shown as separate components, in some embodiments, some components and/or operations shown separately may be integrated into a single component and/or operation and/or some components and/or operations shown as a single component and/or operation may be implemented using multiple components and/or operations.

Some embodiments disclosed above have been described in the context of various implementation details, but the principles of the disclosure are not limited to these or any other specific details. For example, some functions have been described as being implemented by a particular component, but in other embodiments, functions may be distributed among different systems and components in different locations and with various user interfaces. Particular embodiments have been described as having particular processes, operations, etc., but these terms also include embodiments in which a particular process, operation, etc., may be implemented with multiple processes, operations, etc., or in which multiple processes, operations, etc., may be integrated into a single process, operation, etc. References to a component or element may only refer to portions of the component or element. For example, a reference to a block may represent an entire block or one or more sub-blocks. References to components or elements may refer to one or more of the components or elements, and references to multiple components or elements may refer to a single component or element. For example, a reference to a resource may represent one or more resources, and a reference to a resource may represent a single resource. Unless otherwise clear from the context, terms such as "first" and "second" are used in the present disclosure and claims for the purpose of distinguishing between elements that they modify and may not indicate any spatial or temporal order. In some embodiments, a reference to an element may represent at least a portion of the element, e.g., "based on" may represent "based at least in part on" or the like. The reference to a first element does not indicate the presence of a second element. The principles disclosed herein have independent utility and may be embodied separately and not every embodiment may utilize every principle. However, these principles may also be embodied in various combinations, some of which may amplify the benefits of the various principles in a synergistic manner. The various details and embodiments described above may be combined to produce additional embodiments in accordance with the inventive principles of this patent disclosure.

Since the inventive principles of this patent disclosure may be modified in arrangement and detail without departing from the inventive concepts, such changes and modifications are considered to be within the scope of the appended claims.

Claims (20)

1. A storage device, comprising:

at least one storage medium; and

a controller configured to:

receiving a write command, wherein the write command specifies data and a reclamation unit handle;

receiving information about the data; and is also provided with

Based on the reclamation unit handle and the information, data is stored in the reclamation unit of the at least one storage medium.

2. The storage device of claim 1, wherein the information comprises access information.

3. The storage device of claim 1, wherein the information comprises error tolerance information.

4. The storage device of claim 1, wherein the information comprises data attribute information.

5. The storage device of claim 1, wherein the information comprises data type information.

6. The storage device of claim 1, wherein the controller is configured to determine the information based at least in part on a reclamation unit handle.

7. The storage device of claim 1, wherein the controller is configured to:

Receiving an indicator separate from the reclamation unit handle; and is also provided with

The information is determined based at least in part on the indicator.

8. The storage device of claim 1, wherein the controller is configured to select the reclamation unit based on the information.

9. The storage device of claim 8, wherein the controller is configured to select the reclamation unit based on characteristics of at least a portion of the reclamation unit.

10. The memory device of claim 9, wherein the characteristic comprises a number of programming cycles.

11. The storage device of claim 9, wherein the characteristic comprises an error accumulation characteristic.

12. The storage device of claim 1, wherein the controller is configured to compose a reclamation unit based on the information.

13. The storage device of claim 12, wherein the controller is configured to compose the reclamation unit based on characteristics of at least a portion of the reclamation unit.

14. The memory device of claim 13, wherein the characteristic comprises a number of programming cycles.

15. The storage device of claim 13, wherein the characteristic comprises an error accumulation characteristic.

16. An apparatus for forming and selecting a recovery unit, comprising:

Placement logic configured to:

transmitting a write command to the storage device, wherein the write command specifies data and a reclamation unit handle, wherein the reclamation unit handle is to reference a reclamation unit in at least one storage medium of the storage device; and is also provided with

Information about the data is sent to a storage device.

17. The apparatus of claim 16, wherein the information comprises access information.

18. A method of forming and selecting a recovery unit comprising:

receiving a data operation request at a storage device, wherein the data operation request specifies data and a reclamation unit handle;

receiving information about the data at the storage device; and is also provided with

A data operation associated with a reclamation unit of at least one storage medium of the storage device is performed based on the data operation request, the reclamation unit handle, and the information.

19. The method of claim 18, wherein the data operation request comprises a write operation request and the data operation comprises a write operation.

20. The method of claim 19, wherein the write operation request comprises a write command and the write operation comprises storing data in the reclamation unit based on the reclamation unit handle and the information.