CN111209226A

CN111209226A - Method and device for reducing NVMe solid state disk write amplification and computer equipment

Info

Publication number: CN111209226A
Application number: CN202010011209.3A
Authority: CN
Inventors: 付东松; 邹一凡; 张健
Original assignee: Shenzhen Union Memory Information System Co Ltd
Current assignee: Shenzhen Union Memory Information System Co Ltd
Priority date: 2020-01-06
Filing date: 2020-01-06
Publication date: 2020-05-29

Abstract

The application relates to a method and a device for reducing NVMe solid state disk write amplification, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring the number of streams, the Stream ID and the SGS information of each Stream supported by the solid state disk; allocating an independent SGS writing point for each Stream; binding data with different refreshing frequencies into corresponding streams according to the refreshing frequency of the application data in the application program; according to the established binding relationship, assigning Stream IDs for all write commands constructed by the application program; and writing the data into the corresponding independent SGS write point according to the Stream ID specified in the write command. The invention separates the data streams with different refreshing frequencies by using the Stream function of NVMe, and proposes an independent write point for each Stream, so that the data with the same refreshing frequency is only stored in the same SGS write point, thereby greatly reducing the write amplification of SSD.

Description

Method and device for reducing NVMe solid state disk write amplification and computer equipment

Technical Field

The invention relates to the technical field of solid state disks, in particular to a method and a device for reducing NVMe solid state disk write amplification, computer equipment and a storage medium.

Background

At present, the solid state disk of NVMe matched with Nand Flash storage medium is gradually applied to consumer-level and enterprise-level markets by virtue of the advantages of high performance and low power consumption. In some cases, the host may have multiple applications to write user data into the SSD, the data streams of different applications have different refresh frequencies (cold and hot levels), the data of each application may be split by the host into multiple commands to be sent to the SSD, and the commands of different applications may be sent to the SSD across each other, as shown in fig. 1.

In the prior art, since the existing firmware technology cannot effectively distinguish the refresh frequency of the commands (data), the data can only be written into the physical blocks of the flash memory according to the sequence of commands issued by the host, so that data with different refresh frequencies can be written into the same physical block, as shown in fig. 1. Therefore, many garbage blocks (physical blocks contain both valid data and invalid data) are generated subsequently along with frequent refreshing of hot data, and a garbage collection task is frequently triggered, which can lead to a sudden increase in write amplification (ratio of flash memory write data amount to host write data amount) of the SSD, thereby reducing the service life of the SSD.

Disclosure of Invention

In view of the foregoing, it is necessary to provide a method, an apparatus, a computer device, and a storage medium, which can reduce NVMe solid state disk write amplification.

A method for reducing NVMe solid state disk write amplification, the method comprising:

acquiring the number of streams, the Stream ID and the SGS information of each Stream supported by the solid state disk;

allocating an independent SGS writing point for each Stream;

binding data with different refreshing frequencies into corresponding streams according to the refreshing frequency of the application data in the application program;

according to the established binding relationship, assigning Stream IDs for all write commands constructed by the application program;

and writing the data into the corresponding independent SGS write point according to the Stream ID specified in the write command.

In one embodiment, the step of writing data into the corresponding independent SGS write point according to the Stream ID specified in the write command further includes:

receiving a host command and acquiring a direct Specific ID of the command;

judging whether the SGS of the Stream _ Directive Specific write point is full;

if not, writing the data into the write-in point of Stream _ direct Specific.

In one embodiment, after the step of determining whether the SGS of the Stream _ direct Specific write point is full, the method further includes:

if the write is full, migrating the SGS of the Stream _ Directive Specific write point to an SGS data pool;

and obtaining a new SGS from the SGS free pool as a new write-in point of Stream _ direct Specific, and writing data into the new write-in point.

In one embodiment, the SGS indicates a flash write unit preset by the solid state disk for each Stream, and the flash write unit includes a plurality of physical blocks for storing corresponding Stream data, and when the space of the flash write unit is used up, the solid state disk reallocates the flash write unit with the SGS size to the corresponding Stream.

An apparatus for reducing NVMe solid state disk write amplification, the apparatus comprising:

the acquisition module is used for acquiring the number of streams, the Stream ID and the SGS information of each Stream supported by the solid state disk;

the allocation module is used for allocating an independent SGS write point for each Stream;

the binding module is used for binding the data with different refreshing frequencies to the corresponding Stream according to the refreshing frequency of the application data in the application program;

the specifying module is used for specifying Stream IDs for all write commands constructed by the application program according to the established binding relationship;

and the writing module is used for writing the data into the corresponding independent SGS writing point according to the Stream ID specified in the writing command.

In one embodiment, the write module is further configured to:

receiving a host command and acquiring a direct Specific ID of the command;

judging whether the SGS of the Stream _ Directive Specific write point is full;

if not, writing the data into the write-in point of Stream _ direct Specific.

In one embodiment, the write module is further configured to:

A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any of the above methods when executing the computer program.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of any of the methods described above.

According to the method, the device, the computer equipment and the storage medium for reducing the NVMe solid state disk write amplification, the number of streams, the Stream ID and the SGS information of each Stream supported by the solid state disk are obtained; allocating an independent SGS writing point for each Stream; binding data with different refreshing frequencies into corresponding streams according to the refreshing frequency of the application data in the application program; according to the established binding relationship, assigning Stream IDs for all write commands constructed by the application program; and writing the data into the corresponding independent SGS write point according to the Stream ID specified in the write command. The method separates the data streams with different refreshing frequencies by using the Stream function of NVMe, and proposes an independent write-in point for each Stream, so that the data with the same refreshing frequency is only stored in the same SGS write-in point, and the data in the SGS of the write-in point can lose effectiveness at the same time at a large probability, thereby greatly reducing the probability of triggering a garbage recovery task and greatly reducing the write amplification of the SSD.

Drawings

FIG. 1 is a diagram illustrating data writing without separating cold and hot data in the prior art;

FIG. 2 is a schematic flow chart illustrating a method for reducing NVMe solid state disk write amplification in one embodiment;

FIG. 3 is a schematic flow chart illustrating a method for reducing NVMe solid state disk write amplification in another embodiment;

FIG. 4 is a flowchart illustrating a method for reducing NVMe solid state disk write amplification in yet another embodiment;

FIG. 5 is a schematic diagram illustrating the differentiation of hot and cold data by a Stream function in one embodiment;

FIG. 6 is a schematic diagram of an SGS based write flow in one embodiment;

FIG. 7 is a block diagram of an apparatus for reducing NVMe solid state disk write amplification in an embodiment;

FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

At present, different data refresh frequencies of different application programs at a host end are sent to an SSD in a crossed manner, and an existing SSD firmware technology cannot distinguish the refresh frequency of host-issued data (a command describes a section of continuous LBA, and each LBA carries 512B data), and data is written to a physical block of a flash memory through flash memory write points according to a host-issued sequence, as shown in fig. 1, it is assumed that one physical block is composed of 4 physical pages, and each physical page is 512B.

It can be seen that physical block 1 stores part of the data of applications 1 and 2, as does physical block 2. Since the data of application 1 is refreshed faster, the host will soon overwrite application 1, its new data (LBA 0-LBA 3) is written to physical block 3, and the old data previously written in physical blocks 1 and 2 is invalidated due to the characteristics of flash memory: the physical page is the smallest read-write unit, the physical block is the smallest erase unit, and the write operation can be performed only after the physical block is erased integrally.

If the available physical blocks on the flash memory are not enough, the garbage collection task is started inside the SSD, and the valid physical page data (LBA10 to LBA13) in the physical blocks 1 and 2 are moved to the new physical block 4, which needs to be noted that the SSD does extra read and write operations, so that write amplification occurs, and the total life of the flash memory is reduced. Physical blocks 1 and 2, which are all invalid physical pages, are then erased, thus freeing up free physical blocks that can store data that is downloaded by a subsequent host.

Therefore, in the prior art, the refresh frequency of the data stream cannot be distinguished, so that the garbage collection task is frequently triggered, write amplification is caused, and the service life of the SSD is shortened.

Based on the above, the invention provides a method for reducing the write amplification of the NVMe solid state disk, which is expected to utilize the new Stream characteristic in the NVMe1.3 protocol, effectively separate data with different refresh frequencies in SSD firmware, write the data into different physical blocks as much as possible, reduce the trigger probability of garbage recovery tasks, reduce the write amplification of the SSD and prolong the service life of the SSD.

In one embodiment, as shown in fig. 2, there is provided a method for reducing NVMe solid state disk write amplification, the method including:

step 202, acquiring the number of streams, Stream ID and SGS information of each Stream supported by the solid state disk;

step 204, distributing an independent SGS writing point for each Stream;

step 206, binding the data with different refresh frequencies into corresponding streams according to the refresh frequency of the application data in the application program;

step 208, appointing Stream IDs for all write commands constructed by the application program according to the established binding relationship;

and step 210, writing the data into the corresponding independent SGS write point according to the Stream ID specified in the write command.

The crux of the prior art is that data with different refresh frequencies at a host end are stored in the same physical block of an SSD in a crossed manner, so that the SSD firmware frequently triggers a garbage collection task. If the data with different refresh frequencies can be separated, and the data with the same or similar refresh frequencies can be written into the same physical block as intensively as possible, the probability that the physical block contains the data with different refresh frequencies is greatly reduced, so that the probability of garbage recovery triggering is greatly reduced, and the write amplification is greatly reduced.

In one embodiment, the SGS indicates that the solid state disk presets a flash memory write unit for each Stream, and the flash memory write unit includes a plurality of physical blocks for storing corresponding Stream data, and when the space of the flash memory write unit is used up, the solid state disk reallocates the flash memory write unit with the size of the SGS to the corresponding Stream.

In this embodiment, the nvme1.3 protocol adds a Stream property, specifically, the SSD defines the number of supported Streams and Stream ID information, each Stream has an SGS (Stream Granularity size) attribute, the SGS indicates that the SSD firmware is a flash write unit preset for the Stream, and usually includes N physical blocks, and then the data of the Stream is stored in the pre-allocated flash capacity of the SGS size, and when the space of the flash capacity is used up, the firmware allocates a write unit of the SGS size to the Stream again. For convenience of description, it is assumed here that the physical page size is 512B, one physical block is composed of 4 physical pages, the SSD supports 2 streams, IDs are Stream0 and Stream1, respectively, and SGS of Stream0 and Stream1 are defined as 4KB (2 physical blocks).

In a specific embodiment, as shown in fig. 6, the design of Stream inside SSD is as follows:

the size of the SGS is defined as 4KB in this example, and is equal to the size of 2 physical blocks, and the flash memory is abstracted into storage units of SGS, each unit is composed of two physical blocks. The module for managing the SGS resources by the SSD comprises:

SGS free pool: all free available SGSs are managed in this pool;

writing point: the host data is written into the SGS currently on the flash memory and is obtained from the SGS free pool;

SGS data pool: and after the SGS of a certain Stream writing point is full, the SGS is moved to the SGS data pool, and then a new SGS is taken from the SGS free pool.

Specifically, the steps of processing Stream at the host end are as follows:

first, the support of the Stream function is explained in the identification data (SSD basic information) of the SSD.

Then, the host sends Admin commands Return Parameters (Directive Operation 01h) and Get Status (Directive Operation 02h) introduced in section nvme9.3 to obtain the number of streams supported by the SSD, ID information, and SGS information of each Stream.

Then, the application layer or file system of the host decides which Stream the data of each refresh frequency is bound to according to the refresh frequency of various application data, and it is assumed that application 1 binds Stream0 and application 2 binds Stream1 in fig. 1. Certainly, in an actual situation, the number of streams supported by the SSD and the number of refresh frequency steps of various application data at the host end are not the same, which requires the application layer or the file system of the host to establish a binding relationship according to the actual situation, and the good binding relationship can exert the technical effect provided by the present invention.

Finally, the host specifies Stream IDs for all write commands constructed by an application according to the established binding relationship, as shown in fig. 5, in this example, the application 1 binds Stream0, and then the Directive Type of all write commands sent by the application 1 to the SSD is set to 01 (indicating that the Stream function is enabled), and the Directive Specific is set to the corresponding Stream ID, which is 0 here. Similarly, the Directive Type of all write commands sent by the application program 2 to the SSD is set to 01, and the Directive Specific is set to 1. Therefore, in the subsequent writing process, as the SSD firmware writes the data of the command into the Stream write point specified by the field Directive Specific, the write point of Stream0 always stores the host data with Directive Specific of 0, and the write point of Stream1 always stores the host data with Directive Specific of 1, that is, the data with different refresh frequencies are stored in different SGSs, and are not stored across one hundred percent, which can minimize the probability of triggering garbage collection.

In the embodiment, the number of streams, the Stream ID and the SGS information of each Stream supported by the solid state disk are acquired; allocating an independent SGS writing point for each Stream; binding data with different refreshing frequencies into corresponding streams according to the refreshing frequency of the application data in the application program; according to the established binding relationship, assigning Stream IDs for all write commands constructed by the application program; and writing the data into the corresponding independent SGS write point according to the Stream ID specified in the write command. According to the scheme, the data streams with different refreshing frequencies are separated by using the Stream functions of NVMe, and an independent write-in point is proposed for each Stream, so that the data with the same refreshing frequency is only stored to the same SGS write-in point, and thus the data in the SGS of the write-in point can lose effectiveness at the same time at a large rate, the probability of triggering a garbage recovery task is greatly reduced, and the write amplification of the SSD is greatly reduced.

In an embodiment, as shown in fig. 3, a method for reducing NVMe solid state disk write amplification is provided, in which the step of writing data into a corresponding independent SGS write point according to a Stream ID specified in a write command further includes:

step 302, receiving a host command and obtaining a direct Specific ID of the command;

step 304, judging whether the SGS of the Stream _ Directive Specific write point is full;

step 306, if not, writing the data into the write-in point of Stream _ direct Specific.

In one embodiment, as shown in fig. 4, a method for reducing NVMe solid state disk write amplification is provided, where the method further includes, after the step of determining whether the SGS of the Stream _ direct Specific write point is full:

step 402, if the write is full, transferring the SGS of the Stream _ Directive Specific write point to an SGS data pool;

step 404, obtaining a new SGS from the SGS free pool as a new write point of Stream _ direct Specific, and writing data into the new write point.

In the present embodiment, a complete host data write and management flow of the write point SGS is provided. First, a host command is received and a direct Specific ID of the command is obtained. Then, it is judged whether the SGS of the Stream _ directive specific write point is full. If not, writing the data into the write-in point of Stream _ direct Specific. And if the write is full, migrating the SGS of the Stream _ Directive Specific write point to an SGS data pool. And finally, obtaining a new SGS from the SGS free pool as a new write-in point of Stream _ direct Specific, and writing data into the new write-in point.

In the above embodiment, the core idea is to allocate an independent write point, i.e. an independent SGS, for each Stream, SSD firmware writes the data of the command into the Stream write point specified by the field direct Specific, as shown in fig. 6, the write point of Stream0 always stores host data with direct Specific of 0, the write point of Stream1 always stores host data with direct Specific of 1, the number of write points in firmware design is equal to the number of streams supported by SSD, so data with different refresh frequencies are stored into different SGSs, and are hundreds of non-interleaved storage, which can minimize the probability of triggering garbage collection.

It should be understood that although the various steps in the flow charts of fig. 2-4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-4 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.

In one embodiment, as shown in fig. 7, there is provided an apparatus 700 for reducing NVMe solid state disk write amplification, the apparatus comprising:

an obtaining module 701, configured to obtain the number of streams, the Stream ID, and the SGS information of each Stream that are supported by the solid state disk;

an allocating module 702, configured to allocate an independent SGS write point for each Stream;

a binding module 703, configured to bind, according to the refresh frequency of the application data in the application program, data with different refresh frequencies to a corresponding Stream;

an assigning module 704, configured to assign Stream IDs for all write commands constructed by the application program according to the established binding relationship;

a writing module 705, configured to write data into the corresponding independent SGS write point according to the Stream ID specified in the write command.

In one embodiment, the write module 705 is further configured to:

receiving a host command and acquiring a direct Specific ID of the command;

judging whether the SGS of the Stream _ Directive Specific write point is full;

if not, writing the data into the write-in point of Stream _ direct Specific.

In one embodiment, the write module 705 is further configured to:

In an embodiment, the SGS indicates a flash write unit preset by the solid state disk for each Stream, and the flash write unit includes a plurality of physical blocks for storing corresponding Stream data, and when the space of the flash write unit is used up, the solid state disk reallocates the flash write unit with the SGS size to the corresponding Stream.

For specific limitations of the apparatus for reducing NVMe solid state disk write amplification, reference may be made to the above limitations of the method for reducing NVMe solid state disk write amplification, and details are not repeated here.

In one embodiment, a computer device is provided, the internal structure of which may be as shown in FIG. 8. The computer apparatus includes a processor, a memory, and a network interface connected by a device bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The nonvolatile storage medium stores an operating device, a computer program, and a database. The internal memory provides an environment for the operation device in the nonvolatile storage medium and the execution of the computer program. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method for reducing NVMe solid state disk write amplification.

Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method embodiments when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the above respective method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method for reducing NVMe solid state disk write amplification, the method comprising:

allocating an independent SGS writing point for each Stream;

2. The method for reducing NVMe solid state disk write amplification of claim 1, wherein the step of writing data to the corresponding independent SGS write point according to the Stream ID specified in the write command further comprises:

receiving a host command and acquiring a direct Specific ID of the command;

judging whether the SGS of the Stream _ Directive Specific write point is full;

if not, writing the data into the write-in point of Stream _ direct Specific.

3. The method for reducing NVMe solid state disk write amplification of claim 2, further comprising, after the step of determining whether the SGS of the Stream _ direct Specific write point is full:

4. The method for reducing NVMe solid state disk write amplification according to any one of claims 1-3, wherein the SGS indicates flash memory write units preset by the solid state disk for each Stream, and the flash memory write units comprise a plurality of physical blocks for storing corresponding Stream data, and when the space of the flash memory write units is used up, the solid state disk reallocates the flash memory write units with the size of the SGS to the corresponding Stream.

5. An apparatus for reducing NVMe solid state disk write amplification, the apparatus comprising:

6. The apparatus for reducing NVMe solid state disk write amplification of claim 5, wherein the write module is further configured to:

receiving a host command and acquiring a direct Specific ID of the command;

judging whether the SGS of the Stream _ Directive Specific write point is full;

if not, writing the data into the write-in point of Stream _ direct Specific.

7. The apparatus for reducing NVMe solid state disk write amplification of claim 6, wherein the write module is further configured to:

8. The apparatus for reducing NVMe solid state disk write amplification according to any one of claims 5-7, wherein the SGS indicates flash memory write units preset by the solid state disk for each Stream, and the flash memory write units comprise a plurality of physical blocks for storing corresponding Stream data, and when the space of the flash memory write units is used up, the solid state disk reallocates the flash memory write units with the size of the SGS to the corresponding Stream.

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 4 are implemented when the computer program is executed by the processor.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 4.