WO2021068517A1

WO2021068517A1 - Stored data sorting method and device

Info

Publication number: WO2021068517A1
Application number: PCT/CN2020/092840
Authority: WO
Inventors: 史顺玉
Original assignee: 苏州浪潮智能科技有限公司
Priority date: 2019-10-10
Filing date: 2020-05-28
Publication date: 2021-04-15
Also published as: CN110780814A; CN110780814B

Abstract

A stored data sorting method and device. Said method comprises: within erase blocks involved in a defragmentation task request, locking write pages which have not been locked; determining a valid logic block in the write pages of the erase blocks according to a metadata hash table and extracting valid information therefrom, and adding recyclable tags to the write pages; performing an erase operation on the included erase blocks with all the write pages having recyclable tags, and removing the recyclable tags of all the write pages in the erase blocks and locking same; and determining, according to wear leveling, an idle low-wear erase block to store the valid information. The present invention can solve the problem of hard disk access conflict, and improve the service life of a hard disk and the usage experience.

Description

Method and device for sorting stored data

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on October 10, 2019, the application number is 201910957836.3, and the invention title is "A method and device for storing data sorting", the entire content of which is incorporated into this application by reference in.

Technical field

The present invention relates to the field of data storage, and more specifically, to a method and device for organizing stored data.

Background technique

With the continuous reduction of the cost of SSD (flash memory) hard drives and the popularization of applications, the development of storage systems based on all-flash memory has also been put on the agenda. Although SSD has the characteristics of fast reading and writing, light weight, low energy consumption, and small size, it also has obvious disadvantages. For example, its service life is limited, that is, the data storage unit has a limit on the number of erasing and writing. In addition, invalid data erasing cannot be rewritten at will in units of bits like a mechanical disk. For these shortcomings, existing technologies also have optimization methods, such as wear leveling, garbage collection, and bad block mapping, etc., but they will conflict with daily reading and writing, affecting the life of the hard disk and the user experience.

There is currently no effective solution to the problem of conflicts in SSD storage optimization in the prior art.

Summary of the invention

In view of this, the purpose of the embodiments of the present invention is to provide a method and device for arranging stored data, which can solve the problem of hard disk access conflicts and improve the working life and use experience of the hard disk.

Based on the foregoing objectives, the first aspect of the embodiments of the present invention provides a storage data defragmentation method, which executes the following steps in response to receiving a defragmentation task request:

Lock the write pages that have not been locked in the erase block involved in the defragmentation task request;

Determine a valid logical block in the write page of the erase block according to the metadata hash table, extract valid information from it, and add a recyclable mark to the written page;

Perform erasure on erase blocks with recyclable marks on all written pages, and remove recyclable marks and locks on all written pages in the erase block;

A free low-wear erase block is determined according to the wear balance to store effective information.

In some embodiments, the method further includes: in response to receiving the input/output task request, locking the involved write page that is not locked so that it cannot be locked by the defragmentation task request.

In some implementations, locking the write pages that have not been locked in the erase block involved in the defragmentation task request includes: waiting for the write page that has been locked by the input/output task request to be unlocked, and treat it as not yet locked. The locked write page is locked.

In some embodiments, the metadata hash table includes a key-value pair table composed of data volumes and logical block addresses as keys, and metadata as values;

Determining a valid logical block in the write page of the erase block according to the metadata hash table and extracting valid information therefrom includes: obtaining the metadata value of the key located in the write page of the erase block according to the metadata hash table, And read the data as valid information based on the value.

In some embodiments, the recyclable mark is also recorded in the metadata.

A second aspect of the embodiments of the present invention provides a storage data organization device, including:

Flash memory, including multiple erase blocks;

Processor; and

The memory stores program codes that can be run by the processor. When the program codes are run, they perform the following steps in response to receiving a defragmentation task request:

In some embodiments, the step further includes: in response to receiving the input/output task request, locking the involved write page that is not locked so that it cannot be locked by the defragmentation task request.

In some embodiments, the recyclable mark is also recorded in the metadata.

The present invention has the following beneficial technical effects: the storage data arranging method and device provided by the embodiments of the present invention lock the write pages that have not been locked in the erase block involved in the defragmentation task request; Determine the valid logical block in the write page of the erase block and extract valid information from it, and add a recyclable mark to the written page; perform erasing on the erase block that includes all written pages with recyclable marks , And remove the recyclable marks and locks of all written pages in the erase block; determine a free low-wear erase block to store effective information according to the wear balance, which can solve the problem of hard disk access conflicts and improve the working life of the hard disk And experience.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work.

Fig. 1 is a schematic flow chart of a method for organizing stored data provided by the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the following describes the embodiments of the present invention in detail in conjunction with specific embodiments and with reference to the accompanying drawings.

It should be noted that all the expressions "first" and "second" in the embodiments of the present invention are used to distinguish two entities with the same name but not the same or parameters that are not the same, as shown in "first" and "second" Only for the convenience of presentation, it should not be construed as a limitation to the embodiments of the present invention, and subsequent embodiments will not describe this one by one.

Based on the foregoing objectives, the first aspect of the embodiments of the present invention proposes an embodiment of a storage data organization method that can solve the problem of hard disk access conflicts. Figure 1 shows a schematic flow chart of a method for organizing stored data provided by the present invention.

The storage data arranging method, as shown in FIG. 1, includes executing the following steps in response to receiving a defragmentation task request:

Step S101: Lock the written pages that have not been locked in the erase block involved in the defragmentation task request;

Step S103: Determine a valid logical block in the written page of the erase block according to the metadata hash table, extract valid information therefrom, and add a recyclable mark to the written page;

Step S105: Perform erasing on the erase block with recyclable marks on all the written pages included, and remove the recyclable marks and locks of all written pages in the erase block;

Step S107: Determine an idle low-wear erase block to store effective information according to the wear balance.

In the embodiment of the present invention, the minimum erased data block multiple (block) of the SSD is used as the unit (for example, 128Mb) to perform the garbage collection of the all-flash storage system; the defragmentation of data is based on the multiple (page) of the minimum data block written to the flash memory (Such as 4kb). Each time a block that needs to be reclaimed is obtained, the effective fragmented data is copied to the free block in turn. Idle blocks are allocated from the back-end storage array sequentially by the storage system in accordance with the principle of the wear balance of flash disks. In the embodiment of the present invention, a task queue is designed to ensure that the recovery process and IO (input and output) do not conflict with each other.

A person of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be implemented by instructing relevant hardware through a computer program. The program can be stored in a computer readable storage medium, and the program can be executed during execution. At this time, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc. The embodiment of the computer program can achieve the same or similar effect as any of the foregoing corresponding method embodiments.

In some embodiments, the method further includes: in response to receiving the input/output task request, locking the involved write page that is not locked so that it cannot be locked by the defragmentation task request. For example, generating a task queue; in response to receiving an input and output task request, the involved write page that is not locked is locked; in response to the received input and output task request, the write page involved is locked and the input and output task is locked The request is placed in the task queue.

In some implementations, locking the write pages that have not been locked in the erase block involved in the defragmentation task request includes: waiting for the write page that has been locked by the input/output task request to be unlocked, and treat it as not yet locked. The locked write page is locked by the defragmentation task request.

In some embodiments, the metadata hash table includes a key-value pair table composed of data volumes and logical block addresses as keys, and metadata as values; the write-in of the erasure block according to the metadata hash table Determining a valid logical block in the page and extracting valid information therefrom includes: obtaining the metadata value whose key is located in the written page of the erase block according to the metadata hash table, and reading the data as valid information based on the value.

In some embodiments, the recyclable mark is also recorded in the metadata.

The method disclosed according to the embodiment of the present invention may also be implemented as a computer program executed by a CPU, and the computer program may be stored in a computer-readable storage medium. When the computer program is executed by the CPU, it executes the above-mentioned functions defined in the method disclosed in the embodiment of the present invention. The above method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to implement the above steps or unit functions.

The implementation of the present invention is further described below based on specific examples.

First, maintain host IO or GC (defragmentation/garbage cleanup) task requests through a hash (hash) table. In the K-V (key-value pair) searched by Hash, K is the volume ID of the data volume and the LBA of the current IO, and V is the metadata of the task that is performing the operation on the current page. Whenever a host IO/GC operates on a certain page, it is necessary to add a mutex lock (ie, lock) to the current page (the metadata of the page). Subsequent access requests need to wait until the current operation is completed before obtaining the permission to operate the page. Such a mutually exclusive task queue is used to store host IO requests and GC mutually exclusive task requests.

Then, obtain the data storage space block that needs to be reclaimed in the system. Traverse the data read-write unit page in the block in turn to see if there is still valid data. If it is invalid, mark it as recyclable. The detailed implementation steps are as follows:

(1) Get a page and add a mutex lock to it. If the lock is successful, query whether there is a valid LBA in its metadata. If the lock fails, skip to step (4).

(2) If there is no valid LBA. Then mark the current page as recyclable in the corresponding metadata. Prohibit new host IO requests to operate on it. Jump to step (1) and continue to inquire whether the next page can be recycled.

(3) If there is a valid LBA, the data in this page is considered valid, and the data is migrated to a new block, and the current page is marked as recyclable in the metadata. Prohibit new host IO requests to operate on it. Jump to step (1) and continue to inquire whether the next page can be recycled.

(4) If the lock fails, it means that there is a host IO request currently operating on this page. Then put the GC request into the waiting task list.

(5) When the host IO operation is completed, skip to step (2).

(6) When the recovery of the entire block is completed, the block erase operation is executed to complete the recovery of a blcok.

It can be seen from the above embodiment that the storage data defragmentation method provided by the embodiment of the present invention locks the written pages that have not been locked in the erase block involved in the defragmentation task request; Determine the valid logical block in the write page of the block and extract valid information from it, and add a recyclable mark to the written page; erase the erase block that includes all written pages with the recyclable mark, and Remove the recyclable marks and locks of all written pages in the erase block; determine a free low-wear erase block to store effective information according to the wear balance, which can solve the problem of hard disk access conflicts and improve the working life and use of the hard disk Experience.

It should be particularly pointed out that the steps in the various embodiments of the storage data organization method described above can be crossed, replaced, added, or deleted. Therefore, these reasonable permutations and combinations should also belong to the present invention for the storage data organization method. The protection scope of the present invention should not be limited to the above-mentioned embodiments.

Based on the foregoing objectives, the second aspect of the embodiments of the present invention proposes an embodiment of a storage data organization device that can solve the problem of hard disk access conflicts. Storage data sorting devices include:

Flash memory, including multiple erase blocks;

Processor; and

In some embodiments, locking the write page that has not been locked in the erase block involved in the defragmentation task request includes: waiting for the write page that has been locked by the input/output task request to be unlocked, and treat it as not being locked yet The write page is locked.

In some embodiments, the recyclable mark is also recorded in the metadata.

It can be seen from the foregoing embodiment that the storage data defragmentation device provided by the embodiment of the present invention locks the written pages that have not been locked in the erase block involved in the defragmentation task request; and erases the data according to the metadata hash table. Determine the valid logical block in the write page of the block and extract valid information from it, and add a recyclable mark to the written page; erase the erase block that includes all written pages with the recyclable mark, and Remove the recyclable marks and locks of all written pages in the erase block; determine a free low-wear erase block to store effective information according to the wear balance, which can solve the problem of hard disk access conflicts and improve the working life and use of the hard disk Experience.

It should be particularly pointed out that the foregoing embodiment of the storage data organization device uses the embodiment of the storage data organization method to specifically describe the working process of each module. Those skilled in the art can easily think of applying these modules to the In other embodiments of the storage data organization method. Of course, since the steps in the embodiment of the storage data arrangement method can be crossed, replaced, added, and deleted, these reasonable permutations and combinations should also belong to the protection of the present invention for the storage data arrangement device. The scope of the present invention should not be limited to the embodiments described.

The above are exemplary embodiments disclosed by the present invention, but it should be noted that various changes and modifications can be made without departing from the scope of the disclosure of the embodiments of the present invention as defined by the claims. The functions, steps and/or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any specific order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual forms, they may also be understood as plural unless explicitly limited to a singular number.

It should be understood that as used herein, unless the context clearly supports exceptions, the singular form "a" is intended to also include the plural form. It should also be understood that "and/or" as used herein refers to any and all possible combinations including one or more items listed in association. The serial numbers of the disclosed embodiments of the foregoing embodiments of the present invention are only for description, and do not represent the superiority or inferiority of the embodiments.

A person of ordinary skill in the art can understand that all or part of the steps in the above embodiments can be implemented by hardware, or by a program to instruct relevant hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned can be a read-only memory, a magnetic disk or an optical disk, etc.

Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is only exemplary, and is not intended to imply that the scope of disclosure (including the claims) of the embodiments of the present invention is limited to these examples; under the idea of the embodiments of the present invention The above embodiments or the technical features in different embodiments can also be combined, and there are many other changes in different aspects of the embodiments of the present invention as described above, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

A storage data arranging method, which is characterized in that it is applied to a flash memory and includes the following steps in response to receiving a defragmentation task request:

Lock the write pages that have not been locked in the erase block involved in the defragmentation task request;

Determine a valid logical block in the written page of the erase block according to the metadata hash table, extract valid information therefrom, and add a recyclable mark to the written page;

Perform erasing on the erase block with the recyclable mark on all the written pages included, and remove the recyclable mark and lock of all the written pages in the erase block;

A free low-wear erase block is determined according to the wear balance to store the effective information.
The method according to claim 1, further comprising: in response to receiving an input/output task request, locking the involved write page that is not locked so that it cannot be locked by the defragmentation task request.
The method according to claim 2, wherein locking the write page that has not been locked in the erase block involved in the defragmentation task request comprises: waiting for the input and output task request The locked write page is unlocked, and it is regarded as the write page that has not been locked and locked.
The method according to claim 1, wherein the metadata hash table comprises a key-value pair table composed of data volumes and logical block addresses as keys, and metadata as values;

Determining a valid logical block in the written page of the erase block according to the metadata hash table and extracting valid information therefrom includes: obtaining, according to the metadata hash table, that the key is located in the erase block The metadata value written in the page is removed from the block, and data as valid information is read based on the value.
The method of claim 4, wherein the recyclable mark is also recorded in the metadata.
A storage data sorting device, which is characterized in that it comprises:

Flash memory, including multiple erase blocks;

Processor; and

The memory stores program codes that can be run by the processor, and when the program codes are run, they perform the following steps in response to receiving a defragmentation task request:

Lock the write pages that have not been locked in the erase block involved in the defragmentation task request;

Determining a valid logical block in the written page of the erase block according to a metadata hash table, extracting valid information therefrom, and adding a recyclable mark to the written page;

Perform erasing on the erase block with the recyclable mark on all the written pages included, and remove the recyclable mark and lock of all the written pages in the erase block;

A free low-wear erase block is determined according to the wear balance to store the effective information.
The apparatus according to claim 6, wherein the step further comprises: in response to receiving an input/output task request, locking the involved write page that is not locked so that it cannot be defragmented by the defragmentation task. Request a lock.
7. The device according to claim 7, wherein locking the write page that has not yet been locked in the erase block involved in the defragmentation task request comprises: waiting for it to be requested by the input/output task The locked write page is unlocked, and it is regarded as the write page that has not been locked and locked.
The device according to claim 6, wherein the metadata hash table comprises a key-value pair table composed of data volumes and logical block addresses as keys, and metadata as values;

Determining a valid logical block in the written page of the erase block according to the metadata hash table and extracting valid information therefrom includes: obtaining, according to the metadata hash table, that the key is located in the erase block The metadata value written in the page is removed from the block, and data as valid information is read based on the value.
The device according to claim 9, wherein the recyclable mark is also recorded in the metadata.