WO2016041401A1

WO2016041401A1 - Method and device for writing data to cache

Info

Publication number: WO2016041401A1
Application number: PCT/CN2015/083383
Authority: WO
Inventors: 黄福堂
Original assignee: 华为技术有限公司
Priority date: 2014-09-16
Filing date: 2015-07-06
Publication date: 2016-03-24
Also published as: CN104238962A; CN104238962B

Abstract

A method and device for writing data to a cache, comprising: receiving an I/O write request command, the command comprising metadata of data to be written (S410); acquiring a metadata group currently being processed as a first metadata group, one metadata group corresponding to one metadata block and comprising a plurality of free blocks in a memory (S420); determining whether the first metadata group has a residual free block (S430); if so, then writing, into one free block of the residual free blocks and a metadata block corresponding to the first metadata group, the metadata of the data to be written (S440); if not, then determining whether a second metadata group has a residual free block, and if so, then acquiring the second metadata group, and writing, into one free block of the residual free blocks of the second metadata group, the metadata of the data to be written (S450); and writing the metadata of the data to be written into a metadata block corresponding to the second metadata group (S460), thus solving the problem that the service life of flash memory is affected by frequently writing metadata into flash memory.

Description

Method and device for writing data to cache

The present application claims priority to Chinese Patent Application No. 201410471711.7, the entire disclosure of which is hereby incorporated by reference. In this application.

Technical field

The present invention relates to the field of data storage, and in particular, to a method and apparatus for writing data to a cache.

Background technique

In the industry's existing read and write cache (Cache), the commonly used storage medium is based on A Non-Volatile Dual In-line Memory Module (NVDIMM) or non-volatile random access memory ( Non-Volatile Random Access Memory (NVRAM), however, based on the above devices, the capacity of the write cache is usually small, and it is difficult to cope with a large number of random write pressure scenarios.

In the prior art, a flash (Flash) device is used as the Cache. Because of the security of the data, each write request requires 2 IOs, one is to write data IO, and the other is to write metadata IO. The lifetime of Flash is directly related to the number of writes. For example, Multi-Level Cell (MLC) is usually only about 5000-10000 times. The size of the metadata is generally between 30B and 64B. If each metadata is directly IO according to the size of the metadata, the performance of the Flash cannot be fully utilized. If the metadata is IO according to the size of 4K alignment, there may be write amplification. The problem will have a big impact on the lifetime of the entire Flash.

Summary of the invention

Embodiments of the present invention provide a method and an apparatus for writing data into a cache, which can solve frequently Write metadata to Flash, which affects the life of Flash.

In a first aspect, a method for writing data to a cache cache is provided. The storage area of the flash memory of the cache includes a plurality of storage block chunks, and the chunk includes a chunk metadata area and a chunk data area. The one chunk metadata area includes at least one metadata block, the one chunk data area includes at least one data area corresponding to the at least one metadata block, and the metadata block includes a plurality of metadata, the data The area includes a plurality of data blocks, the metadata blocks are in one-to-one correspondence with the data area, and the metadata is in one-to-one correspondence with the data blocks, and the method includes:

Receiving a write IO request command, the write IO request command including metadata of data to be written;

Obtaining a metadata group currently being processed as a first metadata group, and the one metadata group corresponds to one of the metadata blocks, and includes a plurality of free blocks in the memory;

Determining whether there are any remaining free blocks in the first metadata group;

When there are remaining free blocks in the first metadata group, write metadata of the data to be written to one of the remaining free blocks, and correspond to the first metadata group The metadata of the data to be written is written in the metadata block;

When there is no free block remaining in the first metadata group, it is determined whether the second metadata group has remaining free blocks, and the second metadata group is a metadata group that is not currently processed, if the second If the metadata group has remaining free blocks, the second metadata group is acquired, and the metadata of the data to be written is written to one of the remaining free blocks of the second metadata group;

The metadata of the data to be written is written into a metadata block corresponding to the second metadata group.

With reference to the first aspect, in a first implementation manner of the first aspect, the number of the free blocks included in the metadata group is based on a size of the metadata block and a size of the metadata corresponding to the data block. Calculated.

In conjunction with the first aspect, in a second implementation of the first aspect, the free blocks included in the metadata group are consecutively arranged by address.

With reference to the first aspect, in a third implementation manner of the first aspect, the number of the free blocks included in the metadata group is recorded in a multi-level queue of the memory, where each queue in the multi-level queue The number of free blocks included in the recorded second metadata group is different.

With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the second metadata group is recorded according to the number of the free blocks included in the second metadata group In each queue of the level queue, the queues further include a total number of free blocks included in each second metadata group recorded by the queue;

The determining whether the second metadata group has remaining free blocks includes:

It is determined whether the second metadata group has remaining free blocks according to the total number of free blocks included in each second metadata group recorded by each queue.

With reference to the first aspect, or any one of the foregoing four implementation manners of the first aspect, in the fifth implementation manner of the first aspect, after the receiving the write IO request command, the obtaining is currently being performed Before the processed metadata group is the first metadata group, the method further includes:

Determining whether the write IO request command needs to be split according to the size of the data block;

If the splitting is required, splitting the write IO request command according to the size of the data block, and obtaining a plurality of first write IO request commands;

Determining, by the plurality of first write IO request commands, whether metadata of the data to be written included in each first write IO request command is already cached in the memory, and if so, directly performing a write operation If no, the step of obtaining the metadata group currently being processed is the first metadata group.

With reference to the fourth implementation manner of the first aspect, in a sixth implementation manner of the first aspect, when any one of the plurality of queues records a plurality of second metadata groups, according to the second element The number of free blocks included in the data group is arranged for the plurality of second metadata groups.

In conjunction with the third implementation of the first aspect, in a seventh implementation manner of the first aspect, the method further includes:

Receiving a recycle metadata instruction, where the recycle metadata command includes an address of a free block to which the metadata to be recovered belongs;

Obtaining, according to an address of the free block, a metadata group to which the free block belongs;

And if the metadata group is a first metadata group, adding the free block to the first metadata group, and reordering the free blocks in the first metadata group;

If the metadata group is a second metadata group, determining a free block included in the second metadata group Whether the number is greater than a second threshold, and if so, deleting the second metadata group from the queue in which the second metadata group is recorded, and adding the free block to the second metadata group, The second metadata group is recorded in other queues of the multi-level queue, and if not, the free block is added to the second metadata group.

In conjunction with the seventh implementation of the first aspect, in the eighth implementation manner of the first aspect, the recording, by the second metadata group, into another queue of the multi-level queue includes:

The second metadata group is recorded into other queues of the multi-level queue according to the number of free blocks included in the second metadata group.

With reference to the seventh or eighth implementation manner of the first aspect, in a ninth implementation manner of the first aspect, the method further includes:

Deleting the second metadata group from the queue for recording the second metadata group, adding the free block to the second metadata group, and recording the second metadata group to a plurality of After the other queues of the level queue, the method further includes:

Updating the total number of free blocks recorded in the queue of the second metadata group, and updating the total number of free blocks recorded in the queue joined by the second metadata group.

In a second aspect, an embodiment of the present invention provides an apparatus for writing data to a cache cache. The storage area of the flash memory of the apparatus includes a plurality of storage blocks chunk, and the chunk includes a chunk metadata area and a a chunk data area, the one chunk metadata area including at least one metadata block, the one chunk data area including at least one data area corresponding to the at least one metadata block, the metadata block including a plurality of metadata The data area includes a plurality of data blocks, the metadata block is in one-to-one correspondence with the data area, and the metadata is in one-to-one correspondence with the data block, and the apparatus includes: a receiving unit, and a first acquiring unit a determining unit, a first writing unit, a second obtaining unit, and a second writing unit;

The receiving unit is configured to receive a write IO request command, where the write IO request command includes metadata of data to be written;

The first obtaining unit is configured to acquire a metadata group currently being processed into a first metadata group, and the one metadata group corresponds to one of the metadata blocks, and includes a plurality of free blocks in the memory;

The determining unit is configured to determine whether there is any remaining free block in the first metadata group acquired by the first acquiring unit;

The first writing unit is configured to: when the determining unit determines that there are remaining free blocks in the first metadata group, write the data to be written to one of the remaining free blocks Metadata, and writing metadata of the data to be written into a metadata block corresponding to the first metadata group;

The second obtaining unit is configured to: when the determining unit determines that there are no free blocks in the first metadata group, determine whether the second metadata group has remaining free blocks, where the second metadata group is a metadata group currently not being processed, if the second metadata group has remaining free blocks, acquiring the second metadata group, and to a free block in the remaining free blocks of the second metadata group Writing metadata of the data to be written;

The second writing unit is configured to write metadata of the data to be written into a metadata block corresponding to the second metadata group.

With reference to the second aspect, in a first implementation manner of the second aspect, the number of the free blocks included in the metadata group is calculated according to a size of the metadata block and a size of the metadata corresponding to the data block. owned.

In conjunction with the second aspect, in a second implementation of the second aspect, the free blocks included in the metadata group are consecutively arranged by address.

With reference to the second aspect, in a third implementation manner of the second aspect, the number of the free blocks included in the metadata group is recorded in a multi-level queue of the memory, where each queue in the multi-level queue The number of free blocks included in the recorded second metadata group is different.

With the third implementation of the second aspect, in a fourth implementation manner of the second aspect, the device further includes: a recording unit, configured to use, according to the number of free blocks included in the second metadata group, Recording the second metadata group into each queue of the multi-level queue, where each queue further includes a total number of free blocks included in each second metadata group recorded by the queue;

The second acquiring unit is specifically configured to: determine, according to the total number of free blocks included in each second metadata group recorded in each queue, whether the second metadata group has remaining free blocks.

Combining the second aspect or the foregoing four implementation manners of the second aspect, in the second implementation In a fifth implementation manner of the aspect, the device further includes: a splitting unit;

The determining unit is further configured to determine, according to the size of the data block, whether the write IO request command needs to be split;

The splitting unit is configured to: if the determining unit determines that splitting is required, split the write IO request command according to the size of the data block, to obtain a plurality of first write IO request commands ;

The determining unit is further configured to determine, by the splitting unit, the plurality of first write IO request commands, respectively, whether the metadata of the data to be written included in each first write IO request command has been The cache is in the memory, if yes, the write operation is directly performed, and if not, the step of acquiring the metadata group currently being processed is the first metadata group.

With reference to the fourth implementation manner of the second aspect, in a sixth implementation manner of the second aspect, the device further includes: an arranging unit, configured to record a plurality of second elements in any one of the multi-level queues In the case of the data group, the plurality of second metadata groups are arranged according to the number of free blocks included in the second metadata group.

With reference to the third implementation manner of the second aspect, in a seventh implementation manner of the second aspect, the unit is added;

The receiving unit is further configured to receive a recovery metadata instruction, where the recovery metadata instruction includes an address of an associated free block of the metadata to be recovered;

The first obtaining unit is further configured to acquire, according to an address of the free block received by the receiving unit, a metadata group to which the free block belongs;

The joining unit is configured to add the free block to the first metadata group if the metadata group acquired by the first acquiring unit is a first metadata group, and add the first Free block reordering in the metadata group;

The determining unit is further configured to: if the metadata group acquired by the first acquiring unit is a second metadata group, determine whether the number of free blocks included in the second metadata group is greater than a second threshold, If yes, deleting the second metadata group from the queue that records the second metadata group, adding the free block to the second metadata group, and recording the second metadata group to a plurality of In other queues of the level queue, if not, the free block is added to the second metadata group.

With reference to the seventh implementation manner of the second aspect, in the eighth implementation manner of the second aspect, the determining unit is specifically configured to:

With reference to the seventh or eighth implementation manner of the second aspect, in a ninth implementation manner of the second aspect, the device further includes: an updating unit, configured to update a queue that records the second metadata group The total number of free blocks recorded, and the total number of free blocks recorded in the queue to which the second metadata group is added is updated.

A method and apparatus for writing data into a cache provided by an embodiment of the present invention. Recording a free block in the memory by the metadata group, when receiving the write IO request command, acquiring the free block from the above metadata group, and writing the metadata of the data to be written included in the IO request command Into the obtained free block, thereby increasing the merge probability of the metadata, thereby solving the problem of frequently writing metadata to the Flash, resulting in write amplification, which affects the life of the Flash.

DRAWINGS

1 is a schematic diagram of a data layout of a Flash provided by the present invention;

2 is a schematic diagram of another data layout of the Flash provided by the present invention;

FIG. 3 is a schematic diagram showing relationship between a location of a metadata in an in-memory and an actual persistent location in Flash according to the present invention; FIG.

4 is a flowchart of a method for writing data to a Cache according to Embodiment 1 of the present invention;

5 is a schematic diagram of a relationship between a multi-level queue and a second metadata group;

FIG. 6 is a schematic diagram of an apparatus for writing data to a Cache according to Embodiment 2 of the present invention;

FIG. 7 is a schematic structural diagram of a computing and storage integrated machine according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In order to facilitate the understanding of the embodiments of the present invention, the embodiments of the present invention are not to be construed as limiting.

FIG. 1 is a schematic diagram of a data layout of a Flash provided by the present invention. As shown in FIG. 1, the storage area of the Flash may include three parts: a super block, a metadata area, and a data area.

The super block is used to store the global data in the Cache, mainly for the data needed for recovery after the system is down. The global data may include a size of a metadata block, a size of a data block, and the like.

Metadata (data), which maintains the data of the data blocks in the storage system, and describes and organizes the relationship of the data blocks. The size is determined according to the size of the data structure defined in the system.

Metadata area refers to the area used to store metadata.

The data area is used to store real data in the Cache, and may include multiple data blocks below.

2 is a schematic diagram of another data layout of the Flash provided by the present invention. As shown in FIG. 2, the Flash includes a super block and a plurality of chunks, each chunk includes a chunk metadata area and a chunk data area, and each chunk metadata area includes one or more metadata blocks, each of which The chunk data area includes a data area corresponding to the one or more metadata blocks. The data area under each chunk data area has a one-to-one correspondence with the metadata block under the corresponding chunk metadata area. A metadata block maintains a corresponding data area. Specifically, each metadata area includes a plurality of metadata, and each metadata corresponds to one data block under the data area.

In Figure 2, although 1 chunk metadata area contains 4 metadata blocks, it does not mean that 1 chunk metadata area can only contain 4 metadata blocks, and can also be 1, 2 or other values. . Similarly, the number of chunks shown in FIG. 2, the number of data areas in the chunk data area, the number of metadata in the metadata area, and the number of data blocks in the data area are not limited to the values shown in FIG. .

Each data area corresponds to a different metadata block, and the data blocks corresponding to the metadata in one metadata block constitute a data area. That is to say, one data area contains data corresponding to metadata of a metadata block size, the size of one data area is determined by the number of metadata in one metadata block and one The size of the data block is determined.

In Flash, the size of a page write is 4K. Therefore, in a preferred solution, the size of one metadata block is 4K. Let's assume that the size of a metadata is 32B, and the size of a data block is 16K, then the size of a data area is 2M (16K*(4K/32B)).

It should be noted that a chunk metadata area may include one or more metadata blocks, and the size of one chunk metadata area is variable. A preferred solution can take the size of a data block as the size of a chunk metadata area, so that when the Flash load is initialized, the storage area of the Flash is divided according to the size of the data block. For example, if the size of a data block is 4K, the size of a chunk metadata area is also 4K, and the number of metadata blocks is 1 (4K/4K). For another example, if the size of one data block is 16K, the size of one chunk metadata area is also 16K, and the number of metadata blocks is 4 (16K/4K). The size of the data block can be configured by the operating system or the storage device, and the size of the data block can be 4K, 8K, 16K, 32K, or 64K, etc., and the embodiment of the present invention is not limited herein.

Of course, the size of the data block can also be different from the size of the chunk metadata area. For example, the size of the data block may also be 512B, 1K, 2K, etc., and the chunk metadata area includes at least one metadata block, which is obviously larger than one data block.

For another example, when the size of the data block is 8K, 16K, 32K, or 64K, the size of the chunk metadata area may be smaller than the size of the data block, for example, only one metadata block is included in the chunk metadata area.

FIG. 3 is a schematic diagram showing the relationship between the location of the metadata in the memory and the actual persistent location in the Flash provided by the present invention. In Figure 3, a data structure p_head array can be designed in memory to store the metadata corresponding to each data block in the data area. Let's assume that the size of the metadata required for each data block is 64 bytes, then the p_head array can be grouped into 64 (4K/64B=64) metadata.

A metadata in memory corresponds to an element of the p_head array, and its data structure flashcache_wbhead has the following meaning:

As shown in Figure 3, you can also design a data structure metadata_head array in memory, maintain the metadata of the data in memory and the actual storage location of the metadata in the actual persistent device (Flash) (ie the corresponding metadata Block) relationship.

A metadata group in Flash corresponds to a metadata group in memory, corresponding to an element in the metadata_head array. The data structure metadata_head and its meaning can be as follows:

Typedef struct metadata_head

{

List_head_t metadata_pending_q; / / waiting for the merged data IO queue

List_head_t metadata_syncing_q;//Synchronize merge metadata IO queue

Dsw_u64metadata_lba; / / specify the specific location of the metadata in the system in the flash card

Void*buf_address;//Specify the corresponding 4K location in memory

Dsw_u8 is_writing; / / mark head is IO operation

Ptr_heap_t invalid_head_heap;//Manage 4K metadata, sort by metadata address

List_head_t invalid_q_node; / / invalid node, in order to hang this metadata management head into the multi-level queue

Dsw_u32 multi_q_flag; / / mark the current metadata management header into which multi-level queue

Dsw_u32 free_cnt; / / count the current metadata management header, how many free metadata blocks

}metadata_head_t;

As shown in FIG. 3, the metadata information of a data block is stored in the memory by flashCache_wbhead, and the information of the same group metadata (ie, one metadata block) is stored according to the metadata_head, so as to facilitate the writing according to the metadata block. Write.

FIG. 4 is a flowchart of a method for writing data to a Cache according to Embodiment 1 of the present invention. The execution body of the method may be a cache device, and the cache device includes a flash memory Falsh.

Next, the method of FIG. 4 will be described with reference to FIG. 3 and the data structure of the metadata and metadata blocks. Of course, FIG. 4 can also be applied to other manners of the layout of the Flash data including the plurality of metadata areas, which are not limited herein.

S410. Receive a write IO request command, where the write IO request command includes metadata of data to be written.

Optionally, after receiving the write IO request command, the method may further include the following steps:

Specifically, the cache device pre-acquires the size of the data block in the Flash. For example, the size of the acquired data block may be 16K. After receiving the write IO request command, assuming that the write IO request command has a size of 32K (two times the data block), the write IO request command needs to be split. When splitting, when the offset of the write IO request command is 0, the IO request command is split into two first write IO request commands, wherein each first write IO request command The size is 16K; when the offset of the write IO request command is not 0, the IO request command is split into three first write IO request commands. This step can ensure that the size of the data block written by the cache device to the Flash is unified with the size of the data block inside the Flash, thereby improving the efficiency of writing data to the Flash.

It should be noted that, in step S410, the step of determining whether the metadata of the data to be written included in the plurality of first write IO request commands is already cached in the memory may be implemented by referring to the prior art, and the implementation of the present invention is implemented. The example will not be described here.

S420. Acquire a metadata group currently being processed as a first metadata group, and one of the metadata groups corresponds to one of the metadata blocks, and include a plurality of free blocks in the memory.

It should be noted that the total size of the storage space used to store the metadata corresponding to the data block in the memory may be obtained in advance, and the size of the metadata corresponding to the data block in the Flash is obtained, and then the total size of the storage space is divided by the data block. The size of the corresponding metadata, you can get the total number of free blocks in memory. It can be understood that the size of the free block is the size of the metadata. In addition, the metadata group corresponds to the metadata block in the Flash, and the number of the free blocks in the metadata group can be calculated according to the size of the metadata block in the Flash and the size of the metadata corresponding to the data block, for example, The size of the metadata block is 4K, and the size of the metadata corresponding to the data block is 64B. The number of metadata blocks including the number of free blocks is 4K/64B=64, that is, one metadata group can contain 64 free blocks. Optionally, the free blocks contained in the metadata group are consecutively arranged by address. Wherein, each free block is used to store metadata of data to be written. Here, it is explained that the size of the metadata of the data to be written coincides with the size of the metadata corresponding to the data block in the Flash.

It is explained here that the cache device regards the metadata group currently being processed as the first metadata group and the metadata group not currently being processed as the second metadata group. It can be understood that when the cache device processes the first metadata group and starts processing the second metadata group, the second metadata group is used as the first metadata group. That is, the above name is only to distinguish whether the metadata group is being processed. S430, determining the Whether there are remaining free blocks in the first metadata group.

It is explained here that since the free block is metadata for storing data to be written, after the cache device processes the plurality of write IO request commands, the number of free blocks in the first metadata group is correspondingly reduced. As in the foregoing example, 64 metadata blocks may be included in one metadata group. When the cache device writes 64 metadata into the metadata group, the metadata group no longer has remaining free blocks. Therefore, each time the cache device writes metadata to the free block, it is necessary to determine whether the first metadata group has remaining free blocks. Specifically, whether there are remaining free blocks may be determined according to the metadata_head corresponding to the first metadata group.

S440. When there are remaining free blocks in the first metadata group, write metadata of the data to be written to one of the remaining free blocks, and send the metadata to the first metadata. The metadata of the data to be written is written in the metadata block corresponding to the group.

It should be noted that, according to the metadata_head corresponding to the first metadata group, the metadata block corresponding to the first metadata group in the Flash may be obtained, and the metadata of the data to be written is written into the data block.

Optionally, before the metadata of the to-be-written data is written in the metadata block corresponding to the first metadata group, the method may further include:

The cache device determines whether all the first write IO request commands have been processed, and if no processing is completed, continues to process the other first write IO request commands S420-S440; if the processing has been completed, the writes are judged Whether the IO request command can be merged, if it can be merged, the data is written to the Flash according to the merged IO request command. In the present invention, each time a priority is obtained from the first metadata group, since the addresses of the free blocks acquired from the same group are consecutive, the merge probability of writing the IO request command can be increased, thereby reducing The number of times the data is written to Flash.

It should be noted that whether the write IO request command can be merged can be referred to the prior art implementation, and details are not described herein.

S450, when there is no free block remaining in the first metadata group, determining whether the second metadata group has remaining free blocks, where the second metadata group is a metadata group that is not currently processed, if The second metadata group has remaining free blocks, then the second metadata group is acquired, and the metadata of the data to be written is written to one of the remaining free blocks of the second metadata group.

Specifically, the second metadata group is managed by the multi-level queue. See the relationship between the multi-level queue and the second metadata group shown in FIG. 5. In Figure 5, the data structure multi_q_node of the multi-level queue and its meaning are as follows:

Optionally, the method may further include: recording, according to the number of free blocks included in the second metadata group, the second metadata group into each queue of the multi-level queue, where the queues are further The total number of free blocks included in each second metadata group including the queue record;

It should be noted that, according to the data structure invalid_q_type of the type of the queue, the types of the queues in the multi-level queue are different, for example, the number of the free blocks in the first queue is less than or equal to 8 The second metadata group, the second queue only records the second metadata group whose number of free blocks is greater than 8 and less than or equal to 16, and so on. Therefore, if the number of free blocks included in the second metadata group is four, the second metadata group is recorded in the first queue. In addition, the total number of free blocks included in each second metadata group needs to be recorded in the corresponding queue. For example, if the first queue records 5 second metadata groups, each second metadata. The group contains 4, 3, 2, 1, and 1 free blocks, and 11 (4+3+2+1+1) needs to be recorded in the total_num of the first queue. When the cache device determines whether the second metadata group has remaining free blocks, it is determined according to the total_num1 of each queue record in the multi-level queue.

Specifically, referring to FIG. 5, each metadata_head corresponds to a second metadata group, and the multi-level queue includes five queues, wherein the second metadata group is recorded in the first two queues, and the first queue only records the free blocks. A second metadata group with a number less than or equal to 4. As shown in FIG. 5, the second metadata group recorded in the first queue includes 4, 3, 2, 1, and 1 free blocks, and finally records the total number of free blocks included in each second metadata group to the first. In the queue, for example, it is recorded in the total_num of the first queue, and 11 is also recorded in the total_num of the first queue; similarly, the second queue only records the second metadata group whose number of free blocks is greater than 4, The second metadata group of the second queue record includes 7, 6, 5, 5, and 5 free blocks, and finally the total number of free blocks included in each second metadata group in the second queue is also recorded to the second. In the queue, 28 (7+6+5+5+5) is also recorded in the total_num of the second queue. Optionally, when a plurality of the second metadata groups are recorded in any one of the queues of the multi-level queue, the second element is obtained according to the number of free blocks included in the second metadata group. The data sets are arranged.

Referring to FIG. 5, the second metadata group is recorded in the first two queues, and the first queue records five second metadata groups, and each second metadata group includes 4, 3, 2, 1, and 1 respectively. A free block, and each second metadata group in the first queue is sequentially arranged according to the number of free blocks respectively included. Specifically, each second metadata group in the first queue is arranged in descending order according to the number of free blocks included. The second queue records five second metadata groups, each of which contains 7, 6, 5, 5, and 5 free blocks, and each second metadata group in the second queue follows The number of free blocks included in each of the blocks is sequentially arranged, that is, each second metadata group in the second queue is also arranged in descending order of the number of free blocks included.

Preferably, when determining whether there are any remaining free blocks in the second metadata group, the cache device preferentially determines the total_num in the queue with the largest number of recorded free blocks in the multi-level queue, and if the value is 0, In the level queue, the total_num in the queue with the largest number of free blocks is recorded, and so on; if not, the second metadata group having the largest number of free blocks in the queue is obtained, as in the foregoing example. The second metadata group in each queue is sequentially arranged according to the number of free blocks included in each queue. Therefore, the first second metadata group in the queue may be acquired. As shown in FIG. 5, the total_num of the second queue is first determined. If the value is not 0, the first second metadata group in the queue is obtained, and the second metadata group includes 7 free blocks. At the same time, the second metadata group is deleted from the queue, and the total_num of the queue is updated, that is, the updated total_num=total_num-5.

It should be noted that, when it is determined that the total_num of each queue in the multi-level queue is 0, the cache device does not process the write IO request command, and adds the write IO request command to the waiting queue, when the second metadata When there are remaining free blocks in the group, the above write IO request command is processed.

For the second metadata group obtained by the foregoing, including seven free blocks, and one free block is extracted from the seven free blocks, the second metadata group includes six remaining free blocks, and the foregoing The metadata of the data to be written is written in a free block.

S460. Write metadata of the data to be written into a metadata block corresponding to the second metadata group.

Optionally, before S460, the method may further include: the cache device will determine whether all the first write IO request commands are processed, and if no processing is completed, continue to request the other first write IO requests. The processing of S420-S450; if the processing has been completed, it is judged whether these write IO request commands can be merged, and if they can be merged, the data is written to the Flash according to the merged write IO request command. In the present invention, each time a priority is obtained from the first metadata group, since the addresses of the free blocks acquired from the same group are consecutive, the merge probability of writing the IO request command can be increased, thereby reducing The number of times the data is written to Flash.

The method for writing data to the cache provided by the embodiment of the present invention records a free block in the memory by the metadata group, and obtains a free block from the metadata group when receiving the write IO request command, and The metadata of the data to be written included in the write IO request command is written into the obtained free block, so that the merge probability of the metadata can be increased, thereby further solving the frequent writing of metadata to the Flash, and affecting the Flash. The problem of life.

For the free block obtained by the above method, when the metadata written to the free block is invalid metadata, the cache device needs to reclaim the free block into the memory, specifically, re-add the free block to the free block. In the metadata group to which it belongs. The step of recovering the free block may include:

If the metadata group is the second metadata group, determining whether the number of free blocks included in the second metadata group is greater than a second threshold, and if yes, deleting from the queue that records the second metadata group After the second metadata group is added to the second metadata group, the second metadata group is recorded in other queues of the multi-level queue, and if not, the free block is Join the second metadata set.

Specifically, the recording the second metadata group into other queues of the multi-level queue includes:

The second metadata group is recorded into other queues of the multi-level queue according to the number of free blocks included in the second metadata group. It should be noted that the step of determining whether the number of free blocks included in the second metadata group is greater than a second threshold is to ensure that the second metadata can be determined according to the number of free blocks included in the second metadata group. The group is logged to the appropriate queue. For example, suppose there are two queues in the multi-level queue, the first queue records only the second metadata group whose number of free blocks is less than or equal to 8, and the second queue records the second metadata group whose number of free blocks is greater than 8. And assuming that the number of free blocks included in a certain second metadata group is eight, the second metadata group is recorded in the first queue, and when the second metadata group recovers a free block The number of free blocks included in the second metadata group is nine, and the number of free blocks of the second metadata group recorded in the first queue is not satisfied. Therefore, the second metadata set needs to be recorded in the second queue.

Optionally, deleting the second metadata group in the queue from which the second metadata group is recorded, adding the free block to the second metadata group, and then using the second metadata After the group is recorded in other queues of the multi-level queue, the method further includes:

As in the foregoing example, the total_num of the first queue is updated, that is, the total_num=total_num-8 after the first queue is updated; and the total_num of the second queue is updated, that is, the total_num=total_num+8+1 after the second queue is updated; Where 1 is a free block newly collected by the second metadata group.

The reclaimed free block can be reclaimed into the metadata group to which it belongs, so that the address of the obtained free block is consecutive when the free block is obtained by the method of the present invention. Thereby, the merge probability of the metadata can be increased, and the problem that the metadata is frequently written into the Flash and affects the lifetime of the Flash can be solved.

The method provided by the embodiment of the invention, through metadata management, combined with metadata merging, greatly increases the probability of metadata merging, thereby greatly reducing the metadata IO, and has a significant effect on the performance of the providing system, and simultaneously constructing the IO. The number of times, a large reduction in the operation of IO due to metadata, has a significant effect on improving the life of the FLASH card.

For large IOs, through metadata management, as far as possible for large IOs, the location of allocated free blocks in the physical card is also as close as possible. This can be achieved through the WriteV interface. IO performs merge writing. In a large-scale scenario, the number of IOs can be reduced a lot, which can greatly reduce the CPU utilization of the system. Through actual testing, CPU utilization can be reduced in large scenarios. -30%.

Since the physical positions of the IOs are connected as much as possible when the free blocks are allocated, the effect of reading and merging is greatly increased in the scenes of the large blocks or in the sequential IO when the disk is swiped; A large number of requests to reduce IO read are very effective in reducing CPU utilization.

FIG. 6 is a schematic diagram of an apparatus for writing data to a Cache according to Embodiment 2 of the present invention. The apparatus is for performing the method described in FIG. In FIG. 6, the storage area of the flash memory of the device includes a plurality of memory blocks chunk, and the chunk includes a chunk metadata area and a chunk data. a chunk metadata area including at least one metadata block, the one chunk data area including at least one data area corresponding to the at least one metadata block, the metadata block including a plurality of metadata, The data area includes a plurality of data blocks, the metadata block is in one-to-one correspondence with the data area, and the metadata is in one-to-one correspondence with the data block, and includes: a receiving unit 601, a first obtaining unit 602, and a determining unit. 603. A first writing unit 604, a second obtaining unit 605, and a second writing unit 606.

The receiving unit 601 is configured to receive a write IO request command, where the write IO request command includes metadata of the data to be written.

The first obtaining unit 602 is configured to acquire a metadata group currently being processed into a first metadata group, where the one metadata group corresponds to one of the metadata blocks, and includes a plurality of free blocks in the memory.

The determining unit 603 is configured to determine whether there are any remaining free blocks in the first metadata group acquired by the first obtaining unit 602.

a first writing unit 604, configured to: when the determining unit 603 determines that there are remaining free blocks in the first metadata group, write the element of the data to be written to one of the remaining free blocks Data, and writing metadata of the data to be written into a metadata block corresponding to the first metadata group.

The second obtaining unit 605 is configured to: when the determining unit 603 determines that there are no free blocks remaining in the first metadata group, determine whether the second metadata group has remaining free blocks, where the second metadata group is currently not In the processed metadata group, if the second metadata group has remaining free blocks, acquiring the second metadata group and writing to one of the remaining free blocks of the second metadata group The metadata of the data to be written.

The second writing unit 606 is configured to write the metadata of the data to be written into the metadata block corresponding to the second metadata group.

Optionally, the number of free blocks included in the metadata group is calculated according to a size of the metadata block and a size of metadata corresponding to the data block.

Optionally, the free blocks included in the metadata group are consecutively arranged by address.

Optionally, the number of free blocks included in the second metadata group recorded by each queue in the multi-level queue is different.

Optionally, the device further includes: a recording unit 607, configured to perform, according to the second metadata group package The number of the free blocks included, the second metadata group is recorded in each queue of the multi-level queue, and the queues further include the total number of free blocks included in each second metadata group recorded by the queue;

The second obtaining unit 605 is specifically configured to: determine, according to the total number of free blocks included in each second metadata group recorded in each queue, whether the second metadata group has remaining free blocks.

Optionally, the device further includes: a splitting unit 608.

The determining unit 603 is further configured to determine, according to the size of the data block, whether the write IO request command needs to be split.

The splitting unit 608 is configured to, if the determining unit 603 determines that splitting is required, split the write IO request command according to the size of the data block to obtain a plurality of first write IO request commands.

The determining unit 603 is further configured to determine, by the splitting unit 608, the plurality of first write IO request commands, respectively, whether the metadata of the data to be written included in each of the first write IO request commands has been The cache is in the memory, if yes, the write operation is directly performed, and if not, the step of acquiring the metadata group currently being processed is the first metadata group.

Optionally, the device further includes: an arranging unit 609, configured to: when any one of the plurality of queues records a plurality of second metadata groups, according to the free blocks included in the second metadata group The plurality of second metadata groups are arranged.

Optionally, the device further includes: an adding unit 610.

The receiving unit 601 is further configured to receive a reclaiming metadata instruction, where the reclaiming metadata instruction includes an address of an associated free block of the metadata to be reclaimed.

The first obtaining unit 602 is further configured to acquire, according to the address of the free block received by the receiving unit 601, a metadata group to which the free block belongs.

The joining unit 610 is configured to add the free block to the first metadata group if the metadata group acquired by the first obtaining unit 602 is a first metadata group, and add the first metadata to the first metadata group Free blocks in the group are reordered.

The determining unit 603 is further configured to: if the metadata group acquired by the first obtaining unit 602 is a second metadata group, determine whether the number of free blocks included in the second metadata group is greater than a second threshold, and if yes, And deleting the second metadata group from the queue that records the second metadata group, adding the free block After entering the second metadata group, the second metadata group is recorded into other queues of the multi-level queue, and if not, the free block is added to the second metadata group.

Optionally, the determining unit 603 is specifically configured to:

Optionally, the device further includes: an updating unit 611, configured to update a total number of free blocks recorded in a queue in which the second metadata group is recorded, and update a record in a queue in which the second metadata group is added The total number of free blocks.

The device for writing data to the cache provided by the embodiment of the present invention records a free block in the memory through a metadata group, and when receiving the write IO request command, acquires a free block from the metadata group, and writes the above The metadata of the data to be written included in the IO request command is written into the obtained free block, thereby increasing the merge probability of the metadata, thereby solving the problem of frequently writing metadata to the Flash and affecting the life of the Flash. The problem.

Correspondingly, the embodiment of the present invention further provides a computing and storage machine. As shown in FIG. 7, the cache includes a flash memory 701 (only the Flash 701 is shown), the memory 702, and the processor 703.

The storage area of the flash Flash 701 includes a plurality of storage block chunks, one of the chunks includes a chunk metadata area and a chunk data area, the one chunk metadata area includes at least one metadata block, and the one chunk data area includes And at least one data area corresponding to the at least one metadata block, the metadata block includes a plurality of metadata, the data area includes a plurality of data blocks, and the metadata block has a one-to-one correspondence with the data area, The metadata corresponds to the data block one-to-one.

The processor 703 receives a write IO request command, the write IO request command including metadata of the data to be written;

The processor 703 acquires a metadata group currently being processed into a first metadata group, and the one metadata group corresponds to one of the metadata blocks, and includes a plurality of free blocks in the memory 702;

The processor 703 determines whether there are any remaining free blocks in the first metadata group;

When there are remaining free blocks in the first metadata group, the processor 703 writes the metadata of the data to be written to one of the remaining free blocks, and the first metadata is Group corresponding Writing metadata of the data to be written in the metadata block;

When there is no free block remaining in the first metadata group, the processor 703 determines whether the second metadata group has remaining free blocks, and the second metadata group is a metadata group that is not currently processed, if And the second metadata group has the remaining free blocks, the second metadata group is acquired, and the metadata of the to-be-written data is written to one of the remaining free blocks of the second metadata group;

The processor 703 writes the metadata of the data to be written into the metadata block corresponding to the second metadata group.

Further, the number of free blocks included in the metadata group is calculated by the processor 703 according to the size of the metadata block and the size of the metadata corresponding to the data block.

Further, the free blocks included in the metadata group are consecutively arranged by address.

Further, the number of free blocks included in the metadata group is recorded in a multi-level queue of the memory 702, wherein the second metadata group of each queue record in the multi-level queue includes a free block The numbers are not the same.

Further, the processor 703 records the second metadata group into each queue of the multi-level queue according to the number of free blocks included in the second metadata group, where each queue further includes the queue record. The total number of free blocks included in each second metadata group;

The processor 703 determines whether the second metadata group has remaining free blocks according to the total number of free blocks included in each second metadata group recorded in each queue.

Further, after the processor 703 receives the write IO request command, and before acquiring the metadata group currently being processed as the first metadata group, the processor 703 further determines, according to the size of the data block, whether the Write the IO request command for splitting;

If the splitting is required, the processor 703 splits the write IO request command according to the size of the data block to obtain a plurality of first write IO request commands.

The processor 703 requests the plurality of first write IOs to determine whether the metadata of the data to be written included in each first write IO request command is cached in the memory, and if so, directly A write operation is performed, and if not, a step of acquiring the metadata group currently being processed as the first metadata group is performed.

Further, when any one of the multi-level queues records a plurality of second metadata groups, the processor 703 pairs the plurality of second elements according to the number of free blocks included in the second metadata group. The data sets are arranged.

Further, the processor 703 receives the reclaimed metadata instruction, where the reclaimed metadata instruction includes an address of the associated free block of the metadata to be recovered; and acquires the metadata group to which the free block belongs according to the address of the free block; If the metadata group is a first metadata group, adding the free block to the first metadata group, and reordering the free blocks in the first metadata group; if the metadata If the group is a second metadata group, the processor 703 determines whether the number of free blocks included in the second metadata group is greater than a second threshold, and if so, the processor 703 records from the queue that records the second metadata group. Deleting the second metadata group, adding the free metadata block to the second metadata group, and recording the second metadata group to other queues of the multi-level queue, if not, the processor 703 The free block is added to the second metadata group. The recording, by the second metadata group to the other queues of the multi-level queue, includes: according to the number of free blocks included in the second metadata group, the processor 703 records the second metadata group to In other queues of multi-level queues.

Still further, after the processor 703 records the second metadata group in other queues of the multi-level queue, the processor 703 updates the total number of free blocks recorded in the queue in which the second metadata group is recorded, and updates The total number of free blocks recorded in the queue to which the second metadata group is added.

The computing and storage integrated machine provided by the embodiment of the present invention records a free block in the memory through the metadata group, and when receiving the write IO request command, acquires the free block from the metadata group, and writes the above IO request command. The metadata included in the data to be written is written into the obtained free block, so that the merge probability of the metadata can be increased, thereby solving the problem that the metadata is frequently written into the Flash and affecting the lifetime of the Flash.

A person skilled in the art should further appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, in order to clearly illustrate hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. Professional technicians can apply to each specific application Different methods are used to implement the described functionality, but such implementation should not be considered to be beyond the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or technical field. Any other form of storage medium known.

The specific embodiments of the present invention have been described in detail with reference to the preferred embodiments of the present invention. All modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

A method for writing data to a cache cache, wherein a storage area of the flash memory of the cache includes a plurality of storage blocks chunk, and the chunk includes a chunk metadata area and a chunk data area, A chunk metadata area includes at least one metadata block, the one chunk data area includes at least one data area corresponding to the at least one metadata block, the metadata block includes a plurality of metadata, and the data area includes a plurality of data blocks, the metadata block is in one-to-one correspondence with the data area, and the metadata is in one-to-one correspondence with the data block, and the method includes:

Receiving a write IO request command, the write IO request command including metadata of data to be written;

Obtaining a metadata group currently being processed as a first metadata group, and the one metadata group corresponds to one of the metadata blocks, and includes a plurality of free blocks in the memory;

Determining whether there are any remaining free blocks in the first metadata group;

When there are remaining free blocks in the first metadata group, write metadata of the data to be written to one of the remaining free blocks, and correspond to the first metadata group The metadata of the data to be written is written in the metadata block;

When there is no free block remaining in the first metadata group, it is determined whether the second metadata group has remaining free blocks, and the second metadata group is a metadata group that is not currently processed, if the second If the metadata group has remaining free blocks, the second metadata group is acquired, and the metadata of the data to be written is written to one of the remaining free blocks of the second metadata group;

The metadata of the data to be written is written into a metadata block corresponding to the second metadata group.
The method according to claim 1, wherein the number of free blocks included in the metadata group is calculated according to a size of the metadata block and a size of metadata corresponding to the data block.
The method according to claim 1, wherein the free blocks included in the metadata group are consecutively arranged by address.
The method according to claim 1, wherein the number of free blocks included in the metadata group is recorded in a multi-level queue of a memory, wherein a second record of each queue in the multi-level queue The number of free blocks contained in the metadata group is different.
The method of claim 4, wherein the method further comprises:

And the second metadata group is recorded in each queue of the multi-level queue according to the number of the free blocks included in the second metadata group, where each queue further includes each second metadata group recorded by the queue The total number of free blocks included;

The determining whether the second metadata group has remaining free blocks includes:

It is determined whether the second metadata group has remaining free blocks according to the total number of free blocks included in each second metadata group recorded by each queue.
The method according to any one of claims 1 to 5, characterized in that, after the receiving a write IO request command, before the acquiring the metadata group currently being processed is the first metadata group, the method Also includes:

Determining whether the write IO request command needs to be split according to the size of the data block;

If the splitting is required, splitting the write IO request command according to the size of the data block, and obtaining a plurality of first write IO request commands;

Determining, by the plurality of first write IO request commands, whether metadata of the data to be written included in each first write IO request command is already cached in the memory, and if so, directly performing a write operation If no, the step of obtaining the metadata group currently being processed is the first metadata group.
The method according to claim 5, wherein when any one of the plurality of queues records a plurality of second metadata groups, according to the number of free blocks included in the second metadata group, Arranging the plurality of second metadata groups.
The method of claim 4, wherein the method further comprises:

Receiving a recycle metadata instruction, where the recycle metadata command includes an address of a free block to which the metadata to be recovered belongs;

Obtaining, according to an address of the free block, a metadata group to which the free block belongs;

And if the metadata group is a first metadata group, adding the free block to the first metadata group, and reordering the free blocks in the first metadata group;

If the metadata group is the second metadata group, determining whether the number of free blocks included in the second metadata group is greater than a second threshold, and if yes, deleting from the queue that records the second metadata group After the second metadata group is added to the second metadata group, the second metadata group is recorded in other queues of the multi-level queue, and if not, the free block is Join the second metadata set.
The method according to claim 8, wherein the recording the second metadata group to other queues of the multi-level queue comprises:

The second metadata group is recorded into other queues of the multi-level queue according to the number of free blocks included in the second metadata group.
The method according to claim 8 or 9, wherein said second metadata group is deleted from said queue for recording said second metadata group, said free block being added to said second element After the data group is recorded in the other queues of the multi-level queue, the method further includes:

Updating the total number of free blocks recorded in the queue of the second metadata group, and updating the total number of free blocks recorded in the queue joined by the second metadata group.
An apparatus for writing data to a cache cache, wherein a storage area of a flash memory of the device includes a plurality of storage blocks chunk, and the chunk includes a chunk metadata area and a chunk data area, A chunk metadata area includes at least one metadata Block, the one chunk data area includes at least one data area corresponding to the at least one metadata block, the metadata block includes a plurality of metadata, and the data area includes a plurality of data blocks, the metadata block Corresponding to the data area, the metadata is in one-to-one correspondence with the data block, and the apparatus includes: a receiving unit, a first acquiring unit, a determining unit, a first writing unit, a second acquiring unit, and a first Two write units;

The receiving unit is configured to receive a write IO request command, where the write IO request command includes metadata of data to be written;

The first obtaining unit is configured to acquire a metadata group currently being processed into a first metadata group, and the one metadata group corresponds to one of the metadata blocks, and includes a plurality of free blocks in the memory;

The determining unit is configured to determine whether there is any remaining free block in the first metadata group acquired by the first acquiring unit;

The first writing unit is configured to: when the determining unit determines that there are remaining free blocks in the first metadata group, write the data to be written to one of the remaining free blocks Metadata, and writing metadata of the data to be written into a metadata block corresponding to the first metadata group;

The second obtaining unit is configured to: when the determining unit determines that there are no free blocks in the first metadata group, determine whether the second metadata group has remaining free blocks, where the second metadata group is a metadata group currently not being processed, if the second metadata group has remaining free blocks, acquiring the second metadata group, and to a free block in the remaining free blocks of the second metadata group Writing metadata of the data to be written;

The second writing unit is configured to write metadata of the data to be written into a metadata block corresponding to the second metadata group.
The apparatus according to claim 11, wherein the number of free blocks included in the metadata group is calculated according to a size of the metadata block and a size of metadata corresponding to the data block.
The apparatus according to claim 11, wherein said metadata group contains an empty space The free blocks are arranged consecutively by address.
The method according to claim 11, wherein the number of free blocks included in the metadata group is recorded in a multi-level queue of the memory, wherein the second record of each queue in the multi-level queue The number of free blocks contained in the metadata group is different.
The apparatus according to claim 14, wherein the apparatus further comprises: a recording unit, configured to record the second metadata group according to the number of free blocks included in the second metadata group In each queue of the multi-level queue, the queues further include a total number of free blocks included in each second metadata group recorded by the queue;

The second acquiring unit is specifically configured to: determine, according to the total number of free blocks included in each second metadata group recorded in each queue, whether the second metadata group has remaining free blocks.
The device according to any one of claims 11-15, wherein the device further comprises: a splitting unit;

The determining unit is further configured to determine, according to the size of the data block, whether the write IO request command needs to be split;

The splitting unit is configured to: if the determining unit determines that splitting is required, split the write IO request command according to the size of the data block, to obtain a plurality of first write IO request commands ;

The determining unit is further configured to determine, by the splitting unit, the plurality of first write IO request commands, respectively, whether the metadata of the data to be written included in each first write IO request command has been The cache is in the memory, if yes, the write operation is directly performed, and if not, the step of acquiring the metadata group currently being processed is the first metadata group.
The device according to claim 15, wherein the device further comprises: an arranging unit, configured to: when any one of the plurality of queues records a plurality of second metadata groups, according to the second The number of free blocks included in the metadata group is arranged for the plurality of second metadata groups.
The device according to claim 14, wherein the device further comprises: a joining unit;

The receiving unit is further configured to receive a recovery metadata instruction, where the recovery metadata instruction includes an address of an associated free block of the metadata to be recovered;

The first obtaining unit is further configured to acquire, according to an address of the free block received by the receiving unit, a metadata group to which the free block belongs;

The joining unit is configured to add the free block to the first metadata group if the metadata group acquired by the first acquiring unit is a first metadata group, and add the first Free block reordering in the metadata group;

The determining unit is further configured to: if the metadata group acquired by the first acquiring unit is a second metadata group, determine whether the number of free blocks included in the second metadata group is greater than a second threshold, If yes, deleting the second metadata group from the queue that records the second metadata group, adding the free block to the second metadata group, and recording the second metadata group to a plurality of In other queues of the level queue, if not, the free block is added to the second metadata group.
The device according to claim 18, wherein the determining unit is specifically configured to:

The second metadata group is recorded into other queues of the multi-level queue according to the number of free blocks included in the second metadata group.
The apparatus according to claim 18 or 19, wherein the apparatus further comprises: an updating unit, configured to update a total number of free blocks recorded in a queue in which the second metadata group is recorded, and update the The total number of free blocks recorded in the queue joined by the second metadata group.