WO2021082422A1

WO2021082422A1 - Compatibility processing method, system and device for space reclamation of storage system

Info

Publication number: WO2021082422A1
Application number: PCT/CN2020/093085
Authority: WO
Inventors: 赵书龙
Original assignee: 苏州浪潮智能科技有限公司
Priority date: 2019-10-31
Filing date: 2020-05-29
Publication date: 2021-05-06
Also published as: CN111007985B; CN111007985A

Abstract

A compatibility processing method, system and device for space reclamation of a storage system. The method comprises: a host IO simultaneously executing a write operation on a thin volume and a common volume in a storage system; and during writing by means of unmap IO, deleting metadata of the thin volume according to a preset deletion policy in order to perform space reclamation, and writing all-zero IO in the common volume to complete writing data in a disk. According to the present invention, the utilization rate of a storage space of the system can be greatly improved, such that back-end SSD storage resources achieve the maximum read-write efficiency, and the system performance is improved. In addition, the functions of a traditional storage product can be expanded, thereby improving the product competitiveness.

Description

Compatible processing method, system and equipment for storage system space recovery

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on October 31, 2019, the application number is 201911052313.0, and the invention title is "a compatible processing method, system and equipment for storage system space recovery", and its entire content Incorporated in this application by reference.

Technical field

The present invention relates to the technical field of storage systems, and more specifically to a compatible processing method, system and equipment for reclaiming storage system space.

Background technique

As the amount of user data continues to increase, the demand for storage space continues to increase. Improving the utilization of storage space is a problem that many products need to solve. The implementation principle of thin provisioning is to provide storage space for users on demand from a thin storage pool. Compared with traditional non-thin provisioning, there is no need to allocate unused disk capacity in advance, so storage utilization is higher. Traditional Thin LUNs have no space reclamation function, and the continuous allocation of space will cause storage space to run out. Therefore, the space reclamation function of the thin volume is indispensable. When deleting files in the Thin LUN, the space reclamation can be realized through the host unmap command, so as to realize the repeated use of storage space and greatly improve the utilization of storage space.

At the same time, for all-flash storage systems where the back-end disk media is SSD, due to the read and write characteristics of NAND flash, space reclamation operations are indispensable for improving performance and space utilization.

Traditional storage systems (HDD mechanical disks) do not consider the space reclamation function, but with the iteration from traditional storage systems to hybrid flash/full flash storage systems, with the explosive growth of data volume, the demand for space reclamation has become more prominent , The realization of space reclamation cannot bypass the compatibility strategy. How to effectively utilize the space reclamation capabilities of traditional and all-flash storage systems. Enhancing the functionality of traditional storage and improving the storage efficiency of all-flash systems are issues that we urgently need to solve.

Summary of the invention

In view of the above problems, the purpose of the present invention is to provide a compatible processing method, system and equipment for storage system space recovery, which can effectively utilize the space recovery capabilities of traditional and all-flash storage systems. Enhance the functionality of traditional storage and improve the storage efficiency of the all-flash system.

In order to achieve the above objective, the present invention is achieved through the following technical solutions: a compatible processing method for storage system space recovery, including:

The host IO simultaneously performs write operations to the thin volume and the ordinary volume in the storage system. When writing through unmapIO, the metadata of the thin volume is deleted according to the preset deletion strategy, space is reclaimed, and all zero IO is written to the ordinary volume. Complete the order;

When synchronizing the data written in the ordinary volume to the thin volume, an all-zero IO detection mechanism is added to identify the all-zero IO in the ordinary volume.

Further, it also includes:

During data migration and copy synchronization, processing is performed in accordance with the scenario of unmapIO. Further, the preset deletion strategy includes:

The storage system uses a mechanical disk and an SSD disk, the mechanical disk uses a thin volume and a normal volume, and the SSD disk uses a thin volume;

The mechanical disk does not need to be erased when deleting. When deleting a data block, the metadata corresponding to the data block is directly deleted; at the same time, the actual physical address corresponding to the metadata is recycled and added to the unused linked list for recycling;

When an SSD disk deletes a data block, it needs to erase and write NANDFlash. When deleting a data block, first mark the metadata as deleting; then through the background garbage collection module, the SSD is asynchronously erased and written through trim for space recovery.

Further, there is no metadata maintained in the ordinary volume, and the space of the ordinary volume is pre-allocated through the LBA to PBA mapping relationship. When the ordinary volume receives the operating-copde as op_unmapIO, it is written to the back-end disk as all-zero data.

Further, before the deletion operation of the ordinary volume in the mechanical disk, the grainSize space is set to determine the recycling granularity.

Correspondingly, the present invention also discloses a compatible processing system for reclaiming storage system space, including: a delete operation unit for the host IO to simultaneously perform write operations to the thin volume and the ordinary volume in the storage system, and when writing through unmapIO , Delete the metadata of the thin volume according to the preset strategy, reclaim the space, write all zero IO to the ordinary volume, and complete the disk placement;

The all-zero IO identification unit is used to synchronize the data written in the ordinary volume to the thin volume, adding an all-zero IO detection mechanism to identify all-zero IOs placed in the ordinary volume.

Further, it also includes:

The data processing unit is used for processing in accordance with the scenario of unmapIO during data migration and copy synchronization.

Correspondingly, the present invention also discloses a compatible processing device for reclaiming storage system space, including: a memory for storing computer programs;

The processor is configured to implement the steps of the compatible processing method for reclaiming storage system space as described in any one of the above when executing the computer program.

Compared with the prior art, the present invention has the beneficial effects that: the present invention provides a compatible processing method, system and equipment for storage system space recovery, realizes that traditional/hybrid flash/full flash storage systems use the same software version, while supporting traditional storage systems. Mechanical disks and SSD storage disks greatly increase the complexity of system design. The space recovery function of the present invention is based on a reduced volume with an automatic thin configuration function, and uses the metadata management and space management capabilities of the reduced volume to realize data space recovery; because the metadata management of the full flash and the traditional reduced volume is different, the present invention is clear A compatible strategy for space reclamation has been introduced.

The invention can greatly improve the storage space utilization rate of the system, enable the back-end SSD storage resources to maximize the read and write efficiency, and improve the system performance. And it can expand the functions of traditional storage products and enhance product competitiveness.

It can be seen that, compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, without creative work, other drawings can be obtained according to the provided drawings.

Figure 1 is a schematic diagram of the method of the present invention.

Fig. 2 is a schematic diagram of the method for preset deletion measurement of the present invention.

Figure 3 is a system structure diagram of the present invention.

Detailed ways

The specific embodiments of the present invention will be described below with reference to the accompanying drawings.

As shown in Figure 1, a compatible processing method for reclaiming storage system space includes:

The host IO simultaneously performs write operations to the thin and ordinary volumes in the storage system. When writing through unmap IO, the metadata of the thin volume is deleted according to the preset deletion strategy, space is reclaimed, and all zero IO is written to the ordinary volume. , Complete the order placement;

During data migration and copy synchronization, processing is performed in accordance with the scenario of unmap IO. As shown in Figure 2, the preset deletion strategy includes:

The storage system uses mechanical disks and SSD disks, mechanical disks use reduced volumes and ordinary volumes, and SSD disks use reduced volumes. There is no metadata maintained in the ordinary volume. The space of the ordinary volume is pre-allocated through the LBA to PBA mapping relationship. When the ordinary volume receives the operating-copde as op_unmap IO, it is written to the back-end disk as all-zero data. Before deleting the ordinary volume in the mechanical disk, the grainSize space is set to determine the recycling granularity.

The mechanical disk does not need to be erased when deleting. When deleting a data block, the metadata corresponding to the data block is directly deleted; at the same time, the actual physical address corresponding to the metadata is recycled and added to the unused linked list for recycling.

When the SSD disk deletes the data block, it needs to erase the NAND Flash. When deleting the data block, first mark the metadata as deleting; then through the background garbage collection module, the SSD is erased asynchronously through trim for space recovery.

It can be seen that space reclamation can be achieved through the metadata management of the thin volume. The metadata of the corresponding data block needs to be deleted during unmap IO processing. The ordinary pool only deletes the metadata, while the full flash thin pool needs to be first based on the characteristics of SSDs. Delete the metadata, and trim the data on the disk through a background task. At the same time, storage space reclamation supports the volume copy service feature of compatible storage, which can effectively realize the space reclamation function during data migration and backup.

Correspondingly, as shown in FIG. 3, the present invention also discloses a compatible processing system for reclaiming storage system space, including:

The delete operation unit is used for the host IO to simultaneously perform write operations to the thin volume and the ordinary volume in the storage system. When writing through unmap IO, the metadata of the thin volume is deleted according to the preset strategy, and the space is reclaimed. Write all zero IO to complete the disk placement.

The data processing unit is used for processing in accordance with the scenario of unmap IO during data migration and copy synchronization.

Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention can be embodied in the form of software products in essence or the parts that contribute to the prior art. The computer software products are stored in a storage medium such as a USB flash drive or a mobile device. Hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes, including several instructions to make a computer terminal (It can be a personal computer, a server, or a second terminal, a network terminal, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present invention. The same or similar parts in the various embodiments in this specification can be referred to each other. In particular, as for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the description in the method embodiment.

In the several embodiments provided by the present invention, it should be understood that the disclosed system, system, and method may be implemented in other ways. For example, the system embodiment described above is only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, systems or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional modules in the various embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules may be integrated into one unit.

In the same way, the processing units in the various embodiments of the present invention may be integrated into one functional module, or each processing unit may physically exist, or two or more processing units may be integrated into one functional module.

The present invention will be further described with reference to the drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the teachings of the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the present application.

Claims

A compatible processing method for reclaiming storage system space, which is characterized in that it includes:

The host IO simultaneously performs write operations to the thin volume and the ordinary volume in the storage system. When writing through unmapIO, the metadata of the thin volume is deleted according to the preset deletion strategy, space is reclaimed, and all zero IO is written to the ordinary volume. Complete the order;

When synchronizing the data written in the ordinary volume to the thin volume, an all-zero IO detection mechanism is added to identify the all-zero IO in the ordinary volume.
The compatible processing method for reclaiming storage system space according to claim 1, further comprising:

During data migration and copy synchronization, processing is performed in accordance with the scenario of unmapIO.
The compatible processing method for reclaiming storage system space according to claim 1, wherein the preset deletion strategy comprises:

The storage system uses a mechanical disk and an SSD disk, the mechanical disk uses a thin volume and a normal volume, and the SSD disk uses a thin volume;

The mechanical disk does not need to be erased when deleting. When deleting a data block, the metadata corresponding to the data block is directly deleted; at the same time, the actual physical address corresponding to the metadata is recycled and added to the unused linked list for recycling;

When an SSD disk deletes a data block, it needs to erase and write NANDFlash. When deleting a data block, first mark the metadata as deleting; then through the background garbage collection module, the SSD is asynchronously erased and written through trim for space recovery.
The compatible processing method of storage system space reclamation according to claim 3, characterized in that there is no metadata maintained in the ordinary volume, and the space of the ordinary volume is pre-allocated through the LBA to PBA mapping relationship. When the ordinary volume receives the operating -When copde is op_unmapIO, write it as all zero data to the back-end disk.
The compatible processing method for reclaiming storage system space according to claim 3, characterized in that the reclaiming granularity of the ordinary volume in the mechanical disk is determined by setting the grainSize space before the deletion operation.
A compatible processing system for reclaiming storage system space, which is characterized in that it comprises:

The delete operation unit is used for the host IO to simultaneously write to the thin volume and the ordinary volume in the storage system. When writing through unmapIO, the metadata of the thin volume is deleted according to the preset strategy, space is reclaimed, and the ordinary volume is written Enter all zero IO and complete the order placement;

The all-zero IO identification unit is used to synchronize the data written in the ordinary volume to the thin volume, adding an all-zero IO detection mechanism to identify all-zero IOs placed in the ordinary volume.
The compatible processing system for reclaiming storage system space according to claim 6, further comprising:

The data processing unit is used for processing in accordance with the scenario of unmapIO during data migration and copy synchronization.
A compatible processing device for reclaiming storage system space, which is characterized in that it includes:

Memory, used to store computer programs;

The processor is configured to implement the steps of the compatible processing method for reclaiming the storage system space according to any one of claims 1 to 5 when the computer program is executed.