CN113254365A

CN113254365A - Method, apparatus, device, medium, and program product for managing storage space

Info

Publication number: CN113254365A
Application number: CN202110641207.7A
Authority: CN
Inventors: 姜哓庆; 王洪浩
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing Volcano Engine Technology Co Ltd
Priority date: 2021-06-09
Filing date: 2021-06-09
Publication date: 2021-08-13

Abstract

The present invention relates to a method, apparatus, device, medium, and program product for managing storage space. In the technical scheme, the candidate physical address identifications corresponding to the multiple candidate storage units are recorded in the storage space cache, and the candidate storage units are clean storage units, so that the storage space allocated to the first logical address identification is determined, the first physical address identification is obtained from the storage space cache, a first corresponding relation between the first logical address identification and the first physical address identification is established, the first corresponding relation is written into an address mapping table in the memory, and the first corresponding relation is written into the address mapping table in the storage pool. Therefore, the storage units corresponding to the allocated first physical addresses are all clean storage units, and the safety of data storage is improved.

Description

Method, apparatus, device, medium, and program product for managing storage space

Technical Field

The present disclosure relates to the field of data storage technologies, and in particular, to a method, an apparatus, a device, a medium, and a program product for managing a storage space.

Background

The size of data stored globally increases exponentially, and the storage demand for data also increases rapidly.

In order to improve the utilization rate of the physical disks of the data center, the data center pools the storage space, allocates the logical storage space according to the capacity planned and requested by the user, and allocates the physical storage space according to the actual use amount of the user.

However, the existing way of allocating physical space may cause the allocated physical storage space to contain legacy data of the front-end user, so that the security of data storage is not high.

Disclosure of Invention

Embodiments of the present invention provide a method, an apparatus, a device, a medium, and a program product for managing a storage space, which can solve the problem of low security of data storage in the existing storage space allocation method.

In a first aspect, the present disclosure provides a method for managing a storage space, including:

determining to allocate a storage space to the first logical address identifier;

acquiring a first physical address identifier from a storage space cache, wherein the storage space cache records candidate physical address identifiers corresponding to a plurality of candidate storage units, and the candidate storage units are all clean storage units;

establishing a first corresponding relation between a first logic address identifier and a first physical address identifier, writing the first corresponding relation into an address mapping table in a memory, and writing the first corresponding relation into the address mapping table in a storage pool.

Optionally, before obtaining the first physical address identifier from the storage space cache, the method further includes:

acquiring candidate physical address identifications from the physical address identifications of unallocated memory cells according to an allocation bitmap, wherein each bit of the allocation bitmap corresponds to one memory cell, and the value of each bit is used for indicating whether the corresponding memory cell is allocated or unallocated;

erasing the candidate storage unit corresponding to the candidate physical address identifier to enable the candidate storage unit to be a clean storage unit;

and adding the candidate physical address identification to the storage space cache.

Optionally, the method further includes: and updating the allocation bitmap.

Optionally, determining to allocate a storage space to the first logical address identifier includes:

receiving a write request, wherein the write request comprises target data and a logical address identifier;

searching an address mapping table in a memory;

under the condition that no physical address identifier corresponding to the logical address identifier exists in an address mapping table in the memory, determining to allocate a storage space to the first logical address identifier;

writing a first correspondence to an address mapping table in a storage pool, comprising:

and simultaneously writing the target data into the target storage unit corresponding to the first physical address identifier and writing the first corresponding relation into an address mapping table in the storage pool.

Optionally, the method further includes:

if the target data is successfully written into the target storage unit and the first corresponding relation is successfully written into the address mapping table in the storage pool, returning a successful response of the write request;

if the target data is successfully written into the target storage unit and the first corresponding relation is unsuccessfully written into the address mapping table in the storage pool, returning a write request failure response and deleting the first corresponding relation from the address mapping table in the memory;

if the target data is failed to be written into the target storage unit and the first corresponding relation is successfully written into the address mapping table in the storage pool, returning a write request failure response;

and if the target data is failed to be written into the target storage unit and the first corresponding relation is failed to be written into the address mapping table in the storage pool, returning a write request failure response and deleting the first corresponding relation from the address mapping table in the memory.

Optionally, the method further includes:

and under the condition that a first physical address identifier corresponding to the first logical address identifier exists in an address mapping table in the memory, writing target data into a target storage unit corresponding to the first physical address identifier.

Optionally, the method further includes:

receiving a recovery request, wherein the recovery request comprises: the second logic address identification is triggered after the user deletes the data in the storage unit corresponding to the second logic address identification;

Acquiring a second physical address identifier corresponding to the second logical address identifier, and erasing data of a storage unit corresponding to the second physical address identifier;

deleting a second corresponding relation between a second logic address identifier and a second physical address identifier in an address mapping table in the storage pool;

and deleting the second corresponding relation in the address mapping table in the memory.

Optionally, the method further includes: and outputting an over-allocation alarm when detecting that the storage space corresponding to the allocated total logical address is larger than the total storage space of the storage pool.

Optionally, the method further includes: if the allocation bitmap is lost, the address mapping table in the storage pool is scanned to generate the allocation bitmap.

In a second aspect, the present disclosure provides a management apparatus for a storage space, including:

the processing module is used for determining that the storage space is allocated to the first logic address identifier;

the thin allocation module is used for acquiring a first physical address identifier from a storage space cache, wherein the storage space cache records candidate physical address identifiers corresponding to a plurality of candidate storage units, and the candidate storage units are all clean storage units;

and the metadata storage module is used for establishing a first corresponding relation between the first logic address identifier and the first physical address identifier, writing the first corresponding relation into an address mapping table in the memory, and writing the first corresponding relation into the address mapping table in the storage pool.

Optionally, the thin allocation module is further configured to: acquiring candidate physical address identifications from the physical address identifications of unallocated memory cells according to an allocation bitmap, wherein each bit of the allocation bitmap corresponds to one memory cell, and the value of each bit is used for indicating whether the corresponding memory cell is allocated or unallocated; erasing the candidate storage unit corresponding to the candidate physical address identifier to enable the candidate storage unit to be a clean storage unit; and adding the candidate physical address identification to the storage space cache.

Optionally, the metadata storage module is further configured to: and updating the allocation bitmap.

Optionally, the processing module is specifically configured to:

searching an address mapping table in a memory;

the metadata storage module is specifically configured to:

Optionally, the metadata storage module is further configured to:

Optionally, the writing module is further configured to: and under the condition that a first physical address identifier corresponding to the first logical address identifier exists in an address mapping table in the memory, writing target data into a target storage unit corresponding to the first physical address identifier.

Optionally, the apparatus further comprises:

A receiving module, configured to receive a recovery request, where the recovery request includes: the second logic address identification is triggered after the user deletes the data in the storage unit corresponding to the second logic address identification;

the thin distribution cluster recovery module is used for acquiring a second physical address identifier corresponding to the second logical address identifier and erasing data of the storage unit corresponding to the second physical address identifier;

the metadata storage module is further to: deleting a second corresponding relation between a second logic address identifier and a second physical address identifier in an address mapping table in the storage pool; and deleting the second corresponding relation in the address mapping table in the memory.

Optionally, the apparatus further comprises:

and the space management module is used for outputting an over-allocation alarm when detecting that the storage space corresponding to the allocated total logical address is larger than the total storage space of the storage pool.

Optionally, the metadata storage module is further configured to: if the allocation bitmap is lost, the address mapping table in the storage pool is scanned to generate the allocation bitmap.

In a third aspect, the present disclosure provides an electronic device comprising: and a processor for executing the computer program stored in the memory, wherein the processor implements the method for managing the storage space according to the first aspect when executing the computer program.

In a fourth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of managing storage space as described in the first aspect above.

In a fifth aspect, the present disclosure provides a computer program product, which when run on a computer, causes the computer to perform the method of managing memory space as described in the first aspect above.

In the technical solution provided in the embodiment of the present invention, candidate physical address identifiers corresponding to a plurality of candidate storage units are recorded in a storage space cache, and the candidate storage units are clean storage units, so that a storage space is determined to be allocated to a first logical address identifier, the first physical address identifier is obtained from the storage space cache, a first correspondence between the first logical address identifier and the first physical address identifier is established, the first correspondence is written into an address mapping table in a memory, and the first correspondence is written into the address mapping table in a storage pool. Therefore, the storage units corresponding to the allocated first physical addresses are all clean storage units, and the safety of data storage is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic architecture diagram of a management system of a storage space provided in the present disclosure;

fig. 2 is a schematic flowchart of a method for managing storage space according to the present disclosure;

FIG. 3 is a schematic diagram illustrating a mapping relationship between logical addresses and physical addresses provided by the present disclosure;

FIG. 4 is a flow chart illustrating another method for managing storage space according to the present disclosure;

FIG. 5 is a schematic illustration of an allocation bitmap provided by the present disclosure;

FIG. 6 is a flow chart illustrating another method for managing storage space according to the present disclosure;

FIG. 7 is a flowchart illustrating another method for managing storage space according to the present disclosure;

FIG. 8 is a schematic structural diagram of a management apparatus for storage space according to the present disclosure;

fig. 9 is a schematic structural diagram of another management system for storage space provided by the present disclosure.

Detailed Description

In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that the embodiments and features of the embodiments of the present disclosure may be combined with each other without conflict.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced in other ways than those described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments.

In order to improve the utilization rate of the physical storage space of the data center, namely the physical disk, in the data center, the physical storage space may be pooled, so that in the process of allocating the storage space, the logical storage space is allocated in advance according to the capacity planned and requested by the user, and in the process of actually using by the user, the corresponding physical storage space is allocated according to the actually used amount of the user, which may also be called thin allocation.

However, in the process of allocating the physical storage space, when the allocated physical storage space is recovered, the secure erase processing cannot be performed, that is, the recovered physical storage space may contain legacy data of the front-end user, that is, dirty data is stored in the physical storage space. For example, in a physical storage space of a data center, a user a uses a part of the physical storage space and then does not use the physical storage space, the part of the physical storage space is recycled, data of the user a cannot be completely erased during recycling, when the part of the physical storage space is redistributed, and if the part of the physical storage space is allocated to a user B, the user B can read the data of the user a which is not completely erased, so that the problem of data security is brought, and the security of data storage is not high.

In order to solve the technical problem, the present disclosure provides a method for managing a storage space, which is applied to a management system of the storage space.

Referring to fig. 1, fig. 1 is a schematic diagram of an architecture of a management system of a storage space provided in the present disclosure, where the management system of the storage space may include: a read-write device 11, a management device 12 for storage space and a storage pool 13.

The read-write device 11 may be a block device or a file management system or the like.

The management device 12 of the storage space is used to implement management on the storage pool 13, and the management device 12 of the storage space may be a server or a computer, which is not limited in this disclosure.

Storage pool 13 is the physical storage space for data storage. The storage pool 13 may include one or more physical disks, and is obtained by pooling the physical disks, and the storage pool 13 may also be referred to as a storage resource pool, a physical storage space, a physical disk, or the like.

The storage pool 13 is divided into clusters to obtain a plurality of storage units. The cluster is the minimum unit of storage space allocation, that is, each cluster only allows one file to be used, and one or more files are not allowed to use the same cluster. The size of the cluster may be set in advance, for example, 512KB, and the size of each divided memory unit is the size of the cluster.

The data stored in the storage pool 13 contains metadata and user data. The metadata is used to record the allocation of the storage space, and the user data is data stored by the user, and may be, for example, system file data or other data.

The metadata may include, among other things, storage pool metadata and user metadata.

Storage pool metadata includes, but is not limited to, configuration information of storage space and metadata space allocation information, etc. The configuration information of the storage space may include information such as cluster size, whether thin allocation is enabled, and a unique identifier of the storage pool.

The user metadata is used for recording the allocation of the user application space, which may include but is not limited to an address mapping table. The address mapping table is used for storing the mapping relation between the logical address and the physical address. The logical address is known, and the corresponding physical address can be determined according to the address mapping table, so that the operations of writing or reading data and the like are realized.

The address mapping table stored in the storage pool 13 may be cached in the memory when the system of the management device 12 of the storage space is started, and the address mapping table is also stored in the memory. When the memory space is managed, corresponding operation is carried out based on the address mapping table in the memory, so that the high efficiency of address conversion and space allocation can be ensured. Meanwhile, after the address mapping table in the memory is updated, the address mapping table in the memory pool 13 needs to be updated correspondingly, so that the address mapping table in the memory pool 13 is updated in real time, wherein the operation of updating the address mapping table in the memory pool 13 may also be referred to as a destage. In this way, in an abnormal situation such as a system power failure or downtime, the address mapping table in the memory may be lost, but the address mapping table in the storage pool 13 maintains the mapping relationship.

For example, the storage data of the storage space is allocated such that the metadata occupies a small proportion of the storage space, that is, the space consumption ratio of the metadata is low. The following description will be given taking an example in which the physical address needs 8B for recording and the size of the memory cell is 512 KB. The primary metadata ratio is about 8B/512KB ≈ 0.015/1000. Assuming that metadata is stored in units of allocation of 4KB metadata pages, in an extreme case, that is, in a case where one metadata page only contains one storage unit, the space occupation ratio of the metadata is 4KB/512KB ≈ 0.0078, that is, about eight per thousand of space is used as metadata, the metadata space consumption ratio is very small and is much smaller than that of an existing file system, for example, a posix file system (about 4%) such as ext 4.

An application scenario of the present disclosure is described below with reference to fig. 1. When the read-write device 11 needs to request allocation of a storage space in the storage pool 13, the read-write device 11 sends a storage space allocation request to the management device 12 of the storage space, the management device 12 of the storage space allocates a clean storage unit with a corresponding size from the storage pool 13 according to the size of the required storage space included in the storage space allocation request, and the read-write device 11 can use the clean storage unit to perform operations such as writing or reading of data. Thereby completing the allocation of the memory space. The storage units allocated from the storage pool 13 are all clean storage units, so that the safety of data storage is improved.

The technical solution of the present disclosure is explained below with specific examples.

Referring to fig. 2, fig. 2 is a schematic flow chart of a storage space management method provided by the present disclosure, and the present embodiment is executed by the storage space management device shown in fig. 1. The method provided by the embodiment is as follows:

s201, determining to allocate a storage space to the first logic address identifier.

The read-write device may send a storage space allocation request to the management device of the storage space, where the request is to allocate a physical storage space, i.e. a physical address, corresponding to the first logical address identifier. The storage space allocation request comprises a first logic address identification. The storage space allocation request is used for requesting allocation of a physical address according to the first logical address identifier.

The read-write device may also send a write request including the first logical address identifier to the management device of the storage space, so that when the management device of the storage space determines that the physical storage space is not currently allocated to the first logical address identifier, the storage space needs to be allocated to the first logical address identifier. The read-write device may be the read-write device 11 shown in fig. 1 described above.

The first logical address identifier may include a first logical address and a logical address length, for example, the logical address space may be divided according to clusters, the first logical address may represent a two-dimensional logical address space (lba, len), the lba represents a lba-th logical unit, and the len represents a length of the acquired logical address.

S202, acquiring a first physical address identifier from the storage space cache.

And according to the determined number of the storage units, acquiring candidate physical address identifications corresponding to a plurality of candidate storage units from the storage space cache.

The storage space cache records candidate physical address identifiers corresponding to a plurality of candidate storage units, and the candidate storage units are all clean storage units. A clean memory cell may also be referred to as a blank memory cell, and is a memory cell that does not store data, for example, when a clean memory cell is read, the read data is "0". The candidate physical address identifications corresponding to the plurality of candidate storage units may be stored in a table form, which may be referred to as a zero cache table.

S203, establishing a first corresponding relation between the first logic address identifier and the first physical address identifier.

S204, writing the first corresponding relation into an address mapping table in the memory.

S205, writing the first corresponding relation into an address mapping table in the storage pool.

The structure and the stored content of the address mapping table in the memory and the address mapping table in the storage pool are the same, because the address mapping table in the memory is stored in the memory, when the corresponding relation between the logical address and the physical address is updated and searched, the address mapping table in the memory is directly used, and after the address mapping table in the memory is changed, the address mapping table stored in the storage pool needs to be correspondingly changed. Establishing a first corresponding relation between the first logic address identifier and the allocated first physical address identifier, writing the first corresponding relation into an address mapping table in the memory, and writing the first corresponding relation into the address mapping table in the storage pool.

Referring to fig. 3, fig. 3 is a schematic diagram illustrating a mapping relationship between a logical address and a physical address according to the present disclosure. The address mapping table 301 exemplarily shows a mapping relationship between a logical address and a physical address, where the address mapping table 301 may be an address mapping table in the above-mentioned memory or an address mapping table in a storage pool, and a logical address 302 stored in user data may find its physical address through the corresponding address mapping table 301, so as to obtain data stored in a storage space corresponding to the physical address. For example, in fig. 3, a logical address 0 is mapped to a physical address 2, a logical address 1 stores no data, stored data is "0", a logical address 2 stores no data, stored data is "0", and a logical address 3 is mapped to a physical address 4.

Fig. 3 exemplarily shows that "0" in the address mapping table 301 indicates that the data stored at the address is empty, and that "0" may be returned when the data at the address is actually read. In some other embodiments, "0" may also represent a 0 physical address in the address mapping table 301.

In this embodiment, candidate physical address identifiers corresponding to a plurality of candidate storage units are recorded in a storage space cache, and the candidate storage units are clean storage units, so that a storage space is determined to be allocated to a first logical address identifier, the first physical address identifier is obtained from the storage space cache, a first corresponding relationship between the first logical address identifier and the first physical address identifier is established, the first corresponding relationship is written into an address mapping table in a memory, and the first corresponding relationship is written into an address mapping table in a storage pool. Therefore, the storage units corresponding to the allocated first physical addresses are all clean storage units, and the safety of data storage is improved.

On the basis of the foregoing embodiment, further, an allocation bitmap may be stored in the memory, where the allocation bitmap is used to record allocation conditions of all memory units included in the current memory space, that is, whether each memory unit has been allocated. The unallocated memory cells can be obtained from the allocation bitmap, and the physical address identifier of the unallocated memory cell is stored in the memory space cache after the erasure processing, so that the candidate memory cells corresponding to the candidate physical address identifier stored in the memory space cache are all clean memory cells. The following description will be made with reference to a specific embodiment shown in fig. 3.

Referring to fig. 4, fig. 4 is a flowchart illustrating another management method for storage space according to the present disclosure, and fig. 4 is based on the embodiment shown in fig. 2, and further, before executing S202, the following steps S401 to S403 may be further executed:

s401, according to the allocation bitmap, acquiring candidate physical address identifications from physical address identifications of unallocated memory cells.

And acquiring the target number of unallocated memory units from the allocation bitmap, wherein the target number of unallocated memory units are candidate memory units, and the physical address identifier corresponding to the target number of memory units is the candidate physical address identifier.

Wherein each bit of the allocation bitmap corresponds to a memory cell, and the value of each bit is used to indicate whether the corresponding memory cell has been allocated or unallocated. For example, the allocation status of the memory cells may be sequentially recorded according to the physical address order of the memory cells, so as to obtain the allocation bitmap. Wherein the value of each bit may indicate that a memory cell has been allocated or not allocated with a preset value.

Referring to fig. 5, fig. 5 is a schematic diagram of an allocation bitmap provided by the present disclosure, where "1" may indicate that the memory cell is allocated, "0" indicates that the memory cell is not allocated, and "0" may be returned when the memory cell indicated by "0" is read.

S402, erasing the candidate storage unit corresponding to the candidate physical address identifier to enable the candidate storage unit to be a clean storage unit.

And erasing the candidate storage unit corresponding to the candidate physical address identifier to enable the candidate storage unit to be a clean storage unit.

The erasing process may be an erasing (discard) function provided by hardware, or may be a zero clearing (write zero) operation.

The erasing speed of the memory unit can be faster by using the erasing function provided by the hardware, and the erasing efficiency is improved.

And S403, adding the candidate physical address identifier to the storage space cache.

Optionally, a first threshold may be set, the number of candidate storage units stored in the storage space cache is detected in real time, and when the number of candidate storage units stored in the storage space cache is smaller than the first threshold, the steps S401 to S403 are executed, so that the number of candidate storage units in the storage space cache is greater than or equal to the first threshold.

According to the embodiment, according to the allocation bitmap, the candidate physical address identifier is obtained from the physical address identifier of the unallocated memory cell, the candidate memory cell corresponding to the candidate physical address identifier is erased, so that the candidate memory cell is a clean memory cell, the candidate physical address identifier is added to the memory space cache, and the candidate memory cells corresponding to the candidate physical address identifier stored in the memory space cache are all clean memory cells.

On the basis of the embodiment shown in fig. 4, further, after executing S205, the following steps may also be executed:

And updating the allocation bitmap.

After the allocation of the memory space is completed each time, the corresponding bit of the memory cell allocated this time in the allocation bitmap is updated to be allocated.

In this embodiment, after the allocation of the memory space is completed, the allocation bitmap is updated, so that the allocation status of the memory cells is accurately recorded in the allocation bitmap. Therefore, when the unallocated memory cells are acquired from the allocation bitmap for erasing processing, the allocated memory cells are acquired and the data stored in the memory cells are erased by mistake.

On the basis of the above embodiments, further, managing the storage space also involves operations such as writing, reading, and/or recycling of data.

In the scenario of writing target data, if the corresponding storage unit is already allocated as in the method of the above embodiment, the data may be directly written in the storage space.

If the corresponding storage unit is not allocated, the corresponding storage unit needs to be allocated, the metadata (namely the corresponding relation between the logical address and the physical address) is written into the address mapping table in the storage pool, and the data is written into the corresponding storage unit.

In the prior art, when data is allocated, an unallocated storage unit may be directly obtained from an allocation bitmap, and corresponding metadata needs to be written into a storage pool first, and after the metadata is successfully written, target data is written into a storage unit in a physical disk. This would avoid the problem of dirty data caused by a failure in writing metadata and a success in writing target data, however, this would make the writing of user data inefficient.

According to the management method of the storage space, the metadata and the user data are written into the storage pool at the same time. This is illustrated in the embodiment shown in fig. 6.

Referring to fig. 6, fig. 6 is a schematic flowchart of another storage space management method provided by the present disclosure, and fig. 6 is a flowchart of the embodiment shown in fig. 2 or fig. 4, further, S201 may include the following steps S2011-S2014, and correspondingly, S205 may include S2051:

s2011, a write request is received.

The management device of the storage space receives a write request sent by a read-write device, which may be the read-write device 11 shown in fig. 1. The write request comprises target data and a first logic address identifier. The write request is used for requesting to write the target data into the storage unit corresponding to the first logical address identification.

S2012, an address mapping table in the memory is searched.

S2013, judging whether a first physical address identifier corresponding to the first logical address identifier is stored in an address mapping table in the memory.

The management device of the storage space searches an address mapping table in the memory stored in the memory, and judges whether a first physical address identifier corresponding to the first logical address identifier is stored in the address mapping table in the memory.

If there is no first physical address identifier corresponding to the first logical address identifier in the address mapping table in the memory, it is described that the first physical address identifier is not allocated to the first logical address before, and the process continues to step S2014.

Optionally, when there is a first physical address identifier corresponding to the first logical address identifier in the address mapping table in the memory, which indicates that the first physical address identifier has been previously allocated to the first logical address, S2015 is continuously performed.

S2014, determining to allocate a storage space to the first logical address identifier.

Further, the allocation of the memory location corresponding to the first physical address identified in the allocation bitmap is updated to allocated.

S2051, writing the target data into the storage unit corresponding to the first physical address identifier and writing the first corresponding relationship into the address mapping table in the storage pool.

And simultaneously writing the first corresponding relation and the target data into a physical disk to realize data falling. For example, physical disks may be written simultaneously by one batch.

S2015, writing the target data into the storage unit corresponding to the first physical address identifier.

In this embodiment, even if the storage unit of the physical disk stores dirty data, because a clean storage unit is stored in the storage space cache, the clean storage unit is obtained from the storage space cache when the storage unit is allocated, and it is ensured that all the allocated storage units are clean storage units each time, when target data is written into the physical disk, the first correspondence and the write operation of the target data can be simultaneously completed.

Based on the embodiment shown in fig. 6, further, in step S2051, the operation of writing the first corresponding relationship and the target data into the storage pool at the same time, the obtained operation result (whether the writing is successful) relates to the following four cases, and the following processing steps in the four cases are exemplarily given in this embodiment:

the first condition is as follows: the target data is successfully written to the target storage unit and the first correspondence is successfully written to the address mapping table in the storage pool.

In the case that both the first corresponding relationship and the target data are successfully written into the storage pool, a write request success response may be returned, where the write request success response is used to indicate that the target data is successfully written into the storage pool.

Case two: the target data is successfully written to the target storage unit and the first correspondence fails to be written to the address mapping table in the storage pool.

In the event that the target data write to the storage pool is successful and the first corresponding relationship write to the storage pool fails, a write request failure response may be returned because the target data write to the storage pool failed. Wherein the write request failure response is to indicate that the target data write to the storage pool failed. Since the first mapping relation is written into the address mapping table in the storage pool and fails even if the target data is successfully written into the storage pool, the target data still cannot be read because the first mapping relation is not stored during subsequent addressing. And the address mapping table in the storage pool needs to be consistent with the address mapping table in the memory, and if the disk drop fails, the first corresponding relation needs to be deleted from the address mapping table in the memory.

In the second case, the target data that has been successfully written does not need to be deleted from the target storage unit. Because the first corresponding relation is not written, and the target data is successfully written into the target storage unit, the target storage unit in which the target data is written becomes dirty data, and the data security problem cannot be brought at this time.

Further, the value of the bit corresponding to the target memory location in the allocation bitmap is also rolled back as unallocated, i.e., the allocation of the target memory location in the allocation bitmap is updated as unallocated.

Case three: the write of the target data to the target storage unit fails and the write of the first correspondence to the address mapping table in the storage pool succeeds.

And if the first corresponding relation is successfully written and the target data is failed to be written, returning a write request failure response.

In case three, the first mapping relation does not need to be deleted from the address mapping table in the storage pool and the address mapping table in the memory. Since the first corresponding relationship is successfully written in both the memory and the storage pool, it indicates that the storage unit is successfully allocated, but the target data is not successfully written, in this case, in a possible case, since a write request failure response is returned, the user may try to rewrite the target data again, at this time, in step S603, it is determined that the corresponding storage unit is already allocated, and then S607 is continued. After the write request failure response is returned, the user does not rewrite the target data, and then if the user wants to read the target data, the read data is "0" because the target data is not written.

Case four: the write of the target data to the target storage unit fails and the write of the first correspondence to the address mapping table in the storage pool fails.

If the first corresponding relation and the target data are both failed to be written, a write request failure response can be returned, and the first corresponding relation is deleted from an address mapping table in the memory.

Further, the value of the bit corresponding to the target memory location in the allocation bitmap is also rolled back to be unallocated, i.e., the allocation of the target memory location in the allocation bitmap is updated to be unallocated.

In this embodiment, corresponding processing manners are given in the four writing operation results, and for a situation that dirty data is stored in a storage unit, which may be caused by simultaneous disk dropping of target data and the first corresponding relationship, a clean storage unit is obtained from a storage space cache when the storage unit is allocated each time, so that the problem of data security is avoided. When the writing fails, only the address mapping table in the memory needs to be returned in the memory, and the processing efficiency is further improved.

In other embodiments, in the data reclamation scenario, the reclamation of the storage unit is not performed in the prior art, or the reclamation operation of the storage unit is not performed with a complete erase process, so that the space of the storage pool cannot be reallocated. The management method for the storage space provided by this embodiment can perform an erasing operation on the storage unit in time and update the corresponding metadata when the user deletes the file data of the file system. The following description will be made with reference to a specific embodiment shown in fig. 7.

Referring to fig. 7, fig. 7 is a schematic flowchart of another management method for a storage space provided by the present disclosure, and fig. 7 is based on the embodiment shown in fig. 2, fig. 4, or fig. 6, and further, the method provided by this embodiment further includes the following steps:

s701, receiving a recovery request.

And the management equipment of the storage space receives the recovery request sent by the read-write equipment. The read-write device may be the read-write device 11 shown in fig. 1.

Wherein, the recovery request includes: and the second logical address identifier, the recovery request is triggered after the user deletes the data in the storage unit corresponding to the second logical address identifier.

S702, acquiring a second physical address identifier corresponding to the second logical address identifier, and erasing data of the memory cell corresponding to the second physical address identifier.

The second physical address identifier corresponding to the second logical address identifier can be quickly found from the address mapping table in the memory, and the data of the storage unit corresponding to the second physical address identifier is erased.

The erasing process may be an erasing (discard) function provided by hardware, or may be a zero clearing (write zero) operation. The erasure speed is faster by using the support of the discard provided by the hardware, and the real-time efficiency of batch recovery (for example, the operation of deleting the volume by the user) is ensured.

Further, the erase process may be a batch commit, where any memory unit fails to erase, and a space reclamation failure response is returned.

S703, deleting the second corresponding relation between the second logical address identifier and the second physical address identifier in the address mapping table in the storage pool.

S704, deleting the second corresponding relation in the address mapping table in the memory.

After the data in the storage unit is deleted, the corresponding relation stored in the address mapping table in the storage pool is deleted, and after the first corresponding relation is successfully deleted, the address mapping table in the storage pool is updated.

Further, the value of the bit corresponding to the second physical address identifier in the allocation bitmap is updated to be unallocated, that is, the allocation status of the memory location corresponding to the second physical address identifier in the allocation bitmap is updated to be unallocated.

It should be noted that the execution of S701-S704 and S201-S205 is not in sequence, for example, S701-S704 may be executed first, then S201-S205 may be executed, or S201-S205 may be executed first, then S701-S704 may be executed, or S701-S704 and S201-S205 may be executed simultaneously, which is not limited in this disclosure.

In this embodiment, after receiving a recovery request triggered by a user, a second physical address identifier corresponding to the second logical address identifier is obtained, data of a storage unit corresponding to the second physical address identifier is erased, a second corresponding relationship between the second logical address identifier and the second physical address identifier in an address mapping table in a storage pool is deleted, and the second corresponding relationship in the address mapping table in the memory is deleted. Therefore, when the user triggers the recovery storage unit, the storage unit is recovered in time, the released storage unit can be used for storing other data, and the utilization rate of the storage space is improved.

Based on the embodiment shown in fig. 7, further, in step S703, if it fails to delete the second corresponding relationship in the address mapping table in the storage pool, a space reclamation failure response may be directly returned. It should be noted that, because the storage unit corresponding to the second logical address identifier in the recovery request issued by the user stores the discarded data, the storage space is successfully erased, the metadata is not deleted, and the metadata is also consistent with the semantics, and the storage unit corresponding to the deleted second logical address identifier does not need to be restored, that is, the rollback processing is not needed.

In step S703, if the deletion of the second mapping relationship in the address mapping table in the storage pool is successful, the allocation bitmap in the cache and the address mapping table in the memory may be safely updated, and a recovery success response may be returned.

It should be noted that, assuming that the memory space of 8B can identify a mapping relationship from a logical address to a physical address, and update a metadata page (assumed to be 4KB), the update of the address mapping entry of 512 memory units (assumed to be 512KB in size) can be completed, corresponding to the recovery of the memory space of 256 MB. It is calculated that recycling 1GB of space only requires writing 16KB of data into the pool. Reclaiming 1TB of space also requires only writing 16MB of data into the storage pool. From the above calculation, the data recovery efficiency is higher.

In some other embodiments, further, in a data reading scenario, the method provided in this embodiment further includes the following steps:

step 1, receiving a read request, wherein the read request includes a fourth logical address identifier, and the read request is used for instructing to acquire data in a storage unit corresponding to the fourth logical address identifier.

And 2, searching an address mapping table in the memory.

And step 3, judging whether a fourth physical address identifier corresponding to the fourth logical address identifier is stored in an address mapping table in the memory.

And the management equipment of the storage space searches an address mapping table in the memory stored in the memory and judges whether a fourth physical address identifier corresponding to the fourth logical address identifier is stored in the address mapping table in the memory.

And under the condition that the address mapping table in the memory does not have a fourth physical address identifier corresponding to the fourth logical address identifier, indicating that the fourth physical address identifier is not allocated to the fourth logical address before, directly returning 0 data.

When there is a fourth physical address identifier corresponding to the fourth logical address identifier in the address mapping table in the memory, it is described that the fourth physical address identifier has been allocated to the fourth logical address before, and step 4 is continuously executed.

And 4, acquiring data from a storage unit corresponding to the fourth physical address identifier according to the inquired fourth physical address identifier, and returning the acquired data.

It should be noted that, the execution of the steps 1 to 4, S701 to S704, and S201 to S204 is not in sequence, for example, the steps 1 to 4 may be executed first, and then the steps 201 to S205 are executed, or the steps 201 to S205 may be executed first, and then the steps 1 to 4 may be executed, or the steps 1 to 4 and the steps 201 to S205 may be executed at the same time, which is not limited in this disclosure.

In this embodiment, when the read request is received, the corresponding data is read according to the fourth logical address identifier, so that the management scheme of the storage space is more complete.

In addition to the above embodiments, in an over-allocation scenario, that is, the storage space corresponding to the allocated total logical address is larger than the total storage space of the storage pool, for example, the size of the storage space of the actual storage pool is 1T, and if the storage space corresponding to the allocated logical address is 2T, at this time, the user performs a write operation on the allocated storage space, which may cause the storage space of the storage pool to be insufficient, that is, over-allocation occurs. In order to solve the above technical problem possibly brought by the over-allocation, the method provided by this embodiment further includes the following steps:

And outputting an over-allocation alarm when detecting that the storage space corresponding to the allocated total logical address is larger than the total storage space of the storage pool.

The size of the storage space corresponding to the allocated total logical address can be detected in real time, and when the fact that the storage space corresponding to the allocated total logical address is larger than the total storage space of the storage pool is detected, the over-allocation condition is obtained, and an over-allocation alarm can be output.

In this embodiment, an over-allocation alarm is output by detecting that the storage space corresponding to the allocated total logical address is greater than the total storage space of the storage pool, and processing is timely prompted under the over-allocation condition.

Further, after the over-allocation alarm is output, a migration operation may be performed on the data stored in the storage pool.

For example, in the case that data stored by the same user is stored in a first storage node, different data pages of the data stored by the same user may be stored in different storage nodes, where failure rates of different target nodes are smaller than a preset threshold, so that data of other storage nodes of the user is not lost under the condition that data is lost due to abnormal conditions such as a shutdown of a certain storage node, and thus all data of the user is not affected under the abnormal conditions.

In the above embodiment, further, the method of this embodiment may further include the following steps:

if the allocation bitmap is lost, the address mapping table in the storage pool is scanned to generate the allocation bitmap.

The allocation bitmap is cached in the memory, and is updated to the memory pool only when the memory pool is normally closed, so that the processing speed of allocation and recovery is higher. If the allocation bitmap is lost due to a system crash or the like, the allocation bitmap may be generated by scanning the address mapping table stored in the storage pool.

In the method of this embodiment, if the allocation bitmap is lost, the address mapping table in the storage pool is scanned, and the allocation bitmap can be generated quickly, so that the allocation bitmap does not need to be updated into the storage pool in real time, the delay on the IO critical path is reduced, and the processing resource consumption caused by frequently updating the allocation bitmap in the storage pool is avoided.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a management device for storage space according to the present disclosure, in which the device provided in this embodiment includes:

a processing module 801, configured to determine to allocate a storage space to the first logical address identifier;

the thin allocation module 802 is configured to obtain a first physical address identifier from a storage space cache, where the storage space cache records candidate physical address identifiers corresponding to multiple candidate storage units, and the candidate storage units are all clean storage units;

The metadata storage module 803 is configured to establish a first mapping relationship between a first logical address identifier and a first physical address identifier, write the first mapping relationship into an address mapping table in the memory, and write the first mapping relationship into the address mapping table in the storage pool.

Optionally, the thin allocation module 802 is further configured to: acquiring candidate physical address identifications from the physical address identifications of unallocated memory cells according to an allocation bitmap, wherein each bit of the allocation bitmap corresponds to one memory cell, and the value of each bit is used for indicating whether the corresponding memory cell is allocated or unallocated; erasing the candidate storage unit corresponding to the candidate physical address identifier to enable the candidate storage unit to be a clean storage unit; and adding the candidate physical address identification to the storage space cache.

Optionally, the metadata storage module 803 is further configured to: and updating the allocation bitmap.

Optionally, the processing module 801 is specifically configured to:

searching an address mapping table in a memory;

The metadata storage module 803 is specifically configured to:

Optionally, the metadata storage module 803 is further configured to:

and if the target data fails to be written into the target storage unit and the first corresponding relation fails to be written into the second address mapping, returning a write request failure response and deleting the first corresponding relation from the address mapping table in the memory.

Optionally, the processing module 801 is further configured to: receiving a recovery request, wherein the recovery request comprises: the second logic address identification is triggered after the user deletes the data in the storage unit corresponding to the second logic address identification;

the device also includes:

the metadata storage module 803 is further configured to: deleting a second corresponding relation between a second logic address identifier and a second physical address identifier in an address mapping table in the storage pool; and deleting the second corresponding relation in the address mapping table in the memory.

Optionally, the apparatus further comprises:

Optionally, the metadata storage module 803 is further configured to: if the allocation bitmap is lost, the address mapping table in the storage pool is scanned to generate the allocation bitmap.

The apparatus of the foregoing embodiment may be configured to implement the technical solution of the foregoing method embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.

The following description will be given taking a system for managing a storage space shown in fig. 9 as an example. Referring to fig. 9, fig. 9 is a schematic structural diagram of another management system for storage space according to the present disclosure. Fig. 9 is a diagram illustrating a system of fig. 1, and further, the management device 12 of the storage space includes: a metadata storage module 121, a thin allocation module 122, a thin allocation cluster reclamation module 123, and a space management module 124.

The read-write device 11 is configured to send a request for reading and writing of a user and space release to the management apparatus of the storage space, where the read-write device 11 may be a block device or a file system on an upper layer thereof.

The metadata storage module 121 is configured to manage metadata managed by the pooling system, where the metadata is cached in a memory, acts on a critical path of a user IO, and is used to cache an address mapping table from a user logical block to a physical block. The metadata storage module 121 may include: an address mapping table and an allocation bitmap in the memory. After changing the address mapping table in the memory, the metadata storage module 121 correspondingly changes the address mapping table in the storage pool 13.

Wherein an address mapping table in the memory is used for the translation from logical addresses to physical addresses. The allocation bitmap is used to record the allocation of memory cells in the memory pool.

The metadata storage module 121 may be the metadata storage module 803 in the embodiment shown in fig. 8.

The thin allocation module 122 is used to allocate memory units (clusters). The thin allocation module 122 may include: memory space cache allocator and memory space cache.

The memory space cache allocator is configured to complete allocation of the memory cells by interacting with an allocation bitmap in the metadata storage module 121. A certain number of clean storage units are cached in the storage space cache, and the storage space cache distributor can ensure that the distributed storage units are storage units without any dirty data under the condition that the erase sub-module in the thin distribution cluster recovery module 123 is abnormal.

The thin distribution cluster recovery module 123 is configured to recover a storage unit, and when the read/write device 11 sends a request for a deletion operation (trim) of a file system or a deletion operation (discard) of a block device, the thin distribution cluster recovery module 123 can quickly erase data in a recovery space by using a disk hardware technology, so as to achieve a purpose of safe erasure. Under extreme conditions (abnormal power failure of the server and the like), the recovered storage units do not complete data erasure, and a safe allocation mechanism of the storage space cache in the thin allocation module 122 can ensure that the storage units are allocated again without data retention of the front-end users.

The space management module 124 is used for space warning and processing of the storage pool, outputting a warning when the space ratio is high, performing automatic data page migration when the preset threshold is exceeded, and migrating allocated space and data of a part of users to other nodes in cooperation with the client.

The present disclosure provides an electronic device, including: and a processor for executing the computer program stored in the memory, wherein the processor implements the method for managing the memory space according to the above method embodiment when executing the computer program.

The present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of managing storage space as in the above-described method embodiments.

The present disclosure provides a computer program product which, when run on a computer, causes the computer to perform the method of managing storage space as in the above-described method embodiments.

It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The foregoing are merely exemplary embodiments of the present disclosure, which enable those skilled in the art to understand or practice the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for managing storage space, comprising:

establishing a first corresponding relationship between the first logical address identifier and the first physical address identifier, writing the first corresponding relationship into an address mapping table in a memory, and writing the first corresponding relationship into the address mapping table in a storage pool.

2. The method of claim 1, wherein before the obtaining the first physical address identifier from the memory space cache, further comprising:

acquiring candidate physical address identifications from physical address identifications of unallocated memory cells according to an allocation bitmap, wherein each bit of the allocation bitmap corresponds to one memory cell, and the value of each bit is used for indicating that the corresponding memory cell is allocated or unallocated;

3. The method of claim 2, further comprising:

updating the allocation bitmap.

4. The method of any of claims 1-3, wherein determining to allocate storage space for the first logical address identification comprises:

searching an address mapping table in the memory;

under the condition that the address mapping table in the memory does not have the physical address identifier corresponding to the logical address identifier, determining to allocate a storage space to the first logical address identifier;

The writing the first correspondence to an address mapping table in a storage pool includes:

5. The method of claim 4, further comprising:

and if the target data fails to be written into the target storage unit and the first corresponding relation fails to be written into the address mapping table in the storage pool, returning a write request failure response and deleting the first corresponding relation from the address mapping table in the storage pool.

6. The method of claim 4, further comprising:

and under the condition that a first physical address identifier corresponding to the first logical address identifier exists in an address mapping table in the memory, writing the target data into a target storage unit corresponding to the first physical address identifier.

7. The method according to any one of claims 1-3, further comprising:

receiving a recovery request, wherein the recovery request comprises: the recovery request is triggered after a user deletes data in a storage unit corresponding to the second logical address identifier;

deleting a second corresponding relation between the second logical address identifier and the second physical address identifier in an address mapping table in the storage pool;

8. The method according to any one of claims 1-3, further comprising:

9. A method according to claim 2 or 3, characterized in that the method further comprises:

and if the allocation bitmap is lost, scanning an address mapping table in the storage pool to generate the allocation bitmap.

10. An apparatus for managing a storage space, comprising:

and the metadata storage module is used for establishing a first corresponding relation between the first logical address identifier and the first physical address identifier, writing the first corresponding relation into an address mapping table in a memory, and writing the first corresponding relation into the address mapping table in a storage pool.

11. An electronic device, comprising: processor for executing a computer program stored in a memory, the processor implementing the method of management of storage space of any of claims 1-9 when executing the computer program.

12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of management of storage space of any one of claims 1 to 9.

13. A computer program product, characterized in that it causes a computer to carry out the method of management of storage space of any one of claims 1-9, when said computer program product is run on said computer.