CN113836051A - Metadata space recovery method, device, equipment and storage medium - Google Patents
Metadata space recovery method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113836051A CN113836051A CN202111427657.2A CN202111427657A CN113836051A CN 113836051 A CN113836051 A CN 113836051A CN 202111427657 A CN202111427657 A CN 202111427657A CN 113836051 A CN113836051 A CN 113836051A
- Authority
- CN
- China
- Prior art keywords
- target
- metadata
- tree
- metadata space
- space address
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000011084 recovery Methods 0.000 title claims abstract description 35
- 238000004064 recycling Methods 0.000 claims abstract description 11
- 230000005055 memory storage Effects 0.000 claims description 30
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012795 verification Methods 0.000 claims description 8
- 125000004122 cyclic group Chemical group 0.000 claims description 7
- 230000000977 initiatory effect Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000007726 management method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a metadata space recovery method, a device, equipment and a storage medium, wherein the method comprises the following steps: allocating a metadata space address for metadata of the full flash memory system, and organizing and managing the metadata through a data structure of a B + tree; when a metadata space address is allocated to the full flash memory system, if a fault node is monitored to appear in the full flash memory system, recording a target data block currently used for allocating the metadata space address; when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block; determining a target B + tree corresponding to the target metadata space address; querying the target B + tree to determine whether the target metadata space address is a valid address, and if not, recycling the target metadata space address. The method and the device effectively reduce the condition of leakage of the corresponding metadata space caused by storage node faults, and improve the metadata space access efficiency.
Description
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a metadata space recovery method, apparatus, device, and storage medium.
Background
Currently, with the rapid development of information technology, massive data is generated, and how to effectively manage and organize the massive data becomes a prominent problem. For large amounts of data stored, the data can only be utilized more efficiently if the query analyzes the data content and data meaning therein. Efficient organization and management of Metadata (Metadata) in a storage system is an effective means to address the above issues, enabling the system to manage and maintain the data. It can be seen that data can be made more valuable by efficiently managing metadata. At present, in the process of managing metadata of a full flash memory storage system, the condition of metadata space leakage is easy to occur, so that the metadata space is insufficient, the metadata space access efficiency is further reduced, and even the unavailability of the whole full flash memory storage system can be caused in severe cases.
Therefore, how to improve the metadata space access efficiency of the full flash storage system is a technical problem to be solved in the art.
Disclosure of Invention
In view of the above, an object of the present application is to provide a method, an apparatus, a device and a storage medium for reclaiming a metadata space, which can improve the metadata space access efficiency of a full flash memory system. The specific scheme is as follows:
in a first aspect, the present application discloses a metadata space recycling method, including:
allocating a metadata space address for metadata of a full flash memory storage system, and organizing and managing the metadata of the full flash memory storage system through a data structure of a B + tree;
when a metadata space address is allocated to the full flash memory system, if a fault node is monitored to occur in the full flash memory system, recording a target data block currently used for allocating the metadata space address;
when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block;
determining the B + tree corresponding to the target metadata space address to obtain a target B + tree;
and querying the target B + tree to determine whether the target metadata space address is an effective address according to a query result, and if not, recovering the target metadata space address.
Optionally, before determining the B + tree corresponding to the target metadata space address, the method further includes:
detecting whether the target metadata space address has been written with metadata;
if the target metadata space address is not written with metadata, the target metadata space address is put into a preset recovery linked list;
if the target metadata space address has been written with metadata, triggering the step of determining the B + tree corresponding to the target metadata space address.
Optionally, the detecting whether the target metadata space address has been written with metadata includes:
initiating a reading operation on a local preset disk aiming at the target metadata space address to determine a target tree node which is recorded in the local preset disk and corresponds to the target metadata space address;
verifying the target tree nodes by using a cyclic redundancy check algorithm;
if the check fails, determining that the target metadata space address has not been written to metadata;
and if the verification is passed, determining that the target metadata space address is written with the metadata once.
Optionally, the determining the B + tree corresponding to the target metadata space address to obtain a target B + tree includes:
acquiring key value pair data values of the target tree nodes;
and determining the B + tree where the target tree node is located according to the key value pair data value of the target tree node to obtain the target B + tree.
Optionally, after the organizing and managing the metadata of the full flash storage system through the data structure of the B + tree, the method further includes:
and recording the mapping relation between each tree node on each B + tree and the corresponding metadata space address to a local disk to obtain the local preset disk.
Optionally, the querying the target B + tree to determine whether the target metadata space address is an effective address according to a query result, and if not, the recovering the target metadata space address includes:
traversing and inquiring the tree nodes of the target B + tree according to the key value pair data values of the target tree nodes so as to determine whether the target metadata space address is an effective address according to the inquiry result;
if the target metadata space address is an invalid address, putting the target metadata space address into the preset recycling linked list;
and if the target metadata space address is an effective address, forbidding to recycle the target metadata space address.
Optionally, the performing traversal query on the tree node of the target B + tree according to the key-value pair data value of the target tree node to determine whether the target metadata space address is an effective address according to a query result includes:
according to the key value pair data value of the target tree node, performing traversal query operation from a root node to a leaf node on the target B + tree so as to compare whether the value of a non-leaf node of the B + tree is consistent with the target metadata space address or not in the traversal query operation process;
if the value of the non-leaf node of the B + tree is inconsistent with the target metadata space address, judging that the target metadata space address is an invalid address at present;
and if the value of the non-leaf node of the B + tree is consistent with the target metadata space address, judging that the target metadata space address is an effective address at present.
In a second aspect, the present application discloses a metadata space reclamation apparatus, comprising:
the metadata management module is used for distributing metadata space addresses for the metadata of the full flash memory storage system and organizing and managing the metadata of the full flash memory storage system through a data structure of a B + tree;
the data block recording module is used for recording a target data block which is currently used for distributing the metadata space address if a fault node is monitored to occur in the full flash memory storage system when the metadata space address is distributed to the full flash memory storage system;
a space address determination module, configured to scan the target data block when it is monitored that the failed node is recovered, so as to determine a target metadata space address corresponding to the target data block;
a target tree determining module, configured to determine the B + tree corresponding to the target metadata space address to obtain a target B + tree;
and the space address recovery module is used for inquiring the target B + tree so as to determine whether the target metadata space address is an effective address according to an inquiry result, and if not, the target metadata space address is recovered.
In a third aspect, the present application discloses an electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the aforementioned metadata space reclamation method.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the aforementioned metadata space reclamation method.
The metadata space recovery scheme comprises the steps of distributing metadata space addresses for metadata of a full flash memory storage system, and organizing and managing the metadata of the full flash memory storage system through a data structure of a B + tree; when a metadata space address is allocated to the full flash memory system, if a fault node is monitored to occur in the full flash memory system, recording a target data block currently used for allocating the metadata space address; when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block; determining the B + tree corresponding to the target metadata space address to obtain a target B + tree; and querying the target B + tree to determine whether the target metadata space address is an effective address according to a query result, and if not, recovering the target metadata space address. Therefore, the metadata of the full flash memory storage system is organized and managed by the B + tree, when a fault node occurs in the full flash memory storage system, the corresponding B + tree is inquired by the metadata space address corresponding to the target data block which is subjected to metadata space address allocation at the moment, whether the metadata space address corresponding to the target data block is an effective address or not is determined according to the inquiry result, and the target data block is selected to be recovered under the condition that the metadata space address is not the effective address, so that the condition that the corresponding metadata space is leaked due to the fault of the storage node is effectively reduced, the occurrence rate of insufficient metadata space is reduced, and the metadata space access efficiency of the full flash memory storage system is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a metadata space reclamation method provided herein;
FIG. 2 is a flow chart of a specific metadata space reclamation method provided herein;
FIG. 3 is a flow chart of a specific metadata space reclamation method provided herein;
FIG. 4 is a flowchart of a specific metadata space reclamation method provided herein;
FIG. 5 is a flowchart of a specific metadata space reclamation method provided herein;
FIG. 6 is a schematic structural diagram of a metadata space recycling apparatus according to the present application;
fig. 7 is a block diagram of an electronic device provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
At present, in the process of managing metadata of a full flash memory storage system, the condition of metadata space leakage is easy to occur, so that the metadata space is insufficient, the metadata space access efficiency is further reduced, and even the unavailability of the whole full flash memory storage system can be caused in severe cases. Therefore, the present application provides a metadata space reclamation scheme to solve the above technical problems.
Referring to fig. 1, an embodiment of the present application discloses a metadata space recycling method, which may specifically include:
step S11: and allocating a metadata space address for the metadata of the full flash memory system, and organizing and managing the metadata of the full flash memory system through a data structure of a B + tree.
In this embodiment, when a space address is allocated to metadata in the full flash storage system, the metadata of the full flash storage system is organized and maintained through a data structure such as a B + tree. It should be noted that, in this embodiment, the tree nodes of the B + tree store information in the form of Key-Value pairs (Key represents a Key, and Value represents a Value). Since the non-leaf nodes of the B + tree are designed as index nodes and not as data storage nodes, the B + tree has very high search efficiency, and from the time complexity, the search time complexity of the B + tree can reach O (logn), and the space utilization rate can reach 75%. Therefore, in the embodiment, the metadata objects of the full flash memory storage system are organized by using the data structure of the B + tree, so that the search efficiency of the metadata objects in the full flash memory storage system can be greatly improved.
It is understood that the full flash memory system in the embodiment is a memory system completely composed of solid-state storage media, and has advantages such as better low latency characteristics compared to a hybrid flash memory system or a conventional memory system completely composed of mechanical hard disks.
Step S12: when the metadata space address is distributed to the full flash memory system, if the situation that a fault node occurs in the full flash memory system is monitored, a target data block which is used for distributing the metadata space address currently is recorded.
It should be noted that, in the process of allocating the metadata space address to the metadata of the full flash memory system by using the data block on the physical memory space, if a storage node in the full flash memory system fails, the failure of the metadata space address on the data block may be caused by these sudden failure events. In order to reduce the number of failed metadata space addresses caused by the above reasons, when a failed node is monitored, a data block, which is used for allocating a metadata space address to metadata to be stored in the full flash memory storage system at this time, is recorded, that is, the target data block is obtained, so that it is determined whether space recovery is required or not by analyzing the validity of the metadata space address on the target data block in the following process, thereby reducing the number of failed metadata space addresses. It is understood that the type of the data Block is a Physical Block, and accordingly, the metadata space Address specifically refers to an Address (i.e., PBA) on the Physical Block, where one data Block may include several metadata space addresses.
Step S13: and when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block.
In this embodiment, after it is monitored that the failed node is repaired, all corresponding metadata space addresses in the target data block may be further determined, so as to obtain the target metadata space address. That is, after the failed node is repaired, the present embodiment performs subsequent space reclamation check on all metadata space addresses in the target data block recorded in step S12, so as to reduce the occurrence of missing. Specifically, in this embodiment, the metadata space addresses in the target data block are scanned in an address scanning manner, so as to determine all the metadata space addresses corresponding to the target data block.
Step S14: and determining the B + tree corresponding to the target metadata space address to obtain a target B + tree.
In this embodiment, the metadata of the full flash memory system is organized and managed by the data structure of the B + tree, so that different metadata space addresses correspond to different B + trees. Furthermore, after the target metadata space address is determined, the target metadata space address can be used to find out a target B + tree corresponding to the target metadata space address, so that the validity of the target metadata space address can be analyzed by using the found target B + tree in the following.
Step S15: and querying the target B + tree to determine whether the target metadata space address is an effective address according to a query result, and if not, recovering the target metadata space address.
It should be noted that, when the target B + tree is queried, whether the target metadata space address is an effective address may be determined based on a result of comparing the consistency between the Value of the queried non-leaf node and the target metadata space address, and if the result is valid, it indicates that the target metadata space address is not leaked, and then recovery is not required; an invalid state indicates that the target metadata space address has leaked, for which reclamation may be performed. In this embodiment, if the Value on the non-leaf node is inconsistent with the target metadata space address, it indicates that the target metadata space address is discarded, so that it may be determined as an invalid address and recycled, thereby reducing the number of invalid metadata space addresses. If the Value on the non-leaf node is consistent with the target metadata space address, the target metadata space address is still in a used state, so that the target metadata space address is judged to be a valid address and is forbidden to be recycled.
Therefore, according to the embodiment of the application, the B + tree is used for organizing and managing the metadata of the full flash memory storage system, when a fault node occurs in the full flash memory storage system, the corresponding B + tree is queried by using the corresponding metadata space address of the target data block which is currently subjected to metadata space address allocation, whether the metadata space address corresponding to the target data block is an effective address or not is determined according to a query result, and the target data block is selected to be recovered under the condition that the metadata space address is not the effective address, so that the condition that the corresponding metadata space is leaked due to the fault of the storage node is effectively reduced, the occurrence rate of insufficient metadata space is reduced, and the metadata space access efficiency of the full flash memory storage system is improved. Moreover, it should be noted that when the metadata space address is recovered by the metadata space recovery scheme of this embodiment, the normal operation of the service of the host can be continuously maintained, the service of the host does not need to be interrupted, and the recovery of the dynamic metadata space address is supported, so that the influence of the metadata space recovery process on the service of the host is greatly reduced, and efficient metadata space access is obtained.
Referring to fig. 2, an embodiment of the present application discloses a metadata space recovery method, which specifically includes:
step S21: and allocating a metadata space address for the metadata of the full flash memory system, and organizing and managing the metadata of the full flash memory system through a data structure of a B + tree.
Step S22: when the metadata space address is distributed to the full flash memory system, if the situation that a fault node occurs in the full flash memory system is monitored, a target data block which is used for distributing the metadata space address currently is recorded.
Step S23: and when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block.
For details of the above steps S21 to S23, reference may be made to the related contents disclosed in the foregoing text, and details are not repeated herein.
Step S24: detecting whether the target metadata space address was written with metadata.
It should be noted that, after the target metadata space address on the target data block is determined, if the target metadata space address has not been written with metadata before, it indicates that the target metadata space address is a free space address, in this case, the target metadata space address can be directly recovered without performing subsequent B + tree query operations for the target metadata space address, thereby simplifying the recovery process of the metadata space address and improving the recovery efficiency of the metadata space address.
Step S25: and if the target metadata space address is not written with metadata, putting the target metadata space address into a preset recovery linked list.
In this embodiment, a recovery linked list is preset, and when it is determined that the target metadata space address has not been written with metadata, it indicates that the target metadata space address is always in an idle state, and therefore, the target metadata space address may be directly placed in the preset recovery linked list, so as to implement recovery processing on the target metadata space address.
Step S26: and if the target metadata space address is written with metadata, determining the B + tree corresponding to the target metadata space address to obtain a target B + tree.
Step S27: and querying the target B + tree to determine whether the target metadata space address is an effective address according to a query result, and if not, recovering the target metadata space address.
In this embodiment, after determining that the target metadata space address was written with metadata, it is further required to determine whether the target metadata space address is currently in a used state by querying the target B + tree. If the target metadata space address has been retired, it may be determined to be an invalid address and recycled, thereby reducing the number of invalid metadata space addresses. If the target metadata space address is still in use, it is determined to be a valid address and is prohibited from being reclaimed.
As can be seen from the above, after the target metadata space address on the target data block is determined, if it is detected that the target metadata space address has not been written with metadata before, the target metadata space address can be directly recovered without performing subsequent B + tree query operations performed on the target metadata space address, so that a recovery process of the metadata space address is simplified, and recovery efficiency of the metadata space address is improved.
Referring to fig. 3, an embodiment of the present application discloses a metadata space recovery method, which specifically includes:
step S31: and allocating a metadata space address for the metadata of the full flash memory system, and organizing and managing the metadata of the full flash memory system through a data structure of a B + tree.
Step S32: when the metadata space address is distributed to the full flash memory system, if the situation that a fault node occurs in the full flash memory system is monitored, a target data block which is used for distributing the metadata space address currently is recorded.
Step S33: and when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block.
For details of the above steps S31 to S33, reference may be made to the related contents disclosed in the foregoing text, and details are not repeated herein.
Step S34: and initiating a reading operation on a local preset disk aiming at the target metadata space address so as to determine a target tree node which is recorded in the local preset disk and corresponds to the target metadata space address.
In this embodiment, after the target metadata space address corresponding to the target data block is determined, the target tree node corresponding to the target metadata space address and pre-recorded on the local preset disk is determined by using information recorded on the local preset disk, so that further analysis is performed subsequently based on the determined target tree node, and it is determined whether the target metadata space address needs to be recovered according to an analysis result.
It can be understood that, after the step of organizing and managing the metadata of the full flash storage system through the data structure of the B + tree, the embodiment may further specifically include: and recording the mapping relation between each tree node on each B + tree and the corresponding metadata space address to a local disk to obtain the local preset disk. That is to say, in this embodiment, when the B + tree is used to organize the metadata, the mapping relationship between the corresponding metadata space address and the corresponding tree node may be recorded in the local disk, so that the information recorded on the disk may be subsequently used to determine the tree node corresponding to the target metadata space address.
Step S35: and checking the target tree nodes by using a cyclic redundancy check algorithm.
Step S36: and if the verification is not passed, judging that the target metadata space address is not written into the metadata, and putting the target metadata space address into a preset recovery linked list.
Step S37: and if the verification is passed, judging that the target metadata space address has been written with metadata, and determining the B + tree corresponding to the target metadata space address to obtain a target B + tree.
In this embodiment, to determine whether the metadata has been written into the target metadata space address, specifically, a Cyclic Redundancy Check (CRC) algorithm is used to Check a target tree node corresponding to the target metadata space address, so as to determine whether the target metadata space address corresponding to the target tree node has been written with the metadata. If the metadata has not been written, it is an indication of a free space address, and it can be reclaimed directly. If the metadata is written once, the metadata cannot be directly recycled, but the subsequent B + tree query operation is carried out to finally determine whether the current target metadata space address is still in a used state or is abandoned, and accordingly whether the metadata can be recycled is determined.
It should be noted that, in this embodiment, the determining the B + tree corresponding to the target metadata space address to obtain a target B + tree specifically may include: and obtaining the key value pair data value of the target tree node, and determining the B + tree where the target tree node is located according to the key value pair data value of the target tree node to obtain the target B + tree. That is, in this embodiment, the B + tree where the target tree node is located may be determined according to the KV value on the target tree node, that is, the target B + tree is obtained.
Step S38: and querying the target B + tree to determine whether the target metadata space address is an effective address according to a query result, and if not, recovering the target metadata space address.
In this embodiment, after it is determined that the target metadata space address has been written with metadata, traversal query may be performed on the tree node of the target B + tree according to the key value of the target tree node, so as to determine whether the target metadata space address is an effective address according to a query result, that is, determine whether the target metadata space address is currently in a used state. If the target metadata space address has been retired, it may be determined to be an invalid address and recycled, thereby reducing the number of invalid metadata space addresses. If the target metadata space address is still in use, it is determined to be a valid address and is prohibited from being reclaimed.
As can be seen from the above, in order to determine whether metadata has been written into a target metadata space address, in this embodiment, a CRC algorithm is specifically used to check a target tree node corresponding to the target metadata space address, so as to determine whether a target metadata space address corresponding to the target tree node has been written with metadata. If the metadata is not written, the metadata is indicated to be a free space address, and then the metadata can be directly recycled without performing subsequent B + tree query operation for the metadata, so that the recycling process of the metadata space address is simplified. If the metadata is written once, the metadata cannot be directly recycled, but the B + tree query operation is performed to finally determine whether the current target metadata space address is still in a used state or is discarded, so as to determine whether the metadata can be recycled.
Referring to fig. 4, an embodiment of the present application discloses a metadata space recovery method, which specifically includes:
step S41: and allocating a metadata space address for the metadata of the full flash memory system, and organizing and managing the metadata of the full flash memory system through a data structure of a B + tree.
Step S42: when the metadata space address is distributed to the full flash memory system, if the situation that a fault node occurs in the full flash memory system is monitored, a target data block which is used for distributing the metadata space address currently is recorded.
Step S43: and when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block.
Step S44: and initiating a reading operation on a local preset disk aiming at the target metadata space address so as to determine a target tree node which is recorded in the local preset disk and corresponds to the target metadata space address.
Step S45: and checking the target tree nodes by using a cyclic redundancy check algorithm.
Step S46: and if the verification is not passed, judging that the target metadata space address is not written into the metadata, and putting the target metadata space address into a preset recovery linked list.
Step S47: and if the verification is passed, judging that the target metadata space address has been written with metadata, and determining the B + tree corresponding to the target metadata space address to obtain a target B + tree.
For details of the above steps S41 to S47, reference may be made to the related contents disclosed in the foregoing text, and details are not repeated herein.
Step S48: and performing traversal query operation from a root node to a leaf node on the target B + tree according to the key value pair data value of the target tree node so as to compare whether the value of a non-leaf node of the B + tree is consistent with the target metadata space address or not in the traversal query operation process, if not, judging that the current target metadata space address is an invalid address, and if so, judging that the current target metadata space address is an effective address.
In this embodiment, when querying a target B + tree, a traversal query operation from a root node to a leaf node is specifically performed on the target B + tree according to a key value pair data value of a target tree node, and in the traversal query operation process, whether a current target metadata space address is valid is determined by comparing whether a value of a non-leaf node of the B + tree is consistent with the target metadata space address. And if the target metadata space address is consistent with the target metadata space address, judging that the target metadata space address is an invalid address.
Step S49: if the target metadata space address is an invalid address, putting the target metadata space address into the preset recycling linked list; and if the target metadata space address is an effective address, forbidding to recycle the target metadata space address.
For example, referring to fig. 5, fig. 5 shows a specific flow of reclamation for metadata space in a full flash storage system. When a node failure event occurs, all space addresses in a data Block currently used for allocating metadata space addresses are determined to be used as metadata space addresses PBA to be recovered. And then, reading a metadata organization unit corresponding to the PBA, namely a corresponding target tree node BtreeNode, from the local disk according to the metadata space address PBA in the Block, and performing CRC (cyclic redundancy check) on the BtreeNode. If the verification is not passed, the metadata is not written into the address, and the address can be directly recycled; if the check passes, this indicates that the address has written metadata content and the first key value pair data value (KV value) of BtreeNode is obtained. And initiating a query operation aiming at the B + tree where the BtreeNode is located according to the acquired KV value, wherein a query path is from the root node to the leaf node. In the query process, whether the Value of the non-leaf node is consistent with the metadata space address PBA is compared. If the address is consistent, the PBA is indicated to be in use and not recyclable; if not, it indicates that the PBA is not in use, and can be reclaimed.
Referring to fig. 6, an embodiment of the present application further discloses a metadata space recycling apparatus, which includes:
the metadata management module 11 is configured to allocate a metadata space address to metadata of the full flash storage system, and organize and manage the metadata of the full flash storage system through a data structure of a B + tree;
a data block recording module 12, configured to record a target data block currently used for allocating a metadata space address if it is monitored that a failure node occurs in the full flash memory system when allocating the metadata space address to the full flash memory system;
a space address determining module 13, configured to scan the target data block when it is monitored that the failed node is recovered, so as to determine a target metadata space address corresponding to the target data block;
a target tree determining module 14, configured to determine the B + tree corresponding to the target metadata space address to obtain a target B + tree;
and the space address recovery module 15 is configured to query the target B + tree, determine whether the target metadata space address is an effective address according to a query result, and recover the target metadata space address if the target metadata space address is not an effective address.
Therefore, according to the embodiment of the application, the B + tree is used for organizing and managing the metadata of the full flash memory storage system, when a fault node occurs in the full flash memory storage system, the corresponding B + tree is queried by using the corresponding metadata space address of the target data block which is currently subjected to metadata space address allocation, whether the metadata space address corresponding to the target data block is an effective address or not is determined according to a query result, and the target data block is selected to be recovered under the condition that the metadata space address is not the effective address, so that the condition that the corresponding metadata space is leaked due to the fault of the storage node is effectively reduced, the occurrence rate of insufficient metadata space is reduced, and the metadata space access efficiency of the full flash memory storage system is improved. Moreover, it should be noted that when the metadata space address is recovered by the metadata space recovery scheme of this embodiment, the normal operation of the service of the host can be continuously maintained, the service of the host does not need to be interrupted, and the recovery of the dynamic metadata space address is supported, so that the influence of the metadata space recovery process on the service of the host is greatly reduced, and efficient metadata space access is obtained.
In one embodiment, the metadata space reclamation apparatus further includes:
a detection module for detecting whether the target metadata space address has been written with metadata;
a first linked list writing unit, configured to, when the target metadata space address has not been written with metadata, place the target metadata space address in a preset recovery linked list;
a triggering module, configured to trigger the step of determining the B + tree corresponding to the target metadata space address when the target metadata space address has been written with metadata.
In one embodiment, the detection module includes:
a disk reading unit, configured to initiate a reading operation on a local preset disk for the target metadata space address to determine a target tree node corresponding to the target metadata space address and recorded in the local preset disk;
the checking unit is used for checking the target tree node by using a cyclic redundancy check algorithm;
a first determination unit configured to determine that the target metadata space address has not been written with metadata when the check fails;
and the second judgment unit is used for judging that the target metadata space address is written with the metadata once when the verification is passed.
In a specific embodiment, the target tree determining module 14 includes:
a key-value pair obtaining unit, configured to obtain a key-value pair data value of the target tree node;
and the target tree determining unit is used for determining the B + tree where the target tree node is located according to the key value pair data value of the target tree node to obtain the target B + tree.
In one embodiment, the metadata space reclamation apparatus further includes:
and the recording module is used for recording the mapping relation between each tree node on each B + tree and the corresponding metadata space address to a local disk so as to obtain the local preset disk.
In a specific embodiment, the spatial address recycling module 15 includes:
the traversal query submodule is used for performing traversal query on the tree nodes of the target B + tree according to the key value data values of the target tree nodes so as to determine whether the target metadata space address is an effective address according to a query result;
a second linked list writing unit, configured to, when the target metadata space address is an invalid address, place the target metadata space address in the preset recovery linked list;
and the recovery prohibiting unit is used for prohibiting the recovery of the target metadata space address when the target metadata space address is an effective address.
In one embodiment, the traversal query submodule includes:
a traversal query unit, configured to perform a traversal query operation from a root node to a leaf node on the target B + tree according to the key-value pair data value of the target tree node, so as to compare whether a value of a non-leaf node of the B + tree is consistent with the target metadata space address in a traversal query operation process;
a third determining unit, configured to determine that the current target metadata space address is an invalid address when a value of a non-leaf node of the B + tree is inconsistent with the target metadata space address;
a fourth determining unit, configured to determine that the current target metadata space address is an effective address when a value of a non-leaf node of the B + tree is consistent with the target metadata space address.
Further, the embodiment of the application also provides electronic equipment. FIG. 7 is a block diagram illustrating an electronic device 20 according to an exemplary embodiment, and the contents of the diagram should not be construed as limiting the scope of use of the present application in any way.
Fig. 7 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic equipment
20, the method specifically comprises the following steps: at least one processor 21, at least one memory 22, a display 23, an input output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. Wherein the memory 22 is used for storing a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the metadata space reclamation method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be specifically an electronic computer.
In this embodiment, the power supply 26 is used for providing an operating voltage for each hardware device on the electronic device 20;
the communication interface 25 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 24 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the memory 22 can be a read-only memory, a random access memory, a flash memory, a,
Magnetic or optical disks, etc., on which the stored resources may include computer programs 221, in a form of transient or persistent storage. The computer program 221 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the metadata space reclamation method performed by the electronic device 20 disclosed in any of the foregoing embodiments.
Further, the embodiment of the application also discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the metadata space reclamation method disclosed above.
For the specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, which are not herein referred to
Further description is given.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The metadata space recycling method, apparatus, device and storage medium provided by the present application are introduced in detail above, and a specific example is applied in the present application to explain the principle and implementation of the present application, and the description of the above embodiment is only used to help understand the method and core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A method for reclaiming metadata space, comprising:
allocating a metadata space address for metadata of a full flash memory storage system, and organizing and managing the metadata of the full flash memory storage system through a data structure of a B + tree;
when a metadata space address is allocated to the full flash memory system, if a fault node is monitored to occur in the full flash memory system, recording a target data block currently used for allocating the metadata space address;
when the fault node is monitored to be recovered, scanning the target data block to determine a target metadata space address corresponding to the target data block;
determining the B + tree corresponding to the target metadata space address to obtain a target B + tree;
and querying the target B + tree to determine whether the target metadata space address is an effective address according to a query result, and if not, recovering the target metadata space address.
2. The method of claim 1, wherein prior to determining the B + tree corresponding to the target metadata space address, further comprising:
detecting whether the target metadata space address has been written with metadata;
if the target metadata space address is not written with metadata, the target metadata space address is put into a preset recovery linked list;
if the target metadata space address has been written with metadata, triggering the step of determining the B + tree corresponding to the target metadata space address.
3. The method of claim 2, wherein the detecting whether the target metadata space address has been written with metadata comprises:
initiating a reading operation on a local preset disk aiming at the target metadata space address to determine a target tree node which is recorded in the local preset disk and corresponds to the target metadata space address;
verifying the target tree nodes by using a cyclic redundancy check algorithm;
if the check fails, determining that the target metadata space address has not been written to metadata;
and if the verification is passed, determining that the target metadata space address is written with the metadata once.
4. The method of claim 3, wherein the determining the B + tree corresponding to the target metadata space address to obtain a target B + tree comprises:
acquiring key value pair data values of the target tree nodes;
and determining the B + tree where the target tree node is located according to the key value pair data value of the target tree node to obtain the target B + tree.
5. The method for reclaiming metadata space according to claim 3, wherein after the organizing and managing the metadata of the full flash storage system through the data structure of the B + tree, the method further comprises:
and recording the mapping relation between each tree node on each B + tree and the corresponding metadata space address to a local disk to obtain the local preset disk.
6. The method as claimed in any one of claims 3 to 5, wherein said querying the target B + tree to determine whether the target metadata space address is a valid address according to the query result, and if not, said retrieving the target metadata space address comprises:
traversing and inquiring the tree nodes of the target B + tree according to the key value pair data values of the target tree nodes so as to determine whether the target metadata space address is an effective address according to the inquiry result;
if the target metadata space address is an invalid address, putting the target metadata space address into the preset recycling linked list;
and if the target metadata space address is an effective address, forbidding to recycle the target metadata space address.
7. The method of claim 6, wherein the performing a traversal query on the tree nodes of the target B + tree according to the key-value pair data values of the target tree nodes to determine whether the target metadata space address is a valid address according to the query result comprises:
according to the key value pair data value of the target tree node, performing traversal query operation from a root node to a leaf node on the target B + tree so as to compare whether the value of a non-leaf node of the B + tree is consistent with the target metadata space address or not in the traversal query operation process;
if the value of the non-leaf node of the B + tree is inconsistent with the target metadata space address, judging that the target metadata space address is an invalid address at present;
and if the value of the non-leaf node of the B + tree is consistent with the target metadata space address, judging that the target metadata space address is an effective address at present.
8. A metadata space reclamation apparatus, comprising:
the metadata management module is used for distributing metadata space addresses for the metadata of the full flash memory storage system and organizing and managing the metadata of the full flash memory storage system through a data structure of a B + tree;
the data block recording module is used for recording a target data block which is currently used for distributing the metadata space address if a fault node is monitored to occur in the full flash memory storage system when the metadata space address is distributed to the full flash memory storage system;
a space address determination module, configured to scan the target data block when it is monitored that the failed node is recovered, so as to determine a target metadata space address corresponding to the target data block;
a target tree determining module, configured to determine the B + tree corresponding to the target metadata space address to obtain a target B + tree;
and the space address recovery module is used for inquiring the target B + tree so as to determine whether the target metadata space address is an effective address according to an inquiry result, and if not, the target metadata space address is recovered.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the metadata space reclamation method as recited in any one of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the metadata space reclamation method as recited in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111427657.2A CN113836051B (en) | 2021-11-29 | 2021-11-29 | Metadata space recovery method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111427657.2A CN113836051B (en) | 2021-11-29 | 2021-11-29 | Metadata space recovery method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113836051A true CN113836051A (en) | 2021-12-24 |
| CN113836051B CN113836051B (en) | 2022-03-22 |
Family
ID=78971800
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111427657.2A Active CN113836051B (en) | 2021-11-29 | 2021-11-29 | Metadata space recovery method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113836051B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110055626A1 (en) * | 2008-05-15 | 2011-03-03 | The Board Of Governors For Higher Education, State Of Rhode Island And Providence Plantations | Systems and methods for recovering information from nand gates array memory systems |
| US20140229767A1 (en) * | 2011-09-29 | 2014-08-14 | Industry-University Cooperation Foundation Hanyang University | Method and apparatus for power loss recovery in a flash memory-based ssd |
| CN109542911A (en) * | 2018-12-03 | 2019-03-29 | 郑州云海信息技术有限公司 | A kind of metadata organization method, system, equipment and computer readable storage medium |
| CN110502457A (en) * | 2019-08-23 | 2019-11-26 | 北京浪潮数据技术有限公司 | A kind of metadata storing method and device |
| CN110928493A (en) * | 2019-10-31 | 2020-03-27 | 苏州浪潮智能科技有限公司 | Metadata module and metadata module processing method |
| CN111125033A (en) * | 2018-10-31 | 2020-05-08 | 深信服科技股份有限公司 | Space recovery method and system based on full flash memory array |
| CN111881064A (en) * | 2020-07-24 | 2020-11-03 | 北京浪潮数据技术有限公司 | Method, device and equipment for processing access request in full flash memory storage system |
| CN112269540A (en) * | 2020-10-16 | 2021-01-26 | 苏州浪潮智能科技有限公司 | Data storage method, system, equipment and computer readable storage medium |
-
2021
- 2021-11-29 CN CN202111427657.2A patent/CN113836051B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110055626A1 (en) * | 2008-05-15 | 2011-03-03 | The Board Of Governors For Higher Education, State Of Rhode Island And Providence Plantations | Systems and methods for recovering information from nand gates array memory systems |
| US20140229767A1 (en) * | 2011-09-29 | 2014-08-14 | Industry-University Cooperation Foundation Hanyang University | Method and apparatus for power loss recovery in a flash memory-based ssd |
| CN111125033A (en) * | 2018-10-31 | 2020-05-08 | 深信服科技股份有限公司 | Space recovery method and system based on full flash memory array |
| CN109542911A (en) * | 2018-12-03 | 2019-03-29 | 郑州云海信息技术有限公司 | A kind of metadata organization method, system, equipment and computer readable storage medium |
| CN110502457A (en) * | 2019-08-23 | 2019-11-26 | 北京浪潮数据技术有限公司 | A kind of metadata storing method and device |
| CN110928493A (en) * | 2019-10-31 | 2020-03-27 | 苏州浪潮智能科技有限公司 | Metadata module and metadata module processing method |
| CN111881064A (en) * | 2020-07-24 | 2020-11-03 | 北京浪潮数据技术有限公司 | Method, device and equipment for processing access request in full flash memory storage system |
| CN112269540A (en) * | 2020-10-16 | 2021-01-26 | 苏州浪潮智能科技有限公司 | Data storage method, system, equipment and computer readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 耿庆田，狄婧等: "基于B+树的数据索引存储", 《吉林大学学报（理学版）》 * |
| 陈游旻，朱博弘等: "一种持久性内存文件系统数据页的混合管理机制", 《计算机研究与发展》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113836051B (en) | 2022-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108319602B (en) | Database management method and database system | |
| US10013312B2 (en) | Method and system for a safe archiving of data | |
| US20100106753A1 (en) | Cyclic commit transaction protocol | |
| CN110888837B (en) | Object storage small file merging method and device | |
| US11176110B2 (en) | Data updating method and device for a distributed database system | |
| US10922234B2 (en) | Method and system for online recovery of logical-to-physical mapping table affected by noise sources in a solid state drive | |
| US10970172B2 (en) | Method to recover metadata in a content aware storage system | |
| CN111177143B (en) | Key value data storage method and device, storage medium and electronic equipment | |
| CN113515501B (en) | Nonvolatile memory database management system recovery method and device and electronic equipment | |
| CN116088770B (en) | Data management method, device, system, electronic equipment and storage medium | |
| CN114090637A (en) | Data access method, device, equipment and storage medium | |
| CN107729182B (en) | Data storage and access method and device | |
| CN113836051B (en) | Metadata space recovery method, device, equipment and storage medium | |
| CN112463795A (en) | Dynamic hash method, device, equipment and storage medium | |
| CN109240804B (en) | Method and device for managing disk resources of virtual machine | |
| CN112825098A (en) | Data protection method and device, computing equipment and storage medium | |
| CN113867628B (en) | Data deleting method, device, storage medium and computing equipment | |
| CN112100022B (en) | Method for instantly recording object distribution points when monitoring Java object memory leakage on android system | |
| CN111581157B (en) | Object storage platform, object operation method, device and server | |
| CN111444114B (en) | Method, device and system for processing data in nonvolatile memory | |
| CN108271420A (en) | Manage method, file system and the server system of file | |
| CN110704412A (en) | Metadata repairing method, device, equipment and storage medium | |
| CN108021647B (en) | File migration method, system, medium and equipment | |
| CN101833488B (en) | Processing method and equipment for snapshot resource | |
| CN115357384B (en) | Space reclamation method and device for repeated data deleting storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |