CN109189343B - Metadata disk-dropping method, device, equipment and computer-readable storage medium - Google Patents
Metadata disk-dropping method, device, equipment and computer-readable storage medium Download PDFInfo
- Publication number
- CN109189343B CN109189343B CN201811074134.2A CN201811074134A CN109189343B CN 109189343 B CN109189343 B CN 109189343B CN 201811074134 A CN201811074134 A CN 201811074134A CN 109189343 B CN109189343 B CN 109189343B
- Authority
- CN
- China
- Prior art keywords
- metadata
- tree
- data
- value
- smaller
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004590 computer program Methods 0.000 claims description 9
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
- G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a metadata disk-dropping method, which comprises the following steps: traversing data in the memory metadata tree and the original metadata tree of the disk until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata falling tree; when traversing the data in the memory metadata tree and the disk original metadata tree, executing the following operations: determining a current minimum value in a memory metadata tree as a first comparison value; determining the current minimum value in the original metadata tree of the disk as a second comparison value; determining a smaller value and a larger value of the first comparison value and the second comparison value; judging whether all data in the node with the smaller value is smaller than the larger value; and if so, inserting all data in the node into the current node of the metadata landing tree. The method and the device can effectively improve metadata dropping efficiency and reduce time consumption. The application also discloses a metadata tray dropping device, equipment and a computer readable storage medium, and the beneficial effects are also achieved.
Description
Technical Field
The present application relates to the field of storage technologies, and in particular, to a metadata destaging method, apparatus, device, and computer-readable storage medium.
Background
Metadata is a type of important data that records details of data in a computer, and is often stored in a particular data structure (e.g., various data trees) in the computer.
Metadata dropping is a common term in the storage technology field, and means that a metadata tree newly generated in a memory due to a disk I/O read-write operation is merged with an originally existing metadata tree in a disk, so as to generate an updated metadata tree and store the updated metadata tree in the disk.
In the prior art, when a metadata drop operation is performed, a minimum value in a memory metadata tree is specifically compared with a minimum value in an original metadata tree of a disk, and a smaller value of the two minimum values is inserted into a newly generated metadata tree, and then the minimum value of the memory metadata tree and the minimum value of the original metadata tree of the disk are continuously updated and obtained, and a new round of comparison and insertion is performed. Therefore, in the existing metadata disk-dropping operation, only one data in the newly generated metadata tree is written each time, so that the efficiency is low.
Therefore, what kind of metadata destaging technology is adopted so as to effectively improve metadata destaging efficiency and reduce time consumption is a technical problem to be solved urgently by those skilled in the art.
Disclosure of Invention
The application aims to provide a metadata destaging method, a metadata destaging device and a computer readable storage medium, so that metadata destaging efficiency is effectively improved, and time consumption is reduced.
In order to solve the above technical problem, the present application provides a metadata destaging method, including:
traversing data in the memory metadata tree and the original metadata tree of the disk until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata falling tree;
when traversing the data in the memory metadata tree and the original metadata tree of the disk each time, specifically executing the following operations:
determining a current minimum value in the memory metadata tree as a first comparison value;
determining the current minimum value in the original metadata tree of the disk as a second comparison value;
determining a smaller and a larger of the first and second comparison values;
judging whether all the data in the node where the smaller value is located are smaller than the larger value;
and if so, inserting all data in the nodes into the current node of the metadata landing tree.
Optionally, after the determining whether all the data in the node where the smaller value is located is smaller than the larger value, the method further includes:
and if not, inserting the smaller value into the current node of the metadata landing tree.
Alternatively,
the memory metadata tree, the original metadata tree of the disk and the metadata landing tree are all B + trees.
Optionally, before traversing the data in the memory metadata tree and the disk original metadata tree, the method further includes:
acquiring the current data volume of the memory metadata tree;
judging whether the current data volume is larger than a preset threshold value or not;
if yes, executing the step of traversing the data in the memory metadata tree and the original metadata tree of the disk;
if not, continuing to execute the step of obtaining the current data volume of the memory metadata tree.
Optionally, the determining whether all the data in the node where the smaller value is located is smaller than the larger value includes:
determining a maximum value among the nodes;
judging whether the maximum value is smaller than the larger value;
if yes, all data in the node where the smaller value is located are judged to be smaller than the larger value;
if not, judging that all the data in the node where the smaller value is located is not smaller than the larger value.
The application also provides a metadata tray falling device, which comprises a traversal module,
the traversal module is used for traversing data in the memory metadata tree and the original metadata tree of the disk until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata landing tree;
wherein, the traversal module specifically comprises:
the determining unit is used for determining that the current minimum value in the memory metadata tree is a first comparison value when data in the memory metadata tree and the original metadata tree of the disk are traversed each time; determining the current minimum value in the original metadata tree of the disk as a second comparison value; determining a smaller and a larger of the first and second comparison values;
the comparison unit is used for judging whether all data in the node where the smaller value is located are smaller than the larger value;
and the tray falling unit is used for inserting all the data in the nodes into the current nodes of the metadata tray falling tree when all the data in the nodes are smaller than the larger value.
Optionally, the method further comprises:
the acquisition module is used for acquiring the current data volume of the memory metadata tree before the traversal module traverses the data in the memory metadata tree and the original metadata tree of the disk each time;
the judging module is used for judging whether the current data volume is larger than a preset threshold value or not; if yes, the traversal module is used for traversing the data in the memory metadata tree and the original metadata tree of the disk; if not, the obtaining module is continuously used for obtaining the current data volume of the memory metadata tree.
Optionally, the comparing unit is specifically configured to:
determining a maximum value among the nodes; judging whether the maximum value is smaller than the larger value; if yes, all data in the node where the smaller value is located are judged to be smaller than the larger value; if not, judging that all the data in the node where the smaller value is located is not smaller than the larger value.
The application also provides a metadata equipment of falling a plate, includes:
a memory: for storing a computer program;
a processor: for executing the computer program to implement the steps of any of the metadata landing methods described above.
The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is configured to implement the steps of any of the metadata landing methods described above.
The metadata disk-dropping method provided by the application comprises the following steps: traversing data in the memory metadata tree and the original metadata tree of the disk until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata falling tree; when traversing the data in the memory metadata tree and the original metadata tree of the disk each time, specifically executing the following operations: determining a current minimum value in the memory metadata tree as a first comparison value; determining the current minimum value in the original metadata tree of the disk as a second comparison value; determining a smaller and a larger of the first and second comparison values; judging whether all the data in the node where the smaller value is located are smaller than the larger value; and if so, inserting all data in the nodes into the current node of the metadata landing tree.
It can be seen that, compared with the prior art, in the metadata landing method provided by the present application, after traversing the memory metadata tree and the original metadata tree of the disk to obtain the first comparison value and the second comparison value, condition judgment is further performed on the node where the smaller value of the first comparison value and the second comparison value is located, so that when data in the node is generally smaller, all data in the node is inserted into the metadata landing tree at one time, and the number of data inserted into the metadata landing tree in a single traversal operation is increased, thereby reducing the number of times of traversing the memory metadata tree and the original metadata tree of the disk, further effectively improving the traversal processing efficiency in the whole metadata landing operation, and reducing time consumption. The metadata disk-dropping device, the equipment and the computer-readable storage medium provided by the application can realize the metadata disk-dropping method, and also have the beneficial effects.
Drawings
In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings that are needed to be used in the description of the prior art and the embodiments of the present application will be briefly described below. Of course, the following description of the drawings related to the embodiments of the present application is only a part of the embodiments of the present application, and it will be obvious to those skilled in the art that other drawings can be obtained from the provided drawings without any creative effort, and the obtained other drawings also belong to the protection scope of the present application.
Fig. 1 is a flowchart of a traversal operation in a metadata landing method provided in the present application;
FIG. 2 is a flowchart illustrating a traversal operation in another metadata destaging method provided by the present application;
fig. 3 is a block diagram illustrating a structure of a metadata dropping apparatus according to the present application.
Detailed Description
The core of the application is to provide a metadata landing method, a metadata landing device, a metadata landing equipment and a computer readable storage medium, so that the metadata landing efficiency is effectively improved, and the time consumption is reduced.
In order to more clearly and completely describe the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The metadata landing method mainly comprises the steps of traversing data in a memory metadata tree and a disk original metadata tree until all data in the memory metadata tree and the disk original metadata tree are inserted into a generated metadata landing tree;
referring to fig. 1, fig. 1 is a flowchart illustrating a traversal operation in a metadata destaging method according to the present application. As shown in fig. 1, when traversing data in the memory metadata tree and the original metadata tree of the disk each time, the following operations are specifically performed:
step 11: determining a current minimum value in a memory metadata tree as a first comparison value;
step 12: determining the current minimum value in the original metadata tree of the disk as a second comparison value;
step 13: determining a smaller value and a larger value of the first comparison value and the second comparison value;
step 14: judging whether all data in the node with the smaller value is smaller than the larger value; if yes, go to step 15;
step 15: and inserting all data in the nodes into the current node of the metadata landing tree.
Specifically, the metadata is also called intermediate data or relay data, and is data describing data, that is, data of data, which is information describing attributes of the data, and is used to support functions such as indicating storage locations, history data, resource searching, file recording, and the like.
Data in any file device is divided into file actual data and metadata, and the metadata is used for describing file characteristics such as access authority, file owner, distribution information of file data blocks and the like. In particular, in a distributed storage cluster, the distribution information includes the location of the file on the disk and the location of the disk in the cluster, and a user needs to manipulate a file to first obtain its metadata to locate the file and obtain the content or related attributes of the file.
When a computer performs disk I/O operations, i.e., read/write operations, the modification of the actual data of the file will generate corresponding new metadata in the memory. The falling operation of the metadata is to merge the newly generated metadata in the memory with the original metadata in the disk, and store the merged metadata in the disk instead of the original metadata in the disk.
Metadata is stored according to a certain data structure, namely a tree structure, so that in the application, the metadata generated in a memory is called a memory metadata tree, the original metadata in a disk is called an original metadata tree of the disk, and the metadata stored in the disk after being combined is called a metadata falling tree.
There are many nodes in the tree structure, including a root node, branch nodes of each order, and leaf nodes, there may be many nodes in each order, and there may be many data stored in each node. Each tree structure is generated according to a specific node rule and the size relationship of each data, and the node rule is different in different tree structures.
Therefore, when merging the memory metadata tree and the disk original metadata tree, it is necessary to determine the size relationship between the data in the memory metadata tree and the disk original metadata tree. Therefore, when traversing the data in the memory metadata tree and the original metadata tree of the disk each time, it is necessary to determine the data that can be inserted into the current node of the metadata landing tree at this time according to the data size relationship, and perform the insertion operation.
Therefore, the metadata tray-dropping method provided by the application can specifically determine the current minimum value in the memory metadata tree when the traversal operation is performed each time, and takes the current minimum value as the first comparison value obtained by the traversal operation; and determining the current minimum value in the original metadata tree of the disk, and taking the current minimum value as a second comparison value of the traversal operation.
It should be noted that, the present application does not limit the execution sequence of step 11 and step 12, that is, a person skilled in the art may also execute the operation of step 12 first and then execute the operation of step 11, which has no influence on the core protection content of the present application.
When the first comparison value and the second comparison value are determined, the smaller value and the larger value of the first comparison value and the second comparison value can be determined, and the data which can be currently inserted into the current node of the metadata landing tree at least comprises the smaller value. Of course, the smaller value may be the first comparison value from the in-memory metadata tree or the second comparison value from the original metadata tree of the disk.
However, in order to further improve the disk dropping efficiency, the data that can be inserted into the metadata disk dropping tree can be determined as much as possible in the process of this traversal, because it takes time to traverse the memory metadata tree to obtain the first comparison value and traverse the original metadata tree of the disk to obtain the second comparison value. Therefore, if a plurality of data which can be landed can be determined according to the data size in one traversal process, the speed and efficiency of the whole metadata landing operation can be improved certainly.
Therefore, after the metadata disk-dropping method provided by the application obtains the magnitude relation between the first comparison value and the second comparison value, the magnitude analysis is further performed on the node where the smaller value of the first comparison value and the second comparison value is: if all the data in the node with the smaller value is smaller than the larger value, the node can be directly used, that is, all the data in the node is inserted into the current node of the metadata landing tree.
Specifically, since both the memory metadata tree and the original metadata tree of the disk are already generated ordered trees, if the smaller value is the first comparison value from the memory metadata tree, the size relationship between the data in the node and the data in other nodes of the memory metadata tree is already in accordance with the generation rule of the tree structure, and if all the data in the node is simultaneously smaller than the second comparison value (i.e., the current minimum value of the original metadata tree of the disk), the original metadata tree of the disk does not affect the ordering of each data in the node, so that all the data in the node can be directly inserted into the current node of the metadata falling-disk tree.
It should be noted that, the specific generation process of the metadata landing tree may be performed according to the content in the prior art, and the division into nodes may be automatically implemented according to the generation rule of the corresponding tree structure to determine the current node.
And when the data determined after the traversal is inserted into the current node of the metadata landing tree, continuing to perform the next traversal, determining a first comparison value and a second comparison value of a new round, and continuing to complete the subsequent steps until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata landing tree.
Therefore, in the metadata landing method provided by the application, after the memory metadata tree and the original metadata tree of the disk are traversed each time to obtain the first comparison value and the second comparison value, the condition judgment is further performed on the node where the smaller value of the first comparison value and the second comparison value is located, so that when the data in the node is generally smaller, all the data in the node is inserted into the metadata landing tree at one time, the number of the data inserted into the metadata landing tree in a single traversal operation is increased, the traversal times of the memory metadata tree and the original metadata tree of the disk are reduced, the processing efficiency in the whole metadata landing operation can be effectively improved, and the time consumption is reduced.
The metadata disk-dropping method provided by the application is based on the above embodiment:
referring to fig. 2, fig. 2 is a flowchart illustrating a traversal operation in another metadata landing method provided in the present application.
As shown in fig. 2, as a preferred embodiment, after determining whether all the data in the node where the smaller value is located is smaller than the larger value, the method further includes:
if not, the step 16 is entered, and the smaller value is inserted into the current node of the metadata landing tree.
Specifically, if the node where the smaller value is located does not satisfy the preset condition, that is, a part of data not smaller than the larger value exists in the node, the part of data not smaller than the larger value cannot be inserted into the current node of the metadata landing tree, so that only the smaller value may be inserted into the current node of the metadata landing tree at this time.
In a preferred embodiment, the memory metadata tree, the disk original metadata tree and the metadata landing tree are all B + trees.
Specifically, a B + Tree, or B + Tree, is a data structure commonly used in the storage field, and has higher query performance than a B Tree, a B-Tree, and the like.
As previously mentioned, the B + tree has a corresponding node rule. For m-order B + Tree, any node of the Tree with m subtrees contains m data; and, the data of all branch nodes and root nodes exist in the child node at the same time; the root node and the branch node do not store data, only are used for indexing, all the data are stored in the leaf nodes, the leaf nodes comprise pointers pointing to data records, and the leaf nodes are sequentially linked according to the sequence of the data from small to large.
As shown in fig. 2, as a preferred embodiment, before traversing the data in the memory metadata tree and the disk original metadata tree, the method further includes:
step 21: acquiring the current data volume of the memory metadata tree;
step 22: judging whether the current data volume is larger than a preset threshold value or not; if yes, entering step 11; if not, proceed to step 21.
Specifically, before performing the metadata destaging operation, the current data size of the memory metadata tree may be determined, so that the metadata destaging operation is performed when the current data size of the memory metadata tree is large enough, and if the current data size is small, the method may temporarily wait until the current data size is larger than a preset threshold. Therefore, the condition that metadata disk-dropping operation is frequently performed when the data volume is small can be effectively avoided, so that the consumption of the metadata disk-dropping operation on the performance and resources of the storage system is effectively reduced, and the service operation performance of the storage system is further ensured.
Of course, the preset threshold can be selected and set by those skilled in the art according to practical application conditions, and those skilled in the art do not limit this.
As a preferred embodiment, the step of determining whether all the data in the node where the smaller value is located is smaller than the larger value includes:
determining a maximum value among the nodes;
judging whether the maximum value is smaller than a larger value;
if yes, all data in the node where the smaller value is located are judged to be smaller than the larger value;
if not, all the data in the node with the smaller value is not smaller than the larger value.
Specifically, when comparing all the data in the node where the smaller value is located with the larger value, the maximum value in the node where the smaller value is located may be specifically utilized. If the maximum value is smaller than the larger value, it must be said that all data in the node is smaller than the larger value, otherwise, the other way around.
The metadata falling device provided by the present application is described below.
Referring to fig. 3, fig. 3 is a block diagram illustrating a metadata dropping apparatus according to the present application; comprises a traversing module 1 which is used for traversing,
the traversing module 1 is used for traversing data in the memory metadata tree and the original metadata tree of the disk until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata falling tree;
the traversing module 1 specifically comprises a determining unit 11, a comparing unit 12 and a tray falling unit 13;
the determining unit 11 is configured to determine, when traversing data in the memory metadata tree and the original metadata tree of the disk each time, that a current minimum value in the memory metadata tree is a first comparison value; determining the current minimum value in the original metadata tree of the disk as a second comparison value; determining a smaller value and a larger value of the first comparison value and the second comparison value;
the comparing unit 12 is configured to determine whether all data in the node where the smaller value is located is smaller than the larger value;
and the falling disk unit 13 is used for inserting all the data in the node into the current node of the metadata falling disk tree when all the data in the node are smaller than the larger value.
Therefore, the metadata landing device provided by the application can further perform condition judgment on the node where the smaller value of the first comparison value and the second comparison value is located after traversing the memory metadata tree and the original metadata tree of the disk to obtain the first comparison value and the second comparison value every time, so that when the data in the node is generally smaller, all the data in the node are inserted into the metadata landing tree at one time, the number of the data inserted into the metadata landing tree in a single traversal operation is increased, the traversal times of the memory metadata tree and the original metadata tree of the disk are reduced, the processing efficiency in the whole metadata landing operation can be effectively improved, and the time consumption is reduced.
The metadata device that falls that this application provided, on the basis of above-mentioned embodiment:
as a preferred embodiment, the landing tray unit 13 is further configured to:
after the comparing unit 12 determines that all the data in the node where the smaller value is located is not smaller than the larger value, the smaller value is inserted into the current node of the metadata landing tree.
In a preferred embodiment, the memory metadata tree, the disk original metadata tree and the metadata landing tree are all B + trees.
As a preferred embodiment, the method further includes an obtaining module 2 and a judging module 3:
the obtaining module 2 is used for obtaining the current data volume of the memory metadata tree before the traversing module traverses the data in the memory metadata tree and the original metadata tree of the disk each time;
the judging module 3 is used for judging whether the current data volume is larger than a preset threshold value; if yes, the traversal module 1 is used for traversing data in the memory metadata tree and the original metadata tree of the disk; if not, the obtaining module 2 continues to obtain the current data size of the memory metadata tree.
As a preferred embodiment, the comparison unit 12 is specifically configured to:
determining a maximum value among the nodes; judging whether the maximum value is smaller than a larger value; if yes, all data in the node where the smaller value is located are judged to be smaller than the larger value; if not, all the data in the node with the smaller value is not smaller than the larger value.
The application also provides a metadata equipment of falling a plate, includes:
a memory: for storing a computer program;
a processor: for executing the computer program to implement the steps of any of the metadata landing methods described above.
The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is configured to implement the steps of any of the metadata landing methods described above.
The specific embodiments of the metadata destaging device, the apparatus, and the computer-readable storage medium provided in the present application and the metadata destaging method described above may be referred to correspondingly, and are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the device and the equipment disclosed by the embodiment, the description is simple because the device and the equipment correspond to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
It is further noted that, throughout this document, relational terms such as "first" and "second" are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The technical solutions provided by the present application are described in detail above. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
Claims (9)
1. A metadata destaging method, comprising:
traversing data in the memory metadata tree and the original metadata tree of the disk until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata falling tree;
when traversing the data in the memory metadata tree and the original metadata tree of the disk each time, specifically executing the following operations:
determining a current minimum value in the memory metadata tree as a first comparison value;
determining the current minimum value in the original metadata tree of the disk as a second comparison value;
determining a smaller and a larger of the first and second comparison values;
judging whether all the data in the node where the smaller value is located are smaller than the larger value;
if yes, inserting all data in the nodes into the current nodes of the metadata landing tree;
and if not, inserting the smaller value into the current node of the metadata landing tree.
2. The metadata destaging method according to claim 1,
the memory metadata tree, the original metadata tree of the disk and the metadata landing tree are all B + trees.
3. The metadata landing method according to claim 1, further comprising, before traversing the data in the memory metadata tree and the disk original metadata tree:
acquiring the current data volume of the memory metadata tree;
judging whether the current data volume is larger than a preset threshold value or not;
if yes, executing the step of traversing the data in the memory metadata tree and the original metadata tree of the disk;
if not, continuing to execute the step of obtaining the current data volume of the memory metadata tree.
4. The metadata landing method according to any one of claims 1 to 3, wherein the determining whether all the data in the node where the smaller value is located is smaller than the larger value comprises:
determining a maximum value among the nodes;
judging whether the maximum value is smaller than the larger value;
if yes, all data in the node where the smaller value is located are judged to be smaller than the larger value;
if not, judging that all the data in the node where the smaller value is located is not smaller than the larger value.
5. The metadata dropping device is characterized by comprising a traversing module,
the traversal module is used for traversing data in the memory metadata tree and the original metadata tree of the disk until all data in the memory metadata tree and the original metadata tree of the disk are inserted into the generated metadata landing tree;
wherein, the traversal module specifically comprises:
the determining unit is used for determining that the current minimum value in the memory metadata tree is a first comparison value when data in the memory metadata tree and the original metadata tree of the disk are traversed each time; determining the current minimum value in the original metadata tree of the disk as a second comparison value; determining a smaller and a larger of the first and second comparison values;
the comparison unit is used for judging whether all data in the node where the smaller value is located are smaller than the larger value;
a step of dropping the data into the current node of the metadata dropping tree when all the data in the node is smaller than the larger value; and when all data in the nodes are not smaller than the large value, inserting the small value into the current node of the metadata landing tree.
6. The metadata landing device of claim 5, further comprising:
the acquisition module is used for acquiring the current data volume of the memory metadata tree before the traversal module traverses the data in the memory metadata tree and the original metadata tree of the disk each time;
the judging module is used for judging whether the current data volume is larger than a preset threshold value or not; if yes, the traversal module is used for traversing the data in the memory metadata tree and the original metadata tree of the disk; if not, the obtaining module is continuously used for obtaining the current data volume of the memory metadata tree.
7. The metadata landing device according to claim 5, wherein the comparing unit is specifically configured to:
determining a maximum value among the nodes; judging whether the maximum value is smaller than the larger value; if yes, all data in the node where the smaller value is located are judged to be smaller than the larger value; if not, judging that all the data in the node where the smaller value is located is not smaller than the larger value.
8. A metadata dropoff device, comprising:
a memory: for storing a computer program;
a processor: steps for executing the computer program to implement a metadata landing method as claimed in any one of claims 1 to 4.
9. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, is configured to carry out the steps of the metadata landing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811074134.2A CN109189343B (en) | 2018-09-14 | 2018-09-14 | Metadata disk-dropping method, device, equipment and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811074134.2A CN109189343B (en) | 2018-09-14 | 2018-09-14 | Metadata disk-dropping method, device, equipment and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109189343A CN109189343A (en) | 2019-01-11 |
| CN109189343B true CN109189343B (en) | 2021-08-10 |
Family
ID=64911125
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811074134.2A Active CN109189343B (en) | 2018-09-14 | 2018-09-14 | Metadata disk-dropping method, device, equipment and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109189343B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110147204B (en) * | 2019-05-22 | 2020-03-10 | 苏州浪潮智能科技有限公司 | Metadata disk-dropping method, device and system and computer-readable storage medium |
| CN113901276A (en) * | 2021-09-30 | 2022-01-07 | 苏州浪潮智能科技有限公司 | Data management method, B + tree loading method, device and electronic equipment |
| CN114297172B (en) * | 2022-01-04 | 2022-07-12 | 北京乐讯科技有限公司 | Cloud-native-based distributed file system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521627A (en) * | 2009-04-13 | 2009-09-02 | 华为技术有限公司 | Method and device for inserting nodes |
| CN108140024A (en) * | 2015-07-07 | 2018-06-08 | 华为技术有限公司 | Merge index structure in MOLAP and keep the mechanism of inquiry consistency |
| CN108140049A (en) * | 2015-10-20 | 2018-06-08 | 国际商业机器公司 | The parallel batch processing of data structure based on tree |
-
2018
- 2018-09-14 CN CN201811074134.2A patent/CN109189343B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521627A (en) * | 2009-04-13 | 2009-09-02 | 华为技术有限公司 | Method and device for inserting nodes |
| CN108140024A (en) * | 2015-07-07 | 2018-06-08 | 华为技术有限公司 | Merge index structure in MOLAP and keep the mechanism of inquiry consistency |
| CN108140049A (en) * | 2015-10-20 | 2018-06-08 | 国际商业机器公司 | The parallel batch processing of data structure based on tree |
Non-Patent Citations (4)
| Title |
|---|
| B-树/B+树的批量插入算法;魏小亮，等.;《中央民族大学学报(自然科学版)》;20010131;第10卷(第1期);第57-62页 * |
| 一种大数据流内存B+树构建方法;杨良怀，等.;《计算机科学》;20180331;第45卷(第3期);第173-181页 * |
| 基于B+树批量加入算法的空间数据库索引优化策略;蒋华.;《计算机科学》;20051231;第32卷(第7期);第328-330页 * |
| 数据流实时存储关键技术;陈彬.;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160415(第04期);第I138-901页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109189343A (en) | 2019-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2199935A2 (en) | Method and system for dynamically partitioning very large database indices on write-once tables | |
| CN107729558B (en) | Method, system and device for defragmenting file system and computer storage medium | |
| CN109189343B (en) | Metadata disk-dropping method, device, equipment and computer-readable storage medium | |
| CN103020204A (en) | Method and system for carrying out multi-dimensional regional inquiry on distribution type sequence table | |
| CN105320775A (en) | Data access method and apparatus | |
| CN106970958B (en) | A kind of inquiry of stream file and storage method and device | |
| CN102999519A (en) | Read-write method and system for database | |
| KR101744892B1 (en) | System and method for data searching using time series tier indexing | |
| CN109240607B (en) | File reading method and device | |
| US20240061712A1 (en) | Method, apparatus, and system for creating training task on ai training platform, and medium | |
| JP5790755B2 (en) | Database management apparatus and database management method | |
| CN112000971B (en) | File permission recording method, system and related device | |
| WO2024078122A1 (en) | Database table scanning method and apparatus, and device | |
| CN111552438B (en) | Method, device, server and storage medium for writing object | |
| CN107315806B (en) | Embedded storage method and device based on file system | |
| CN106648550B (en) | Method and device for concurrently executing tasks | |
| KR102354343B1 (en) | Spatial indexing method and apparatus for blockchain-based geospatial data | |
| CN108984720B (en) | Data query method and device based on column storage, server and storage medium | |
| CN110990394B (en) | Method, device and storage medium for counting number of rows of distributed column database table | |
| JP5953262B2 (en) | DATA INDEX DEVICE, DATA INDEX METHOD, AND PROGRAM | |
| CN116610729B (en) | Database intelligent statistical information management method, system, equipment and medium | |
| CN115809268B (en) | Adaptive query method and device based on fragment index | |
| CN111459411B (en) | Data migration method, device, equipment and storage medium | |
| CN109388613B (en) | File management method and electronic equipment | |
| CN114385584A (en) | Data writing method and data reading method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |