CN110659249A - Metadata subtree migration method, device and equipment and readable storage medium - Google Patents
Metadata subtree migration method, device and equipment and readable storage medium Download PDFInfo
- Publication number
- CN110659249A CN110659249A CN201910911892.3A CN201910911892A CN110659249A CN 110659249 A CN110659249 A CN 110659249A CN 201910911892 A CN201910911892 A CN 201910911892A CN 110659249 A CN110659249 A CN 110659249A
- Authority
- CN
- China
- Prior art keywords
- metadata
- subtree
- subtrees
- target
- cold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000013508 migration Methods 0.000 title claims abstract description 49
- 230000005012 migration Effects 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012544 monitoring process Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 10
- 230000007704 transition Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000004140 cleaning Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/119—Details of migration of file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Abstract
The invention discloses a metadata subtree migration method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: monitoring the cold and hot states of the metadata subtrees and acquiring the total number of the metadata subtrees; judging whether the total quantity is greater than a preset back-transition threshold value; if so, determining a target metadata subtree to be migrated back by using the cold and hot states, and migrating the target metadata subtree back. According to the method, when the total number of the metadata subtrees reaches the preset backover threshold value, the backover operation is carried out, so that the total number of the metadata subtrees is limited within a certain number range, extra calculation caused by excessive number of the metadata subtrees can be avoided, and influence on service performance caused by excessive number of the metadata subtrees can be avoided.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a metadata subtree migration method, apparatus, device, and readable storage medium.
Background
Distributed cluster storage is widely applied under various scenes (such as high performance, video monitoring, broadcast and television media resources and the like) of current big data. Massive data exists in the distributed file system, and the premise of big data calculation is to organize and store the massive data. A plurality of metadata servers form a metadata cluster for providing external services, and the metadata performance can be improved.
In a metadata server, metadata subtrees (subtrees) are divided into metadata clusters to provide a service concurrence precondition, different directories can have different subtrees, and different Subtree services can be simultaneously concurrent, so that the metadata performance is improved. However, too many subtrees also result in extra computations. Therefore, from another perspective, the number of subtrees needs to be limited to prevent the subtrees from being too large, resulting in limited service performance.
In summary, how to effectively solve the problems of controlling the number of metadata subtrees and the like is a technical problem that needs to be solved urgently by those skilled in the art at present.
Disclosure of Invention
The invention aims to provide a metadata subtree migration method, a metadata subtree migration device, metadata subtree migration equipment and a readable storage medium.
In order to solve the technical problems, the invention provides the following technical scheme:
a metadata subtree migration method, comprising:
monitoring the cold and hot states of the metadata subtrees and acquiring the total number of the metadata subtrees;
judging whether the total number is greater than a preset back-transition threshold value;
if so, determining a target metadata subtree to be migrated back by using the cold and hot states, and migrating the target metadata subtree back.
Preferably, the monitoring of the cold and hot states of the metadata subtree comprises:
recording the access times of files and/or directories under the metadata sub-tree directory;
and determining the heat value of the metadata subtree by using the access times.
Preferably, determining the heat value of the metadata sub-tree using the number of accesses comprises:
combining a heat attenuation rule and calculating the heat value by using the access times;
correspondingly, the step of determining the target metadata subtree to be migrated back by using the cold and hot states comprises the following steps: and determining the metadata subtree with the heat value smaller than a preset heat threshold value as the target metadata subtree.
Preferably, the method further comprises the following steps:
counting the quantity of metadata in the metadata sub-tree directory;
and determining the metadata subtree with the metadata number of 0 as the target metadata subtree.
Preferably, after migrating the target metadata subtree, the method includes:
and recording subtree information corresponding to the metadata subtrees which are not migrated back.
Preferably, the method further comprises the following steps:
and when the metadata is played back, playing back the metadata information corresponding to the metadata subtree which is not returned.
Preferably, after migrating the target metadata subtree, the method further comprises:
and clearing the subtree log corresponding to the target metadata subtree.
A metadata subtree migration apparatus, comprising:
the metadata subtree monitoring module is used for monitoring the cold and hot states of the metadata subtrees and acquiring the total number of the metadata subtrees;
the transition judging module is used for judging whether the total number is greater than a preset transition threshold value;
and the backtracking module is used for determining a target metadata subtree to be backtracked by utilizing the cold and hot states and backtracking the target metadata subtree when the total data volume is greater than the preset backtracking threshold value.
A metadata subtree migration apparatus, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the metadata subtree migration method when the computer program is executed.
A readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the above-mentioned metadata sub-tree migration method.
By applying the method provided by the embodiment of the invention, the cold and hot states of the metadata subtrees are monitored, and the total number of the metadata subtrees is obtained; judging whether the total quantity is greater than a preset back-transition threshold value; if so, determining a target metadata subtree to be migrated back by using the cold and hot states, and migrating the target metadata subtree back. And monitoring the cold and hot states of the metadata subtrees, and determining the target metadata subtrees to be migrated back based on the cold and hot states when determining that the total number of the metadata subtrees reaches a preset backout threshold. Then, the target metadata subtree is migrated back. Therefore, by performing the rollback operation when the total number of the metadata subtrees reaches the preset rollback threshold, the total number of the metadata subtrees can be limited within a certain number range, extra calculation caused by the excessive number of the metadata subtrees can be avoided, and the influence on the service performance caused by the excessive number of the metadata subtrees can be avoided. In addition, when the target metadata subtree is determined, screening is carried out based on the cold and hot states of the metadata subtree, and the influence on service performance caused by the return of the metadata subtree can be avoided.
Accordingly, embodiments of the present invention further provide a metadata subtree migration apparatus, a device, and a readable storage medium corresponding to the metadata subtree migration method, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating an implementation of a metadata subtree migration method according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a metadata subtree migration apparatus according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a metadata subtree migration apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a metadata subtree migration apparatus according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
referring to fig. 1, fig. 1 is a flowchart of a metadata subtree migration method in an embodiment of the present invention, which can be applied to a metadata Server (MDS) in a distributed file system, and the method includes the following steps:
s101, monitoring the cold and hot states of the metadata subtrees, and acquiring the total number of the metadata subtrees.
In the embodiment of the present invention, in order to limit the total number of metadata subtrees, the total number of metadata subtrees can be acquired in real time or periodically.
Considering that when a subtree is migrated back, if a metadata subtree with a higher access heat is migrated back, the remaining metadata subtree with a lower access heat may adversely affect service performance. Therefore, in this embodiment, the cold and hot states of the metadata subtree also need to be monitored.
Specifically, the cold and hot states of the metadata subtree can be divided into two states, i.e., a cold state and a hot state, or the cold and hot states of the metadata subtree can be represented by a hot value. When the cold state and the hot state are distinguished, when the metadata subtree is migrated back, the metadata subtree in the cold state can be used as a target metadata subtree for the purpose of migration back. When the hot and cold states of the metadata subtrees are determined by representing the heat values, the metadata subtrees with smaller heat values can be determined as target metadata subtrees according to a judgment threshold value.
The cold and hot states of the metadata subtree are the access conditions of the directory or the file in the metadata subtree directory. It is considered that the more the number of accesses, the higher the heat. Thus, in this embodiment, monitoring the cold and hot status of a metadata sub-tree includes:
recording the access times of files and/or directories under a metadata sub-tree directory;
and step two, determining the heat value of the metadata subtree by using the access times.
For convenience of description, the above two steps will be described in combination.
In this embodiment, the heat value corresponding to each metadata sub-tree may be calculated.
Specifically, when only a file exists in the directory of the metadata subtree A, the access times corresponding to the file in the directory are counted; when only a directory exists under the directory of the metadata subtree B, counting the access times of the directory under the directory; and when the directory of the metadata subtree C has both the directory and the file, accumulating and counting the access times corresponding to the file and the directory under the directory respectively.
After the access times are obtained, a heat value of the metadata sub-tree may be determined based on the access times. Specifically, one visit may be defined, and 1 is added to the heat value, that is, how many times the visit is made, and what the heat value is; it may also be defined that ten accesses are not accumulated, increasing the heat value by 1.
Preferably, it is considered that past distant accesses have less significance to the statistical metadata subtree and are inaccurate. Therefore, in this embodiment, it is also possible to count only the number of accesses within the current period of time. The current period of time may be the last hour, the last day, the last week, and the like. The specific time length can be determined according to the approximate change rule of the metadata access heat. For example, when the access heat degree changes more frequently, the number of accesses in a longer period of time may be counted, and when the change in the access heat degree is more stable, the number of accesses in a shorter period of time may be counted.
Preferably, as the needs of the service and the user may change with time, that is, a certain metadata sub-tree with higher original popularity may appear, and the access is less and less; of course, a certain metadata sub-tree with low heat originally occurs, and the access is less and less. At this time, if the cold and hot states are determined based on the access times, the service performance may be affected after the metadata subtree is migrated back. Based on this, in the present embodiment, it is proposed to calculate the heat value in conjunction with the number of visits in conjunction with the heat decay rule. The heat value can be calculated by combining the heat attenuation rule and utilizing the access times. Specifically, the access heat of each subtree is calculated as follows: accessing a certain file or directory under the subtree directory once, and adding 1 to the heat; the heat decay equation is: new (val) et*ln(0.5)/5Wherein, val is the original heat value, new is the heat value after decaying along with the time, that is, the heat value gradually decays according to the e exponential function curve along with the increasing of the time t. Of course, in other embodiments of the present invention, the heat decay rule (i.e., the heat decay equation may also gradually decay according to other curves).
And S102, judging whether the total quantity is greater than a preset back-transition threshold value.
In the embodiment of the present invention, a limit number of a metadata subtree may be preset, and the limit number is a preset back-migration threshold. The specific value of the preset migration threshold value can be set according to the specific performance of a distributed file system with well strips, and in a system with better specific performance, the preset migration threshold value can be set to be relatively larger; in a system with poor performance, the threshold value of the predetermined back-transition is relatively small. For example, the preset rollback threshold may be set to 1000, i.e., the total number of metadata subtrees is limited to 1000 or less, although in other embodiments of the present invention, the preset rollback threshold may be set to other values, such as 900. That is, the specific value of the preset transition threshold is not limited in this embodiment.
And comparing the total number of the obtained metadata subtrees with a preset backtracking threshold value to determine a judgment result. After the determination of the result, the subsequent execution steps can be selected according to the specific result. Specifically, if the determination result is yes, that is, if the number of the current metadata subtrees is too large, which may affect the service performance, the step S103 is performed; if the determination result is negative, that is, the number of the current metadata subtrees does not affect the service performance, step S104 may be entered, i.e., no operation may be performed.
S103, determining a target metadata sub-tree to be migrated back by using the cold and hot states, and migrating the target metadata sub-tree back.
And service performance is limited in order to avoid the metadata subtrees with higher access heat from being migrated back. Thus, in determining a target metadata sub-tree, a metadata sub-tree of a colder of the corresponding cold-hot state may be determined as the target metadata sub-tree. Specifically, the metadata subtree with the heat value smaller than the preset heat threshold value may be determined as the target metadata subtree. The preset access heat threshold may be set according to the actual application, for example, the preset access heat threshold may be specifically set to 10.
In an embodiment of the present invention, the way to control the total number of metadata subtrees is to migrate back instead of delete. That is, after the target metadata sub-tree is migrated back, the files or directories under the directory of the target metadata sub-tree are not deleted, but merged upward to the parent node or root node of the directory metadata sub-tree, so that the files or directories under the directory of the target metadata sub-tree are still accessible after the target metadata sub-tree is migrated back.
Preferably, in practical applications, after the aging process is processed, the metadata in the metadata sub-tree is empty. In order to avoid the influence of the metadata subtree without data on normal business service, the metadata subtree without metadata can be integrated. The integration may be performed by determining a metadata sub-tree without metadata as the target metadata sub-tree. The specific implementation process comprises the following steps:
step one, counting the quantity of metadata in a metadata sub-tree directory;
and step two, determining the metadata subtree with the metadata quantity of 0 as a target metadata subtree.
A traversal query may be performed on each metadata sub-tree directory to determine the amount of metadata under each metadata sub-tree directory. Then, a metadata sub-tree whose metadata number is 0 is determined as a target metadata sub-tree. Therefore, when the target metadata subtree is migrated back, the metadata subtree without metadata can be migrated back, and the metadata subtree without metadata in the distributed file system can be cleaned.
Preferably, in order to increase the speed of the disk dropping of the metadata log event, after the target metadata subtree is migrated back, subtree information corresponding to the metadata subtree that is not migrated back may be recorded. The metadata of the target metadata sub-tree is migrated back to its following node, and thus the target metadata sub-tree may be considered to be absent when recording the sub-tree information.
Preferably, in order to increase the playback speed of the metadata server, the metadata information corresponding to the metadata subtree which is not migrated back is played back when the playback is performed. The metadata of the target metadata sub-tree is migrated back to the following node, so that the same metadata information does not need to be played back repeatedly when the metadata information is played back, and the playback speed of the metadata server can be improved.
Preferably, in order to increase the metadata playback speed, after the target metadata subtree is migrated back, the subtree log corresponding to the target metadata subtree may also be cleared. Thus, when the metadata is played back, the amount of data played back can be reduced.
By applying the method provided by the embodiment of the invention, the cold and hot states of the metadata subtrees are monitored, and the total number of the metadata subtrees is obtained; judging whether the total quantity is greater than a preset back-transition threshold value; if so, determining a target metadata subtree to be migrated back by using the cold and hot states, and migrating the target metadata subtree back. And monitoring the cold and hot states of the metadata subtrees, and determining the target metadata subtrees to be migrated back based on the cold and hot states when determining that the total number of the metadata subtrees reaches a preset backout threshold. Then, the target metadata subtree is migrated back. Therefore, by performing the rollback operation when the total number of the metadata subtrees reaches the preset rollback threshold, the total number of the metadata subtrees can be limited within a certain number range, extra calculation caused by the excessive number of the metadata subtrees can be avoided, and the influence on the service performance caused by the excessive number of the metadata subtrees can be avoided. In addition, when the target metadata subtree is determined, screening is carried out based on the cold and hot states of the metadata subtree, and the influence on service performance caused by the return of the metadata subtree can be avoided.
Example two:
corresponding to the above method embodiment, an embodiment of the present invention further provides a metadata subtree migration apparatus, which may be applied to a metadata server in a distributed file system, where the metadata subtree migration apparatus described below and the metadata subtree migration method described above may be referred to in a corresponding manner.
Referring to fig. 2, the apparatus includes the following modules:
a metadata sub-tree monitoring module 101, configured to monitor a cold and hot state of a metadata sub-tree and obtain a total number of the metadata sub-trees;
a transition judging module 102, configured to judge whether the total number is greater than a preset transition threshold;
and the migration module 103 is configured to determine a target metadata sub-tree to be migrated back by using a cold and hot state when the total data amount is greater than a preset migration threshold, and migrate the target metadata sub-tree back.
By applying the device provided by the embodiment of the invention, the cold and hot states of the metadata subtrees are monitored, and the total number of the metadata subtrees is obtained; judging whether the total quantity is greater than a preset back-transition threshold value; if so, determining a target metadata subtree to be migrated back by using the cold and hot states, and migrating the target metadata subtree back. And monitoring the cold and hot states of the metadata subtrees, and determining the target metadata subtrees to be migrated back based on the cold and hot states when determining that the total number of the metadata subtrees reaches a preset backout threshold. Then, the target metadata subtree is migrated back. Therefore, by performing the rollback operation when the total number of the metadata subtrees reaches the preset rollback threshold, the total number of the metadata subtrees can be limited within a certain number range, extra calculation caused by the excessive number of the metadata subtrees can be avoided, and the influence on the service performance caused by the excessive number of the metadata subtrees can be avoided. In addition, when the target metadata subtree is determined, screening is carried out based on the cold and hot states of the metadata subtree, and the influence on service performance caused by the return of the metadata subtree can be avoided.
In a specific embodiment of the present invention, the metadata sub-tree monitoring module 101 is specifically configured to record the number of times of accessing files and/or directories under a metadata sub-tree directory; and determining the heat value of the metadata subtree by using the access times.
In an embodiment of the present invention, the metadata sub-tree monitoring module 101 is specifically configured to calculate a heat value by using access times in combination with a heat decay rule;
accordingly, the migration module 103 is specifically configured to determine a metadata sub-tree with a heat value smaller than a preset heat threshold as a target metadata sub-tree.
In one embodiment of the present invention, the method further comprises:
the metadata sub-tree cleaning module is used for counting the number of metadata in a metadata sub-tree directory; a metadata sub-tree having a metadata number of 0 is determined as a target metadata sub-tree.
In a specific embodiment of the present invention, the sub-tree information recording module is configured to record sub-tree information corresponding to a non-migrated metadata sub-tree after the target metadata sub-tree is migrated back.
In one embodiment of the present invention, the method further comprises:
and the playback processing module is used for playing back the metadata information corresponding to the metadata subtree which is not returned when the playback is carried out.
In one embodiment of the present invention, the method further comprises:
and the subtree log cleaning module is used for cleaning the subtree log corresponding to the target metadata subtree after the target metadata subtree is migrated back.
Example three:
corresponding to the above method embodiment, an embodiment of the present invention further provides a metadata subtree migration apparatus, and a metadata subtree migration apparatus described below and a metadata subtree migration method described above may be referred to in a corresponding manner.
Referring to fig. 3, the metadata subtree migration apparatus includes:
a memory D1 for storing computer programs;
a processor D2 for implementing the steps of the metadata subtree migration method of the above method embodiment when executing the computer program.
Specifically, referring to fig. 4, fig. 4 is a schematic diagram illustrating a specific structure of a metadata subtree migration apparatus provided in this embodiment, which may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 322 (e.g., one or more processors) and a memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) storing an application 342 or data 344. Memory 332 and storage media 330 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 330 may include one or more modules (not shown), each of which may include a series of instructions operating on a data processing device. Still further, the central processor 322 may be configured to communicate with the storage medium 330 to perform a series of instruction operations in the storage medium 330 on the metadata sub-tree migration apparatus 301.
The metadata subtree migration apparatus 301 may also include one or more power sources 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341. Such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
The steps in the metadata subtree migration method described above may be implemented by the structure of the metadata subtree migration apparatus.
Example four:
corresponding to the above method embodiment, an embodiment of the present invention further provides a readable storage medium, and a readable storage medium described below and a metadata subtree migration method described above may be referred to correspondingly.
A readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the metadata subtree migration method of the above-mentioned method embodiment.
The readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other readable storage media capable of storing program codes.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Claims (10)
1. A metadata subtree migration method, comprising:
monitoring the cold and hot states of the metadata subtrees and acquiring the total number of the metadata subtrees;
judging whether the total number is greater than a preset back-transition threshold value;
if so, determining a target metadata subtree to be migrated back by using the cold and hot states, and migrating the target metadata subtree back.
2. The method of claim 1, wherein monitoring the cold and hot status of the metadata subtree comprises:
recording the access times of files and/or directories under the metadata sub-tree directory;
and determining the heat value of the metadata subtree by using the access times.
3. The method of claim 2, wherein determining the heat value of the metadata sub-tree using the number of accesses comprises:
combining a heat attenuation rule and calculating the heat value by using the access times;
correspondingly, the step of determining the target metadata subtree to be migrated back by using the cold and hot states comprises the following steps: and determining the metadata subtree with the heat value smaller than a preset heat threshold value as the target metadata subtree.
4. The method of migrating a metadata subtree of claim 1, further comprising:
counting the quantity of metadata in the metadata sub-tree directory;
and determining the metadata subtree with the metadata number of 0 as the target metadata subtree.
5. The metadata subtree migration method according to claim 1, wherein after migrating the target metadata subtree back, comprising:
and recording subtree information corresponding to the metadata subtrees which are not migrated back.
6. The metadata subtree migration method according to claim 5, further comprising:
and when the metadata is played back, playing back the metadata information corresponding to the metadata subtree which is not returned.
7. The metadata subtree migration method according to any of claims 1 to 6, further comprising, after migrating the target metadata subtree back:
and clearing the subtree log corresponding to the target metadata subtree.
8. A metadata subtree migration apparatus, comprising:
the metadata subtree monitoring module is used for monitoring the cold and hot states of the metadata subtrees and acquiring the total number of the metadata subtrees;
the transition judging module is used for judging whether the total number is greater than a preset transition threshold value;
and the backtracking module is used for determining a target metadata subtree to be backtracked by utilizing the cold and hot states and backtracking the target metadata subtree when the total data volume is greater than the preset backtracking threshold value.
9. A metadata subtree migration apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the metadata subtree migration method according to any one of claims 1 to 7 when executing said computer program.
10. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the metadata sub-tree migration method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910911892.3A CN110659249A (en) | 2019-09-25 | 2019-09-25 | Metadata subtree migration method, device and equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910911892.3A CN110659249A (en) | 2019-09-25 | 2019-09-25 | Metadata subtree migration method, device and equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110659249A true CN110659249A (en) | 2020-01-07 |
Family
ID=69039126
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910911892.3A Withdrawn CN110659249A (en) | 2019-09-25 | 2019-09-25 | Metadata subtree migration method, device and equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110659249A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446197A (en) * | 2016-09-30 | 2017-02-22 | 华为数字技术（成都）有限公司 | Data storage method, device and system |
| CN107798104A (en) * | 2017-10-31 | 2018-03-13 | 郑州云海信息技术有限公司 | A kind of catalog management method, device, equipment and computer-readable recording medium |
| CN107943867A (en) * | 2017-11-10 | 2018-04-20 | 中国电子科技集团公司第三十二研究所 | High-performance hierarchical storage system supporting heterogeneous storage |
-
2019
- 2019-09-25 CN CN201910911892.3A patent/CN110659249A/en not_active Withdrawn
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446197A (en) * | 2016-09-30 | 2017-02-22 | 华为数字技术（成都）有限公司 | Data storage method, device and system |
| CN107798104A (en) * | 2017-10-31 | 2018-03-13 | 郑州云海信息技术有限公司 | A kind of catalog management method, device, equipment and computer-readable recording medium |
| CN107943867A (en) * | 2017-11-10 | 2018-04-20 | 中国电子科技集团公司第三十二研究所 | High-performance hierarchical storage system supporting heterogeneous storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10140050B2 (en) | Providing access information to a storage controller to determine a storage tier for storing data | |
| US8745014B2 (en) | Time series data mapping into a key-value database | |
| US6681309B2 (en) | Method and apparatus for measuring and optimizing spatial segmentation of electronic storage workloads | |
| CN107122126B (en) | Data migration method, device and system | |
| US9378305B1 (en) | Selecting pages implementing leaf nodes and internal nodes of a data set index for reuse | |
| US11093471B2 (en) | Large range lookups for Bϵ-tree | |
| Puttaswamy et al. | Frugal storage for cloud file systems | |
| US11048678B2 (en) | Bulk-load for B-trees | |
| US10831371B2 (en) | Quota controlled movement of data in a tiered storage system | |
| US20210303537A1 (en) | Log record identification using aggregated log indexes | |
| CN111857592A (en) | Data storage method and device based on object storage system and electronic equipment | |
| US7113936B1 (en) | Optimizer improved statistics collection | |
| CN113177050A (en) | Data balancing method, device, query system and storage medium | |
| CN110175070B (en) | Distributed database management method, device, system, medium and electronic equipment | |
| CN112988066A (en) | Data processing method and device | |
| CN109189343B (en) | Metadata disk-dropping method, device, equipment and computer-readable storage medium | |
| CN109388550B (en) | Cache hit rate determination method, device, equipment and readable storage medium | |
| CN109947730A (en) | Metadata restoration methods, device, distributed file system and readable storage medium storing program for executing | |
| CN110659249A (en) | Metadata subtree migration method, device and equipment and readable storage medium | |
| CN115993932A (en) | Data processing method, device, storage medium and electronic equipment | |
| CN114564457B (en) | Storage space optimization method and system for database files | |
| CN110708361A (en) | System, method and device for determining grade of digital content publishing user and server | |
| US9354820B2 (en) | VSAM data set tier management | |
| CN107819804B (en) | Cloud storage device system and method for determining data in cache of cloud storage device system | |
| CN111352590A (en) | File storage method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200107 |
|
| WW01 | Invention patent application withdrawn after publication |