WO2014082203A1

WO2014082203A1 - Metadata management method and device

Info

Publication number: WO2014082203A1
Application number: PCT/CN2012/085344
Authority: WO
Inventors: 过晓春
Original assignee: 华为技术有限公司
Priority date: 2012-11-27
Filing date: 2012-11-27
Publication date: 2014-06-05
Also published as: CN103688257B; CN103688257A

Abstract

Provided are a metadata management method and device. The method comprises: acquiring load information to be migrated; looking up directory attributes of directories in metadata, selecting a directory with the directory attributes thereof containing load information matching the load information to be migrated and having a subtree flag as a target directory, and determining all subtrees with the target directory as a root directory as target subtrees to be migrated, the subtree flag being preset in the directory attributes of the root directory identified by the subtree flag; and migrating the target subtrees entirely. The present invention improves the metadata management efficiency of a file system.

Description

TECHNICAL FIELD The present invention relates to storage technologies, and in particular, to a metadata management method and apparatus. Background technique

The cluster file system is a system for uniformly managing files of multiple machines included in the cluster. The file system provides a unified storage space for multiple machines, which is called a namespace. The namespace is used to store each machine in the cluster. The storage path of the file; for example, the storage path of the file a is C/aa/c/a (where C/aa/c may be referred to as a directory); the metadata includes the information for indicating the file storage path. The namespace is divided into multiple shares, each of which is assigned to a machine in the cluster for storing metadata in the corresponding machine; assuming that the space in the C directory of a machine stores two file paths, respectively C/aa/c/a and C/bb/d, C/aa/c/a can be called a subtree, C/bb/d is also called a subtree, and aa/c/a can also be called As a subtree, the subtree is actually metadata, just the image title set according to the hierarchical connection between the directories or files in the metadata.

During the running of the cluster, the file will be accessed by the user's application. The file access amount can be represented by the load. The larger the file access volume of a certain machine, the higher the load of the machine. In order to balance the load of each machine in the cluster, the prior art will migrate part of the load of the machine to another machine (actually file migration) when the load of one machine is heavy; correspondingly, The metadata of the machine is also updated as the load moves, and the subtree corresponding to the migration load is also migrated to another machine, that is, the corresponding metadata of the load migration also migrates, and the distribution of the metadata changes.

However, in the load migration and subtree migration for load balancing, the distribution of metadata in the cluster is scattered. It is possible that the user is supposed to migrate a file from one directory to another. When the two directories are located on different machines, the distribution management of the metadata requires cross-machine operation, that is, the metadata is sent from one machine to another, causing the file system to perform files in the user. The efficiency of metadata distribution management corresponding to the operation is low, which affects the access performance of the cluster to user applications. Summary of the invention The present invention provides a metadata management method and apparatus for improving metadata management efficiency of a file system.

In a first aspect, a metadata management method is provided, including:

Get the load information that will be migrated;

Finding a directory attribute of a directory in the metadata, selecting a directory attribute including load information matching the load information to be migrated and having a subtree flag as a target directory, and determining all the root directory of the target directory a subtree as a target subtree to be migrated; the subtree flag is preset in a directory attribute of a root directory identified by the subtree flag;

The target subtree is migrated as a whole.

With reference to the first aspect, in a first possible implementation, the subtree of the root directory of the subtree identifier identifier includes metadata corresponding to a smallest unit of the service access operation range.

With reference to the first aspect, in a second possible implementation, the subtree of the root directory of the subtree identifier is included in the sub-tree whose access frequency reaches a preset threshold within a preset time period.

In combination with the first aspect, in a third possible implementation, before searching for the directory attribute of the directory in the metadata, the method further includes: selecting, having the same in at least two subtrees of the directory attribute of the root directory having the subtree flag The plurality of subtrees of the load change trend are used as a concurrent group, and the concurrent group flag is set in the directory attribute of the root directory of each subtree in the concurrent group; the selected directory attribute includes the load information to be migrated The matching load information and the directory having the subtree flag are the target directory, including: the selected directory attribute includes not only the load information but also a subtree flag, and a directory including the concurrent group flag as the target directory.

With reference to any one of the first possible implementation of the first aspect to the third possible implementation, in the fourth possible implementation, the selecting the directory attribute includes not only the load information, and a subtree flag, including the directory of the concurrent group flag, as the target directory, including: from a concurrent group including the largest number of subtrees, selecting a directory attribute includes not only the load information, and a subtree flag, but also includes The directory of the concurrent group flag is used as the target directory.

In conjunction with the third possible implementation of the first aspect, in a fifth possible implementation, the directory attribute is an attribute that is received by a user through an application programming interface API, and the attribute includes the sub-tree flag and Concurrent group logo.

In a second aspect, a metadata management apparatus is provided, including:

a load determining unit, configured to acquire load information to be migrated; a subtree searching unit, configured to search for a directory attribute of a directory in the metadata, and selecting a directory attribute including load information matching the load information to be migrated and having a subtree flag as the subtree flag in advance Set in the directory attribute of the root directory identified by the subtree flag; a subtree migration unit, configured to migrate the target subtree as a whole.

With reference to the second aspect, in a first possible implementation, the subtree of the root directory of the subtree identifier identifier includes the metadata corresponding to the smallest unit of the service access operation range.

With reference to the second aspect, in a second possible implementation manner, the subtree of the root directory of the subtree identifier identifier includes metadata that has reached a preset threshold in a preset time period.

With reference to the second aspect, in a third possible implementation, the method further includes: an attribute setting unit, configured to: before the subtree searching unit searches for a directory attribute of a directory in the metadata, the directory attribute in the root directory has a sub Among the at least two subtrees of the tree flag, a plurality of subtrees having the same load change tendency are selected as one concurrent group, and a concurrent group flag is set in a directory attribute of a root directory of each subtree in the concurrent group; a tree search unit, specifically configured to find a directory attribute of a directory in the metadata, where the selected directory attribute includes not only the load information but also a subtree flag, and the subtree as a target subtree to be migrated; the subtree The flag is preset in the directory attribute of the root directory identified by the subtree flag.

With reference to any one of the first possible implementation of the second aspect to the third possible implementation, in a fourth possible implementation, the subtree searching unit is specifically configured to include a subtree In the most concurrent group, the selected directory attribute includes not only the load information, and the subtree flag, but also a directory of the concurrent group flag as the target directory.

In conjunction with the third possible implementation of the second aspect, in a fifth possible implementation, the method further includes: an attribute obtaining unit, configured to receive an attribute set by a user through an application programming interface API, where the attribute includes the Subtree flag and concurrency group flag.

A third aspect provides a computing node for metadata management, where the computing node includes: a processor, a communication interface, a memory, and a bus; wherein the processor, the communication interface, and the memory complete each other through the bus Communication;

The communication interface, the computing node for the metadata management receiving program;

The processor is configured to execute a program; The memory is configured to store a program;

The program includes: a load determining unit, a subtree searching unit, and a subtree migrating unit; and the load determining unit, configured to acquire load information to be migrated;

The subtree searching unit is configured to search for a directory attribute of a directory in the metadata, and select a directory tree that includes load information matching the load information to be migrated and has a subtree flag; the subtree The flag is preset in the directory attribute of the root directory identified by the subtree flag; the subtree migration unit is configured to migrate the target subtree as a whole.

With reference to the third aspect, in a first possible implementation, the subtree of the root directory of the subtree identifier identifier includes the metadata corresponding to the smallest unit of the service access operation range.

With reference to the third aspect, in a second possible implementation manner, the subtree of the root directory of the subtree identifier is included in the sub-tree whose access frequency reaches a preset threshold within a preset time period.

In combination with the third aspect, in a third possible implementation, the method further includes: an attribute setting unit, configured to: before the subtree search unit searches for a directory attribute of a directory in the metadata, the directory attribute in the root directory has a sub Among the at least two subtrees of the tree flag, a plurality of subtrees having the same load change tendency are selected as one concurrent group, and a concurrent group flag is set in a directory attribute of a root directory of each subtree in the concurrent group; a tree search unit, specifically configured to find a directory attribute of a directory in the metadata, where the selected directory attribute includes not only the load information but also a subtree flag, and the subtree as a target subtree to be migrated; the subtree The flag is preset in the directory attribute of the root directory identified by the subtree flag.

With reference to any one of the first possible implementation to the third possible implementation of the third aspect, in a fourth possible implementation, the sub-tree search unit is specifically configured to include a subtree In the most concurrent group, the selected directory attribute includes not only the load information, and the subtree flag, but also a directory of the concurrent group flag as the target directory.

In conjunction with the third possible implementation of the third aspect, in a fifth possible implementation, the method further includes: an attribute obtaining unit, configured to receive an attribute set by a user through an application programming interface API, where the attribute includes the Subtree flag and concurrency group flag.

A fourth aspect, a computer program product for repairing data, comprising a computer readable storage medium storing program code; The program code includes instructions for acquiring load information to be migrated; searching for a directory attribute of a directory in the metadata, and selecting a directory attribute including load information matching the load information to be migrated and having a subtree flag The directory is the target directory, and all subtrees having the target directory as the root directory are determined as the target subtree to be migrated; the subtree flag is preset in the directory attribute of the root directory identified by the subtree flag. ; migrate the target subtree as a whole.

The technical effects of the metadata management method and apparatus provided by the present invention are as follows: by setting a subtree flag in a directory attribute, and all subtrees having a directory having a subtree flag as a root directory are migrated as a target subtree to be migrated as a whole. Compared with the metadata migration mode in the prior art, the metadata operation across the MDS can be effectively reduced, the time for information transmission between different MDSs across the MDS is saved, and the efficiency of metadata distribution management is improved. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a metadata distribution diagram of a file system applied to an embodiment of a metadata management method according to the present invention; FIG. 2 is a flow chart of an embodiment of a metadata management method according to the present invention;

3 is a schematic flowchart of another embodiment of a metadata management method according to the present invention;

4 is a file system metadata distribution diagram of another embodiment of the metadata management method of the present invention; FIG. 5 is a comparison diagram of overall performance of an MDS cluster according to an embodiment of the metadata management method of the present invention; FIG. 6 is a metadata management method of the present invention; The overall performance comparison of the MDS cluster of the embodiment is shown in FIG. 2; FIG. 7 is a diagram of the MDS cluster load distribution before the improvement of the metadata management method embodiment of the present invention; FIG. 8 is a modified MDS cluster load of the metadata management method embodiment of the present invention. FIG. 9 is a schematic structural diagram of an embodiment of a metadata management apparatus according to the present invention;

FIG. 10 is a schematic structural diagram of another embodiment of a metadata management apparatus according to the present invention; FIG.

FIG. 11 is a schematic structural diagram of an embodiment of a computing node for metadata management according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS In order to make the metadata management method of the embodiment of the present invention easier to understand, first, some basic concepts involved in the embodiments of the present invention are described in conjunction with FIG. 1. FIG. 1 is a schematic diagram of an embodiment of a metadata management method according to the present invention. The metadata distribution of the file system. The metadata management of this embodiment actually manages the metadata in the file system, specifically the management of the metadata distribution; the metadata is management information about a directory or a file, for example, a directory or a file. Name, genus Sex, hierarchical relationship, etc.

Referring to FIG. 1 , a Metadata Server (MDS) cluster is taken as an example. In the cluster, multiple MDSs are included, and each MDS is responsible for managing metadata of one machine; the metadata is a graph. A tree-like information composed of respective blocks connected to each other as shown in 1, for example, k1, bucket-1, and the like. In Figure 1, the three storage spaces are enclosed by dashed lines, which belong to MDS1, MDS2 and MDS3 respectively. For example, the dotted box pi is the metadata stored in MDS1, and the dotted box p2 is stored in MDS2. Metadata, in the dashed box p3, is the metadata stored in MDS3. Metadata that is not within the dashed box is metadata on other MDSs.

The metadata of the MDS cluster is stored in a tree structure as shown in FIG. 1. The metadata of the tree structure involves the following concepts, wherein, the directories, files, and subtrees mentioned below are required to be explained. The node and the like refer to the metadata. The embodiment of the present invention discusses the distribution structure of the metadata. For example, the file k-e.avi in FIG. 1 mentioned above is actually the element of the file k-e.avi. Data; In addition, the load mentioned in the embodiment of the present invention also refers to the load for metadata access, but the update of the metadata load is also caused by actual access to files or directories.

Directory: For example, kobe, james, bucket-1, etc. shown in Figure 1;

File: For example, k_e.avi, k-d.avi, etc. shown in Figure 1, the file is located in the directory, for example, k-e.avi is located under directory k3, and the upper directory of the directory k3 is k2 The directory above the directory k2 is kl-1, and so on, the directory at the top of the tree structure is the total root directory; the above files and directories, and the hierarchical relationship between them (that is, the files are located in a directory) The following is the storage path of the file; for example, the storage path of the file k-b.avi is

/kobe/bucket—l/kl/k—b.avi. For another example, the file in the directory bucket-2 in Figure 1 has not been placed yet.

Subtree: If the directory "/" at the top of the tree structure in Figure 1 is likened to the root of a large tree, any branch of the large tree can be called a subtree;

For example, the entirety of all metadata (including all directories and files) of the directory kobe and its subordinates is called a subtree, and the whole of all metadata of the directory kl and its subordinates is called a subtree (ie kl -k_b.avi ), the whole of all metadata of the directory k2 and its subordinates is called a subtree (the subtree includes k2, k3, k_e.avi, k_c.avi). As you can see, the subtree is rooted at a directory, including the directory and all the branches connected to it, which is called a subtree.

In the migration of metadata described later, it is described by the migration of subtrees, because the migration of metadata is in units of subtrees. Node: Each box in Figure 1 is referred to as a node, which may be a directory or a file; for example, k_e.avi, bucket-1, kobe, etc. are all a node;

Node attributes and load: The attributes of the node include, for example, creation time, load value, etc. Each directory or file has a creation time, which is a property of the node; the load value can be seen in the top left of each node in Figure 1. The number indicated by the corner, for example, 31 in the upper left corner of bucket-1, 71 in the upper left corner of kobe, and so on. The load value of the node indicates the current load of the node, which is a parameter that indicates the access volume of the file under the node. Generally, the larger the file access amount, the higher the load, so the load value of the node is actually the MDS where the node is located. The load that is being carried, of course, the load of the entire MDS is the sum of the load values of all the nodes located on it.

For example, the load value of bucket-1 is 31, and the load value of kobe is 71; the load value of a directory is the sum of the load values of the next-level nodes (directories or files) connected to the directory, for example, from the figure 1 It can be seen that the file k_e.avi is in the directory k3, and the load value of the file k_e.avi is 2 (the load value is a characterization value calculated according to an algorithm according to the file access amount or the access frequency) The load value of the directory k3 is the same as the load value of the file k-e.avi; and the load value of the directory k2 is the sum of the load values of the file k_e.avi and the file k-c.avi of the subordinate, the load value of k2 Load value of =k3 +k - load value of c.avi.

Subtree migration: Figure 1 includes many subtrees, and these subtrees belong to different MDSs respectively; for example, subtree bucket-1/kl/k-b.avi is located on MDS1, which means file k_b The metadata of .avi, directory kl, and directory bucket-1 are managed and served by MDS1. The large amount of access to the metadata of the file k_b.avi, directory kl, and directory bucket-1 will make the load of MDS1 larger. The subtree k2 - 1 / k - d. avi is stored in MDS2, that is, the metadata representing the file k - d.avi and the directory k2 - 1 is stored in MDS2, the file k_d.avi or the directory k2 A large amount of access to 1 will make the load of MDS2 larger. The load of the MDS refers to the load of the metadata on the MDS, and the load of the metadata is also caused by the access of the actual directory or file corresponding to the metadata. For example: Assume that the actual access to the file k_b.avi, the corresponding load value of the metadata of the file k_b.avi stored on MDS1 will increase, and the load value of the metadata of the file k_b.avi increases. After that, the load value of the upper directory will also increase, for example, the metadata load value of the directory bucket-1/kl increases; or, if the access to the directory k2-1 is accessed, the corresponding storage on the MDS2 The load value of the metadata of the directory k2-1 will increase.

In order to maintain load balancing between these multiple MDSs, the load on the current MDS is usually large. When migrating a portion of the load to another MDS, it actually migrates the metadata of certain files and directories to another MDS, so that another MDS accesses the migrated files and directories, reducing the current MDS. load. When the load is migrated, the storage location of the metadata of files and directories changes, such as MDS2 to MDS3. It should be noted that the migration of metadata is based on a subtree, for example, the subtree k2 - 1 / k - d.avi can only be migrated. The subtree migration is to change the storage location of the subtree, which is equivalent to enclosing the subtree k2-1/k-d.avi in the dashed box p3 in Figure 1, but the connection relationship of the subtree in the entire tree structure There is no change. For example, the subtree k2—1/k—d.avi is still connected to the directory kl-1.

Based on the above description, the metadata management method of the embodiment of the present invention is described in detail below: Embodiment 1

2 is a schematic flowchart of an embodiment of a metadata management method according to the present invention. As shown in FIG. 2, the method may include:

201. Obtain load information to be migrated;

The load information refers to a load value, such as the number indicated in the upper left corner of each node as shown in FIG. 1, for example, 31 in the upper left corner of the bucket-1.

This example obtains the load information to be migrated. For example, the cluster includes three MDSs. Each MDS updates the load value of the corresponding metadata according to the access information of files and directories that it is responsible for. And periodically detecting the difference between the total load value carried by itself and other MDS load values. When the difference exceeds a certain threshold, the load distribution between the MDSs is uneven, and the load is heavier. MDS will initiate load balancing processing and migrate some of its own load to other MDSs. In fact, the migration of the load is a migration of a subtree in the metadata, and the access corresponding to the subtree is migrated to another MDS, and the access is served by another MDS, and the corresponding heavily loaded MDS is The load carried is reduced.

In this embodiment, for example, the load information that is acquired by an MDS is 7 load values, that is, 7 load values need to be migrated to other MDSs.

202. Search for a directory attribute of a directory in the metadata, select a directory attribute that includes load information that matches the migrated load information, and a directory that has a subtree flag as a target directory, and determine that all the target directory is the root directory. The subtree is the target subtree to be migrated; the subtree flag is pre-set in the directory attribute of the root directory identified by the subtree flag;

Wherein, after determining the load information to be migrated in 201, the directory in the metadata is searched for. a directory attribute, including a directory attribute including a load information matching the load information to be migrated and having a subtree flag as a target directory; load information in a directory attribute of the target directory indicating that the target directory is a root directory The maximum subtree corresponds to the load value. It should be noted that the matching refers to that the load value corresponding to the subtree is not necessarily equal to the load information, and may be similar, and the specific numerical similar range may be determined by a commonly used load balancing fuzzy matching algorithm. The embodiments of the present invention are not described in detail.

For example: See Figure 1 for the assumption that MDS2 will migrate its own 7 load values. As you can see in Figure 2, the load of subtree k2 - 1 / k - d.avi is 7 load values (directory The load value marked in the upper left corner of k2—1 is 7. The load value is also one of the attributes of the directory k2-1, which can be called the load attribute. At this time, the load value corresponding to the subtree is equal to the load information. If MDS2 wants to migrate its own 8 load values and assume that it does not have subtrees with exactly equal load values, the subtree k2-1/k-d.avi with the above load value of 7 can also be used. After migration, the values of 8 and 7 are similar, and the load value corresponding to the subtree can be considered to match the load information.

It should be noted that, in the subtree of the migration selected in this embodiment, not only the load value matches the load information, but also the root directory of the subtree, that is, the attribute of the directory k2-1, including the subtree flag attribute.

The concept of a subtree has been described above with reference to Figure 1. The subtree is rooted at a directory, including the directory and all branches connected to the directory. This whole is called a subtree; then the root directory is here. Called the root directory. For example, the root directory of the subtree k2 - 1 / k - d.avi is the directory k2 - 1; the subtree consisting of two branches k2 / k3 / k_e.avi and k2 / k - c.avi, the root directory is the directory K2. The subtree to be migrated selected in this embodiment must have a subtree flag in its root directory.

The subtree flag is used to indicate that all subtrees with the target directory as the root directory can only be moved by the target subtree. The feature: the metadata included in the target subtree is the smallest unit corresponding to the scope of the business access operation. For example, when the user performs a certain service, an application (that is, an application used by the user) is used, and the application pair The access of the directory or file corresponding to the subtree is basically an internal access of the subtree, and the target subtree includes all the metadata corresponding to the application access operation (the all metadata may be equivalent to the smallest unit of the service access operation range) ). Or, when the user is performing a certain business, Two applications A and B are used, A involves a part of the metadata change, B involves another part of the metadata change, and the two parts of the metadata have an intersection, then the metadata related to A and B can be involved. The metadata as a whole (the whole can be equivalent to the smallest unit of the business access operation range), the whole is completely included in the target subtree. Alternatively, if there is no intersection between the metadata involved in the two applications A and B, the metadata involved in the application A may be used as a whole (the whole may be equivalent to the smallest unit of the service access operation range), The target subtree completely includes the whole. The above is only an example, and the flexibility in the specific implementation is not limited to this.

The subtree in FIG. 1 can be used as an example to illustrate an optional meta-data included in the target sub-tree: For example, FIG. 1 includes k2/k3/k-e.avi and k2/k- C.avi sub-tree of two branches, the sub-tree k2 of the sub-tree is set to the sub-tree flag attribute, then the sub-tree is a whole; the user application access to the sub-tree includes, for example: the user will file k-e. Avi moves from the directory k3 to the directory k2, or moves the file k_c.avi from the directory k2 to the directory k3, or creates a file under the directory k3. It can be seen that these operations are all operations inside the subtree, and do not involve other subtrees other than the subtree, and the subtree already includes metadata corresponding to the application access operation, for example, an application access operation. K_e.avi is moved from the directory k3 to the directory k2, and the metadata involved is k_e.avi, k3, k2, and the metadata is inside the subtree; suppose the directory k3 is set as a subtree The flag, the corresponding subtree k3/k-e.avi does not actually include the metadata k2 involved in the application access operation, that is, only a part of the metadata is included, which does not conform to the target subtree described in this embodiment. The case of metadata included. The following features: The metadata included in the target subtree corresponds to the rule of experience migration; that is, the subtree flag can be set according to the characteristics of the metadata changes in the respective MDS, for example, if a certain part of the metadata is found frequently It is accessed simultaneously for a period of time, and the access frequency of the part of the metadata reaches a preset threshold within a preset time period. For example, the preset threshold is 50 times, and the access frequency of the metadata is within a preset time period. When the degree reaches 50 times, it can be determined that the whole subtree including these metadata is taken as the target subtree.

The overall migration of the subtree corresponding to the directory in which the subtree flag is set means that the subtree including the two branches k2/k3/k_e.avi and k2/k_c.avi is used as an example, and the subtree is used as an example. The role of the flag attribute is explained: Assuming that the subtree k2 of the subtree does not have the subtree flag attribute set, the branch of the subtree can also be divided, for example, the branch k3/k-e.avi is separately migrated (the The branch is actually a Subtree), that is, two load values are migrated; however, if the root directory k2 is set to the subtree flag attribute, it indicates that the above includes k2/k3/k-e.avi and k2/k-c.avi The subtrees of the branches can no longer be split, but can only be migrated as a whole, and no more of them can be migrated.

In a specific implementation, when searching for a subtree to be migrated, the embodiment is performed by searching a directory in the metadata, if the attribute of a certain directory includes a subtree flag, and the load attribute of the directory and the load If the information matches, the load value of the subtree with the directory as the root directory is the load value to be migrated, and the subtree can only be migrated in a holistic manner. This is the subtree to be migrated. This embodiment will search for The root directory of the subtree is called the target directory.

203. The target subtree is migrated as a whole.

The metadata management method of the embodiment is implemented to improve the metadata management efficiency of the file system by: setting a subtree flag in the directory attribute, and using all the subtrees having the subtree flag as the root directory as The overall migration of the target subtree of the migration, compared to the migration of the metadata in the prior art, since the metadata included in the target subtree is always migrated in a holistic manner, it is inevitable to effectively reduce the metadata operations across the MDS, for example When some metadata related to the change is actually the metadata inside the target subtree, it only changes within the target subtree, and does not involve operations across the MDS, thereby saving the difference across the MDS. The time of information transmission between MDS, the efficiency of metadata distribution management is improved.

That is, on the one hand, the distribution of metadata is managed by the MDS itself, and the MDS is responsible for load migration when the load is heavy; when the metadata is distributed among the MDSs, the child with the root directory of the subtree tag attribute is located. The tree is placed in the same MDS as a whole, and the parts in this type of subtree are not placed in different MDSs. Therefore, when the metadata is distributed, it can be based on the subtree tag attributes. The subtree tag attribute is located in the same MDS as the subtree of the root directory. On the other hand, in the process of load balancing, even if the subtree including the target directory is to be migrated, the subtree is migrated as a whole, and only a part of the subtree is migrated, so that the whole of the subtree can be guaranteed. Always in the same MDS, not in different MDS.

Then, the metadata included in the target subtree is an example of all the metadata involved in an application. The assumption is that the access operation of a user application is to move the file k_e.avi from the directory k3 to the directory k2. - 1 , correspondingly, because the storage path of the file changes, the corresponding metadata (which can be understood as metadata means the storage path of the file) also changes accordingly, as shown in Figure 1, the tree in Figure 1 The k-e.avi connection in the structure is changed to the directory k2-1 in the directory k3; The fruit tree k2—1/k—d.avi is located on MDS3, and the subtree k3/k-e.avi is located on MDS2. When the distribution of the meta-metadata is changed, MDS2 will pass the metadata of the file k_e.avi. The MDS operation is sent to the MDS3, and the updated metadata is stored by the MDS3, that is, the subtree k2-1/k-e.avi, and the file storage path service is provided by the MDS3 when the subsequent file k_e.avi is accessed. Access actually becomes the load of MDS3. If the subtree k2—1/k—d.avi and the subtree k3/k—e.avi are both located in the MDS2, the above metadata distribution change can be performed inside the MDS, eliminating the need for different MDSs across the MDS. The time of information transmission, the efficiency of metadata distribution update is improved.

In the following Embodiment 2 and Embodiment 3, the implementation of the metadata distribution management method of the embodiment of the present invention will be described in detail through two optional specific examples.

Embodiment 2

3 is a schematic flowchart of another embodiment of a metadata management method according to the present invention. This embodiment mainly describes how to migrate a certain load value from a certain MDS. FIG. 4 is a file of another embodiment of the metadata management method of the present invention. System metadata distribution diagram, which shows the metadata distribution structure on one of the MDSs in the cluster. Referring to Figures 3 and 4, the method includes:

301. Set a subtree flag attribute for the directory;

The cluster file system that includes multiple MDSs in the cluster and uniformly manages the metadata of the cluster may first set the attributes of the metadata before the cluster metadata is distributed among the plurality of MDSs, where the metadata is set. The work of the attribute is performed by the cluster's metadata control module; the metadata control module is responsible for setting attributes for the metadata and distributing the metadata to multiple MDSs in the cluster. After the initial metadata distribution is completed, the metadata distribution processing in the subsequent cluster running process, such as load balancing, is performed by each MDS, and each MDS manages the metadata stored by itself, and manages the metadata according to the load. Distribution.

In this embodiment, the metadata control module sets a subtree flag attribute for the directory in the metadata; the setting principle of the subtree flag attribute is as follows: the metadata involved in the application access is used as a As a whole, the subtree flag attribute of the subtree corresponding to the whole is set to ensure that the access of the application corresponds to the metadata change inside the subtree whose root directory is the root directory.

For example: When an application is enabled, the corresponding access operation of the application includes moving the file k_e.avi from the directory k3 to the directory k2-1, or moving the file k_c.avi from the directory k2 to Directory k3, or create a new file under directory k3. According to the access characteristics of the application, if it will include kl-1, k2-1, k2, k3, k_e.avi, k-e.avi and The metadata of k_d.avi as a whole, then the access of the above application corresponds to the internal metadata change of the whole; the whole is a subtree, and the whole root directory kl-1 is set to the subtree flag attribute. The subtree flag is used to indicate that the subtree corresponding to the root directory k1-1 can only be migrated in a holistic manner.

It can also be seen from the above that the subtree flag attribute is actually set according to the access characteristics of the application. Based on this, the following two attribute setting modes are set in this embodiment:

One way is to provide a user application programming interface (API), and the user directly sets the attributes of the directory according to the access characteristics of the user's application; that is, if the application has the above access characteristics, The user can then set the properties of the directory to the subtree flag.

Another way is that the setting of the directory attribute is automatically performed by the metadata control module; for example, the attribute setting policy can be set by the user for the metadata control module, for example, the above-mentioned "access to the corresponding application" The meta-data is changed to the sub-tree internal sub-tree set to a whole "etc. After the policy is set, the metadata control module can perform the application access operation that occurs during the running of the cluster, and detect the access characteristics of the application access operation. If the above policy is met, the shell J metadata control module can set the attribute of the root directory of the whole subtree to the subtree flag according to the policy.

In a specific implementation, referring to FIG. 4, in order to facilitate subsequent searching for which directories have sub-tree flag attributes, a sub-tree queue may be set, and each MDS may internally establish a sub-tree queue corresponding to its own storage metadata, and the sub-tree queue All directories that have the subtree flag attribute set are included; or, MDS sets a subtree queue that includes all of its metadata, and inserts a pointer to the subtree queue that points to the directory in which the subtree flag attributes are set. For example, in this embodiment, the directories k1-1, j1, and j2 set the subtree flag attributes.

The metadata control module also sets other attributes for the metadata, for example, the creation time of the directory or the file, the load value, etc.; after setting the attributes for the cluster metadata according to the above rules, the metadata control module distributes the metadata in each MDS, wherein The subtree corresponding to the root directory of the above subtree flag attribute is set to be allocated in the same MDS as a whole.

It should be noted that, in this embodiment, the subtree tag attribute is initially set as an example for description; however, the specific implementation is not limited thereto, and the setting of the metadata attribute includes, for example, the setting of the subtree tag attribute. There is no limit to the setup time, and it can also be applied at any time during the operation of the MDS system. Features Set this subtree tag property. When the subtree tag attribute is set in the system running, if the subtree whose root directory is the directory of the subtree tag attribute is distributed on different MDSs, the system will mark the attribute according to the subtree tag attribute. The subtree migrates to the same MDS as a whole. The way to set the subtree tag attribute in the system operation is the same as the previous method, such as API mode or automatic setting.

302. Obtain load information that will be migrated;

In the running process of the cluster, each MDS manages the distribution of the stored metadata and performs load balancing processing between the MDSs.

Taking the MDS1 shown in FIG. 4 as an example, the MDS1 updates the load value attribute of the metadata stored by the application according to the access of the application. For example, an application enabled by the user currently accesses the file k_b.avi multiple times, and the MDS1 will be The file access provides a path service, that is, the application is accessed to the file k-b.avi according to the stored file path, and correspondingly, the MDS1 accesses the load value of the metadata involved in the access according to the file access. Specifically, for example, the load value of the update file k_b.avi is increased from 10 to 12. Correspondingly, the upper directory of the file k-b.avi, such as the directory kl, bucket-1, kobe, and / will update the load value. Increase the two load values.

In addition to updating the load value, MDS1 also periodically detects the load difference between each MDS. In a cluster including multiple MDSs, each MDS communicates with each other, and can exchange their respective load information, etc., MDS1 is able to obtain load information of other MDSs, and compares its own load information with load information of other MDSs to determine whether the trigger condition for performing load balancing is reached. For example, it can be set that if the load difference between the load of MDS1 and another MDS reaches 20 load values, load balancing is triggered to balance the load distribution between the MDSs. In this embodiment, it is assumed that the MDS1 is detected to determine that the load is too heavy, and eight load values need to be migrated to other MDSs, that is, the acquired load information to be migrated is eight load values.

303. Search for a target directory, the attribute includes a subtree flag, and the load attribute matches the load information.

Specifically, the step is to find whether a target directory exists in the metadata, the target directory attribute includes a subtree flag, and the load attribute of the target directory matches the load information.

The MDS1 in Figure 4 is still taken as an example. After determining the load information to be migrated, the MDS1 will first search for the directory in which the subtree flag attribute is set in the stored metadata, and determine the load attribute of the directory ( That is, the load value) matches the load information to be migrated. In this embodiment, the subtree having the load value equal to the load information will be preferentially searched. For example: MDS1 will look in the directories kl-1, jl and j2 first, because the three directories have the subtree flag attribute set; determine whether there are load attributes and the load information to be migrated in the three directories. Match the directory. After judging, the load values of these three directories are not 8, therefore, there is no suitable directory.

In this step, if the target directory exists after the search, that is, the attribute is a sub-tree flag whose load attribute is the same as the load information, proceed to execute 305, and the entire sub-tree with the target directory as the root directory is migrated. Otherwise, if it is found that it is found that the target directory does not exist, then execution 304 is continued.

304, starting from the dynamic subtree root of the MDS, searching for a suitable directory, and entering recursion; wherein, the dynamic subtree root of the MDS refers to the directory kobe and the directory james in the MDS1, from FIG. It can also be seen that these two directories are the starting root of all the metadata in MDS1, and other directories or files are extended from the two directories, so it can be called a subtree root; and, because of this The subtrees of the embodiment are all separable. For example, dividing a directory into two subdirectories, etc., is called a dynamic subtree root.

The traversal finds a suitable directory, enters recursion, and refers to, along the directory hierarchy shown in FIG. 4, step by step to find whether there is a directory of load values matching the load information; when encountered in the search process When the directory with the subtree flag attribute is stopped, it returns to the previous directory.

For example: When the load information to be migrated is 8 load values, MDS1 starts looking down from the directory kobe and judges whether the load value of the directory is 8, for example, in the following order, kobe bucket-1 kl k_b. Avi, kobe bucket - 1 kl-1, etc., in fact, in the kobe - bucket - 1 - kl - k_b.avi path, it has been found that the load value of the directory kl is 8, then continue Execute 305.

Suppose MDS1 first searches for the kobe-bucket-1-kl-1 path. When it finds the directory kl-1, it finds that the directory kl-1 has the subtree flag attribute, so it will not continue to search for the directory kl-1. Subdirectories or files (such as directory k2, file k_e.avi, etc.), because the subtree of the root directory with the subtree flag attribute is a whole, there is no need to continue to look inside the whole subtree. Then MDS1 will stop at the directory kl-1, return to the previous directory, bucket-1, and then look up the bucket-1 along the path of bucket-1, kl-k_b.avi.

305. Select the subtree that matches the migrated load information and perform the migration.

Wherein, as described above, the load value of the directory kl is 8, then determining the subtree kl_k_b.avi is The subtree of the migration.

As can be seen from the above process, when a certain load value is to be migrated from an MDS, the directory in which the subtree flag attribute is set is preferentially searched; and, in the subtree of the root directory having the subtree flag attribute, The subtree is not selected, that is, the subtree with the root of the subtree flag attribute is a whole, and the whole is migrated, which is equivalent to the "static subtree" because the subtree is no longer split. . In this way, the metadata segmentation is prevented from being too fragmented, and the metadata corresponding to the application access operation is included in the same subtree, so that the metadata change of the application access is performed in one MDS, and there is no case of crossing the MDS. Effectively reduce the frequency of subtree migration.

Embodiment 3

This embodiment also describes how to migrate a certain load value from a certain MDS, but the main difference from the second embodiment is that the embodiment also sets the concept of the concurrent group to improve the efficiency of metadata distribution management. On, further improve the effect of load balancing.

First, the concept of the concurrent group is explained: among the at least two subtrees in which the subtree flag attribute is set, multiple subtrees having the same load change trend are selected as one concurrent group, and the root directory of each subtree in the concurrent group is set. Concurrent group properties. On the one hand, the concurrency group includes multiple subtrees, and the root directory of each subtree is a directory in which the subtree flag attributes are set. On the other hand, the characteristics of the multiple subtrees in the concurrent group are that they have the same load change trend.

The same load change trend refers to, for example, assuming that there are two subtrees in the concurrent group, and the load values of the two subtrees always rise sharply in a certain period of time, or drop sharply in a certain period of time. For example, in a certain hour interval, the load values of the two subtrees are increased by 10, or a subtree can be increased by 9 load values, and the other subtree is increased by 10 load values, that is, only the emphasis is emphasized. The load values of the two subtrees all rise, and the specific values of the rise can be somewhat deviated, while the other subtrees are basically unchanged. Or, the load values of the two subtrees are reduced by 8 in a certain period of time. Load values.

The same load change trend usually indicates that the files corresponding to the two subtrees are usually accessed at the same time, so the load changes at the same time. For example, in Figure 4, the subtree with the directory kl-1 as the root directory, and The subtree with the directory jl as the root directory belongs to a concurrent group. When the user enables the application, both the file ke.avi and the file james.avi are usually accessed, because the two files are It is related, so the load values of the above two subtrees always appear synchronously rising or falling, which means "concurrent", the user application is at the same time Access multiple subtrees within the same group.

If there is a concurrent group in the same MDS, it indicates that the load of the MDS changes by 4, for example, if there is a concurrent group including 5 subtrees in the MDS, the 5 subtrees may appear in a certain period of time. The load value is increased, so that the MDS will quickly appear a large load value increase, and the burden of the MDS will be 4 ;; therefore, this embodiment sets the concurrent group attribute to distinguish such subtrees, and the set rule is, if To migrate subtrees, try to prioritize the subtrees in the concurrent group for migration, so that subtrees in the concurrent group can increase the burden of MDS in the same MDS.

For example, in the subtree in which the load value of the load is to be migrated in FIG. 4, the search process of the migrated subtree is the same as that in the second embodiment, and is not detailed. Only some steps related to the concurrent group in the search process are described: In the process of preferentially finding the directory in which the subtree flag is set, it is found that the directory kl-1, the directory jl, and the directory j2 are all set with the subtree flag attribute, and the load values of the directory kl-1 and the directory j2 are 15, At this point, it is necessary to determine whether to select the subtree corresponding to the directory kl-1 or the subtree corresponding to the directory j2.

Specifically, it is determined whether the directory k1-1 and the directory j2 have a concurrent group attribute. If the concurrent group attribute is present, it indicates that the subtree with the directory as the root directory is a subtree in the concurrent group, and the subtree is preferentially selected. For example, the directory kl-1 has the concurrency group attribute, and the subtree corresponding to the directory k1-1 is the same concurrency group as the subtree corresponding to the directory jl; then the subtree with the directory kl-1 as the root directory is preferentially selected. For migration, the directory kl-1 is also referred to as the target directory to be looked up.

By setting the concurrent group attribute for the directory and preferentially migrating the subtrees in the concurrent group, the burden of the MDS can be divided; for example, the subtree corresponding to the directory k1-1 in FIG. 4 above and the subtree corresponding to the directory j1, if the directory kl The subtree corresponding to -1 is moved out, even if the load trends of the two subtrees are the same, for example, the load value increases during a certain period of time, which is increased in the two MDSs respectively, and is not increased only in MDS 1. The distribution of the load before each MDS is made more balanced. The setting manner of the concurrent group attribute in this embodiment is the same as the setting method of the sub-tree flag attribute, and details are not described herein again.

Further, there may be multiple concurrent groups in the MDS, and there are two sub-groups with suitable sub-trees, that is, the load attribute is the same as the load information to be migrated. How to select the two concurrent groups The rule set in this embodiment is that the concurrent group in which the selected subtree is located is the concurrent group including the largest number of subtrees in the plurality of concurrent groups; for example, one of the concurrent groups includes five subtrees. If another sub-tree includes 2 sub-trees, the sub-trees in the concurrent group including 5 sub-trees are preferred, because the more sub-trees are included, the greater the MDS burden caused when the load increases.

The embodiment of the present invention makes metadata by setting a subtree flag attribute and a concurrency group attribute for a directory. The distribution management efficiency is higher, the load balancing effect is better, and the overall performance of the MDS cluster is improved. For example, by setting the subtree flag attribute, the subtree corresponding to the root directory of the subtree flag attribute is set in the same MDS as a whole, and the operation across the MDS can be reduced, and the metadata change is performed inside the MDS, thus saving MDS is able to handle more metadata distribution management tasks when information is transmitted across MDS operations.

5 and FIG. 6, FIG. 5 is a comparison diagram of the overall performance of the MDS cluster according to the embodiment of the metadata management method of the present invention, and FIG. 6 is a comparison chart 2 of the overall performance of the MDS cluster according to the embodiment of the metadata management method of the present invention. The ordinate of the table indicates the load value. It can be seen from the figure that after the scheme of the embodiment of the present invention is adopted (i.e., after the improvement), the load values of the indexes before the implementation (i.e., before the improvement) are improved. For example, the file creation load, file removal load, directory creation load, directory removal load, tree creation load, and subtree deletion (shown in the figure) Tree removal ) load, and file start load, directory start load, these loads are the load corresponding to the MDS cluster receiving and processing related application access operations, such as processing file creation operations. load. After the improvement, the above load increase before the improvement indicates that the application access operation task of the improved MDS cluster processing is increased compared with the previous one. For example, only two file creation operations can be processed before the improvement in a certain period of time, and five file creations can be processed after the improvement. The operation has significantly improved the performance of the MDS.

For example, by setting the concurrent group attribute, the sub-trees belonging to the same concurrent group are preferentially selected for migration, so that multiple sub-trees in the concurrent group can be distributed in multiple MDSs to avoid concentration in the same MDS. The MDS burden is too large. After such load balancing processing, the burden of the MDS is alleviated, the metadata processing efficiency of the MDS is improved, and higher application access operations can be handled; for example, the MDS 1 in FIG. 4 has a concurrent group before the improvement. As a result, the load of the MDS 1 is heavy, and the processing efficiency of the MDS1 is slowed down. After the improvement, the load of the MDS1 is reduced due to load balancing, the metadata processing efficiency is improved, and the amount of processing tasks is more.

Referring to FIG. 7 and FIG. 8, FIG. 7 is a diagram showing an improved MDS cluster load distribution chart of the metadata management method embodiment of the present invention, and FIG. 8 is a modified MDS cluster load distribution chart of the metadata management method embodiment of the present invention. The ordinates of both tables represent the load value. It can be seen from the figure that the load distribution of MDS0 and MDS1 before the improvement is very unbalanced, the columnar figure is jagged, the processing load value of the ordinate display is also low, indicating that the processing task is small; and after the improvement, MDS0 and MDS1 The load distribution has been balanced, the load balancing effect is significantly improved compared to before the improvement, and the processing load value displayed on the ordinate is also greatly improved before the relative improvement, which improves the performance of the MDS.

It should be noted that the embodiment of the present invention is directed to the MDS of the cluster file system, and the method for managing the distribution of the metadata is described. However, the specific implementation is not limited to the MDS scenario, and the method of the embodiment of the present invention can be used for other management needs. A system for distributing file system metadata services.

Embodiment 4

FIG. 9 is a schematic structural diagram of an embodiment of a metadata management apparatus according to the present invention. The apparatus may perform the method according to any embodiment of the present invention. As shown in FIG. 9, the apparatus may include: a load determining unit 91, a subtree searching unit 92, and a sub Tree migration unit 93; wherein

a load determining unit 91, configured to acquire load information to be migrated;

The subtree searching unit 92 is configured to search for a directory attribute of a directory in the metadata, and select, in the directory attribute, load information that matches the load information to be migrated, and the directory having the subtree flag is the subtree flag in advance. Set in the directory attribute of the root directory identified by the subtree flag; the subtree migration unit 93 is configured to migrate the target subtree as a whole.

Further, the subtree of the root directory of the subtree identifier identifier includes metadata corresponding to the smallest unit of the service access operation range.

Further, the subtree of the root directory of the subtree flag identifier includes metadata whose access frequency reaches a preset threshold within a preset time period.

10 is a schematic structural diagram of another embodiment of the metadata management apparatus of the present invention. The present embodiment is based on the structure shown in FIG. 9. The metadata management apparatus further includes: an attribute setting unit 94, configured to search in the subtree. Before the unit searches the directory attribute of the directory in the metadata, in the at least two subtrees of the root directory whose directory attribute has the subtree flag, select multiple subtrees having the same load change trend as one concurrent group, and the concurrent group Set the concurrency group flag in the directory attribute of the root directory of each subtree;

The subtree searching unit 92 is specifically configured to search for a directory attribute of a directory in the metadata, where the selected directory attribute includes not only the load information but also a subtree flag, and the target subtree of the concurrent group flag shift; The subtree flag is preset in a directory attribute of a root directory identified by the subtree flag. Further, the subtree searching unit 92 is specifically configured to select, from the concurrent group including the largest number of subtrees, the directory attribute to include not only the load information, and the subtree flag, but also the directory of the concurrent group flag. The target directory.

Further, the metadata management apparatus of this embodiment further includes: an attribute obtaining unit 95, configured to receive an attribute set by a user through an application programming interface API, where the attribute includes the subtree flag and a concurrent group flag.

Embodiment 5

FIG. 11 is a schematic structural diagram of an embodiment of a computing node for metadata management according to the present invention. As shown in FIG. 11, this embodiment provides a schematic diagram of a computing node 700. The computing node 700 may be a host server including computing power, or a personal computer (PC), or a portable computer or terminal, etc., and the specific embodiment of the present invention does not implement a computing node. Make a limit.

The compute node 700 includes a processor 710, a communications interface 720, a memory 730, and a bus 740. The processor 710, the communication interface 720, and the memory 730 complete communication with each other via the bus 740.

The communication interface 720 is configured to receive a program by communicating with the network element.

The processor 710 is configured to execute the program 732. In particular, program 732 can include program code, the program code including computer operating instructions.

The processor 710 may be a central processing unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more configured to implement the embodiments of the present invention. integrated circuit.

The memory 730 is configured to store the program 732. The memory 730 may include a random access memory (RAM) and may also include a non-volatile memory such as at least one disk storage. The program 732 may specifically include: a load determining unit 91, configured to acquire load information to be migrated;

The subtree searching unit 92 is configured to search for a directory attribute of a directory in the metadata, and select, in the directory attribute, load information that matches the load information to be migrated, and the directory having the subtree flag is the subtree flag in advance. Set in the directory attribute of the root directory identified by the subtree flag; the subtree migration unit 93 is configured to migrate the target subtree as a whole. For the specific implementation of each unit in the program 732, refer to the corresponding units in the embodiment shown in FIG. 9 to FIG. 10, and details are not described herein.

Further, the program further includes:

And an attribute setting unit, configured to: before the sub-tree search unit finds the directory attribute of the directory in the metadata, select at least two sub-trees having the same load change trend in at least two sub-trees of the root directory having the sub-tree flag The tree acts as a concurrent group, and sets the concurrent group flag in the directory attribute of the root directory of each subtree in the concurrent group;

The subtree search unit is specifically configured to search for a directory attribute of a directory in the metadata, and the selected directory attribute includes not only the load information but also a subtree flag, and a target subtree of the concurrent group flag; The subtree flag is preset in the directory attribute of the root directory identified by the subtree flag.

Further, the subtree searching unit is specifically configured to select, from the concurrent group that includes the largest number of subtrees, the directory attribute, not only the load information, and the subtree flag, but also the directory of the concurrent group flag. The target directory.

Further, the program further includes:

An attribute obtaining unit, configured to receive an attribute set by a user through an application programming interface API, where the attribute includes the subtree flag and a concurrent group flag.

The embodiment of the invention further provides a computer program product for repairing data, comprising a computer readable storage medium storing the program code;

The program code includes instructions for acquiring load information to be migrated; searching for a directory attribute of a directory in the metadata, and selecting a directory attribute including load information matching the load information to be migrated and having a subtree flag The directory is the target directory, and all subtrees having the target directory as the root directory are determined as the target subtree to be migrated; the subtree flag is preset in the directory attribute of the root directory identified by the subtree flag. ; migrate the target subtree as a whole.

It will be apparent to those skilled in the art that the above description is convenient and concise for the description. For a specific working process of the system, the device, and the unit, reference may be made to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not executed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some communication interface, device or unit, and may be in electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM for short), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. The medium of the program code.

Finally, it should be noted that the above embodiments are only for explaining the technical solutions of the present invention, and are not intended to be limiting thereof; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. range.

Claims

claims

1. A metadata management method, characterized by including:

Obtain the load information to be migrated;

Search the directory attributes of the directory in the metadata, select the directory whose directory attributes include load information matching the load information to be migrated and have a subtree flag as the target directory, and determine all directories with the target directory as the root directory. The subtree serves as the target subtree to be migrated; the subtree flag is preset in the directory attribute of the root directory identified by the subtree flag;

Migrate the target subtree as a whole.

2. The metadata management method according to claim 1, wherein the subtree of the root directory identified by the subtree flag includes metadata corresponding to the smallest unit of the business access operation range.

3. The metadata management method according to claim 1, wherein the subtree of the root directory identified by the subtree flag includes metadata whose access frequency reaches a preset threshold within a preset time period.

4. The metadata management method according to claim 1, characterized in that, before searching the directory attributes of the directory in the metadata, it further includes:

Among at least two subtrees whose directory attribute of the root directory has a subtree flag, select multiple subtrees with the same load change trend as a concurrency group, and add the directory attribute of each subtree in the concurrency group to the directory attribute of the root directory. Set the concurrent group flag;

The selected directory attribute includes the load information that matches the load information to be migrated and has a subtree flag as the target directory, including: the selected directory attribute not only includes the load information and has a subtree flag, The directory containing the concurrent group flag is also included as the target directory.

5. The metadata management method according to any one of claims 2 to 4, characterized in that the selected directory attribute not only includes the load information and the subtree flag, but also includes the directory of the concurrent group flag as the The target directory includes:

From the concurrent group that includes the largest number of subtrees, select a directory whose directory attributes include not only the load information and the subtree flag, but also the concurrent group flag as the target directory.

6. The metadata management method according to claim 4, wherein the directory attribute is an attribute set by a user through an application programming interface API, and the attribute includes the subtree flag and the concurrent group flag.

7. A metadata management device, characterized by including: Load determination unit, used to obtain load information to be migrated;

A subtree search unit, used to search the directory attributes of the directory in the metadata, and select a directory whose directory attributes include load information matching the load information to be migrated and have a subtree flag, and the subtree flag is preset. Set in the directory attribute of the root directory identified by the subtree flag; a subtree migration unit, used to migrate the target subtree as a whole.

8. The metadata management device according to claim 7, wherein the subtree of the root directory identified by the subtree flag includes metadata corresponding to the smallest unit of the business access operation range.

9. The metadata management device according to claim 7, wherein the subtree of the root directory identified by the subtree flag includes metadata whose access frequency reaches a preset threshold within a preset time period.

10. The metadata management device according to claim 7, further comprising: an attribute setting unit, configured to set the directory attribute in the root directory before the subtree search unit searches for the directory attribute of the directory in the metadata. Among at least two subtrees whose attributes have subtree flags, select multiple subtrees with the same load change trend as a concurrent group, and set the concurrent group flag in the directory attribute of the root directory of each subtree in the concurrent group;

The subtree search unit is specifically used to search the directory attributes of the directory in the metadata, and select the directory attributes that not only include the load information and have the subtree flag, but also include the target subtree of the concurrent group flag; so The subtree flag is preset in the directory attribute of the root directory identified by the subtree flag.

11. The metadata management device according to any one of claims 8 to 10, characterized in that the subtree search unit is specifically used to select from the concurrent group including the largest number of subtrees the directory attributes that include not only all The load information, subtree flag, and directory of the concurrent group flag are also included as the target directory.

12. The metadata management device according to claim 10, further comprising: an attribute acquisition unit, configured to receive attributes set by the user through an application programming interface API, where the attributes include the subtree flag and concurrency Group logo.

13. A computing node for metadata management, characterized in that the computing node includes: a processor, a communication interface, a memory and a bus; wherein the processor, communication interface and memory pass Complete mutual communication through the bus;

The communication interface is a computing node receiving program used for the metadata management;

The processor is used to execute programs;

The memory is used to store programs;

The program includes: a load determination unit, a subtree search unit and a subtree migration unit; the load determination unit is used to obtain load information to be migrated;

The subtree search unit is used to search the directory attributes of the directory in the metadata, and select a directory tree whose directory attributes include load information matching the load information to be migrated and have a subtree mark; the subtree The flag is preset in the directory attribute of the root directory identified by the subtree flag; the subtree migration unit is used to migrate the target subtree as a whole.

14. The computing node for metadata management according to claim 13, characterized in that the subtree of the root directory identified by the subtree flag includes metadata corresponding to the smallest unit of the business access operation range.

15. The computing node for metadata management according to claim 13, characterized in that: the metadata.

16. The computing node for metadata management according to claim 13, characterized in that the program further includes:

The attribute setting unit is configured to select multiple subtrees with the same load change trend among at least two subtrees whose directory attributes of the root directory have subtree flags before the subtree search unit searches for the directory attribute of the directory in the metadata. tree as a concurrent group, and set the concurrent group flag in the directory attribute of the root directory of each subtree in the concurrent group;

17. The computing node for metadata management according to any one of claims 14-16, characterized in that, The subtree search unit is specifically used to select a directory whose directory attributes include not only the load information and the subtree flag, but also the concurrent group flag from the concurrent group that includes the largest number of subtrees as the target. Table of contents.

18. The computing node for metadata management according to claim 16, characterized in that the program further includes:

An attribute acquisition unit is used to receive attributes set by the user through the application programming interface API, where the attributes include the subtree flag and the concurrent group flag.

19. A computer program product for repairing data, characterized by including a computer-readable storage medium storing program code;

The program code includes instructions for obtaining the load information to be migrated; searching for directory attributes of the directory in the metadata; and selecting the directory attributes that include load information that matches the load information to be migrated and has a subtree flag. The directory is the target directory, and all subtrees with the target directory as the root directory are determined as the target subtree to be migrated; the subtree flag is preset in the directory attribute of the root directory identified by the subtree flag. ; Migrate the target subtree as a whole.