CN106331075B

CN106331075B - Method for storing file, metadata server and manager

Info

Publication number: CN106331075B
Application number: CN201610682509.8A
Authority: CN
Inventors: 彭瑞林
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2016-08-18
Filing date: 2016-08-18
Publication date: 2020-01-17
Anticipated expiration: 2036-08-18
Also published as: CN106331075A

Abstract

The embodiment of the application provides a method for storing files, a metadata server and a manager. The method is applied to a system comprising an MDS and a plurality of SNs, wherein each SN has at least one characteristic, each characteristic in each SN corresponds to a label, the same characteristic in the SNs corresponds to the same label, different characteristics in the SNs correspond to different labels, the MDS stores a mapping relation between a directory and a label condition, and the label condition is used for matching at least one label to determine the SN, and the method comprises the following steps: the MDS receives a request message sent by a client, wherein the request message is used for requesting a target file in a target directory to store a target data block; the MDS determines a target label condition corresponding to the target directory according to the mapping relation between the directory and the label condition and the request message; the MDS determines a target SN that matches the target tag condition, the target SN to store the target data block.

Description

Method for storing file, metadata server and manager

Technical Field

The present application relates to the field of information technology, and more particularly, to a method, a metadata server, and a manager for storing files.

Background

In a Distributed File System (DFS), a Metadata server (MDS) may store data onto a Storage Node (SN). Since SNs run on different types of nodes, different nodes are different in hardware resources, for example, different in terms of a Central Processing Unit (CPU), a memory, a hard disk, a network card, and a motherboard, and in order to enable a file to be stored in an SN that can provide a high quality service, the SNs may be grouped.

In the prior art, the SN is classified according to the disk type, and the MDS matches the SN for the file according to the disk type, so that the disk resource of the SN can be fully utilized.

Disclosure of Invention

The embodiment of the application provides a method for storing a file, a metadata server and a manager, which can improve the resource utilization rate of an SN.

In a first aspect, a method for storing a file is provided. The method is applied to a system comprising a metadata server MDS and a plurality of storage nodes SN, each of the SNs having at least one characteristic, each of the characteristics in each of the SNs corresponding to a tag, the same characteristics in the SNs corresponding to the same tag, different characteristics in the SNs corresponding to different tags, the MDS storing a mapping of a directory to tag conditions, the tag conditions being used to match at least one of the tags to determine the SN, although the tag conditions may also be used to match at least two of the tags to determine the SN, the method comprises: the MDS receives a request message sent by a client, wherein the request message is used for requesting a target file in a target directory to store a target data block; the MDS determines a target label condition corresponding to the target directory according to the mapping relation between the directory and the label condition and the request message; the MDS determines a target SN that matches the target tag condition, the target SN to store the target data block.

The MDS receives a request message which is sent by a client and used for requesting a target file in a target directory to store a target data block, determines a target label condition corresponding to the target directory according to the request message and a mapping relation between the directory and the label condition stored by the MDS, and determines a target SN matched with the target label condition and provided with at least one label according to the target label condition.

In some possible implementations, the target tag condition includes any one of the following operators: and or; the matching operation matched with the target label condition comprises any one of the following operations: if the target label condition comprises the AND, the SN of the label with the AND on both sides is the target SN; if the target tag condition includes the or, the SN of any of the tags having the or both sides is the target SN.

The target tag condition includes any one of operators of (& &) or (| |), wherein & & indicates that two tags corresponding to both sides of the operator are to exist simultaneously, that is, an SN having characteristics corresponding to the two tags at the same time is taken as a target SN; if one label exists on two sides of the operator, the requirement can be met, namely the SN with any one label of the two labels can be used as the target SN, so that the MDS can match the SN for the target file according to the labels corresponding to at least two characteristics, and the resource utilization rate of the SN is further improved.

In some possible implementations, the target tag condition includes any one of the following operators: none, greater than, equal to, and less than; the matching operation matched with the target label condition comprises any one of the following operations: if the target tag condition includes the non, the SN of the tag except the SN with the non back is the target SN; if the target tag condition includes that the SN of which the value of the tag is greater than the first threshold value is the target SN; if the target tag condition includes the equality, the SN with the value of the tag equal to a second threshold is the target SN; if the target tag condition includes the value less than, the SN of the tag whose value is less than the third threshold is the target SN.

The target tag condition includes any one of a not (|), greater than (>), less than (<), and equal to (═ where | is! SN representing that the label after the operator is not included can be used as the target SN; SNs indicating that the value of the tag is greater than a preset threshold (indicated as a first threshold) may be the target SNs, SNs indicating that the value of the corresponding tag is equal to a preset threshold (indicated as a second threshold) may be the target SNs, and SNs < indicating that the value of the corresponding tag is less than a preset threshold (indicated as a third threshold) may be the target SNs. In this way, the MDS can match the SN for the target file according to the corresponding relation between the at least one label and the preset threshold value, and therefore the resource utilization rate of the SN is further improved.

In some possible implementations, the target tag condition includes any one or a combination of the following operators: and, or, not, greater than, equal to, less than, and parenthesis; if the target tag condition includes the small bracket, the tag in the small bracket participates in the matching operation outside the small bracket as a whole after the tag in the small bracket is preferentially matched.

The target tag condition may include any one of and, or, not, greater than, equal to, less than, and parenthesis, or may be any combination of the above operators. If the target tag condition includes a small bracket, then the matching operation within the small bracket participates as a whole in matching operations outside the small bracket. Therefore, the MDS can match the SN for the target file according to the label condition comprising one or more operators, and the resource utilization rate of the SN is further improved.

In some possible implementations, before the MDS determines the target tag condition corresponding to the target directory according to the mapping relationship between the directory and the tag condition and the request message, the method further includes: the MDS receives the mapping relation between the directory and the label condition sent by the manager.

The MDS receives the mapping relation between the directory and the label condition sent by the manager, and determines the label condition (expressed as a target label condition) corresponding to the target directory according to the mapping relation, so that the target SN matched with the target label condition can be determined, and the target SN can improve the service quality of file storage. The mapping relation between the directory and the label condition can be generated by the manager according to the user requirement, and the flexibility of SN grouping configuration can be improved.

In some possible implementations, before the MDS determines the SN for storing the target file according to the target tag condition, the method further includes: the MDS receives the tag sent by each SN associated with the MDS.

In the DFS system, the MDS is associated with all SNs in the DFS system, all SNs in the DFS system send the labels of the SNs to the MDS, and the MDS receives the labels of the SNs sent by all the SNs. Therefore, the MDS can match the SN meeting the target label condition from all the SNs according to the target label condition, and the service quality of file storage is improved.

In some possible implementations, the request message further includes the target data block; the method further comprises the following steps: the MDS stores the target block of data in the target SN.

The MDS receives the request message sent by the client and can also comprise a target data block, and after the MDS determines the target SN matched with the target label condition, the target data block can be directly stored in the target SN, so that the situation that the client establishes connection with the SN and then stores the data block is avoided, and power consumption is saved.

In some possible implementations, the method further includes: the MDS instructs the client to store the target data block in the target SN.

After the MDS matches the target SN meeting the target tag condition, the client may be instructed to store the target data block in the target SN. For example, the MDS may send indication information to the client, which may instruct the client to store the target data block in the matched target SN. The indication information may be in the form of a list, and the MDS may list the identifiers of the matched target SNs and send the list to the client, so that the client establishes a connection with each matched SN according to the indication information, and writes a target data block into each matched target SN, thereby enabling file storage to obtain high-quality service.

In some possible implementations, the request message further includes a number of the target data blocks requested to be stored; wherein the MDS determining the target SN matched with the target tag condition comprises: the MDS determines the number of the target SNs from the SNs that match the target tag condition.

To ensure the reliability of the storage of the data blocks, the MDS may also maintain a copy of each data block. The MDS receiving the request message sent by the client may further include the number of target data blocks requested to be stored (i.e., the number of copies), and selects, as the target SN, the SN of the number of target data blocks meeting the requirement, from among the plurality of SNs determined by the MDS to be matched with the target tag condition. The target SN can be an SN which is randomly selected from a plurality of matched SNs and accords with the number of copies; or the target SN can be an SN with less load and according with the number of copies according to a load balancing principle, so that the service quality of file storage is improved, and the resource utilization rate of the SN is improved.

In some possible implementations, if the characteristic is a resource type, the tag corresponding to the resource type is any one of the following: a Central Processing Unit (CPU) label, a memory label, a hard disk label, a network label, a server type label and an Operating System (OS) type label; if the characteristic is a preset characteristic, the label corresponding to the preset characteristic is any one of the following: the method comprises the steps of user equipment type labels, node belonging area labels, service type labels, node operation service labels and storage data destination labels.

If the SN characteristic is a preset characteristic, the MDS may match the target data block to an SN corresponding to the preset characteristic included in the condition of meeting the target tag. If the characteristics of the SNs are resource type characteristics, the target tag condition may be any one of a CPU tag, a memory tag, a hard disk tag, a network tag, a server type tag, and an OS type tag, and the MDS may match the SNs corresponding to the tags of the resource types included in the target tag condition for the target data block, thereby improving the resource utilization of the SNs.

In some possible implementations, the request message further includes a number of the target data blocks requested to be stored; the method further comprises the following steps: the MDS determines a SN that does not match the target tag condition, and determines the number of the target SNs from the plurality of SNs.

If the number of the SNs which are matched by the MDS and meet the target tag condition is smaller than the number of the copies of the request, the MDS can add and select SNs in unmatched SNs, and the SNs which meet the target tag condition and the added and selected SNs serve as target SNs. If the MDS determines that the SN meeting the target label condition is not matched at all, the MDS can randomly select the SN meeting the copy number from all the SNs as the target SN, or can select the SN meeting the copy number with less load according to a load balancing principle, so that the resource utilization rate of the SN is improved.

In some possible implementations, the method further includes: the MDS determines that the SN which is not matched with the target label condition and determines each level of father directory on the target directory; the MDS determines the label condition mapped with the father catalog of each level according to the mapping relation between the catalogs and the label condition; and the MDS judges whether SN matched with the label condition mapped by the father directories of each level exists from bottom to top step by step according to the upper and lower levels of the father directories of each level until the SN is matched, and determines the target SN from the matched SN.

And if the MDS does not match the SN meeting the target label condition corresponding to the target directory requested by the client, the MDS matches the upper-level directory of the target directory. The MDS acquires the label condition corresponding to the parent directory according to the mapping relation between the directory and the label condition, and further can determine whether SN meeting the label condition corresponding to the parent directory of the target directory exists or not, and if the SN does not exist, the SN continues to be searched for the upper level of the parent directory; if so, determining the SN meeting the label condition of the level directory as the target SN. Therefore, the MDS can match the target SN for the target file as far as possible by expanding the matching range after the SN is not matched, and the matched SN is relatively high in resource utilization rate.

In some possible implementations, the request message further includes a number of the target data blocks requested to be stored; the determining the target SN from the matching SNs includes the MDS determining the number of the target SNs from the matching SNs.

The MDS receives the request message sent by the client and can also comprise the number of target data blocks requested to be stored, the MDS determines a plurality of SNs meeting the conditions of the target tags according to the parent directory, and the SNs meeting the number of copies are selected from the SNs as the target SNs. The target SN can be an SN which is randomly selected from a plurality of matched SNs and accords with the number of copies; or the target SN can be the SN with less load and according with the number of copies according to the load balancing principle, so that the resource utilization rate of the SN is improved.

In a second aspect, a method for storing a file is provided. The method is applied to a system comprising a metadata server MDS and a plurality of storage nodes SN, each of the SNs having at least one characteristic, each of the characteristics in each of the SNs corresponding to a tag, the same characteristics in the plurality of SNs corresponding to the same tag, and different characteristics in the plurality of SNs corresponding to different tags, the method comprising: the manager generates a mapping relationship between the directory and the label condition, where the label condition is used to match at least one label, and the label condition can also be used to match at least two labels to determine the SN; the manager sends the mapping relation to the MDS, wherein the mapping relation is used for the MDS to determine a target storage node SN of a target file storage target data block in a target directory, and the target SN is matched with a target label condition corresponding to the target directory.

The manager generates a mapping relation between a directory and a label condition and sends the mapping relation to the MDS, so that the MDS determines a storage node SN of a target file storage target data block in a target directory according to the mapping relation, the target SN has at least one characteristic matched with the target label condition corresponding to the target directory, and each characteristic of the SN corresponds to one label, so that the MDS can match the SN for the target file according to the label corresponding to the characteristic of the SN, and the resource utilization rate of the SN is improved.

The target tag condition includes any one of operators of (& &) or (| |), wherein & & indicates that two tags corresponding to both sides of the operator are to exist simultaneously, that is, an SN having characteristics corresponding to the two tags at the same time is taken as a target SN; if one label exists on both sides of the operator, the requirement can be met, namely the SN of any one label of the two labels can be used as the target SN. The manager sends the mapping relation between the directory and the target label condition to the MDS, so that the MDS can match the SN for the target file according to the labels corresponding to the at least two characteristics, and the resource utilization rate of the SN is further improved.

The target tag condition includes any one of a not (|), greater than (>), less than (<), and equal to (═ where | is! SN representing that the label after the operator is not included can be used as the target SN; SNs indicating that the value of the tag is greater than a preset threshold (indicated as a first threshold) may be the target SNs, SNs indicating that the value of the corresponding tag is equal to a preset threshold (indicated as a second threshold) may be the target SNs, and SNs < indicating that the value of the corresponding tag is less than a preset threshold (indicated as a third threshold) may be the target SNs. The manager sends the mapping relation between the directory and the target label condition to the MDS, and the MDS can match the SN for the target file according to the corresponding relation between at least one label and the preset threshold value, so that the resource utilization rate of the SN is further improved.

The target tag condition may include any one of and, or, not, greater than, equal to, less than, and parenthesis, or may be any combination of the above operators. If the target tag condition includes a small bracket, then the matching operation within the small bracket participates as a whole in matching operations outside the small bracket. The manager sends the mapping relation between the directory and the target label condition to the MDS, so that the MDS can match the SN for the target file according to the label condition comprising one or more operators, and the resource utilization rate of the SN is further improved.

In some possible implementations, the method further includes: the manager receiving the label for each SN; the method for generating the mapping relation between the catalogue and the label condition by the manager comprises the following steps: the manager generates the mapping relationship according to the label and the user requirement of each SN.

The manager receives the labels of all SN which can provide service for the client, so that the manager can configure label conditions for different catalogs according to the received labels of the SN and user requirements, generate the mapping relation between the catalogs and the label conditions, and send the mapping relation between the catalogs and the label conditions to the MDS. Therefore, the user can configure label conditions for different catalogs according to requirements, habits and the like, and the flexibility of SN grouping configuration is improved.

In some possible implementations, the receiving, by the manager, the tag for each SN includes: the manager receives the label corresponding to the resource type characteristic sent by each SN; and/or the manager receives the label corresponding to the preset characteristic of each SN input by the user.

The manager can receive a label corresponding to the SN-collected resource type characteristics reported by the SN; or the manager may directly receive the tag corresponding to the preset characteristic input by the user, and the user may apply a custom tag (represented as a tag corresponding to the preset characteristic) such as a related scene according to the service type, the service requirement, and the like. That is, the label received by the manager may be a label corresponding to the resource type characteristic reported by the received SN, or may be a custom label received by the user; or the labels received by the manager are all labels corresponding to the resource type characteristics reported by the SN; or the received tags may all be from custom tags entered by the user. Therefore, a user can configure different label conditions according to various types of labels, and the flexibility of configuring the mapping relation between the catalog and the label conditions is improved.

The manager can collect the characteristics of different SNs to configure the tags, or the user can also customize various tags, so that the user can flexibly define the tags according to the requirements and habits, and the flexibility of SN grouping configuration is improved.

In a third aspect, the present application provides a method for storing a file. The method is applied to a system comprising a metadata server MDS and a plurality of storage nodes SN, wherein each SN has at least one characteristic, each characteristic in each SN corresponds to one label, the same characteristic in the SNs corresponds to the same label, different characteristics in the SNs correspond to different labels, the MDS stores a mapping relation between a file and a label condition, and the label condition is used for matching at least one label to determine the SN, and the method comprises the following steps: the MDS receives a request message sent by a client, wherein the request message is used for requesting to store a target data block in a target file; the MDS determines a target label condition corresponding to the target file according to the mapping relation between the file and the label condition and the request message; the MDS determines a target SN that matches the target tag condition, the target SN to store the target data block.

The MDS receives a request message sent by a client and used for requesting to store a target data block in a target file, determines a target label condition corresponding to the target file according to the request message and a mapping relation between the file stored by the MDS and the label condition, and determines a target SN matched with the target label condition and provided with at least one label according to the target label condition, so that the MDS can match the SN for the target file according to the label corresponding to the SN characteristic, and the resource utilization rate of the SN is improved.

In some possible implementations, before the MDS determines the target tag condition corresponding to the target file according to the mapping relationship between the file and the tag condition and the request message, the method further includes: the MDS receives the mapping relation between the file and the label condition sent by the manager.

In some possible implementations, the request message further includes a number of the target data blocks requested to be stored; the method also includes the MDS determining a SN that does not match the target tag condition, determining the number of the target SN from the plurality of SNs.

In a fourth aspect, a method for storing a file is provided. The method is applied to a system comprising a metadata server MDS and a plurality of storage nodes SN, each of the SNs having at least one characteristic, each of the characteristics in each of the SNs corresponding to a tag, the same characteristics in the plurality of SNs corresponding to the same tag, and different characteristics in the plurality of SNs corresponding to different tags, the method comprising: the manager generates a mapping relationship between the file and the label condition, where the label condition is used to match at least one label, and of course, the label condition may also be used to match at least two labels to determine the SN; the manager sends the mapping relation to the MDS, wherein the mapping relation is used for the MDS to determine a target storage node SN for storing a target data block in a target file, and the target SN is matched with a target label condition corresponding to the target file.

The manager generates a mapping relation between a file and a label condition, the label condition is used for matching at least one label, and the mapping relation is sent to the MDS, so that the MDS determines a storage node SN for storing a target data block in a target file according to the mapping relation, the target SN is matched with a target label condition corresponding to the target file, the MDS can match the SN for the target file according to the label corresponding to the characteristics of the SN, and therefore the resource utilization rate of the SN is improved.

In some possible implementations, the method includes: the manager receiving the label for each SN; the method for generating the mapping relation between the file and the label condition by the manager comprises the following steps: the manager generates the mapping relationship according to the label and the user requirement of each SN.

In a fifth aspect, the present application provides an MDS comprising means for performing the method of the first aspect or any possible implementation manner of the first aspect.

In a sixth aspect, the present application provides a manager comprising means for performing the method of the second aspect or any possible implementation of the second aspect.

In a seventh aspect, the present application provides an MDS comprising means for performing the method of the third aspect or any possible implementation manner of the third aspect.

In an eighth aspect, the present application provides a manager comprising means for performing the method of the fourth aspect or any possible implementation manner of the fourth aspect.

In a ninth aspect, there is provided a file storage system comprising:

the MDS of the fifth aspect, the manager of the sixth aspect, the plurality of SNs, and the client.

In a tenth aspect, there is provided a file storage system comprising:

the MDS of the seventh aspect, the manager of the eighth aspect, the plurality of SNs, and the client.

In an eleventh aspect, a computer storage medium is provided, in which a program code is stored, where the program code is used to instruct execution of instructions of the method for storing files in the first aspect or any one of the possible implementation manners of the first aspect.

In a twelfth aspect, a computer storage medium is provided, in which a program code is stored, the program code being used for instructing to execute instructions of the method for storing files in the second aspect or any possible implementation manner of the second aspect.

In a thirteenth aspect, a computer storage medium is provided, in which a program code is stored, and the program code is used for instructing to execute an instruction of a method for storing a file in the third aspect or any possible implementation manner of the third aspect.

In a fourteenth aspect, a computer storage medium is provided, wherein the computer storage medium has program code stored therein, and the program code is used for instructing to execute the instructions of the method for storing files in the fourth aspect or any possible implementation manner of the fourth aspect.

Based on the technical scheme, the MDS receives a request message sent by a client and used for requesting a target file in a target directory to store a target data block, determines a target label condition corresponding to the target directory according to the request message and a mapping relation between the directory and the label condition stored by the MDS, and determines a target SN matched with the target label condition and provided with at least one label according to the target label condition.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of a distributed file system architecture;

FIG. 2 is a schematic diagram of a prior art method for storing files;

FIG. 3 is an interaction flow diagram of a method for storing files according to one embodiment of the present application;

FIG. 4 is a schematic diagram of a method for storing files according to yet another embodiment of the present application;

FIG. 5 is an interactive flow diagram of a method for storing files according to yet another embodiment of the present application;

FIG. 6 is an interactive flow diagram of a method for storing files according to yet another embodiment of the present application;

figure 7 is a schematic block diagram of a MDS of one embodiment of the present application;

FIG. 8 is a schematic block diagram of a manager of one embodiment of the present application;

FIG. 9 is a schematic block diagram of a file storage system of an embodiment of the present application;

figure 10 is a schematic block diagram of a MDS of a further embodiment of the present application;

FIG. 11 is a schematic block diagram of a manager of yet another embodiment of the present application;

FIG. 12 is a schematic block diagram of a file storage system of an embodiment of the present application;

figure 13 is a schematic diagram of the structure of a MDS according to one embodiment of the present application;

FIG. 14 is a block diagram of a manager according to yet another embodiment of the present application;

figure 15 is a schematic diagram of the structure of a MDS according to one embodiment of the present application;

fig. 16 is a schematic structural diagram of a manager according to another embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

To facilitate understanding of the embodiments of the present application, the following elements are first introduced before the embodiments of the present application are described herein.

Metadata is data describing data, mainly information describing data attributes, and is used for indicating functions such as storage location, resource search, file record, file authority and the like. The metadata may be considered an electronic catalog; for the purpose of cataloguing, the metadata has the content and characteristics of describing and collecting data, and is convenient for data retrieval.

Distributed file systems refer to a file system that manages physical storage resources that are not necessarily directly connected to a local Storage Node (SN), but rather are connected to the SN through a computer network.

The manager (manager) manages software for Operation Maintenance (OM), and is responsible for installation, deployment, configuration, issuing and the like of the cluster.

The hard operating system (OS _ hard way) is an underlying entity, and specifically includes server HARDWARE, an operating system, and the like.

FIG. 1 illustrates a distributed file system architecture based on metadata management. As shown in fig. 1, the distributed file system architecture based on metadata management mainly includes three parts: a metadata server (MDS), an SN, and a client (client).

The MDS is used to store and manage metadata information in the distributed file system. A copy of this metadata information may be maintained at the SN for reliability. The MDS is mainly used to record a directory (also referred to as a file directory) and the file is specifically divided into several data blocks (data blocks), and the mapping relationship between the several data blocks and the SNs (i.e., on which SNs each data block is specifically stored). When the file needs to be read or modified subsequently, the position of the data block corresponding to the file can be found in the system according to the directory.

The metadata information of the general record is: data block (datablock) identification (Identity, ID)1 … N (namely, the file is divided into N data blocks, and the corresponding IDs are respectively 1-N) - > SN1 … N (namely, the file contents divided into N data blocks are respectively stored on N SNs, and the node numbers are 1-N); here, "- >" indicates that there is a correspondence relationship. That is, the file "in the directory"/path/to/"is divided into N data blocks 1 to N, and the N data blocks are correspondingly stored on N SNs (SN1 … N).

For example, file1 is divided into 3 blocks, block1, block2 and block3, wherein block1 has 3 copies, which are stored in SN1, SN3 and SN4, respectively, and multiple copies are for reliability, and similarly, block2 also has 3 copies, block3 also has 3 copies in SN2, SN3 and SN 5, and SN1, SN2 and SN 6 are represented as follows:

i.e./file 1- > block1- > (SN1, SN3, SN4)

block2->(SN2，SN3，SN5)

block3->(SN1，SN2，SN6)。

The SN is used to store a data block of a file and record information of the file and information of the data block of the file (e.g., an identification ID of the data block), and the SN will report the recorded information to the MDS.

A client (client) is an external interface of the distributed file system, and specifically may be a standard interface provided for an Application program (APP); the standard interfaces provided by the client include: portable Operating System Interface (POSIX) such as create, open, write, read, close, mkdir, chmod, chown, stat, etc. Data interaction can be carried out between the client and the MDS or the SN, for example, data writing is realized through a write interface, and data reading is realized through a read interface.

In one possible design, the distributed file system stores data blocks (data blocks) and copy/Erasure codes (Erasure codes). A client (client) of a Distributed File System (DFS) divides a file to be stored by a service into data blocks according to a fixed size, and stores each data block to a plurality of SNs in a copy or erasure code mode. For example, the block size (block size) of a data block is 1M, there are 3 SNs in the system, and when a service requires to store a file of 3M size, the DFS divides the file into 3 data blocks of 1M, including block1, block2, block 3; three copies (including an original block and two copied blocks) of each block are obtained, and the SN of each copy is selected through a certain algorithm or rule.

The write flow of data in DFS is as follows:

1. clients provide standard interfaces, such as open, read, write, etc., which are provided for APP use; the APP calls the interfaces to create, read and write files.

2. When the APP needs to create a file, a create interface provided by the client is called to enable the client to send a file creating request to the MDS, wherein the file creating request can carry the number of copies to be created.

3. After receiving a file creating request from a client, the MDS determines SN from the recorded SN list according to a file path (which can also be called as a file directory or a directory) of a file specified by the file creating request and the number of the copies, wherein the number of the determined SN is the same as the number of the copies; and the MDS returns the node information of the determined SN to the client.

4. And after receiving the node information of the SN, the client communicates with the SN according to the node information so as to correspondingly write the data of the APP calling write interface into the SN determined by the MDS according to the copy number.

5. And after the SN finishes writing data, informing the MDS of the mapping relation between the path of the persistent file and the SN of the storage data block.

In DFS, SN refers to a node running SN services; generally, hardware resources of the node are different, for example, there are differences in CPU, memory, hard disk, network card, and motherboard, and in order to make the service better adapt to different hardware, the nodes of the SN may be grouped:

(1) in order to better enable the service to utilize the resources of the underlying hardware, the types of the hardware can be distinguished, and different SNs are determined according to the types of the hardware to provide services for different services.

(2) To ensure timeliness of some particular services, one or more SNs can be isolated for use by the particular service.

Fig. 2 is a method of storing a file according to the related art. As shown in fig. 2, in the conventional hierarchical storage system, SNs are grouped according to disk types, so that files of different directories are stored on different types of storage media. The flow of storing data is as follows:

1. the SN collects disk types, such as Random Access Memory (RAM), Solid State Drives (SSD), Serial Attached Small computer System Interface (SAS), Serial Advanced Technology Attachment (SATA), and reports disk types to the MDS.

2. The MDS groups SNs by disc type.

3. The MDS configures the disk type that the directory needs to store, e.g., SSD for file directory/dir 1 and SAS for directory/dir 2.

4. When the client desires to write the file, the MDS allocates the SN of the disk type corresponding to the directory to which the file belongs to the client, so that the client writes the file to the disk of the corresponding disk type. For example, the client desires to write a file to/dir 1, and the MDS allocates an SN with an SSD for the client, so that the client is serviced by the SN with an SSD medium.

The prior art can only configure the SN according to the difference of the hard disk types, and the configuration mode can cause that the utilization rate of other characteristics of the SN is low.

FIG. 3 shows an illustrative interaction flow diagram of a method for storing a file according to one embodiment of the present application. The method comprises the following steps:

301, the manager generates a mapping relationship between the directory and the tag condition. A directory may also be referred to as a file directory.

Before storing files, a user firstly installs a DFS cluster system through a manager, and respectively installs software components MDS, SN, client and the like on different nodes in a cluster. The manager installs the SN software to the OS _ HARDWARE of the node, and the node where the SN software is installed becomes the SN. The DFS comprises an MDS and a plurality of SNs, each SN has at least one characteristic, each characteristic in each SN corresponds to one label, the same characteristics in different SNs correspond to the same label, the different characteristics in different SNs correspond to different labels, and the label condition can be used for matching at least one label, so that the SN with the at least one label meets the requirement. The label condition corresponding to each directory may be configured currently by the user, may also be configured automatically by the manager, or may be configured in advance by being saved, which is not limited in the present application.

It should be understood that the software components MDS, SN and client may be installed manually by a user, or may be installed automatically and periodically, which is not limited in this application.

Optionally, after the MDS receives the mapping relationship between the directory and the tag condition, the mapping relationship can be saved, and the mapping relationship can be directly used subsequently, so that the situation that the mapping relationship is not changed is avoided and the mapping relationship is still sent to the MDS for many times, and the power consumption is reduced.

It should be understood that the mapping relationship between the directories and the label conditions, that is, each directory has a corresponding label condition, and the label conditions corresponding to different directories may be the same or different, which is not limited in the present invention.

Optionally, the manager receives the tags of all SNs associated with the MDS in the DFS, so that the user can configure tag conditions for different directories according to the received tags of the SNs and user requirements, and generate a mapping relationship between the directories and the tag conditions. Therefore, the user can configure label conditions for different catalogs according to requirements, habits and the like, and the flexibility of SN grouping configuration is improved.

Optionally, the receiving, by the manager, the tag of each SN of the plurality of SNs includes: the manager receives the label corresponding to the resource type characteristic sent by each SN; and/or the manager receives a label corresponding to the preset characteristic of each SN input by the user.

Specifically, after the manager completes the installation of the SN software, the SN collects a tag corresponding to the resource type characteristic on the OS _ hard trace through an interface provided by the operating system, and sends the tag corresponding to the collected resource type characteristic to the manager. Or the manager can directly receive the label corresponding to the preset characteristic, wherein the preset characteristic refers to the application-related characteristic according to the service type, the service requirement and the like, and the label corresponding to the preset characteristic is a user-defined label. That is, part of the tags received by the manager may be tags corresponding to the resource type characteristics reported by the SN, and part of the tags may be custom tags (tags corresponding to the preset characteristics) received from the user; or the labels received by the manager are all labels corresponding to the resource type characteristics reported by the SN; or the received tags are all custom tags input by the user.

It should be noted that, one SN collects all the labels corresponding to the resource type characteristics of all the SNs in a unified manner, and sends all the collected labels to the manager. Or, each SN may collect respective tags corresponding to a plurality of resource type characteristics, and report the respective tags respectively.

Optionally, if the SN is a resource type, the label corresponding to the resource type is any one of the following: a Central Processing Unit (CPU) label, a memory label, a hard disk label, a network label, a server type label and an Operating System (OS) type label; if the SN characteristic is a preset characteristic, the label corresponding to the preset characteristic is any one of the following: the method comprises the steps of user equipment type labels, node belonging area labels, service type labels, node operation service labels and storage data destination labels.

Specifically, each characteristic of each SN corresponds to a tag, and the tag may be a tag corresponding to a preset characteristic or a tag corresponding to a resource type characteristic. Therefore, a user can configure different label conditions according to various types of labels, and the flexibility of configuring the mapping relation between the catalog and the label conditions is improved.

The tag corresponding to the resource type characteristic may be at least one of a CPU tag, a memory tag, a hard disk tag, a network tag, a server type tag, and an OS type tag. The CPU tag may be regarded as a tag related to the CPU, and specifically may be a tag indicating a CPU type, a tag indicating a CPU core number, a tag indicating a CPU frequency, and the like, which is not limited in the present application. For example, the types of the CPU include INTEL (INTEL), Advanced Micro Devices (AMD), flash family, etc., the core number of the CPU is 32 or others, and the frequency of the CPU is 2.0 or others.

Accordingly, the memory tag may also be a tag related to the memory, specifically, a tag indicating the memory capacity (e.g., RAM 32G, RAM 64G, etc.) and a tag indicating the memory type (e.g., RAM Double data rate 4, RAM DDR3), etc., which is not limited in this application.

The hard DISK tag may also be a tag indicating a type of the hard DISK (e.g., SSD, SATA, SAS), a tag indicating a type of a DISK array (RAID Card, RAID) (e.g., RAID _ LSI), a tag indicating the number of hard DISKs (e.g., DISK _ NUM ═ 12), a tag indicating a capacity of the hard DISK (e.g., DISK _ CAP ═ 4T), and the like.

The network tag may also be a tag indicating the number of network cards (e.g., NIC _ NUM ═ 4), a tag indicating the type of network card (e.g., NIC _ ETHERNET, NIC _ IB), a tag indicating the SPEED of the network card (e.g., NIC _ SPEED ═ 10000M), or the like.

The SERVER type tag may also be a tag representing a SERVER vendor (e.g., SERVER _ IBM, SERVER _ huaboei, SERVER _ HP). Operating System (OS) types include red hat software (Redhat) (e.g., OS _ Redhat ═ 6.4, OS _ Redhat ═ 7.2), SuSE (e.g., OS _ SuSE _11SP1), ubbant (e.g., OS _ ubuntu), and so on.

It should be understood that a tag condition comprising >, < and > indicates that there are particular values that can be applied to the operators in the tag condition. For example, if the RAM has two types of memory capacities, i.e., 32G and 64G, the user can set the tag condition with >, < or ═ or the like, for example, the RAM is set to be >32G, and then the RAM having the memory capacity of 64G satisfies the tag condition.

The label corresponding to the preset characteristic may be at least one type of label among a user type label, a node belonging area label, a service type label, a node operation service label and a data storage destination label. The user can configure the label according to the requirement, thereby realizing the purpose of flexibly defining and using the storage resource.

The user type tag may be a tag indicating a user type. The user types may include customers, EMPLOYEEs, and user GROUPs, etc., represented using labels such as FOR _ VIP _ USERS, FOR _ entity _ USERS, FOR _ MANAGER _ USERS, FOR _ GROUP1, etc.

The node belonging area label may be a label indicating an area to which the node belongs. The area to which the node belongs may be divided into a RACK in which the node belongs, a DATA center to which the node belongs, and the like, and is represented by using tags of a RACK (RACK)1, a RACK2, a DATA _ CERTER1, a DATA _ CERTER2, and the like.

The service type label may be a label indicating a service type. The traffic type may be classified into a traffic using DFS, etc. FOR example, the SERVICE type is represented by a label of FOR _ BIGDATA, FOR _ USER _ INFO, FOR _ WEB _ SERVICE, FOR _ STORAGE _ SERVICE, or the like.

The service label run by the node may be a label representing the service run by the node. The services that the node runs may be differentiated by whether a service is already running on the node. For example, service tags run for the nodes respectively through tags such as HTTP _ SERVER, hbsase, ORACLE and the like.

The label of the data storage purpose may represent a label of the data storage purpose, and the data storage purpose may be classified into temporary archive, archival storage, hot spot storage, and the like. FOR example, the corresponding tags representing temporary ARCHIVE, archival storage, and hotspot storage may be FOR TEMP STORE, FOR ARCHIVE STORE, FOR HOTDATA STORE.

For example, if the user is an employee, the user may select a tag that includes a tag that is an employee when configuring the tag conditions.

It should be understood that, when the label condition refers to labels of multiple SNs, for example, the label condition is a label & & B label, and the a label and the B label may both be labels corresponding to the resource type characteristic, may both be labels corresponding to the preset characteristic, or may be one label corresponding to the resource type characteristic and the other label corresponding to the preset characteristic.

It should also be understood that for user-defined tags, which are used only for matching of tags, DFS may not be understood literally.

Optionally, in an embodiment of the present application, the target label condition includes any one of the following operators: and or; the matching operation matched with the target label condition comprises any one of the following operations: if the target label condition comprises the AND, the SN of the label with the AND on both sides is the target SN; if the target tag condition includes the or, the SN of any of the tags having the or both sides is the target SN.

The target label condition may be any one of the label conditions in the directory and label condition mapping relationship. The target tag condition includes any one of operators of (& &) or (| |), wherein & & indicates that two tags corresponding to both sides of the operator are to exist simultaneously, that is, an SN having characteristics corresponding to the two tags at the same time is taken as a target SN; if one label exists on both sides of the operator, the requirement can be met, namely the SN of any one label of the two labels can be used as the target SN.

Optionally, in an embodiment of the present application, the target label condition includes any one of the following operators: none, greater than, equal to, and less than; the matching operation matched with the target label condition comprises any one of the following operations: if the target tag condition includes the non, the SN of the tag except the SN with the non back is the target SN; if the target tag condition includes that the SN of which the value of the tag is greater than the first threshold value is the target SN; if the target tag condition includes the equality, the SN with the value of the tag equal to a second threshold is the target SN; if the target tag condition includes the value less than, the SN of the tag whose value is less than the third threshold is the target SN.

The target tag condition includes any one of a not (|), greater than (>), less than (<), and equal to (═ where | is! SN representing that the label after the operator is not included can be used as the target SN; SNs indicating that the value of the tag is greater than a preset threshold (indicated as a first threshold) may be the target SNs, SNs indicating that the value of the corresponding tag is equal to a preset threshold (indicated as a second threshold) may be the target SNs, and SNs < indicating that the value of the corresponding tag is less than a preset threshold (indicated as a third threshold) may be the target SNs.

It should be noted that the tag condition may be generated by User input, specifically, the User may use a command line for input, or may use a User Interface (UI) for pointing, which is not limited in this application.

For example, when the user clicks the input through the UI, the UI displays all the tags and the selectable dependencies for the user to select.

Optionally, in an embodiment of the present application, the target label condition includes any one or a combination of the following operators: and, or, not, greater than, equal to, less than, and parenthesis; if the target tag condition includes the small bracket, the tag in the small bracket participates in the matching operation outside the small bracket as a whole after the tag in the small bracket is preferentially matched.

Specifically, the tag condition may further include a plurality of operators, for example, the tag condition may be configured as an a tag & & B tag & & C tag, or a first preset threshold < a tag value < a second preset threshold, and the like, which is not limited in the present application.

The operator may further include "()", and when there are a plurality of operators, an object to be preferentially executed may be determined based on "()". For example, the tag condition is configured such that the a tag & (B tag | | | C tag) represents two combinations of an a tag and a B tag, and an a tag and a C tag, and the a tag & & B tag | | C tag represents two combinations of an a tag and a B tag, C tag.

It should also be understood that, according to the requirement, the user may change the mapping relationship between the configured directory and the tag condition, and store the configured mapping relationship between the directory and the tag condition, so as to facilitate subsequent reading.

302. The manager sends the mapping relationship between the directory and the tag to the MDS.

303. The client sends a request message to the MDS.

When the client needs to store data, the client sends a request message for requesting to store a target data block in a target file in a target directory to the MDS, and the MDS receives the request message sent by the client. The MDS comprises a plurality of directories, and the target directory can be any one of the directories. The target file may comprise only one target data block.

304. And the MDS determines the target label condition corresponding to the target directory according to the mapping relation between the directory and the label condition and the request message.

The MDS, upon receiving the request message, may determine the target tag condition corresponding to the target directory according to the mapping relationship between the directory and the tag condition and the target directory requested in the request message.

Optionally, the client may further carry the required tag information when sending the request message to the MDS, and the MDS may further narrow the range of the tag condition according to whether the required tag information is a subset of the target tag condition, if so, so that the file may be more accurately stored in the SN providing the high-quality service.

For example, the target tag condition is an a tag | | B tag, and if the tag information requested by the client is an a tag & & B tag, the MDS matches the SN with the a tag & & B tag.

Optionally, in an embodiment of the present application, before the MDS determines, according to the target tag condition, an SN for storing the target file, the method further includes: the MDS receives the tag sent by each SN associated with the MDS.

Specifically, after the MDS receives the mapping relationship between the directory and the tag condition, the MDS may also receive a corresponding tag sent by each SN associated with the MDS. That is, the MDS receives the tags sent by all SNs in the DFS system that are capable of serving clients. Thus, the MDS can match the SN meeting the target tag condition from all SNs according to the target tag condition.

For example, as shown in fig. 4, the client requests the target file in the first directory (which may be denoted as/dir 1) to store the target data block, and if the tag condition corresponding to the first directory in the mapping relationship sent by the manager is SSD & & NIC _ IB, the policy that the MDS stores all files in the directory/dir 1 is to put them on SNs having two tags, SSD and NIC _ IB (i.e., SNs 2 and SN4 in fig. 4).

For another example, as shown in fig. 4, the client requests to store the target file in the second directory (which may be represented as/dir 2), and if the label condition corresponding to the second directory in the mapping relationship sent by the manager is SAS | | RACK1, the policy that the MDS stores all files in the directory/dir 2 is to put them on SNs having two labels, that is, SN1, SN2, SN3, and SN4 in fig. 4, in both SAS and RACK 1.

305. The MDS determines a target SN that matches the target tag condition.

The label condition is used for matching at least one label to determine the SN, and the MDS determines the SN with at least one label matched with the target label condition according to the target label condition corresponding to the target directory. Because the label corresponds to the characteristics of the SN, the MDS can match the SN for the target file according to the characteristics of the SN, and therefore the resource utilization rate of the SN is improved.

Alternatively, if there are multiple target SNs, the target data block may be stored in each target SN.

Optionally, the target data block may select any one target SN from the multiple target SNs, and store the target data block in the target SN, or store the target data block in any one or more target SNs with a smaller load according to a load balancing principle, which is not limited in this application.

Optionally, in an embodiment of the present application, the request message further includes the target data block; the method further comprises the following steps: the MDS stores the target block of data in the target SN.

The request message sent by the client received by the MDS may further include a target data block, and after the MDS determines a target SN matched with the target tag condition, the target data block may be directly stored in the target SN.

Optionally, the method further comprises: the MDS instructs the client to store the target data block in the target SN.

After the MDS matches the SN meeting the target tag condition, indication information may be sent to the client, where the indication information may instruct the client to store the target data block in the matched target SN. After receiving the indication information, the client may establish a connection with each selected target SN through a write interface provided by the client (i.e., the client may communicate with each SN), and then write the target data block into each target SN. The indication information may be in the form of a list, for example, the MDS may list the identifiers of the matched target SNs, and send the list to the client, so that the client establishes a connection with each matched target SN according to the indication information, and writes a target data block into each matched target SN, thereby enabling file storage to obtain high-quality service, and improving resource utilization rate of the SNs.

Optionally, in an embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; wherein the MDS determining the target SN matched with the target tag condition comprises: the MDS determines the number of the target SNs from the SNs that match the target tag condition.

In particular, to ensure the reliability of the storage of the data blocks, the MDS may also maintain a copy of each data block. The number of target data blocks requested to be stored (namely, the number of copies) can also be included in the request message sent by the client received by the MDS, and among the plurality of SNs determined by the MDS to be matched with the target tag condition, the SN meeting the number of copies is selected as the target SN. The target SN can be an SN which is randomly selected from a plurality of matched SNs and accords with the number of copies; or the target SN may be an SN that meets the number of copies with a smaller load according to a load balancing principle, so that the resource utilization rate of the SN can be improved.

For example, in the above embodiment, if four of SN1, SN2, SN3, and SN4 meet the tag condition, and the number of data blocks required by the target file is 3, the MDS can select any three.

It should be understood that the target data blocks correspond to the target SNs one-to-one, i.e., the target data blocks are stored in different target SNs, respectively.

Optionally, in an embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; the method also includes the MDS determining a SN that does not match the target tag condition, determining the number of the target SN from the plurality of SNs.

If the number of SNs which are matched by the MDS and meet the target label condition is less than the number of requested target data blocks, part of target SNs in the multiple target SNs of the stored target data blocks can be any SNs in the DFS; or one SN meeting the target label condition of the MDS is not matched, the MDS can randomly select the SN meeting the copy number in all the SNs in the DFS as the target SN, or can select the SN meeting the copy number with less load according to the load balancing principle, so that the resource utilization rate of the SN can be improved.

Optionally, in an embodiment of the present application, the method further includes the MDS determining that there is no SN matching the target tag condition, determining each level of parent directory above the target directory; the MDS determines the label condition mapped with the father catalog of each level according to the mapping relation between the catalogs and the label condition; and the MDS judges whether SN matched with the label condition mapped by the father directories of each level exists from bottom to top step by step according to the upper and lower levels of the father directories of each level until the SN is matched, and determines the target SN from the matched SN.

And if the MDS does not match the SN meeting the target label condition corresponding to the target directory requested by the client, the MDS matches the upper-level directory of the target directory. The MDS can acquire the label condition corresponding to the parent directory according to the mapping relation between the directories and the label condition, and further can determine whether SN meeting the label condition corresponding to the parent directory of the target directory exists or not, and if the SN does not exist, the MDS can search the label condition corresponding to the directory of the same level as the parent directory or continue searching the parent directory above; if so, determining the SN meeting the label condition of the level directory as the target SN.

It should be understood that, in general, the tag condition of the target directory is a subset of the tag condition of the parent directory of the target directory, and the MDS searches for SNs meeting the tag condition one level up, which is equivalent to expanding the range of the tag condition; alternatively, the tag condition corresponding to the parent directory may not have a correspondence with the target directory, which is not limited in the present application.

Optionally, in an embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; the determining the target SN from the matching SNs includes the MDS determining the number of the target SNs from the matching SNs.

The MDS receives the request message sent by the client and can also comprise the number of target data blocks requested to be stored, the MDS determines a plurality of SNs meeting the conditions of the target tags according to the parent directory, and the SNs meeting the required number of the target data blocks are selected as the target SNs. The target SN can be an SN randomly selected from a plurality of matched SNs and meeting the number of required target data blocks; or the target SN may be an SN with a smaller load according to the load balancing principle and meeting the number of the required target data blocks, so that the resource utilization rate of the SN can be improved.

Optionally, after the target data block is completely written, the SN may return a write completion message to the client, and after receiving the write completion message, the client notifies, through a close interface provided by the client, that the target data block has been completely written, the MDS closes the related program, and then sends a close completion confirmation message to the client.

It should be understood that, in various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Therefore, in the method for storing a file according to the embodiment of the present application, the MDS receives a request message sent by the client for requesting the target file in the target directory to store the target data block, determines the target tag condition corresponding to the target directory according to the request message and the mapping relationship between the directory and the tag condition stored by the MDS, and determines the target SN having at least one tag matched with the target tag condition according to the target tag condition, so that the MDS can match the SN for the target file according to the tag corresponding to the characteristic of the SN, thereby improving the resource utilization rate of the SN.

FIG. 5 shows an interactive flow diagram of a method for storing files according to one embodiment of the present application. The meanings of various terms in the embodiments of the present application are the same as those of the previous embodiments.

It should be noted that this is only for helping the skilled person to better understand the embodiments of the present application, and does not limit the scope of the embodiments of the present application.

The manager receives 501 a label for a SN.

In the embodiments of the present application, three SN (SN1, SN2, and SN3, respectively) are used as examples for explanation, but the present application is not limited thereto. The manager may receive the tag reported by the SN, or may be a tag input by the user.

502, the manager generates a mapping relationship between the catalog and the label condition according to the label and the user requirement.

The manager sends 503 the mapping to the MDS.

The client sends a request message to the MDS requesting creation of a target file in the target directory 504.

The sequence of step 504 and the above steps 501 to 503 is not limited in this application.

505, the SN reports the SN label to the MDS.

The sequence of step 505 and the above steps 501 to 504 is not limited in this application.

Optionally, the MDS may store the SN tag reported by the SN, and if the tag corresponding to the preset characteristic input by the user does not change, the SN may not report the current SN tag, and at this time, the MDS may perform subsequent operations according to the stored tag.

It should be understood that the tag of the SN currently received by the MDS may overwrite the tag already saved by the MDS.

And 506, the MDS determines the target label condition corresponding to the target directory according to the request message and the mapping relation.

The MDS determines the SN that matches the target tag condition 507. For example, SN2 and SN3 satisfy the target tag condition, i.e., SN2 and SN3 are both target SNs.

Optionally, if the MDS determines that there is no SN that matches the target tag condition, the MDS may select, as the target SN, an SN of any number of copies that meets the requirement from all SNs.

508, the MDS sends indication information to the client indicating that the target file is stored to SN2 and SN 3.

509, the client establishes network connections with SN2 and SN 3.

510, the client writes the target data block into SN2 and SN3, respectively.

Alternatively, if the request message includes the target data block, the MDS may store the target data block directly into SN2 and SN3, which may eliminate steps 508 through 510 described above.

511, after the data writing is completed, the SN returns a writing completion message to the client.

The client sends a close request message to the MDS 512.

513, the MDS replies to the client with a close complete message.

It should be understood that, for the sake of brevity, detailed description is omitted here for specific indication manners of the corresponding information mentioned above with reference to the foregoing embodiments.

It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

FIG. 6 shows an interactive flow diagram of a method for storing files according to one embodiment of the present application. The meanings of various terms in the embodiments of the present application are the same as those of the previous embodiments. The process of determining, by the MDS, the SN matched with the target tag condition according to the mapping relationship between the request message and the stored file and the tag condition may be the same as the process of the embodiment shown in fig. 5, and in order to avoid repetition, detailed description is not repeated in this application.

601, the manager receives the label of the SN.

The manager generates a mapping relationship between the file and the label condition according to the label and the user requirement 602.

603, the manager sends the mapping to the MDS.

The client sends a request message to the MDS requesting storage of the target data block in the target file 604.

The sequence of step 604 and the above steps 601 to 603 is not limited in the present application.

605, the SN reports the SN tag to the MDS.

The sequence of step 605 and the above steps 601 to 604 is not limited in this application.

And 606, the MDS determines the target label condition corresponding to the target file according to the request message and the mapping relation.

The MDS determines 607 a SN that matches the target tag condition. For example, SN2 and SN3 match target tag conditions.

608, the MDS sends indication information to the client indicating that the target file is stored to SN2 and SN 3.

609, the client establishes a network connection with SN2 and SN 3.

610, the client writes the target data block into SN2 and SN3, respectively.

Alternatively, if the request message includes the target data block, the MDS may store the target data block directly into SN2 and SN3, which may eliminate steps 608 through 610 described above.

611, after the data writing is completed, the SN returns a writing completion message to the client.

The client sends a close request message to the MDS 612.

613, the MDS replies to the client with a close complete message.

Therefore, in the method for storing the file according to the embodiment of the application, the MDS receives the request message sent by the client for requesting the storage of the target data block in the target file, determines the target tag condition corresponding to the target file according to the request message and the mapping relationship between the file stored by the MDS and the tag condition, and determines the target SN having at least one tag matched with the target tag condition according to the target tag condition, so that the MDS can match the SN for the target file according to the tag corresponding to the characteristic of the SN, thereby improving the resource utilization rate of the SN.

Having described the method for storing files in detail above according to embodiments of the present application, the MDS and manager according to embodiments of the present application will be described below.

Figure 7 illustrates a schematic block diagram of a MDS700 according to an embodiment of the present application. As shown in fig. 7, the MDS700 is applied to a system including a metadata server MDS and a plurality of storage nodes SN, each of the SNs has at least one property, each of the properties in each of the SNs corresponds to a tag, the same property in the SNs corresponds to the same tag, the different property in the SNs corresponds to different tags, the MDS stores a mapping relationship between a directory and a tag condition, and the tag condition is used for matching at least one of the tags to determine the SN, the MDS700 includes:

a first receiving module 710, configured to receive a request message sent by a client, where the request message is used to request a target file in a target directory to store a target data block;

a first determining module 720, configured to determine a target tag condition corresponding to the target directory according to the mapping relationship between the directory and the tag condition and the request message received by the first receiving module 710;

a second determination module 730, configured to determine a target SN that matches the target tag condition determined by the first determination module 720, where the target SN is used to store the target data block.

Therefore, the MDS in the embodiment of the application receives a request message sent by a client for requesting a target file in a target directory to store a target data block, determines a target tag condition corresponding to the target directory according to the request message and a mapping relationship between the directory and the tag condition stored by the MDS, and determines a target SN having at least one tag matched with the target tag condition according to the target tag condition, so that the MDS can match the SN for the target file according to the tag corresponding to the characteristic of the SN, thereby improving the resource utilization rate of the SN.

Optionally, in this embodiment of the present application, the target tag condition includes any one of the following operators: and or; the matching operation matched with the target label condition comprises any one of the following operations: if the target label condition comprises the AND, the SN of the label with the AND on both sides is the target SN; if the target tag condition includes the or, the SN of any of the tags having the or both sides is the target SN.

Optionally, in this embodiment of the present application, the target tag condition includes any one of the following operators: none, greater than, equal to, and less than; the matching operation matched with the target label condition comprises any one of the following operations: if the target tag condition includes the non, the SN of the tag except the SN with the non back is the target SN; if the target tag condition includes that the SN of which the value of the tag is greater than the first threshold value is the target SN; if the target tag condition includes the equality, the SN with the value of the tag equal to a second threshold is the target SN; if the target tag condition includes the value less than, the SN of the tag whose value is less than the third threshold is the target SN.

Optionally, in this embodiment of the present application, the target label condition includes any one or a combination of the following operators: and, or, not, greater than, equal to, less than, and parenthesis; if the target tag condition includes the small bracket, the tag in the small bracket participates in the matching operation outside the small bracket as a whole after the tag in the small bracket is preferentially matched.

Optionally, in an embodiment of the present application, the MDS700 further comprises:

and the second receiving module is used for receiving the mapping relation between the catalogue and the label condition sent by the manager.

and the first processing module is used for instructing the client to store the target data block into the target SN.

Optionally, in this embodiment of the present application, the request message further includes the target data block; the MDS700 further comprises: and the second processing module is used for storing the target data block into the target SN.

Optionally, in this embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; the second determining module 730 is specifically configured to: the number of the target SNs is determined from the SNs that match the target tag condition.

Optionally, in this embodiment of the application, if the characteristic is a resource type, the tag corresponding to the resource type is any one of the following: a Central Processing Unit (CPU) label, a memory label, a hard disk label, a network label, a server type label and an Operating System (OS) type label; if the characteristic is a preset characteristic, the label corresponding to the preset characteristic is any one of the following: the method comprises the steps of user equipment type labels, node belonging area labels, service type labels, node operation service labels and storage data destination labels.

Optionally, in this embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; the MDS700 further includes a third determining module for determining the SNs that do not match the target tag condition, determining the number of the target SNs from the plurality of SNs.

Optionally, in this embodiment of the present application, the MDS700 further includes a fourth determining module, configured to determine SNs that do not match the target tag condition, and determine each level of parent directory above the target directory; a fifth determining module, configured to determine, according to a mapping relationship between the directory and the tag condition, a tag condition mapped with the parent directory at each level; and the sixth determining module is used for judging whether the SN matched with the label condition mapped by the father directory of each level exists or not step by step from bottom to top according to the upper level and the lower level of the father directory of each level until the SN is matched, and determining the target SN from the matched SN.

Optionally, in this embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; the sixth processing module is specifically configured to determine the number of the target SNs from the matching SNs.

MDS700 according to an embodiment of the present application may correspond to an MDS of a method for storing a file according to an embodiment of the present application, and the above and other operations and/or functions of each module in MDS700 are respectively for implementing corresponding flows of the foregoing methods, and are not described herein again for brevity.

Fig. 8 shows a schematic block diagram of a manager 800 according to an embodiment of the present application. As shown in fig. 8, the manager 800 includes:

a generating module 810, configured to generate a mapping relationship between a directory and a tag condition, where the tag condition is used to match at least one tag;

a sending module 820, configured to send the mapping relationship generated by the generating module to the MDS, where the mapping relationship is used for the MDS to determine a target storage node SN of a target data block stored in a target file in a target directory, and the target SN is matched with a target tag condition corresponding to the target directory.

Therefore, the manager in the embodiment of the application generates a mapping relationship between the directory and the tag condition, where the tag condition is used to match at least one tag, and sends the mapping relationship to the MDS, where the mapping relationship is used for the MDS to determine a storage node SN of a target data block stored in a target file in the target directory, and the target SN is matched with a target tag condition corresponding to the target directory, so that the MDS can match an SN for the target file according to the tag corresponding to the characteristic of the SN, thereby improving the resource utilization rate of the SN.

Optionally, in an embodiment of the present application, the manager includes: a receiving module, configured to receive the tag of each SN; the generation module is specifically configured to: and generating the mapping relation according to the label of each SN and the user requirement.

Optionally, in this embodiment of the present application, the receiving module is specifically configured to: receiving the label corresponding to the resource type characteristic sent by each SN; and/or receiving the label corresponding to the preset characteristic of each SN input by the user.

The manager 800 according to the embodiment of the present application may correspond to a manager of a method for storing a file according to the embodiment of the present application, and the above and other operations and/or functions of each module in the manager 800 are respectively to implement corresponding flows of the foregoing methods, and are not described herein again for brevity.

Fig. 9 illustrates a file storage system 900 of the present application, the file storage system 900 including:

MDS700, manager 800, SN 910, and client 920 of embodiments of the present application.

Figure 10 shows a schematic block diagram of an MDS 1000 of an embodiment of the present application. As shown in fig. 10, the MDS 1000 is applied to a system including a metadata server MDS and a plurality of storage nodes SN, each of the SNs has at least one characteristic, each of the characteristics in each of the SNs corresponds to a tag, the same characteristics in the SNs correspond to the same tag, different characteristics in the SNs correspond to different tags, the MDS stores a mapping relationship between a file and a tag condition, and the tag condition is used for matching at least one of the tags to determine the SN, the MDS 1000 includes:

a first receiving module 1010, configured to receive a request message sent by a client, where the request message is used to request a target file in a target file to store a target data block;

a first determining module 1020, configured to determine a target tag condition corresponding to the target file according to the mapping relationship between the file and the tag condition and the request message received by the first receiving module 1010;

a second determining module 1030 configured to determine a target SN matching the target tag condition determined by the first determining module 1020, the target SN being used for storing the target data block.

Optionally, in this embodiment of the present application, the target tag condition includes any one of the following operators: none, greater than, equal to, and less than; the matching operation matched with the target label condition comprises any one of the following operations: if the target tag condition includes the non, the SN of the tag other than the SN with the non-back is the target SN; if the target tag condition includes that the SN of which the value of the tag is greater than the first threshold value is the target SN; if the target tag condition includes the equality, the SN with the value of the tag equal to a second threshold is the target SN; if the target tag condition includes the value less than, the SN of the tag whose value is less than the third threshold is the target SN.

Optionally, in an embodiment of the present application, the MDS 1000 further comprises: and the second receiving module is used for receiving the mapping relation between the file and the label condition sent by the manager.

Optionally, in an embodiment of the present application, the MDS 1000 further comprises: and the first processing module is used for instructing the client to store the target data block into the target SN.

Optionally, in this embodiment of the present application, the request message further includes the target data block; the MDS further comprises: and the second processing module is used for storing the target data block into the target SN.

Optionally, in this embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; the second determining module is specifically configured to: the number of the target SNs is determined from the SNs that match the target tag condition.

Optionally, in this embodiment of the present application, the request message further includes the number of the target data blocks requested to be stored; the MDS 1000 also includes a third determining module for determining the SNs that do not match the target tag condition, determining the number of the target SNs from the plurality of SNs.

Therefore, the MDS in the embodiment of the application, by receiving a request message sent by a client for requesting storage of a target data block in a target file, determines a target tag condition corresponding to the target file according to the request message and a mapping relationship between a file stored by the MDS and the tag condition, and determines a target SN having at least one tag matched with the target tag condition according to the target tag condition, so that the MDS can match SNs for the target file according to the tags corresponding to the characteristics of the SNs, thereby improving the resource utilization rate of the SNs.

Fig. 11 shows a schematic block diagram of a manager 1100 according to an embodiment of the present application. As shown in fig. 11, the manager 1100 is applied to a system including a metadata server MDS and a plurality of storage nodes SN, each of the SNs having at least one characteristic, each of the characteristics of each of the SNs corresponding to a tag, the same characteristics of the SNs corresponding to the same tag, and different characteristics of the SNs corresponding to different tags, the manager 1100 includes:

a generating module 1111, configured to generate a mapping relationship between a file and a tag condition, where the tag condition is used to match at least one tag;

a sending module 1120, configured to send the mapping relationship generated by the generating module to the MDS, where the mapping relationship is used for the MDS to determine a target storage node SN for storing a target data block in a target file, and the target SN is matched with a target tag condition corresponding to the target file.

Therefore, the manager in the embodiment of the application, by generating a mapping relationship between a file and a tag condition, where the tag condition is used to match at least one tag, and sending the mapping relationship to the MDS, enables the MDS to determine, according to the mapping relationship, a storage node SN where a target data block is stored in a target file, where the target SN is matched with a target tag condition corresponding to the target file, and enables the MDS to match an SN for the target file according to a tag corresponding to a characteristic of the SN, thereby improving a resource utilization rate of the SN.

Fig. 12 shows a file storage system 1200 of the present application, the file storage system 1200 comprising:

MDS 1000, manager 1200, SN 1210, and client 1220 of the embodiment of this application.

Figure 13 illustrates an architecture of an MDS provided by an embodiment of the invention, which includes at least one processor 1302 (e.g., a CPU), at least one network interface 1305 or other communication interface, a memory 1306, and at least one communication bus 1303 for enabling connectivity communications between these devices. The processor 1302 is configured to execute executable modules, such as computer programs, stored in the memory 1306. The Memory 1306 may include a Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection with at least one other network element is implemented via at least one network interface 1305 (which may be wired or wireless).

In some embodiments, memory 1306 stores program 13061 and processor 1302 executes program 13061 to:

receiving a request message sent by a client through a network interface 1305, wherein the request message is used for requesting a target file in a target directory to store a target data block;

determining a target label condition corresponding to the target directory according to the mapping relation between the directory and the label condition and the request message;

and determining a target SN matched with the target label condition, wherein the target SN is used for storing the target data block.

It should be noted that the MDS may be embodied as the MDS in the above embodiment, and may be used to execute each step and/or flow corresponding to the MDS in the above method embodiment.

It can be seen from the above technical solutions provided in the embodiments of the present invention that a request message sent by a client for requesting a target file in a target directory to store a target data block is received, a target tag condition corresponding to the target directory is determined according to the request message and a mapping relationship between a directory and a tag condition stored by an MDS, and a target SN having at least one tag and matching with the target tag condition is determined according to the target tag condition, so that the MDS can match SNs for the target file according to the tag corresponding to the characteristic of the SN, thereby improving the resource utilization rate of the SNs.

Fig. 14 shows a structure of a manager provided by a further embodiment of the present invention, which includes at least one processor 1402 (e.g., CPU), at least one network interface 1405 or other communication interface, a memory 1406, and at least one communication bus 1403 for implementing connection communication between these devices. The processor 1402 is used to execute executable modules, such as computer programs, stored in the memory 1406. Memory 1406 may comprise a high-speed Random Access Memory (RAM) and may also include a non-volatile memory, such as at least one disk memory. The communication connection with at least one other network element is realized through at least one network interface 1405 (which may be wired or wireless).

In some embodiments, the memory 1406 stores the program 14061, and the processor 1402 executes the program 14061 to perform the operations of:

generating a mapping relation between a catalogue and a label condition, wherein the label condition is used for matching at least one label;

the mapping relationship is sent to the MDS through the network interface 1405, and the mapping relationship is used for the MDS to determine a target storage node SN of a target file storage target data block in a target directory, and the target SN matches a target label condition corresponding to the target directory.

It should be noted that the manager may be embodied as the manager in the foregoing embodiment, and may be configured to execute each step and/or flow corresponding to the manager in the foregoing method embodiment.

It can be seen from the above technical solutions provided in the embodiments of the present invention that by generating a mapping relationship between a directory and a tag condition, where the tag condition is used to match at least one tag, and sending the mapping relationship to an MDS, where the mapping relationship is used for the MDS to determine a storage node SN of a target data block stored in a target file in a target directory, and the target SN is matched with a target tag condition corresponding to the target directory, the MDS can match an SN for the target file according to the tag corresponding to the characteristic of the SN, thereby improving the resource utilization rate of the SN.

Figure 15 illustrates the structure of the MDS provided by another embodiment of the present invention, which includes at least one processor 1502 (e.g., a CPU), at least one network interface 1505 or other communication interface, memory 1506, and at least one communication bus 1503 for enabling the connective communication between these devices. The processor 1502 is configured to execute executable modules, such as computer programs, stored in the memory 1506. The memory 1506 may include a Random Access Memory (RAM) and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection with at least one other network element is realized via at least one network interface 1505, which may be wired or wireless.

In some embodiments, the memory 1506 stores the programs 15061, and the processor 1502 executes the programs 15061 to:

receiving a request message sent by a client through the network interface 1505, the request message being used for requesting to store a target data block in a target file;

determining a target label condition corresponding to the target file according to the mapping relation between the file and the label condition and the request message;

It can be seen from the above technical solutions provided in the embodiments of the present invention that, by receiving a request message sent by a client for requesting storage of a target data block in a target file, a target tag condition corresponding to the target file is determined according to the request message and a mapping relationship between a file stored by an MDS and the tag condition, and a target SN having at least one tag and matched with the target tag condition is determined according to the target tag condition, so that the MDS can match SNs for the target file according to the tag corresponding to the characteristic of the SN, thereby improving the resource utilization rate of the SNs.

Fig. 16 shows a structure of a manager provided by a further embodiment of the present invention, which includes at least one processor 1602 (e.g., a CPU), at least one network interface 1605 or other communication interfaces, a memory 1606, and at least one communication bus 1603 for enabling connection communication between these devices. The processor 1602 is used to execute executable modules, such as computer programs, stored in the memory 1606. The memory 1606 may include a high-speed Random Access Memory (RAM), and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection with at least one other network element is realized through at least one network interface 1605 (which may be wired or wireless).

In some embodiments, the memory 1606 stores a program 16061, and the processor 1602 executes the program 16061 for performing the following:

generating a mapping relation between the file and the label condition;

the mapping relationship is sent to the MDS through the network interface 1605, and the mapping relationship is used for the MDS to determine a target storage node SN where a target data block is stored in a target file, and the target SN matches with a target label condition corresponding to the target file.

According to the technical scheme provided by the embodiment of the invention, the mapping relation between the file and the label condition is generated, the label condition is used for matching at least one label, and the mapping relation is sent to the MDS, so that the MDS determines the storage node SN for storing the target data block in the target file according to the mapping relation, the target SN is matched with the target label condition corresponding to the target file, the MDS can match the SN for the target file according to the label corresponding to the characteristics of the SN, and the resource utilization rate of the SN is improved.

Embodiments of the present application also provide a computer storage medium that can store program instructions for instructing any one of the methods described above.

Alternatively, the storage medium may be embodied as the memory 1306, 1406, 1506, or 1606.

It should be understood that the specific examples are provided only for helping those skilled in the art to better understand the embodiments of the present application and are not intended to limit the scope of the embodiments of the present application.

It should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the unit is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for storing a file, applied to a system including a metadata server MDS and a plurality of storage nodes SN, each of the SNs having at least one characteristic, each of the characteristics in each of the SNs corresponding to a tag, the same characteristics in the SNs corresponding to the same tag, different characteristics in the SNs corresponding to different tags, the MDS storing a mapping relationship between a directory and a tag condition, the tag condition being used to match at least one of the tags to determine the SN, the directory being used to indicate a storage location of the file, the method comprising:

the MDS receives a request message sent by a client, wherein the request message is used for requesting a target file in a target directory to store a target data block;

the MDS determines a target label condition corresponding to the target directory according to the mapping relation between the directory and the label condition and the request message;

the MDS determines a target SN that matches the target tag condition, the target SN to store the target data block.

2. The method of claim 1, wherein the target tag condition comprises any one of the following operators: and or;

the matching operation matched with the target label condition comprises any one of the following operations:

if the target label condition comprises the AND, the SN of the label with the AND on both sides is the target SN;

if the target label condition includes the or, the SN of any one of the labels with the or both sides is the target SN.

3. The method of claim 1, wherein the target tag condition comprises any one of the following operators: none, greater than, equal to, and less than;

if the target tag condition includes the NOT, the SNs of the plurality of SNs other than the SN with the tag after the NOT are the target SN;

if the target tag condition includes that the SN with the tag value greater than a first threshold is the target SN;

if the target tag condition includes the equality, the SN with the value of the tag equal to a second threshold is the target SN;

and if the target label condition comprises that the SN of the label with the value smaller than a third threshold value is the target SN.

4. The method of claim 2 or 3, wherein the target label condition comprises any one or a combination of the following operators: and, or, not, greater than, equal to, less than, and parenthesis;

if the target label condition comprises the small brackets, the labels in the small brackets participate in matching operation outside the small brackets as a whole after the prior matching operation.

5. The method according to any of claims 1 to 3, wherein before the MDS determines the target tag condition corresponding to the target directory according to the mapping relationship between the directory and the tag condition and the request message, the method further comprises:

and the MDS receives the mapping relation between the directory and the label condition sent by the manager.

6. The method according to any one of claims 1 to 3, further comprising:

the MDS instructs the client to store the target data block into the target SN.

7. The method according to any one of claims 1 to 3, wherein the request message further comprises the target data block;

the method further comprises the following steps:

the MDS stores the target block of data into the target SN.

8. The method according to any one of claims 1 to 3, wherein the request message further comprises a number of the target data blocks requested to be stored;

wherein the MDS determines a target SN that matches the target tag condition, including:

the MDS determines the number of the target SNs from SNs that match the target tag condition.

9. The method according to any one of claims 1 to 3, wherein if the characteristic is a resource type, the tag corresponding to the resource type is any one of the following: a Central Processing Unit (CPU) label, a memory label, a hard disk label, a network label, a server type label and an Operating System (OS) type label;

if the characteristic is a preset characteristic, the label corresponding to the preset characteristic is any one of the following labels: the method comprises the steps of user equipment type labels, node belonging area labels, service type labels, node operation service labels and storage data destination labels.

10. The method according to any one of claims 1 to 3, wherein the request message further comprises a number of the target data blocks requested to be stored;

the method further comprises the following steps:

the MDS determines a SN that does not match the target tag condition, determining the number of the target SNs from the plurality of SNs.

11. The method according to any one of claims 1 to 3, further comprising:

the MDS determines that the SN which is not matched with the target label condition is not obtained, and determines each level of father directory on the target directory;

the MDS determines the label conditions mapped with the parent directories of each level according to the mapping relation between the directories and the label conditions;

and the MDS judges whether SN matched with the label condition mapped by each level of the father directory exists or not step by step from bottom to top according to the upper level and the lower level of each level of the father directory until the SN is matched, and the target SN is determined from the matched SN.

12. The method of claim 11, wherein the request message further includes a number of the target data blocks requested to be stored;

the determining the target SN from the matched SNs includes:

the MDS determines the number of the target SNs from the matching SNs.

13. A method for storing a file, applied to a system comprising a metadata server MDS and a plurality of storage nodes SN, each of the SNs having at least one characteristic, each characteristic of each of the SNs corresponding to a tag, the same characteristics of the plurality of SNs corresponding to the same tag, and different characteristics of the plurality of SNs corresponding to different tags, the method comprising:

the manager generates a mapping relation between a directory and a label condition, wherein the label condition is used for matching at least one label, and the directory is used for indicating the storage position of a file;

and the manager sends the mapping relation to the MDS, wherein the mapping relation is used for the MDS to determine a target SN of a target file storage target data block in a target directory, and the target SN is matched with a target label condition corresponding to the target directory.

14. The method of claim 13, wherein the target tag condition comprises any one of the following operators: and or;

15. The method of claim 14, wherein the target tag condition comprises any of the following operators: none, greater than, equal to, and less than;

16. The method of claim 14 or 15, wherein the target label condition comprises any one or a combination of the following operators: and, or, not, greater than, equal to, less than, and parenthesis;

17. The method according to any one of claims 13 to 15, characterized in that it comprises:

the manager receiving the label for each of the SNs;

the method for generating the mapping relationship between the catalogue and the label condition by the manager comprises the following steps:

and the manager generates the mapping relation according to the label of each SN and the user requirement.

18. The method of claim 17, wherein the manager receiving the label for each of the SNs comprises:

the manager receives the label corresponding to the resource type characteristic sent by each SN; and/or

And the manager receives the label corresponding to the preset characteristics of each SN input by a user.

19. The method according to any of claims 13 to 15, wherein if the property is a resource type, the label corresponding to the resource type is any of the following: a Central Processing Unit (CPU) label, a memory label, a hard disk label, a network label, a server type label and an Operating System (OS) type label;

20. A metadata server MDS, applied to a system including a metadata server MDS and a plurality of storage nodes SN, each of the SNs having at least one characteristic, each characteristic in each of the SNs corresponding to a tag, a same characteristic in the SNs corresponding to a same tag, a different characteristic in the SNs corresponding to a different tag, the MDS storing a mapping of a directory to a tag condition, the tag condition being used to match at least one of the tags to determine the SN, the directory being used to indicate a storage location of a file, the MDS comprising:

the system comprises a first receiving module, a second receiving module and a third receiving module, wherein the first receiving module is used for receiving a request message sent by a client, and the request message is used for requesting a target file in a target directory to store a target data block;

a first determining module, configured to determine a target tag condition corresponding to the target directory according to a mapping relationship between the directory and the tag condition and the request message received by the first receiving module;

a second determination module, configured to determine a target SN that matches the target tag condition determined by the first determination module, where the target SN is used to store the target data block.

21. The MDS of claim 20 wherein the target tag condition comprises any one of the following operators: and or;

22. The MDS of claim 20 wherein the target tag condition comprises any one of the following operators: none, greater than, equal to, and less than;

23. The MDS of claim 21 or 22, wherein the target tag condition comprises any one or a combination of the following operators: and, or, not, greater than, equal to, less than, and parenthesis;

24. The MDS according to any one of claims 20 to 22, further comprising:

25. The MDS according to any one of claims 20 to 22, further comprising:

a first processing module, configured to instruct the client to store the target data block in the target SN.

26. The MDS of any one of claims 20 to 22, wherein the request message further comprises the target data block;

the MDS further comprises:

and the second processing module is used for storing the target data block into the target SN.

27. The MDS of any one of claims 20 to 22, wherein the request message further comprises a number of the target data blocks requested to be stored;

the second determining module is specifically configured to:

determining the number of the target SNs from SNs that match the target tag condition.

28. The MDS of any one of claims 20 to 22, wherein if the characteristic is a resource type, the tag corresponding to the resource type is any one of: a Central Processing Unit (CPU) label, a memory label, a hard disk label, a network label, a server type label and an Operating System (OS) type label;

29. The MDS of any one of claims 20 to 22, wherein the request message further comprises a number of the target data blocks requested to be stored;

the MDS further comprises:

a third determining module for determining the SN that does not match the target tag condition, the number of the target SNs being determined from the plurality of SNs.

30. The MDS according to any one of claims 20 to 22, further comprising:

the fourth determining module is used for determining the SN which is not matched with the target label condition and determining each level of parent directory on the target directory;

a fifth determining module, configured to determine, according to a mapping relationship between a directory and a tag condition, a tag condition mapped with the parent directory at each level;

and the sixth determining module is used for judging whether SN matched with the label condition mapped by each level of the father directory exists or not step by step from bottom to top according to the upper level and the lower level of each level of the father directory until the SN is matched, and determining the target SN from the matched SN.

31. The MDS of claim 30 wherein the request message further includes a number of the target data blocks requested to be stored;

the sixth determining module is specifically configured to:

determining the number of the target SNs from the matching SNs.

32. A manager, for use in a system comprising a metadata server MDS and a plurality of storage nodes SN, each of said SNs having at least one characteristic, each characteristic in each of said SNs corresponding to a tag, the same characteristics in said plurality of SNs corresponding to the same tag, and different characteristics in said plurality of SNs corresponding to different tags, said manager comprising:

the generating module is used for generating a mapping relation between a directory and a label condition, wherein the label condition is used for matching at least one label, and the directory is used for indicating the storage position of a file;

and the sending module is used for sending the mapping relation generated by the generating module to the MDS, the mapping relation is used for the MDS to determine a target storage node SN of a target file storage target data block in a target directory, and the target SN is matched with a target label condition corresponding to the target directory.

33. The manager according to claim 32, wherein the target tag condition comprises any one of the following operators: and or;

34. The manager according to claim 33, wherein the target tag condition comprises any of the following operators: none, greater than, equal to, and less than;

35. The manager according to claim 33 or 34, wherein the target label condition comprises any one or a combination of the following operators: and, or, not, greater than, equal to, less than, and parenthesis;

36. The manager according to any of the claims 32 to 34, characterized in that it comprises:

a receiving module, configured to receive the tag of each SN;

the generation module is specifically configured to:

and generating the mapping relation according to the label of each SN.

37. The manager according to claim 36, wherein the receiving module is specifically configured to:

receiving the label corresponding to the resource type characteristic sent by each SN; and/or

And receiving the label corresponding to the preset characteristics of each SN input by a user.

38. The manager according to any of the claims 32 to 34, wherein if the property is a resource type, the tag corresponding to the resource type is any of the following: a Central Processing Unit (CPU) label, a memory label, a hard disk label, a network label, a server type label and an Operating System (OS) type label;