WO2015043305A1

WO2015043305A1 - Metadata processing method and metadata server

Info

Publication number: WO2015043305A1
Application number: PCT/CN2014/083065
Authority: WO
Inventors: 谢晓芹
Original assignee: 华为技术有限公司
Priority date: 2013-09-30
Filing date: 2014-07-25
Publication date: 2015-04-02
Also published as: CN103501339A; CN103501339B

Abstract

Embodiments of the present invention provide a metadata processing method and a metadata server. The method comprises: caching at least one operation request for the same catalog sent by a plurality of clients to a request queue, determining whether a lock of the catalog is authorized, traversing each operation request in the request queue if the lock is authorized, returning a returning result of each operation request, and processing metadata corresponding to the at least one operation request according to each returning result. In the procedure, for a plurality of concurrent operation requests under the same catalog, such as a creation operation, a deletion operation and a modification operation, the metadata server first obtains the lock of the catalog and combines, at the catalog level, the operation requests sent by the plurality of clients if the lock is authorized, so that the efficiency of processing metadata is improved.

Description

The present invention claims the priority of the Chinese patent application entitled "metadata processing method and metadata server", which is filed on September 30, 2013 by the Chinese Patent Office, application number 201310464137. The entire contents of this application are incorporated herein by reference.

TECHNICAL FIELD Embodiments of the present invention relate to communication technologies, and in particular, to a metadata processing method and a metadata server.

Background technique

A distributed file system is a file system based on a client/server model. The system includes a client and a metadata server. Metadata and metadata information are stored in the metadata server. When the client needs to manipulate the file, it must first query the metadata information of the file from the metadata server, access the metadata server according to the metadata information, and perform operations, and then send the processed metadata information to the metadata server after the operation. At present, the concurrent modification operation of a certain metadata is performed serially. For example, when a client's write operation A obtains a write lock of a certain metadata X, if there are multiple client write operations at this time {Bi } Sending a write lock request for the metadata X to the metadata server, then the write operations {Bi: are in a wait state because they are not authorized. For the client, waiting for a long time has greatly affected the efficiency of data access. In the prior art, in order to improve the efficiency of modifying metadata, the metadata server caches multiple modification operations that arrive at the same time to the loop operation queue, and stores and writes the modified data corresponding to the last operation in the queue, thereby to a certain extent Improve data access efficiency. However, the above technical solution is only a concurrent modification operation for the overlay attribute of the same metadata, such as the time of last modification of the file or directory (mt ime ), the time of the last access to the file or directory (at ime ), etc. . However, in addition to the concurrent modification of the overlay attribute of the same metadata, the processing of the metadata includes modification of the extended attribute of the same metadata, such as an access control list (Aces s Control L ist , ACL ) entry. Add, delete, etc., and in the same In the directory, concurrently create multiple metadata operations, multiple concurrent modification operations on multiple metadata in the same directory, etc. At this time, only serial execution mode can be used, resulting in long response time and completion in unit time. The operation request is small, the degree of concurrency is not high, and the processing efficiency is low. SUMMARY OF THE INVENTION Embodiments of the present invention provide a metadata processing method and a metadata server to improve processing efficiency of metadata. The first aspect of the present invention provides a metadata processing method, including: buffering, by a plurality of clients, at least one operation request for the same directory to a request queue; determining whether the lock of the directory is authorized; The lock is authorized to traverse each operation request in the request queue, returning a return result of each operation request; processing metadata corresponding to the at least one operation request according to each of the returned results . In a first possible implementation manner of the first aspect, if the lock is authorized, traversing each operation request in the request queue, and returning a return result of each operation request, includes: For each operation request, determining a category of the operation request, the category includes: a first category, a second category, and a third category, where the first category includes a read operation on the metadata, the second The category includes a new operation, a modification operation, and a deletion operation on the metadata, and the third category includes a movement operation on the metadata, a new link operation, and a delete link operation; and returning the operation request according to the category to which the operation request belongs Corresponding return results. With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the returning result corresponding to the operation request is returned according to a category to which the operation request belongs, The method includes: if the operation request belongs to the first category, returning the operation request as a return result of the read request. With reference to the first possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the returning result corresponding to the operation request is returned according to a category to which the operation request belongs, The method includes: if the operation request belongs to the second category, determining whether the operation request is legal for the operation of the metadata; if not, returning the operation request as a return result of the error request; otherwise, if legal, Determining whether there is an operation request of the third category in the merge queue, where the merge queue includes an operation request corresponding to the mergeable return result; if the operation request of the third category exists in the merge queue, returning to the The operation request is a return result of the mutual exclusion request; otherwise, if the operation request of the third category does not exist in the merge queue, the operation request of the second category is added to the merge queue, Metadata of the second category of operation request operations is added to the buffered data queue, and the operation request is returned as a mergeable request Results. With reference to the first possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the returning result corresponding to the operation request is returned according to a category to which the operation request belongs, The method includes: if the operation request belongs to the third category, determining whether the operation request is legal for the operation of the metadata; if not, returning the operation request as a return result of the error request; otherwise, if legal, Determining whether there is an operation request of the second category in the merge queue, or an operation request of the third category, where the merge queue includes an operation request corresponding to the mergeable return result; if the merge queue exists The operation request of the second category, or the operation request of the third category, returns the return result of the operation request as a mutual exclusion request; Otherwise, if the operation request of the second category does not exist in the merge queue, and the operation request of the third category does not exist, the operation request of the third category is added to the merge queue. The metadata of the third category of operation request operations is added to the buffered data queue, and the result of the operation is returned as a result of the mergeable request. With reference to the second possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, the performing, according to each of the returning results, performing metadata corresponding to the at least one operation request Processing, comprising: moving the operation request out of the request queue; the port responds to the operation request. In conjunction with the third or fourth possible implementation of the first aspect, in a sixth possible implementation manner of the first aspect, if the operation request is returned as an error request, And processing, according to each of the returned results, the metadata corresponding to the at least one operation request, including: moving the operation request out of the request queue; and responding to the operation request. With reference to the third or fourth possible implementation of the first aspect, in a seventh possible implementation manner of the first aspect, if the operation request is returned as a return result of the mutual exclusion request, And processing, according to each of the returned results, the metadata corresponding to the at least one operation request, including: saving the operation request in the request queue. With reference to the third, fourth or seventh possible implementation of the first aspect, in an eighth possible implementation manner of the first aspect, Processing, by the metadata corresponding to the operation request, the metadata in the buffered data queue is sent to the storage node, so that the storage node writes the metadata to obtain a transaction result; And according to the result of the transaction, the corresponding client sends an operation port response to the operation request in the merge queue. In conjunction with the eighth possible implementation of the first aspect, in a ninth possible implementation manner of the first aspect, the requesting, by the transaction result, the corresponding client to the operation in the merge queue After sending an action response, it also includes:

Determining whether there is still an operation request in the request queue; if present, traversing each operation request in the request queue, returning a return result of each operation request; otherwise, if not, releasing the lock . In a second aspect, the embodiment of the present invention provides a metadata server, including: a request queue module, configured to cache at least one operation request for the same directory sent by multiple clients to a request queue; Determining whether the lock of the directory is authorized; the merge decision module, configured to: if the directory lock module determines that the lock is authorized, traverse each operation request in the request queue, and return each operation request The result of the return; the request processing module is configured to process the metadata corresponding to the at least one operation request according to each of the returned results.

In a first possible implementation manner of the second aspect, the merging decision module determines, for each operation request, a category of the operation request, where the category includes: a first category, a second category, and a third a category, where the first category includes a read operation on the metadata, the second category includes a new operation, a modify operation, and a delete operation on the metadata, and the third category includes a move operation on the metadata, and a new The link operation and the delete link operation return a return result corresponding to the operation request according to the category to which the operation request belongs.

Combining the first possible implementation of the second aspect, the second possible aspect of the second aspect In an implementation manner, the merge decision module is configured to: if the operation request belongs to the first category, return the operation request as a return result of the read request.

With reference to the first possible implementation of the second aspect, in a third possible implementation manner of the second aspect, the merging decision module is configured to: if the operation request belongs to the second category, Determining whether the operation request is legal for the operation of the metadata, and if it is determined that the operation request is invalid for the operation of the metadata, returning the operation request as a return result of the error request; otherwise, if the operation request is determined If the operation of the metadata is legal, it is determined whether there is an operation request of the third category in the merge queue, where the merge queue includes an operation request corresponding to the mergeable return result, and if it is determined that the The operation request of the three categories returns the return result of the operation request as a mutual exclusion request;

Otherwise, if it is determined that the operation request of the third category does not exist in the merge queue, the request processing module is configured to add the operation request of the second category to the merge queue, where the The metadata of the two types of operation request operations is added to the buffer data queue, and the merge decision module is configured to return the return result of the operation request as a mergeable request.

With reference to the first possible implementation of the second aspect, in a fourth possible implementation manner of the second aspect, the merging decision module is configured to: if the operation request belongs to the third category, Determining whether the operation request is legal for the operation of the metadata, and if it is determined that the operation request is invalid for the operation of the metadata, returning the operation request as a return result of the error request; otherwise, if the operation request is determined If the operation of the metadata is legal, it is determined whether the operation request of the second category exists in the merge queue, or the operation request of the third category, the merge queue includes an operation request corresponding to the mergeable return result; Determining that the operation request of the second category exists in the merge queue, or the operation request of the third category returns a return result of the operation request as a mutual exclusion request; otherwise, if the operation queue is in the merge queue There is no operation request of the second category, and if there is no operation request of the third category, the third category is Be added to the combined request queue The metadata of the operation request operation of the third category is added to the buffer data queue, and the result of the operation is returned as a result of the mergeable request.

With reference to the second possible implementation of the second aspect, in a fifth possible implementation manner of the second aspect, the request processing module is configured to remove the operation request from the request queue, and The operation request should be as described.

With reference to the third or fourth possible implementation of the second aspect, in a sixth possible implementation of the second aspect, the request processing module is configured to remove the operation request from the request Queue, and respond to the operation request.

In conjunction with the third or fourth possible implementation of the second aspect, in a seventh possible implementation of the second aspect, the request processing module is configured to save the operation request in the In the request queue.

In conjunction with the third, fourth, or seventh possible implementation of the second aspect, in an eighth possible implementation of the second aspect, the request processing module is configured to: The metadata in the data queue is sent to the storage node, so that the storage node writes the metadata to obtain a transaction result;

The metadata server further includes a sending module, configured to send an operation response to the corresponding client in the operation queue in the merge queue according to the transaction result.

In conjunction with the eighth possible implementation of the second aspect, in a ninth possible implementation manner of the second aspect, the directory lock module is further configured to determine whether an operation request still exists in the request queue;

The merge decision module is further configured to: if the directory lock module determines that an operation request still exists in the request queue, traverse each operation request in the request queue, and return a return result of each operation request ;

The request processing module is further configured to: if the directory lock module determines that the request queue does not exist Upon operation of the request, the lock is released.

Multiple concurrent operation requests in the same directory, such as a new operation, a delete operation, a modification operation, etc., first acquire a lock of the directory, and if the lock is authorized, perform a directory level merge on the operation requests sent by multiple clients, thereby Improve the efficiency of processing metadata.

BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. The drawings are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive labor.

1 is a flowchart of Embodiment 1 of a metadata processing method according to the present invention;

2A is a schematic diagram of a main process 1 of MDS processing in a metadata processing method according to the present invention;

2B is a schematic diagram of a sub-process 2 in the metadata processing method of the present invention; FIG. 2C is a schematic diagram of a sub-process 3 in the metadata processing method of the present invention; FIG. 3 is a schematic structural diagram of a first embodiment of the metadata server according to the present invention; A schematic diagram of the structure of the second embodiment of the metadata server of the present invention.

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. The embodiments are a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without departing from the inventive scope are the scope of the present invention. FIG. 1 is a flowchart of Embodiment 1 of a metadata processing method according to the present invention. The execution subject of this embodiment is a metadata server. Specifically, the embodiment includes the following steps: 101. Cache at least one operation request sent by multiple clients to the same directory to the request queue.

The distributed file system includes a plurality of metadata servers (Metadata Servers, MDS), and all metadata is assigned to the specified metadata server according to the metadata identifier of its home directory to ensure metadata under the directory, for example The operation of subdirectory items (including directories and files, etc.) is handled by the MDS, facilitating the consolidation of directory-level metadata operations. In this step, multiple clients (CLIENT) send an operation request, and the metadata identifier of the metadata home directory operated by the case operation request caches at least one operation request of the same directory to the request queue.

For example, the directory is /mountfs/2013, and its metadata identifier is dirFIDl. The existing files in this directory include:

I mountfs/2013/fileOOl, the metadata identifier of the directory fileOOl is FID1;

I mountfs/2013/file002, the metadata of the directory f i le002 is FID2;

I mountfs/2013/file003, the metadata identifier of the directory f i le003 is FID3;

I mountfs/2013/file004, the metadata of the directory f i le004 is FID4. The above existing files can be expressed as {dirFID existing metadata} according to the metadata identifier. The operation requests belonging to /mountfs/2013 are processed by the metadata server corresponding to the directory whose metadata identifier is dirFIDl, that is, the operation requests of multiple clients concurrently arrive at the request queue of the metadata server. E.g:

CLIENT1. create (dirFID, newfile005 ), which means that the new file newf ile005 is in the /mountf s/2013 directory;

CLIENT2. create (dirFID, newfile006 ), which means that the new file newfile006 is created in the /mountf s/2013 directory.

CLIENT3. unlink (dirFID, newfileOOl), which means to delete the file fileOOl in the /mountf s/2013 directory; CLIENT4. unl ink ( di rFID, newf i le002 ), which means to delete the file fi le002 in the /mountf s/2013 directory;

CLIENT5. setat tr ( FID3 , f i le003 , Add ACL ), indicating to /mountfs/2013/f i le003 , adding ACL; CLIENT6. setat tr ( FID3 , f i le004 , Remove ACL ), indicating pair

/mountfs/2013/f i le004 , delete the ACL.

102. Determine whether the lock of the directory is authorized. In the distributed file system, the concept of locks was introduced. If the client wants to operate a certain metadata, for example, a directory or subdirectory, etc., it needs to apply for the lock of the metadata first, so that multiple clients can prevent the data from being modified and read inconsistently when operating the metadata. Sex. In this step, the metadata server corresponding to the directory determines whether the lock of the directory is authorized, and performs subsequent processing according to the result of the lock authorization. If the lock of the directory is not authorized, the metadata corresponding to the operation request is not processed; otherwise, if the lock corresponding to the directory is authorized, step 103 is performed. The metadata server can determine whether the lock of the directory is authorized. For details, refer to the related art, and details are not described herein. It should be noted that the execution of 101 and 102 above is not strictly sequential.

103. If the lock is authorized, traverse each operation request in the request queue, and return the return result of each operation request. If the metadata server determines that the lock of the directory has been authorized, it traverses each operation request in the request queue, returns a return result of each operation request according to a preset rule, etc., and the return result indicates processing of the operation request by the metadata server This way, for example, moves the operation request out of the request queue, continues to be saved in the request queue, responds to the operation request, and so on.

104. Process metadata corresponding to the at least one operation request according to each return result. In this step, the metadata server processes the metadata corresponding to each operation request according to the return result corresponding to each operation request. For example, if the result of the return is to move the operation request out of the request queue, then If the operation request is a read request, the client that sends the read request responds, and if the operation request is an error request, the response to the client that sent the operation request fails, and continues to traverse the request queue; if the returned result is mergeable, The operation request is added to the merge queue, and the metadata corresponding to the operation request in the merge queue is added to the buffer data queue, so that after the request queue is traversed, the merged metadata in the buffer data queue is submitted to the storage at one time. The node writes the disk. The metadata processing method provided by the embodiment of the present invention, the metadata server requests multiple concurrent operations in the same directory, such as a new operation, a delete operation, a modification operation, etc., first acquires a lock of the directory, and if the lock is authorized, Operation requests sent by multiple clients are merged at the directory level, thereby improving the efficiency of processing metadata. 2A is a schematic diagram of a main process 1 of MDS processing in the metadata processing method of the present invention. Hereinafter, for the sake of clarity of description, the process is referred to as [main process 1-MDS processing main process]. The execution subject of this embodiment is a metadata server. Specifically, the embodiment includes the following steps:

201. The metadata server caches at least one operation request to the request queue. For a certain directory, such as di rFID, after the specified metadata server receives the operation request of the multiple clients for the directory, the operation request is cached to the request queue of the directory, for example, {dirFID request queue}, if the subsequent continues Upon receiving an operation request for the directory dirFID, the operation requests are further cached to the {dirFID request queue}.

202. The metadata server acquires a lock of the directory and determines whether the lock is authorized. The metadata server corresponding to the directory determines whether the lock of the directory is authorized. If the lock is not authorized, execute 203; otherwise, if the lock is authorized, execute 204.

203. Waiting for processing. If the metadata server determines that the lock is not authorized, it waits for processing.

204. Execute [sub-process 2 - request merge sub-process]. If the metadata server determines that the lock is authorized, execute [Sub-Process 2 - Request Merge Sub-Process]. In this step, the MDS traverses each operation request in the {di rFID request queue}, and processes each operation request according to the [sub-process 2 - request merge sub-process], and returns the return result of each operation request. [Sub-process 2 - request merge sub-process] can be seen in FIG. 2B.

205. Process metadata corresponding to the operation request according to the returned result. Specifically, this step includes the following optional sub-steps (not shown in the figure):

2051. The result of the read request. If the result of the return is a read request (REQ_IS_READ), the operation request corresponding to the returned result is moved out of the {dirFID request queue}, and if the operation request is hit in the cache of the metadata server, the client that initiated the operation request is directly Respond; otherwise, if it does not, then the storage node responds to the client after reading the operation request, and continues to traverse the {di rFID request queue}.

2052. The result of the error request. If the result of the return is an error request (REQ_ IS_ERR0R), the originating client of the operation request corresponding to the returned result responds with an error and continues to traverse the {di rFID request queue}.

2053. The result of the mergeable return. If the returned result is mergeable (REQ_ IS_MERGED ), the operation request corresponding to the returned result is added to the merge queue, for example, {di rFID merge queue}, and the metadata operated by the operation request is added to the buffer data queue. Medium, such as {dirFID buffered data queue}, continues to traverse {dirFID request queue}. The metadata operated by the operation request is, for example, new metadata, deleted original data, modified metadata, and the like.

2054, the result of mutual exclusion. If the result of the return is mutual exclusion (REQ_IS_MUTEXED), the operation request corresponding to the returned result is saved in the {di rFID request queue}, the loop is exited, and step 205 is executed.

2055, traversal is completed. The metadata server determines whether to traverse the {dirFID request queue}, if it has not been traversed, then Continue the loop; otherwise, if the traversal is completed, exit the loop and go to step 205.

206. Send metadata in the buffered data queue to the storage node. The metadata server sends the data in the buffered data queue to the storage node at one time, so that the storage node guarantees the atomicity of multiple metadata writes through the transaction mechanism, waiting for the transaction result.

207. Send an operation response to the client corresponding to the operation request in the merge queue. After the metadata server waits for the result of the transaction, it requests all operations in the {dirFID merge queue} and responds according to the result of the transaction. For example, if the transaction is successful, a successful response is sent; otherwise, if it fails, a failure response is sent. After the client response to all operation requests in the {dirFID merge queue} is completed, the merge operation request ends.

208. Determine whether an operation request still exists in the request queue. The metadata server continues to check the {dirFID request queue}, and if there are still operation requests waiting in the {dirFID request queue}, it returns 204; otherwise, it executes 209.

209. Release the lock. If there is no pending operation request in the {dirFID request queue}, the lock is released. 2A is a detailed description of the present invention from the perspective of [main process 1-MDS processing main process]. Next, the [sub-process 2 - request merge sub-process] in step 203 will be described in detail. In the embodiment of the present invention, the operation request for the metadata may be classified into the following three categories, the first category: a read operation on the metadata; the second category: a new operation, a modification operation, and a deletion operation on the metadata; : the operation operation of the metadata, the new link operation, and the deletion, wherein the operation request of the first category and the operation request of the second category are mutually exclusive, that is, the operation of the first category in the process of traversing the {di rFID request team 歹W The request, if the second category of operation request already exists in the {di rFID merge queue}, does not add the operation request currently being traversed to the {dirFID merge queue}, but returns the return result of the mutual exclusion request (REQ_IS_MERGED) Similarly, if the operation request currently traversed belongs to the second category, and the operation request of the first category or the second category already exists in the {dirFID merge queue}, the operation request is not taken. The second category of operation requests that were previously traversed are added to the {di rFID merge queue}, and the result of the mutex request is returned. Specifically, refer to Table 1, which is a classification table of operation requests of the present invention. Table 1

Optionally, in the embodiment of the present invention, for the operation request in the first category, there is no need to merge; for each operation request of the first category, try to merge; for the operation request of the third category, since it may involve at least two directories The operation is therefore not merged. For details, refer to FIG. 2B. FIG. 2B is a schematic diagram of sub-process 2 in the metadata processing method of the present invention, that is, a schematic diagram of [sub-process 2-request merge sub-process]. Referring to FIG. 2B, the embodiment includes the following steps:

301. For each operation request, return a corresponding return result.

Specifically, this step includes the following optional sub-steps:

3011. Return a corresponding return result for the operation request belonging to the first category. In the process of traversing {di rFID request queue ,', for an operation request belonging to the first category, the result of the return is a read request (REQ_ IS_READ ).

3012. Return a corresponding return result for the operation request belonging to the second category. In the process of traversing the {di rFID request queue ,W, for the operation request belonging to the second category, the [sub-process 3 - request legality judgment sub-process] is executed. In the process of traversing the {di rFID request queue 歹W, for the operation request belonging to the second category, the metadata server calls [sub-process 3 - request legality judgment sub-process] to judge the legality of the operation request, according to whether it is legal or not, Returning different return results, wherein [Sub-Process 3 - Request Legitimacy Judgment Sub-Process] Referring to FIG. 2C, the legality judging process for the operation request belonging to the second category can be referred to. Specifically, if it is not legal, the operation result is returned as an error request (REQ_IS_ERROR); if it is legal, it is determined whether there is an operation request belonging to the third category in the {dirFID merge queue}. If the {dirFID merge queue} already has an operation request belonging to the third category, since the operation requests of the first category and the second category are mutually exclusive, the operation result is returned as a result of the mutual exclusion request (REQ_IS_MUTEXED); otherwise, if { dirFID merge team 歹 'There is no operation request belonging to the third category, then the traversal operation request belonging to the second category is added to the {di rFID merge queue}, and the metadata corresponding to the operation request is added to the {di rFID buffers the data queue} and returns the result of the operation as a result of the mergeable request (REQ_ IS_MERGED). For example, if you first traverse the {dirFID request queue}, the first traversed operation request belongs to the second category. At this time, {dirFID merge queue 歹' J} and {di rFID buffer data queue 歹'J} are both Empty, that is, there is no operation request belonging to the second category in {dirFID merge queue}. At this time, the current operation request is added to the {dirFID merge queue}, and the metadata of its operation is added to the {dirFID buffer data queue. }, and return the result of the operation as a result of the mergeable request (REQ_ IS_MERGED ), continue to traverse. If the operation request traversed subsequently belongs to the third category, the operation request is returned as a return result of the mutual exclusion request (REQ-IS-MERGED); otherwise, if the traversed operation request belongs to the second category, the operation request is continuously added. Go to {dirFID merge queue} and continue the process.

3013. Return a corresponding return result for the operation request belonging to the third category. In the process of traversing the {di rFID request queue ,, for the operation request belonging to the third category, the [sub-process 3 - request legality judgment sub-process] is executed. In the process of traversing {di rFID request queue ,, for the operation request belonging to the third category, the metadata server calls [sub-process 3 - request legality judgment sub-process] to judge the legality of the operation request, root According to the lawfulness or not, different return results are returned, wherein [Sub-Process 3 - Request Legitimacy Judgment Sub-Process] can refer to the legality judging process of the operation request belonging to the third category in FIG. 2C. Specifically, if it is not legal, the operation result is returned as an error request (REQ_IS_ERROR); if it is legal, it is determined whether there is an operation request belonging to the second category or the third category in the {dirFID merge queue}. If there is an operation request belonging to the second category or the third category in the {dirFID merge queue}, the return operation request is a return result of the mutual exclusion request (REQ-IS-MUTEXED), otherwise, if the {dirFID merge queue} does not exist, The operation request of the second category or the third category adds the current operation request to the {dirFID merge queue}, adds the metadata of its operation to the {dirFID buffer data queue 歹' J}, and returns the operation request. Returns the result of the mergeable request (REQ_ IS_MERGED ). For example, if the first traversed operation request belongs to the third category when traversing the {dirFID request queue} at the beginning, then {dirFID merge queue} and {di rFID buffer data queue} are both empty, ie {dirFID merge There are no operation requests belonging to the second category or the third category in the queue}. At this point, the current operation request is added to the {di rFID merge queue}, the metadata of its operation is added to the {dirFID buffer data queue}, and the return result of the operation request as the mergeable request (REQ_ IS_MERGED) is returned. , continue to traverse. If the operation request traversed subsequently belongs to the second category or the third category, the return operation request is a return result of the mutual exclusion request (REQ_IS_MUTEXED). The above FIG. 2A illustrates the present invention from the perspective of [main process 1-MDS processing main process], FIG. 2B is from [sub-process 2 - request merge sub-process], and below, in the steps 3012 and 3013 Process 3 - Request Legitimacy Judgment Sub-Process] is described in detail. For details, refer to FIG. 2C. FIG. 2C is a schematic diagram of sub-process 3 in the metadata processing method of the present invention, that is, a schematic diagram of [sub-process 3 - request legality judging sub-process]. Referring to FIG. 2C, the embodiment includes the following steps:

401. Determine the legality of the operation request belonging to the second category. Optionally, this step may include:

(1) For the setta ttr request, the return is legal; (2) For operation requests such as create/mkdir/symlink/mknod, the metadata corresponding to the operation request and the metadata existing in the directory, such as {dirFID existing metadata} and {dirFID buffer data queue} The list of directory entry names is matched, and whether it is legal or not depends on whether it is a duplicate name. If there is no duplicate name, it returns legal; otherwise, if it is duplicated, the return is invalid. For example, /mountfs/2013/f ileOOl, fileOOl exists in the list of directory entry names for the {dirFID existing metadata} of the directory /mountf s/2013. In this case, if CLIENT1. create ( dirFID, newfileOOl ), it means / The new file newf i leOOl is created in the mountf s/2013 directory. At this time, due to the duplicate name, the return is invalid.

(3) For the unlink request, match the directory entry name corresponding to the deletion with the directory entry name list in the {dirFID existing metadata} to determine whether the deleted name exists. If it exists, it returns legal; otherwise, if it does not exist, the return is invalid.

(4) Matching the rmdir operation request, matching the deleted directory entry corresponding to the operation request with the directory entry name list in the {dirFID existing metadata}, determining whether the deleted name exists and is an empty directory. If it exists, and it is an empty directory, it returns legal; otherwise, if it does not exist, or is not empty, the return is invalid.

402. Judging the legality of the operation request belonging to the third category. Optionally, this step may include:

(1) A request for a move operation, such as a rename request, matching the renamed directory entry corresponding to the operation request with a list of directory entry names in {dirFID existing metadata}, determining whether the original, ie, the old name exists, If it exists, it returns legal; otherwise, if it does not exist, it returns illegal.

(2) Matching the link request, matching the newly created directory entry name with the directory entry name list in {dirFID existing metadata}, determining whether the newly created directory entry exists, and if it exists, returning is invalid. Otherwise, if it exists, it returns legal.

FIG. 3 is a schematic structural diagram of Embodiment 1 of a metadata server according to the present invention. The number of elements provided in this embodiment According to a server, it is possible to implement the various steps of the method applied to the metadata server provided by any of the embodiments of the present invention. Specifically, the metadata server 100 provided in this embodiment specifically includes: a request queue module 11 configured to cache at least one operation request for the same directory sent by the multiple clients to the request queue; the directory lock module 12 is configured to determine Whether the lock of the directory is authorized; the merge decision module 13 is configured to: if the directory lock module 12 determines that the lock is authorized, traverse each operation request in the request queue, return a return result of each operation request; request processing module 14 And for processing metadata corresponding to the at least one operation request according to each returned result. The metadata server provided by the embodiment of the present invention acquires the lock of the directory by using multiple concurrent operation requests in the same directory, such as a new operation, a delete operation, a modification operation, etc., and if the lock is authorized, the multiple clients are The operation request sent by the terminal performs directory level merging, thereby improving the processing efficiency of the metadata. Further, the merge decision module 13 determines, for each operation request, a category of the operation request, the category includes: a first category, a second category, and a third category, wherein the first category includes a read operation on the metadata, The second category includes a new operation, a modification operation, and a deletion operation on the metadata, and the third category includes a movement operation on the metadata, a new link operation, and a delete link operation, and returns a return result corresponding to the operation request according to the category to which the operation request belongs. Further, the merge decision module 13 is configured to return an operation request as a return result of the read request if the operation request belongs to the first category. Further, the merge decision module 13 is configured to determine, if the operation request belongs to the second category, whether the operation request is legal for the operation of the metadata, and if it is determined that the operation request is invalid for the operation of the metadata, the operation request is returned as an error. The result of the request is returned; otherwise, if it is determined that the operation request is legal for the operation of the metadata, it is determined whether the merge queue is stored. In the operation request of the third category, the merge queue includes an operation request corresponding to the mergeable return result, and if it is determined that the operation request of the third category exists in the merge queue, the return operation request is a return result of the mutual exclusion request; otherwise, if If it is determined that there is no operation request of the third category in the merge queue, the request processing module is requested.

14 is used to add the operation request of the second category to the merge queue, add the metadata of the operation request operation of the second category to the buffer data queue, and the merge decision module 13 is configured to return the operation request as the mergeable request. Returns the result. Further, the merge decision module 13 is configured to determine, if the operation request belongs to the third category, whether the operation of the operation request is legal, and if it is determined that the operation request is invalid for the operation of the metadata, the operation request is returned as an error. If the operation request determines that the operation of the metadata is legal, then it is determined whether there is a second category of operation request in the merge queue, or the third category of operation request, and the merge queue includes the mergeable return result. Corresponding operation request; if it is determined that there is an operation request of the second category in the merge queue, or an operation request of the third category, the operation request is returned as a return result of the mutual exclusion request; otherwise, if there is no second in the merge queue The operation request of the category, and, if there is no operation request of the third category, adding the operation request of the third category to the merge queue, adding the metadata of the operation request operation of the third category to the buffer data queue, and The result of the return operation is the result of the mergeable request. Further, the request processing module 14 is configured to move the operation request out of the request queue and respond to the operation request.

Further, the request processing module 14 is configured to move the operation request out of the request queue and respond to the operation request.

Further, the request processing module 14 is configured to save the operation request in the request queue. Further, the request processing module 14 is configured to send the metadata in the buffered data queue to the storage node, so that the storage node writes the metadata to obtain the transaction result, and requests the operation in the merge queue according to the transaction result. The corresponding client sends an action response. Further, the directory lock module 12 is further configured to determine whether an operation request still exists in the request queue. The merge decision module 13 is further configured to: if the directory lock module 12 determines that an operation request still exists in the request queue, traverse the request queue. Each operation request returns a return result of each operation request; the request processing module 14 is further configured to release the lock if the directory lock module 12 determines that there is no operation request in the request queue.

4 is a schematic structural diagram of Embodiment 2 of a metadata server according to the present invention. As shown in FIG. 4, the metadata server 200 of this embodiment includes: a memory 21 and a processor 22; a memory 21, code for storing a metadata processing method; and a processor 22 for using the code stored in the memory 21, At least one operation request sent by the multiple clients to the same directory is cached to the request queue; determining whether the lock of the directory is authorized; if the lock is authorized, traversing each operation request in the request queue, returning the return of each operation request Result: processing metadata corresponding to the at least one operation request according to each returned result. Optionally, the processor 22 is configured to determine, if the lock of the directory is authorized, a category of the operation request for each operation request, the category includes: a first category, a second category, and a third category, where the first category Including the reading operation on the metadata, the second category includes a new operation, a modification operation, and a deletion operation on the metadata, and the third category includes a movement operation on the metadata, a new link operation, and a delete link operation; according to the category to which the operation request belongs , returns the result returned by the operation request. Optionally, the processor 22 is configured to: if the operation request belongs to the first category, return an operation request as a return result of the read request. Optionally, the processor 22 is configured to: if the operation request belongs to the second category, determine whether the operation request is legal for the operation of the metadata, and if not, return the operation request as a return result of the error request; Then, if it is legal, it is determined whether there is an operation request of the third category in the merge queue, and the merge queue includes an operation request corresponding to the mergeable return result; if there is an operation request of the third category in the merge queue, the operation request is returned as mutual The result of the request is rejected; otherwise, if there is no operation request of the third category in the merge queue, the operation request of the second category is added to the merge queue, and the metadata of the operation request operation of the second category is added to the buffer. In the data queue, and return the result of the operation as a result of the mergeable request. Optionally, the processor 22 is configured to: if the operation request belongs to the third category, determine whether the operation request is legal for the operation of the metadata; if not, return the operation request as a return result of the error request; otherwise, if legal, Determining whether there is a second category of operation request in the merge queue, or an operation request of the third category, the merge queue includes an operation request corresponding to the mergeable return result; if there is a second category of operation request in the merge queue, or The third category of operation request returns the operation request as the return result of the mutual exclusion request; otherwise, if there is no operation request of the second category in the merge queue, and there is no operation request of the third category, the third category is The operation request is added to the merge queue, the metadata of the operation operation of the third category is added to the buffer data queue, and the result of the operation is returned as a result of the mergeable request.

Optionally, the processor 22 is configured to move the operation request out of the request queue and respond to the operation request. Optionally, the processor 22 is configured to: if it is not legal, return the operation request as a return result of the error request, then move the operation request out of the request queue, and respond to the operation request.

Optionally, the processor 22 saves the operation request in the request queue if the return operation request is a return result of the mutual exclusion request. Optionally, the processor 22 is configured to send the metadata in the buffered data queue to the storage node, so that the storage node writes the metadata to obtain a transaction result, and requests the operation in the merge queue according to the transaction result. The corresponding client sends an action response.

Optionally, the processor 22 is configured to determine whether an operation request still exists in the request queue; if yes, traverse each operation request in the request queue, and return a return knot of each operation request If not, if it does not exist, release the lock. It will be understood by those skilled in the art that all or part of the steps of implementing the above method embodiments may be performed by hardware related to the program instructions. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps including the foregoing method embodiments are performed; and the foregoing storage medium includes: various media that can store program codes, such as ROM, RAM, disk or optical disk. It should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. range.

Claims

Rights request

1. A metadata processing method, characterized by including:

Cache at least one operation request for the same directory sent by multiple clients to the request queue;

Determine whether the lock of the directory is authorized; if the lock is authorized, traverse each operation request in the request queue and return the return result of each operation request;

According to each of the returned results, the metadata corresponding to the at least one operation request is processed.

2. The method according to claim 1, characterized in that if the lock is authorized, each operation request in the request queue is traversed and the return result of each operation request is returned, including:

For each operation request, determine the category of the operation request. The categories include: a first category, a second category and a third category, where the first category includes a read operation of metadata, and the second category Categories include new operations, modification operations, and deletion operations on metadata. The third category includes move operations, new link operations, and deletion link operations on metadata. Return the operation request according to the category to which the operation request belongs. The corresponding return result.

3. The method according to claim 2, characterized in that, returning a return result corresponding to the operation request according to the category to which the operation request belongs, including:

If the operation request belongs to the first category, a return result indicating that the operation request is a read request is returned.

4. The method according to claim 2, characterized in that, returning a return result corresponding to the operation request according to the category to which the operation request belongs includes: If the operation request belongs to the second category, then determine whether the operation request on the metadata is legal; if it is not legal, then return the result that the operation request is an error request; otherwise, if it is legal, then determine Whether there is an operation request of the third category in the merge queue. The merge queue includes an operation request corresponding to the return result that can be merged. If there is an operation request of the third category in the merge queue, return the operation request. The request is the return result of a mutually exclusive request; otherwise, if there is no operation request of the third category in the merge queue, the operation request of the second category is added to the merge queue, and the operation request of the third category is added to the merge queue. The metadata of the second category operation request operation is added to the buffer data queue, and the return result of the operation request being a mergeable request is returned.

5. The method according to claim 2, characterized in that: returning a return result corresponding to the operation request according to the category to which the operation request belongs includes: if the operation request belongs to the third category, Then determine whether the operation request on the metadata is legal; if it is not legal, return the result that the operation request is an error request; otherwise, if it is legal, then determine whether there is an operation of the second category in the merge queue request, or, the operation request of the third category, the merge queue includes the operation request corresponding to the return result that can be merged; if there is the operation request of the second category in the merge queue, or, the third category If there is an operation request of the second category, then the return result is that the operation request is a mutually exclusive request; otherwise, if there is no operation request of the second category in the merge queue, and there is no operation request of the third category , then the operation request of the third category is added to the merge queue, and the metadata of the operation request of the third category is added to the buffer. in the data queue, and return the operation result as the return result of the merge request.

6. The method according to claim 3, characterized in that, processing the metadata corresponding to the at least one operation request according to each of the returned results includes: moving the operation request out of the request queue ;

Oral direction in response to the operation request.

7. The method according to claim 4 or 5, characterized in that, if the operation request is illegal and the return result of the operation request is an error request, then according to each of the return results, the at least one operation request is Corresponding metadata is processed, including:

Remove the operation request from the request queue;

Oral direction in response to the operation request.

8. The method according to claim 4 or 5, characterized in that, if the returned operation request is a return result of a mutually exclusive request, then according to each of the returned results, the at least one operation request corresponding to Metadata is processed, including:

The operation request is saved in the request queue.

9. The method according to claim 4, 5 or 8, characterized in that, processing the metadata corresponding to the at least one operation request according to each of the returned results includes: converting the buffered data The metadata in the queue is sent to the storage node, so that the storage node writes the metadata to obtain the transaction result;

According to the transaction result, an operation response is sent to the client corresponding to the operation request in the merge queue.

10. The method according to claim 9, characterized in that, after sending an operation response to the client corresponding to the operation request in the merge queue according to the transaction result, it further includes: Determine whether there is still an operation request in the request queue; if it exists, traverse each operation request in the request queue and return the return result of each operation request; otherwise, if it does not exist, release the lock .

11. A metadata server, characterized in that it includes: a request queue module, used to cache at least one operation request for the same directory sent by multiple clients to the request queue; a directory lock module, used to determine the directory whether the lock is authorized; a combined judgment module, configured to, if the directory lock module determines that the lock is authorized, traverse each operation request in the request queue and return the return result of each operation request; A request processing module, configured to process metadata corresponding to the at least one operation request according to each returned result.

12. The metadata server according to claim 11, characterized in that, for each operation request, the merge judgment module determines the category of the operation request, and the categories include: a first category, a second category and The third category, wherein the first category includes read operations on metadata, the second category includes new operations, modification operations and delete operations on metadata, and the third category includes move operations on metadata. , create a link operation and delete a link operation, and return the return result corresponding to the operation request according to the category to which the operation request belongs.

13. The metadata server according to claim 12, wherein the merge decision module is configured to return a return result that the operation request is a read request if the operation request belongs to the first category.

14. The metadata server according to claim 12, characterized in that, The merge judgment module is used to determine whether the operation request on metadata is legal if the operation request belongs to the second category. If it is determined that the operation on metadata by the operation request is illegal, then The return result of the operation request being an error request is returned; otherwise, if it is determined that the operation request on the metadata is legal, then it is determined whether there is an operation request of the third category in the merge queue, and the merge queue includes possible For the operation request corresponding to the merged return result, if it is determined that the operation request of the third category exists in the merge queue, then the return result of the operation request being a mutually exclusive request will be returned; otherwise, if it is determined that the merge queue If there is no operation request of the third category in , the request processing module is configured to add the operation request of the second category to the merge queue, and add the metadata of the operation request of the second category Added to the buffered data queue, the merging decision module is configured to return a return result that the operation request is a mergeable request.

15. The metadata server according to claim 12, characterized in that, the merge judgment module is used to judge whether the operation request on metadata is legal if the operation request belongs to the third category. , if it is determined that the operation request on the metadata is illegal, then the return result that the operation request is an error request is returned; otherwise, if it is determined that the operation request on the metadata is legal, then it is determined that the operation request in the merge queue is Whether there is an operation request of the second category, or an operation request of the third category, and the merge queue includes operation requests corresponding to the return results that can be merged; if it is determined that the second category of requests exists in the merge queue. If the operation request is an operation request of the third category, or the operation request of the third category is returned, the return result is that the operation request is a mutually exclusive request; otherwise, if there is no operation request of the second category in the merge queue, and , if there is no operation request of the third category, add the operation request of the third category to the merge queue, and add the metadata of the operation request of the third category to the buffer data queue. , and return the operation result as the return result of the mergeable request.

16. The metadata server according to claim 13, characterized in that,

The request processing module is used to remove the operation request from the request queue and respond to the operation request.

17. The metadata server according to claim 14 or 15, characterized in that the request processing module is used to move the operation request out of the request queue and respond to the operation request.

18. The metadata server according to claim 14 or 15, characterized in that the request processing module is configured to save the operation request in the request queue.

19. The metadata server according to claim 14, 15 or 18, characterized in that the request processing module is used to send the metadata in the buffer data queue to the storage node, so that the storage node The node writes the metadata to obtain the transaction result; the metadata server also includes a sending module, configured to send an operation response to the client corresponding to the operation request in the merge queue according to the transaction result.

20. The metadata server according to claim 19, characterized in that: the directory lock module is also used to determine whether there are still operation requests in the request queue; the merge decision module is also used to determine if the operation request exists in the request queue; The directory lock module determines that there are still operation requests in the request queue, then traverses each operation request in the request queue, and returns the return result of each operation request;

The request processing module is also configured to release the lock if the directory lock module determines that there is no operation request in the request queue.