WO2017092673A1 - Metadata access method, related apparatus and system - Google Patents

Metadata access method, related apparatus and system Download PDF

Info

Publication number
WO2017092673A1
WO2017092673A1 PCT/CN2016/107979 CN2016107979W WO2017092673A1 WO 2017092673 A1 WO2017092673 A1 WO 2017092673A1 CN 2016107979 W CN2016107979 W CN 2016107979W WO 2017092673 A1 WO2017092673 A1 WO 2017092673A1
Authority
WO
WIPO (PCT)
Prior art keywords
metadata
target
directory
client
server
Prior art date
Application number
PCT/CN2016/107979
Other languages
French (fr)
Chinese (zh)
Inventor
王晓锋
方新
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2017092673A1 publication Critical patent/WO2017092673A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/164File meta data generation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • G06F16/1824Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • G06F16/183Provision of network file services by network file servers, e.g. by using NFS, CIFS

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a metadata access method, related device, and system.
  • the metadata access portion of the GFS includes three subsystems: a metadata client subsystem 10, a metadata service subsystem 20, and a cluster service subsystem 30.
  • the metadata client subsystem 10 includes a plurality of clients (Client).
  • the client 101-103 is taken as an example in FIG. 1;
  • the metadata service subsystem 20 includes a plurality of metadata servers, and the metadata servers 201-203 are taken as an example in FIG. 1.
  • the cluster service subsystem 30 mainly includes a cluster management server. 301.
  • the principle of the client access metadata in FIG. 1 is as shown in FIG. 2.
  • the distributed consistency server 302 is first sent exclusive access rights to the target metadata.
  • the client 101 is the only client among all the clients 101-103 that can directly request the metadata server 201-203 to operate the target metadata, thus ensuring the target. Metadata cache coherency.
  • the client 101 queries the metadata routing server 303 for the attribution information of the target metadata, that is, queries which metadata server operation the target metadata belongs to; the metadata routing server 303 queries according to the mapping relationship between the stored metadata and the metadata server.
  • the attribution information of the target metadata is sent to the client 101; the client 101 analyzes the attribution information, and when the target metadata is analyzed and belongs to the metadata server 201, the metadata server is sent to the metadata server.
  • 201 issues a request to request access to the target metadata
  • the metadata server 201 receives the request of the client 101, and operates on the target metadata according to the request, and then feeds back the operation result to the client 101, and accordingly, the client 101 receives the element.
  • the result of the operation sent by the data server 201 is cached locally, and the client 101 accesses the target metadata at this point.
  • the above-mentioned distributed consistency server 302 grants the client 101 exclusive access rights to the target metadata, which can be implemented by a distributed lock, a distributed protocol, etc., when distributed through the distributed lock, the distributed consistency server 302 and the client
  • the interaction between the terminals 101 is required to perform the lock application, the lock release, and the lock recall operation.
  • the communication overhead between the nodes is large in the process; especially in the scenario of Lot of Small Files (LoSF), distributed
  • the metadata of the small files in the file system (GFS) is very large.
  • the client 101 accesses the metadata of each small file, the lock application, the lock release, and the lock recall operation are performed with the distributed consistency server.
  • the huge communication overhead generated reduces the efficiency of client access to metadata.
  • the embodiment of the invention discloses a method for accessing metadata, a related device and a system, which can reduce the overhead of maintaining metadata cache consistency in the process of accessing metadata by the client, and improve the access efficiency of the metadata.
  • an embodiment of the present invention provides a method for accessing metadata, where the method includes:
  • the first client When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target directory For storing a directory having a plurality of metadata, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the first a client accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
  • the first client accesses the first target metadata when the first client applies for exclusive access to all metadata in the target directory.
  • the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client when the first client does not apply for exclusive access rights to all metadata in the target directory, the first client Receiving, by the terminal, the node information of the second client sent by the directory lock server, and sending a request message to the second client according to the node information, to access the first target metadata by using the second client
  • the second client is a client that applies to the directory lock server for exclusive access to all metadata in the target directory.
  • the first client accesses the first target through the second client that has obtained exclusive access rights of all metadata in the target directory when the application does not have exclusive access rights to all the metadata in the target directory.
  • the metadata enables the first client to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
  • the first client is requesting access to the first target metadata in the target directory
  • the first application message is sent to the directory lock server to apply for exclusive access to all metadata in the target directory, the following:
  • the first client obtains metadata describing the target directory when requesting access to the first target metadata in the target directory, and the metadata describing the target directory is configured to identify whether the target directory is stored. Identification information of a directory of multiple metadata;
  • a first application message is sent to the directory lock server to request exclusive access to all metadata in the target directory.
  • the method further includes:
  • the first client accesses the first target metadata according to the request message, and sends an access result of accessing the first target metadata to the third client.
  • the third client After performing the above steps, after the first client obtains exclusive access rights to all metadata in the target directory, if the third client has a requirement to access the first target metadata in the target directory, the third client will access the request. Sending to the first client, accessing the first target metadata by the first client, ensuring that the client that does not apply for exclusive access rights of all metadata in the target directory can also access the first target metadata normally. .
  • the method when the first client applies exclusive access rights to all metadata in the target directory, after the first client accesses the first target metadata, the method further includes:
  • the first client when requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata in the target directory other than the first target metadata;
  • the first client accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
  • the second target metadata can be directly accessed, and the access permission is not required to be applied again.
  • the efficiency of a client's access to metadata in the target directory is not required to be applied again.
  • the first client accessing the first target metadata includes:
  • the first client receives the home information of the target directory sent by the metadata routing server, and the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
  • the operation result is that the target metadata server applies to the directory lock server to perform exclusive operation on all metadata in the target directory. After the permission, the result of the operation of the first target metadata.
  • the target metadata server After performing the above steps, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server first locks the directory before operating the first target metadata in the target directory.
  • the server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata retention consistency of the target directory is ensured.
  • the first client accessing the first target metadata includes:
  • the first client sends a second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory;
  • Receiving, by the first client, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for
  • the first client operates the first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
  • the first client operates the first target metadata according to the exclusive operation authority of all metadata in the target directory, and obtains an operation result.
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, the first client does not need to interact with the remote metadata server. To request the metadata server to operate on the first target metadata, the overhead between the nodes is reduced, and the access efficiency of accessing the first target metadata is improved.
  • the first client when the first client applies exclusive access rights to all metadata in the target directory, After the first client accesses the first target metadata, the method further includes:
  • the first client releases exclusive access rights of all metadata in the target directory, so that the directory lock server grants exclusive access rights of all metadata in the target directory to the fourth client end.
  • the exclusive access permission of all metadata in the target directory is automatically released, and the fourth client with lower load is obtained to obtain the row of all metadata in the target directory. It accesses the load to achieve the purpose of sharing the load; or automatically releases all metadata in the target directory when the first client accesses the metadata in the target directory less frequently than the metadata in other clients accessing the target directory.
  • Exclusive access rights allow clients with higher frequency of access to obtain exclusive access rights to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
  • an embodiment of the present invention provides a metadata access method, where the method includes:
  • the directory lock server receives the first application message sent by the first client when the first client accesses the target metadata in the target directory, and the first application message is used to apply for all metadata in the target directory.
  • Exclusive access rights the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the directory lock server determines, according to the first application message, whether exclusive access rights of all metadata in the target directory have been granted to the second client, and exclusive access rights of all metadata in the target directory are used for Providing, by the first client, target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
  • the directory lock server grants exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, Transmitting, by the directory lock server, node information of the second client to the first client, so that the first client accesses the second client by using node information of the second client The target metadata.
  • the first client accesses the target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can successfully access the target.
  • Metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the directory lock server performs exclusive access of all metadata in the target directory. After the rights are granted to the first client, the method further includes:
  • the directory lock server sends the node information of the first client to the third client, so that the third client passes the first client based on the node information where the first client is located. Access the target metadata.
  • the third client After performing the above steps, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement to access the target metadata in the target directory, the third client sends the access request to the third client.
  • the first client accesses the target metadata through the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the target metadata normally.
  • the directory lock server After the exclusive access rights of all the metadata in the target directory are granted to the first client, the method further includes:
  • the directory lock server grants exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server operates the target metadata And feeding back the operation result to the first client, wherein the exclusive operation authority of all the metadata in the target directory is used for the target metadata server to operate the target metadata in the target directory, and the operation Metadata in the target directory other than the target metadata.
  • the target metadata server After performing the above steps, after receiving the request for the target metadata in the target directory sent by the first client, the target metadata server requests the target directory from the directory lock server before performing the operation on the target metadata in the target directory.
  • the exclusive operation permission of all the metadata in the metadata makes that only one metadata server in all the metadata servers has the operation authority for the target directory, which ensures the persistent preservation consistency of the metadata in the target directory.
  • the directory lock server After the exclusive access rights of all the metadata in the target directory are granted to the first client, the method further includes:
  • the directory lock server receives a second application message sent by the first client, where the second application message is used to apply for exclusive operation rights on all metadata in the target directory;
  • the directory lock server grants exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client operates the target metadata And an exclusive operation authority of all metadata in the target directory for the first client to operate target metadata in the target directory, and operating the target directory except the target metadata External metadata.
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, the first client does not need to interact with the remote metadata server. To request the metadata server to operate on the target metadata, the overhead between the nodes is reduced, and the access efficiency of the access target metadata is improved.
  • an embodiment of the present invention provides a terminal, where the terminal is a first client, and the first client includes a sending device, a receiving device, a memory, and a processor, where the processor invokes A program for accessing metadata in the memory for performing the following operations:
  • the server sends a first application message to apply for exclusive access rights to all metadata in the target directory, wherein the target directory is a directory in which a plurality of metadata is stored, and the first target metadata is the processing Metadata currently required to be accessed by the device; exclusive access rights for all metadata in the target directory for the processor to access first target metadata in the target directory, and access to the target directory Metadata other than the first target metadata;
  • the first target metadata is accessed when applying for exclusive access to all metadata in the target directory.
  • the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the directory lock server is received by the receiving device And sending, by the sending device, a request message to the second client, according to the node information, to access metadata in the target directory by using the second client,
  • the second client is a client that applies to the directory lock server for exclusive access to all metadata in the target directory.
  • the first client accesses the first target through the second client that has acquired exclusive access rights of all metadata in the target directory when applying for exclusive access rights of all metadata in the target directory.
  • the metadata enables the first client to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
  • the processor sends a first application message to the directory lock server to apply for the
  • the exclusive access rights of all metadata in the target directory are as follows:
  • the first application message is sent to the directory lock server by the sending device to apply for exclusive access to all metadata in the target directory.
  • the processor when applying to the opposite The processor is further configured to: after the process accesses the first target metadata, when the process accesses the first target metadata in the target directory,
  • the directory lock server applies exclusive access rights to all metadata in the target directory, and after receiving the node information of the first client returned by the directory lock server, according to the a message sent by the node information of the client to the first client;
  • the third client After performing the above operation, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement to access the first target metadata in the target directory, the third client will access the request. Sending to the first client, accessing the first target metadata by the first client, ensuring that the client that does not apply for exclusive access rights of all metadata in the target directory can also access the first target metadata normally. .
  • the processor after applying the exclusive access right to all metadata in the target directory, after the processor accesses the first target metadata, the processor is further used to :
  • the processor accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
  • the second target metadata can be directly accessed, and the access permission is not required to be applied again.
  • the efficiency of a client's access to metadata in the target directory is not required to be applied again.
  • the manner in which the processor accesses the first target metadata is specifically:
  • the receiving device Receiving, by the receiving device, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server for exclusive operation permission on all metadata in the target directory. Thereafter, the result of operating the first target metadata.
  • the target metadata server After performing the above operation, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server first locks the directory before operating the first target metadata in the target directory.
  • the server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata retention consistency of the target directory is ensured.
  • the manner in which the processor accesses the first target metadata is specifically:
  • Receiving, by the receiving device, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for the processing
  • the first target metadata in the target directory is operated, and metadata other than the first target metadata in the target directory is operated;
  • the first target metadata is operated according to the exclusive operation authority of all the metadata in the target directory, and the operation result is obtained.
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, it does not need to interact with the remote metadata server. To request the metadata server to operate on the first target metadata, the overhead between the nodes is reduced, and the access efficiency of accessing the first target metadata is improved.
  • the processor accesses the first target when applying exclusive access rights to all metadata in the target directory After the metadata, the processor is also used to:
  • the exclusive access permission of all metadata in the target directory is automatically released, and the fourth client with lower load is obtained to obtain the row of all metadata in the target directory. It accesses the load to achieve the purpose of sharing the load; or automatically releases all metadata in the target directory when the first client accesses the metadata in the target directory less frequently than the metadata in other clients accessing the target directory.
  • Exclusive access rights allow clients with higher frequency of access to obtain exclusive access rights to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
  • an embodiment of the present invention provides a terminal, where the terminal is a directory lock server, and the directory lock server includes a sending device, a receiving device, a memory, and a processor, where The processor calls a program for accessing metadata in the memory to perform the following operations:
  • the receiving device Receiving, by the receiving device, a first application message sent by the first client, where the first client accesses the target metadata in the target directory, where the first application message is used to apply for all the target directories.
  • Exclusive access rights of the metadata the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the second client If the second client is not granted, grant exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, pass the sending device to Sending, by the first client, node information where the second client is located, so that the first client accesses the target metadata by using the second client according to node information of the second client .
  • the application lock access authority of all the metadata in the target directory is requested to the directory lock server at one time, so that the first client can successfully access the target. Metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the processor grants exclusive access rights of all metadata in the target directory to the After the first client, the processor is further configured to:
  • the third client After performing the above operation, after the first client obtains the exclusive access right of all the metadata in the target directory, if the third client has the requirement of accessing the target metadata in the target directory, the third client sends the access request to the third client.
  • the first client accesses the target metadata through the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the target metadata normally.
  • the processor After the exclusive access rights of all metadata in the directory are granted to the first client, the processor is further configured to:
  • the target metadata server is configured to cause the target metadata server to operate on the target metadata and feed back an operation result to the first client, wherein all metadata in the target directory is exclusive
  • the operation authority is for the target metadata server to operate the target metadata in the target directory, and to operate metadata in the target directory other than the target metadata.
  • the target metadata server After performing the above operation, after receiving the request for the target metadata in the target directory sent by the first client, the target metadata server requests the target directory from the directory lock server before operating the target metadata in the target directory.
  • the exclusive operation permission of all the metadata in the metadata makes that only one metadata server in all the metadata servers has the operation authority for the target directory, which ensures the persistent preservation consistency of the metadata in the target directory.
  • the processor After the exclusive access rights of all metadata in the directory are granted to the first client, the processor is further configured to:
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, it does not need to interact with the remote metadata server. To request the metadata server to operate on the target metadata, the overhead between the nodes is reduced, and the access efficiency of the access target metadata is improved.
  • an embodiment of the present invention provides a distributed file system, where the system includes a first client, a second client, and a directory lock server, where:
  • the first client When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target directory For storing a directory having a plurality of metadata, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the first a client accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
  • the directory lock server grants exclusive access rights of all metadata in the target directory to the first client And sending a notification message to the first client to notify the first client of exclusive access rights of all metadata in the target directory;
  • the first client receives the notification message and accesses the first target metadata according to the notification message.
  • the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client is the first client described in any possible implementation manner of the third aspect
  • the directory lock server is a directory lock server as described in any of the possible implementations of the fourth aspect.
  • the directory in which the plurality of metadata is stored includes a directory in which a plurality of metadata for describing a small number of files is stored.
  • the sending the first application message to the directory lock server to apply for exclusive access to all metadata in the target directory includes:
  • the lock is a non-preemptable distributed lock.
  • the sending the second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory includes:
  • the determining, by the first client, whether the load generated by accessing the metadata in the target directory meets a preset condition comprises: the first client records actively accessing the target directory by itself The first load generated by the metadata, and the second load generated by accessing the metadata in the target directory according to the request message of the other client; the terminal determines whether the relative size of the first load and the second load is Meet the preset conditions.
  • an embodiment of the present invention provides an apparatus for accessing metadata, where the apparatus includes a functional unit for performing some or all of the steps of any of the first aspects of the embodiments of the present invention.
  • an embodiment of the present invention provides an apparatus for accessing metadata, where the apparatus includes a functional unit for performing some or all of the steps of any of the second aspects of the embodiments of the present invention.
  • the first client When the first client accesses the first target metadata in the target directory, the first client requests the exclusive access permission of all the metadata in the target directory to the directory lock server, so that the first client The first target metadata can be successfully accessed, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • FIG. 1 is a schematic structural diagram of a distributed file system in the prior art
  • FIG. 2 is a schematic diagram of the principle of the distributed file system shown in FIG. 1;
  • FIG. 3 is a schematic flowchart of a metadata access method according to an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention.
  • FIG. 5A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention.
  • FIG. 5B is a schematic flowchart of interaction between devices in FIG. 5A; FIG.
  • FIG. 6 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a principle of metadata access according to an embodiment of the present invention.
  • FIG. 6B is a schematic flowchart of interaction between devices in FIG. 6A;
  • FIG. 7 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of still another terminal according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of a distributed file system according to an embodiment of the present disclosure.
  • FIG. 11 is a schematic structural diagram of still another terminal according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic structural diagram of still another terminal according to an embodiment of the present invention.
  • FIGS. 3-7 are used to describe the method of the embodiments of the present invention.
  • FIG. 3 is a schematic flowchart of a metadata access method according to an embodiment of the present invention, which describes how the first client obtains exclusive access rights of all metadata in the target directory, including but not limited to the following: step.
  • Step S303 When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, where
  • the target directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for The first client accesses first target metadata in the target directory and accesses metadata in the target directory other than the first target metadata.
  • the distributed file system exists to store multiple elements.
  • the target directory of the data, the first target metadata in the target directory is the metadata that the first client currently needs to access.
  • the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the directory lock server coordinates the access rights of each client to the target directory by granting a lock to the target directory, if the directory lock server accesses the target directory The lock is granted to the first client, and the first client has access to all metadata in the target directory.
  • a distributed lock service such as a server carrying the zookeeper service
  • the distributed lock service can be set to preemptive distributed locks and non-preemptive distributed Lock; if set to a preemptible distributed lock, when the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server will release the access target directory according to the preemption mechanism. Lock, and re-grant the lock to the target directory to the first client, if the lock accessing the target directory is not granted to other clients, then the directory lock server directly grants the lock to the target directory to the first client; if not set to Preemptive distributed lock.
  • the directory lock server When the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server will not grant the lock to the target directory according to the non-preemptive mechanism.
  • the first client if the lock accessing the target directory is not granted to other clients, the directory lock server directly grants the lock accessing the target directory to the first client.
  • the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the first client applies for the lock to access the target directory, it has all the metadata in the target directory.
  • Exclusive access rights that is, other clients except the first client do not have access to the metadata in the target directory (ie, access to the first target metadata and metadata other than the first target metadata) ) unless the first client actively releases the lock accessing the target directory.
  • the method further includes: when the first client requests access to the second target metadata in the target directory, if the second target metadata belongs to the target directory except the first target metadata Metadata; then the first client accesses the second target metadata according to the obtained exclusive access rights to the second target metadata.
  • the original client when the first client requests access to the first target metadata in the target directory, the original client only needs to apply exclusive access rights of the first target metadata, but the first client actually applies for the target directory.
  • Exclusive access rights for all metadata such that when the first client requests access to metadata other than the first target metadata (eg, second target metadata) in the target directory, it determines that it has acquired The right to access the second target metadata, so there is no need to apply for exclusive access rights of the second target metadata, but directly access the second target metadata.
  • the first client when the first client requests access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for all metadata in the target directory.
  • the exclusive access permission includes: when the first client requests to access the first target metadata in the target directory, acquiring metadata describing the target directory, where the metadata describing the target directory is configured to identify the Whether the target directory is the identification information of the directory in which the plurality of metadata is stored; determining whether the target directory is a directory in which the plurality of metadata is stored according to the metadata of the target directory; if yes, the directory lock The server sends a first application message to request exclusive access to all metadata in the target directory.
  • the plurality of metadata may be specifically metadata describing a small number of small files (each small file corresponds to its own metadata).
  • the target directory can also be regarded as a file, so it also corresponds to the metadata describing the target directory, and the metadata describing the target directory is stored in the upper directory of the target directory.
  • the metadata routing server receives the query request of the first client, Searching, from a mapping relationship between the stored directory and the metadata server, a metadata server to which the upper-level directory belongs, wherein the metadata server belongs to the metadata server, and the metadata server has the upper-level directory
  • the operation authority the attribution information of the upper-level directory is fed back to the first client, and the first client receives the home information sent by the metadata routing server, and analyzes the attribution information to obtain the element belonging to the upper-level directory.
  • the data server then requests, from the metadata server to which the upper-level directory belongs, metadata stored in the upper-level directory for describing the target directory, where the metadata server to which the upper-level directory belongs is stored and stored. Metadata used to describe the target directory in the primary directory, and the metadata of the read target directory is read To the first client.
  • directory B is the target directory
  • directory A is the upper directory of directory B
  • directory B stores the first target metadata describing a file.
  • the directory A stores the metadata describing the directory B; when the first client wants to access the first target metadata f, first needs to find the metadata server to which the directory A belongs to the metadata routing server, and then to the element
  • the data server requests the metadata describing the directory B under the directory A. Accordingly, the metadata server receives the request of the first client, reads the metadata describing the directory B in the directory A, and reads the element describing the directory B. The data is sent to the first client.
  • the first client analyzes the metadata describing the target directory, and determines whether the target directory stores multiple
  • the metadata describing each directory has identification information for defining the directory attribute of each directory, so the first client can be based on describing the metadata in the target directory.
  • the identification information "derives the directory attribute of the target directory, and further determines whether the target directory is a directory in which a plurality of metadata is stored.
  • the metadata describing the target directory may include "identification information" to identify the directory attribute of the target directory.
  • identification information When the identification information is set to "1", it indicates that the target directory is the above-mentioned directory storing a plurality of metadata, when When the identification information is set to "0", it indicates that the target directory is not the directory in which the plurality of metadata are stored as described above.
  • the "identification information” may be set by the user when the target directory is created, or may be modified by the user after the target directory is created. After the first client receives the metadata describing the target directory, it is determined whether the "identification information" is "1” or "0", and whether the target directory is a directory storing a plurality of metadata.
  • Step S304 When the first client applies for exclusive access rights to all metadata in the target directory, the first client accesses the first target metadata.
  • the directory lock server receives the first application message sent by the first client, analyzes the first application message, and obtains that the first application message is used to apply for exclusive access rights of all metadata in the target directory, so The directory lock server determines whether the exclusive access rights of all metadata in the target directory have been granted to other clients, and if not, the directory lock server will all metadata in the target directory.
  • the exclusive access permission is granted to the first client, so that only the first client of all the clients has the right to access the metadata in the target directory, and the first client subsequently accesses any metadata in the target directory.
  • the first client does not need to apply for access to the arbitrary metadata.
  • the first client accesses the metadata in the target directory.
  • the operations of reading and writing metadata belong to the access metadata.
  • the process of accessing the metadata includes requesting the metadata server to read the metadata and receiving the metadata server.
  • the metadata is read; when the metadata is written, the process of accessing the metadata includes requesting the metadata server to write content (including modifications) to the metadata and receiving the write result sent by the metadata server.
  • the metadata server is a device with operational metadata rights.
  • the target directory is directory B
  • the upper directory of directory B is directory A
  • the metadata of files is d, e, and f under directory B
  • the first client can access the metadata f, and can access the metadata e or d subsequently.
  • Direct access without having to apply for access to directory B, and no need to apply for access to e and d.
  • the first client accesses the first target metadata in the target directory
  • the first client accesses the directory lock server for all the metadata access rights in the target directory, so that the first client can be smoothly performed by performing steps S303-S304.
  • Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client when the first client does not apply for exclusive access to all metadata in the target directory, the first client receives the first sent by the directory lock server. And the node information where the client is located, and sending a request message to the second client according to the node information to access the first target metadata by using the second client, where the second client is a
  • the directory lock server applies to the client for exclusive access to all metadata in the target directory.
  • the directory lock server determines that the exclusive access rights of all metadata in the target directory have been granted to other clients (such as the second client)
  • the directory lock server does not queue all the metadata in the target directory.
  • the access permission is granted to the first client, but the directory lock server sends the node information including the second client to the first client, and correspondingly, the first client receives the node information sent by the directory lock server, and from the node Parsing out the node of the second client in the node information, and then sending a request to access the target directory to the second client based on the node, and correspondingly, the second client receives the request for accessing the target directory sent by the first client, according to the request The first target metadata under the target directory is accessed, and the access result is sent to the first client.
  • steps S303-S304 After performing the above steps S303-S304, if the first client acquires the rights of all the metadata in the target directory, the first client may also perform steps S305 and S306 for other third clients that cannot obtain the access rights of the target directory.
  • the terminal provides a service for accessing metadata in the first target directory, and the specific flowchart is as shown in FIG. 4. The detailed description of steps S305 and S306 is as follows:
  • Step S305 The first client receives a request message sent by a third client, where the request is cancelled.
  • the information is used to request access to the first target metadata by using the first client, and the request message is for the third client to apply to the directory lock server for exclusive use of all metadata in the target directory.
  • the directory lock server when the directory lock server grants exclusive access rights of all metadata in the target directory to the first client, the directory lock server also receives exclusive access rights of all metadata in the target directory of the third client application.
  • Request the third client refers to a type of client that does not have exclusive access to all metadata in the target directory
  • the directory lock server since the exclusive access has been granted to the first client, the directory lock server does not The exclusive access to all metadata in the target directory is granted to the third client, but in order to ensure that the third client can normally access the metadata in the target directory, the directory lock server sends the third client to the third client.
  • the third client receives the node information sent by the directory lock server, and analyzes, according to the node information, the first target metadata that needs to be accessed through the first client, and then sends the first target metadata to the first client.
  • An application message where the application message includes information about the first target metadata in the target directory that the third client needs to access,
  • the first client application receives a third message sent by the client.
  • Step S306 The first client accesses the first target metadata according to the request message, and sends an access result of accessing the first target metadata to the third client.
  • the first client analyzes the application message, and obtains that the metadata that the third client needs to access is the first target metadata, and further determines whether the first target metadata is in the cache, and if yes, the cached A target metadata is read, written, etc., and the operation result is fed back to the third client; if not in the cache, the metadata server to which the target directory belongs is requested to read and write the first target metadata, The result of the operation is then fed back to the third client.
  • the third client After performing the steps S305 and S306, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement of accessing the first target metadata in the target directory, the third client will The access request is sent to the first client, and the first target metadata is accessed by the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the first target normally. Metadata.
  • the first client After the first client obtains the exclusive access right of all the metadata in the target directory, the first client accesses the first target metadata by using the steps S307-S310; As shown in Figure 5.
  • the detailed description of steps S307-S310 is as follows:
  • Step S307 The first client requests the home information of the target directory from the metadata routing server, and the metadata routing server prestores the home information of the directory.
  • the first client sends a query request to the metadata routing server to query the home information of the target directory; the metadata routing server receives the query request of the first client, and searches from the mapping relationship between the stored directory and the metadata server.
  • the target metadata server corresponding to the target directory and then sending the home information to the first client, indicating that the target directory belongs to the target metadata server.
  • Step S308 The first client receives the home information of the target directory sent by the metadata routing server, and the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server.
  • the first client receives the home information sent by the metadata routing server, and analyzes the attribution information to obtain that the operation authority of the target directory belongs to the target metadata server.
  • Step S309 The first client sends an operation request message to the target metadata server, requesting the target metadata server to operate the first according to the exclusive access right to all metadata in the target directory.
  • Target metadata The first client sends an operation request message to the target metadata server, requesting the target metadata server to operate the first according to the exclusive access right to all metadata in the target directory.
  • the first client since the first client has obtained exclusive access rights of all metadata in the target directory, the first client has the right to send an operation request message to the target metadata server, requesting the target metadata server to the first target metadata. The operation is performed, and other clients that do not have exclusive access to all metadata in the target directory cannot request the target metadata server to operate on the first target metadata.
  • Step S310 The first client receives an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server to all metadata in the target directory. The result of operating the first target metadata after exclusive operation permission.
  • the target metadata server receives the operation request message sent by the first client, and parses the request message to obtain that the metadata of the first client request operation is the first target metadata in the target directory, and thus the directory lock
  • the server sends an application request to apply for exclusive operation permission of all metadata in the target directory; accordingly, the directory lock server receives the application request sent by the metadata server, and at this time, regardless of whether the exclusive operation permission of all metadata in the target directory is granted
  • the directory lock server grants the exclusive operation permission of all metadata in the target directory to the target metadata server (if the exclusive operation permission of the target directory is specifically implemented by distributed locks, then when the target directory is operated When the lock has been granted to other clients, the directory lock server recalls the lock of the operation target directory, and then grants the lock of the operation target directory to the first client; when the lock of the operation target directory is not granted to other clients, the directory lock server directly The lock of the operation target directory is granted to the first client), so that all Only the target metadata server capable of operating on metadata in the target directory metadata server.
  • the target metadata server operates the first target metadata in the target directory, and then sends an operation result of operating the first target metadata to the first client, and correspondingly, the first client receives the target metadata.
  • the exclusive operation permission of all metadata in the application target directory and the exclusive access permission of all metadata in the application target directory can be realized by the distributed lock based on the directory granularity, wherein all the application target directories are
  • the exclusive operation permission of metadata uses a preemptible distributed lock.
  • the exclusive access permission for all metadata in the target directory is a non-preemptable distributed lock.
  • the target metadata server After performing the steps S307-S310, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server performs the operation on the first target metadata in the target directory.
  • the directory lock server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata of the target directory is consistently saved. Sex.
  • the first client After the first client obtains the exclusive access rights of all the metadata in the target directory, the first client accesses the first target metadata by using the steps S311-S315; As shown in Figure 6.
  • the detailed description of steps S311 to S315 is as follows:
  • Step S311 The first client sends a handover message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client.
  • the first client sends a handover message to the metadata routing server
  • the metadata routing server receives the handover message sent by the first client, and analyzes the handover message, and analyzes the handover message to indicate that the target is to be
  • the metadata routing server modifies the mapping relationship in the routing table, so that the target directory corresponds to the first client, that is, the operation authority of the target directory is attributed to the first client.
  • Step S312 The first client receives the notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client.
  • the metadata routing server sends a notification message that the route switching succeeds to the first client, and the first client receives the notification message sent by the metadata routing server, and analyzes the notification message to know that the target directory has belonged to the first client. end.
  • a client proxy and a metadata service proxy may be set in the first client, and the client proxy is used to provide a cache of metadata and an exclusive access right to apply for all metadata in the target directory to the directory lock server;
  • the Metadata Service Agent is used to store the target directory and to apply exclusive access to the Directory Lock Server for all metadata in the target directory.
  • Step S313 The first client sends a second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory.
  • the first client when the first client knows that the target directory belongs to itself through the notification message, the first client sends a second application message to the directory lock server to apply for exclusive operation permission of all metadata in the target directory; correspondingly, the directory
  • the lock server receives the second application message sent by the first client, and the directory lock server grants the operation authority of the target directory to the first client, regardless of whether another metadata server operates the target directory.
  • a client sends a grant message to notify all the metadata in the target directory that the operational rights have been granted to the first client.
  • Step S314 The first client receives a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used. Providing, by the first client, first target metadata in the target directory, and operating metadata in the target directory other than the first target metadata.
  • the first client receives the grant message sent by the directory lock server, and after analyzing the grant message, can learn that it has acquired exclusive operation rights of all metadata in the target directory (including the first target metadata and the target directory). In this case, the exclusive operation permission of the metadata other than the first target metadata), so that the first client can operate on the metadata in the target directory without requesting an additional metadata server for the element in the target directory.
  • the data is manipulated.
  • Step S315 The first client operates the first target metadata according to the exclusive operation authority of all the metadata in the target directory, and obtains an operation result.
  • the first client is based on the exclusive operation rights of all metadata in the target directory that has been acquired.
  • the operation is performed on the first target metadata in the target directory, and the operation result is obtained.
  • the first client completes the access to the first target metadata in the target directory.
  • the result of the operation includes whether the reading is successful, and the read metadata is included when the reading is successful; when the form of the access is written (or modified), the result of the operation includes whether to write or not Success, the rest of the forms of access here are not examples.
  • the first client functions as both a client and a metadata server, and the first client does not need to have a remote metadata server after obtaining exclusive access rights of all metadata in the target directory.
  • Interacting to request the metadata server to operate on the first target metadata reduces the overhead between the nodes and improves the access efficiency of accessing the first target metadata.
  • the first client may also perform steps S316 and S317 to balance the load of each client, specifically, the flowchart is as shown in FIG. 7. Shown. The detailed description of steps S316 and S317 is as follows:
  • Step S316 The first client determines whether the load generated by accessing the metadata in the target directory satisfies a preset condition.
  • the first client after accessing the metadata in the target directory, the first client detects the overall load of the runtime in real time, and compares the load value of the overall load with a preset threshold to determine the overall load of the first client. Whether the preset threshold is exceeded or not, and the preset threshold is a load value used for reference comparison according to actual needs.
  • the first client records the first load generated by the user actively accessing the metadata in the target directory, and the first data generated by accessing the metadata in the target directory according to the request message of other clients.
  • the second client further determines whether the relative size of the first load and the second load meets a preset condition, for example, the first load is smaller than the second load.
  • Step S317 if satisfied, the first client releases exclusive access rights of all metadata in the target directory, so that the directory lock server grants exclusive access rights of all metadata in the target directory. Fourth client.
  • the fourth client is You can apply to the directory lock server for exclusive access to all metadata in the target directory. After the fourth client applies for exclusive access to all metadata in the target directory, the first client needs to access the target directory if it needs to access the target directory. The fourth client comes to visit.
  • the third client and the fourth client in the embodiment of the present invention may be the same client or may not be the same client.
  • steps S303 to S317 may be combined to form another solution, for example, steps S303 to S306 are combined with steps S307 to S310; steps S303 to S306 are combined with steps S311 to S315; and steps S303 to S310 and S316 are combined.
  • S317 is combined; steps S303-S306 are combined with steps S311-S317, etc., and the beneficial effects generated by the combination can be referred to the related descriptions of the above steps, and details are not described herein again.
  • FIG. 5A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention
  • FIG. 5B is a schematic flowchart of interaction between devices in FIG. 5A.
  • FIG. 5A includes a first client 101, a second client 102, and a third client 103 (the number of clients can be deployed according to actual needs), a directory lock server 305, a metadata routing server 303, and a first metadata server 201.
  • the second metadata server 202 and the third metadata server 203 are examples of the metadata server 203.
  • the first client 101 includes a metadata cache unit and a lock unit for accessing the directory, and the metadata cache unit is used for The metadata sent by the metadata server is cached, and the lock unit of the access directory is used to apply exclusive access rights to all metadata in the directory to the lock unit of the directory lock server 305;
  • the metadata routing server 303 includes the routing service unit a home query service for providing a directory for the first client 101;
  • the first metadata server 201, the second metadata server 202, and the third metadata server 203 each include a metadata service unit and a lock unit of the operation directory, and a meta
  • the data service unit is used to implement consistent persistence of metadata, and the lock unit of the operation directory is used.
  • the flow when the first client 101 accesses the path ".../A/B/f" is as shown in steps S501 to S517, wherein the directory B is a directory under the directory A, and the first target metadata f is a plurality of directories B. An item in the metadata.
  • Step S501 The first client 101 requests the metadata routing server 303 for the home information of the directory A.
  • Step S502 The metadata routing server 303 searches the first metadata server 201 to which the directory A belongs from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
  • Step S503 The first client 101 requests the first metadata server 201 for metadata describing the directory B under the directory A.
  • Step S504 The first metadata server 201 sends an application request to the directory lock server 305, requesting the operation of the lock of the directory A (for the convenience of description, may be referred to as "the home lock of the directory A") to obtain the exclusive information of all the metadata in the directory A. access permission.
  • Step S505 the directory lock server 305 grants the home lock of the directory A to the first metadata server 201, so that only the first metadata server 201 of all the metadata servers is provided in the directory A. Operational authority for all metadata.
  • Step S506 The first metadata server 201 reads the metadata describing the directory B from the directory A.
  • Step S507 The first metadata server 201 sends the read metadata describing the directory B to the first client 101.
  • Step S508 The first client 101 analyzes the metadata describing the directory B, and determines whether the directory B is a directory in which a plurality of metadata is stored.
  • Step S509 If it is determined that the directory B is a directory storing a plurality of metadata, the first client 101 sends an application request to the directory lock server 305, requesting access to the lock of the directory B (for the convenience of description, it may be referred to as "the acceleration of the directory B”. Lock”) to get exclusive access to all metadata in directory B.
  • Step S510 When the directory lock server 305 finds that the acceleration lock of the directory B is not granted to other clients, the directory lock server 305 grants the acceleration lock of the directory B to the first client 101, so that only the first client 101 among all the clients Have access to all metadata in directory B. Subsequent steps S511 to S517 are specific processes for the first client to access the metadata in the directory B.
  • Step S511 The first client 101 requests the metadata routing server 303 for the home information of the directory B.
  • Step S512 The metadata routing server 303 searches the second metadata server 202 corresponding to the directory B (that is, the home metadata server 202) from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
  • Step S513 The first client 101 requests the second metadata server 202 to access the first target metadata f under the directory B.
  • Step S514 The second metadata server 202 applies to the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • Step S515 Based on the preemptive distributed lock rule, regardless of whether the home lock of the directory B is granted to other metadata servers, the directory lock server 305 grants the home lock of the directory B to the second metadata server 202, and informs the second The metadata server 202 enables only the second metadata server 202 to have exclusive operational rights to all metadata in the target directory among all metadata servers.
  • Step S516 The second metadata server 202 operates on the first target metadata f in the directory B.
  • Step S517 The second metadata server 202 transmits the operation result of operating the first target metadata f to the first client 101.
  • FIG. 6A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention
  • FIG. 6B is a schematic flowchart of interaction between devices in FIG. 6A.
  • FIG. 6A includes a first client 101, a second client 102, a third client 103 (the number of clients can be deployed according to actual needs), a directory lock server 305, and a metadata routing server 303.
  • the first client 101 includes a metadata cache unit and access.
  • the lock unit of the directory, the metadata service unit, and the lock unit of the operation directory The metadata cache unit is configured to cache the metadata sent by the metadata server, and the lock unit of the access directory is used to apply for exclusive access rights of the directory to the lock unit of the directory lock server 305, and the metadata service unit is used for To implement operations such as reading, writing, and persisting metadata, the lock unit of the operation directory is used to apply the exclusive operation authority of the directory to the lock unit of the operation directory of the directory lock server 305;
  • the metadata routing server 303 includes the routing service unit and The routing switching unit is configured to provide a first query service for the first client 101, and the routing switching unit is configured to attribute the directory to the client.
  • the flow when the first client 101 accesses the path ".../A/B/f" is as shown in steps S601-S615, wherein the directory B is a directory under the directory A, and the first target metadata f is a plurality of directories B. An item in the metadata.
  • Step S601 The first client 101 requests the home information of the directory A from the metadata routing service 303.
  • Step S602 The metadata routing server 303 finds the target metadata server to which the directory A belongs from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
  • Step S603 The first client 101 requests the target metadata server for the metadata describing the directory B under the directory A.
  • Step S604 The target metadata server sends an application request to the directory lock server 305, requesting the lock of the operation directory A (referred to as "home lock of directory A" for convenience of description) to obtain exclusive access rights of all metadata in the directory A. .
  • Step S605 The directory lock server 305 grants the home lock of the directory A to the target metadata server, so that only the target metadata server of all the metadata servers has the operation authority for all the metadata in the directory A.
  • Step S606 The target metadata server reads the metadata describing the directory B from the directory A.
  • Step S607 The target metadata server sends the read metadata describing the directory B to the first client 101.
  • Step S608 The first client 101 analyzes the metadata describing the directory B, and determines whether the directory B is a directory in which a plurality of metadata is stored.
  • Step S609 If it is determined that the directory B is a directory storing a plurality of metadata, the first client 101 sends an application request to the directory lock server 305, requesting access to the lock of the directory B (for the convenience of description, it may be referred to as "the acceleration of the directory B”. Lock”) to get exclusive access to all metadata in directory B.
  • Step S610 When the directory lock server 305 finds that the acceleration lock of the directory B is not granted to other clients, the directory lock server 305 grants the acceleration lock of the directory B to the first client 101, so that only the first client 101 among all the clients Have access to all metadata in directory B. Subsequent steps S611 to S615 are specific processes in which the first client 101 accesses the metadata in the directory B.
  • Step S611 The first client 101 sends a request message to the metadata routing server 303, requesting the metadata routing server to modify the correspondence between the directory and the target metadata server, and mapping the attribution of the directory B to the first client 101.
  • Step S612 The metadata routing server 303 modifies the correspondence between the directory and the target metadata server, attributes the directory B to the first client 101, and notifies the first client 101.
  • Step S613 The first client 101 applies to the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • Step S614 Based on the preemptive distributed lock rule, regardless of whether the home lock of the directory B is granted to other target metadata servers, the directory lock server 305 grants the home lock of the directory B to the first client 101, and informs the first The client 101 is such that only the first client 101 itself has the operational authority to the metadata in the target directory.
  • Step S615 The first client 101 operates on the first target metadata f process in the directory B, and obtains an operation result.
  • steps S601-S615 may be referred to the steps S302-S304 and the descriptions of the steps S311-S315, and details are not described herein again.
  • FIG. 8 is a schematic structural diagram of a terminal 80 according to an embodiment of the present invention.
  • the terminal 80 is a first client 80, which may include a receiving device 801, a sending device 802, and a memory 803.
  • the processor 804 (the processor may be one or more, and one processor in FIG. 8 is taken as an example), the receiving device 801, the transmitting device 802, the memory 803 and the processor 804 are respectively connected to the bus 805, wherein the memory 803 A set of metadata access program code is stored, and the processor 804 calls the metadata access program code stored in the memory 803 for performing the following operations:
  • the directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the processor 804 currently needs to access; exclusive access rights of all metadata in the target directory are used for the processing
  • the device 804 accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
  • the first target metadata is accessed when applying for exclusive access to all metadata in the target directory.
  • the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the directory lock server coordinates the access rights of each client to the target directory by granting a lock to the target directory, if the directory lock server will access The lock of the target directory is granted to the processor 804, then the processor 804 has the right to access all the metadata in the target directory; further, the distributed lock service can be set to preemptive distributed locks and non-preemptable distributed locks; For a preemptible distributed lock, when the processor 804 requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server releases the lock accessing the target directory according to the preemption mechanism, and re- The lock accessing the target directory is granted to the processor 804.
  • a distributed lock service such as a server carrying the zookeeper service
  • the directory lock server directly grants the lock accessing the target directory to the processor 804; if it is set to a non-preemptable distributed lock,
  • the directory server will be based on the lock mechanism can not be seized, the lock access to the target directory does not grant the first client 80, if access to the target directory The lock is not granted to other clients, and the directory lock server directly grants the lock to the target directory to the processor 804.
  • the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the processor 804 applies for the lock to access the target directory, it has the row of all the metadata in the target directory. It has access rights, that is, other clients than the processor 804 have no access to the metadata in the target directory unless the processor 804 actively releases the lock accessing the target directory.
  • the processor 804 performs the foregoing operation, and when accessing the first target metadata in the target directory, requests the directory lock server for exclusive access rights of all metadata in the target directory, so that the first client 80 can Successful access to the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply access rights, saving the overhead of maintaining cache coherency.
  • the receiving device 801 receives the node where the second client sent by the directory lock server is located. And sending, by the sending device 802, a request message to the second client according to the node information to access metadata in the target directory by using the second client, where the second client is The directory lock server applies to a client that has exclusive access to all metadata in the target directory.
  • the processor 704 by performing the above operation, accesses the first target element through the second client that has acquired exclusive access rights of all metadata in the target directory when the application does not obtain exclusive access rights of all metadata in the target directory.
  • the data enables the first client 80 to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
  • the manner in which the processor 804 sends the first application message to the directory lock server by using the sending device 802 to apply for exclusive access rights to all metadata in the target directory is specifically:
  • the first application message is sent to the directory lock server by the sending device 802 to apply for exclusive access rights to all metadata in the target directory.
  • the processor 804 determines whether the target directory is a directory for storing multiple metadata by acquiring metadata describing the target directory, and adding a “directory attribute” based on the existing metadata.
  • the value of the "directory attribute" indicates that the target directory is a different attribute, and one attribute is used to indicate that the target directory is a directory storing a plurality of metadata, and accordingly, the processor 804 analyzes the "directory attribute". It can be concluded whether the target directory is a directory that stores multiple metadata.
  • the processor 804 further uses to:
  • the processor 804 obtains the exclusive access permission of all the metadata in the target directory by performing the above operation, if the third client has the requirement of accessing the first target metadata in the target directory, the access target is provided for the third client.
  • the service of the first target metadata in the directory ensures that the client that does not apply for exclusive access to all metadata in the target directory can also access the first target metadata normally.
  • the processor 804 accesses the first target metadata
  • the processor 804 is requested to apply exclusive access rights to all metadata in the target directory. Also used for:
  • the second target metadata is accessed in accordance with the obtained exclusive access rights to the second target metadata.
  • the processor 804 can directly access the second target metadata when accessing other metadata (such as the second target metadata) in the target directory, and does not need to apply for access permission again, thereby improving the processor. 804 access to metadata in the target directory is efficient.
  • the manner in which the processor 804 accesses the first target metadata is specifically:
  • the receiving device 801 Receiving, by the receiving device 801, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server for exclusive operation on all metadata in the target directory. After the permission, the result of the operation of the first target metadata.
  • the target metadata server After the processor 804 performs the above operation, after receiving the request of the first target metadata in the target directory sent by the processor 804, the target metadata server performs the operation on the first target metadata in the target directory.
  • the directory lock server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata of the target directory is consistently saved. Sex.
  • the manner in which the processor 804 accesses the first target metadata is specifically:
  • the notification message sent by the metadata routing server is used to indicate that the operation authority of the target directory belongs to the first client 80;
  • Receiving, by the receiving device 801, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for
  • the processor 804 operates the first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
  • the first target metadata is operated according to the exclusive operation authority of all the metadata in the target directory, and the operation result is obtained.
  • the processor 804 performs the above operations to enable the first client 80 to function as both the client and the metadata server. After the processor 804 obtains exclusive access rights to all metadata in the target directory, the remote client does not need to be associated with the remote terminal.
  • the data server interacts to request the metadata server to operate on the first target metadata, reducing the overhead between the nodes, and improving the access efficiency of accessing the first target metadata.
  • the processor 804 after the processor 804 accesses the first target metadata, when the processor 804 accesses the exclusive access rights to all the metadata in the target directory, the processor 804 further uses to:
  • the processor 804 detects the load generated by accessing the metadata in the target directory in real time, and the detection manner includes but is not limited to the following two cases:
  • the processor 804 detects the overall load of the first client 80 in real time, and compares the load value of the overall load with a preset threshold to determine whether the overall load of the first client 80 exceeds a preset.
  • Threshold value which is a load value used for reference comparison according to actual needs.
  • the processor 804 records the first load generated by the first client 80 actively accessing the metadata in the target directory, and accesses the metadata in the target directory according to the request message of other clients.
  • the processor 804 can also refer to the corresponding description of the embodiment of the embodiment of the present invention, and details are not described herein again.
  • the processor 804 automatically releases the exclusive access rights of all the metadata in the target directory when the load on the first client 80 is too high, and allows the fourth client with a lower load to obtain the target directory.
  • Exclusive access to all metadata for load sharing purposes; or access to metadata in the target directory at the first client is more frequent than other clients accessing the target directory
  • the exclusive access permission of all metadata in the target directory is automatically released, so that the client with higher access frequency obtains exclusive access rights of all metadata in the target directory, so as to reduce the load generated by forwarding access. Improves the efficiency of accessing metadata in the target directory.
  • FIG. 9 is a schematic structural diagram of still another terminal 90 according to an embodiment of the present invention.
  • the terminal 90 is a directory lock server 90, which may include a receiving device 901, a sending device 902, and a memory 903.
  • the processor 904 (the processor may be one or more, and one processor in FIG. 9 is taken as an example), the receiving device 901, the transmitting device 902, the memory 903 and the processor 904 are respectively connected to the bus 905, wherein the memory 903 A set of metadata access program code is stored, and the processor 904 calls the metadata access program code stored in the memory 903 for performing the following operations:
  • a first application message sent by the first client where the first client accesses the target metadata in the target directory, where the first application message is used to apply to the target directory.
  • Exclusive access rights of all metadata the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the second client If the second client is not granted, grant exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, pass the sending device 902 Sending the node information of the second client to the first client, so that the first client accesses the target element by using the second client based on the node information of the second client data.
  • the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the processor 904 can coordinate the access of each client to the target directory by granting a lock accessing the target directory based on the distributed lock service.
  • a distributed lock service such as a server carrying the zookeeper service
  • the processor 904 Permission, if the processor 904 grants the lock accessing the target directory to the first client, the first client has the right to access all the metadata in the target directory; further, the distributed lock service can be set to preemptively distribute Lock and non-preemptive distributed lock; if set to preemptible distributed lock, when the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the processor 904 will preempt according to the preemption The mechanism releases the lock accessing the target directory, and re-grants the lock accessing the target directory to the first client. If the lock accessing the target directory is not granted to other clients, the processor 904 directly grants the lock accessing the target directory to the first.
  • the processor 904 if set to a non-preemptable distributed lock, the first client requests access to the target When the lock is recorded, if the lock accessing the target directory has been granted to other clients, the processor 904 will not grant the lock to the target client to the first client according to the non-preemptive mechanism, if the lock of the target directory is not accessed. Granting to other clients, the processor 904 directly grants the lock accessing the target directory to the first client.
  • the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the first client applies for the lock to access the target directory, it has all the metadata in the target directory. Exclusive access rights, that is, other customers besides the first client The terminal does not have access to the metadata in the target directory unless the first client actively releases the lock accessing the target directory.
  • the processor 904 performs the above operation, when the first client accesses the target metadata in the target directory, the processor 904 grants all the exclusive access rights of all the metadata in the target directory to the first client at one time to maintain the cache consistency. Sex, in this way, the first client can successfully access the target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • the processor 904 grants exclusive access rights of all metadata in the target directory to the first client, the processor 904 is also used to:
  • the end accesses the target metadata.
  • the processor 904 After the processor 904 performs the above operation, after the first client obtains the exclusive access right of all the metadata in the target directory, if the third client has the requirement to access the target directory target metadata, the third client may be based on the processing.
  • the node information sent by the device 904 sends the access request to the first client, and accesses the target metadata through the first client, so that the client that does not apply for exclusive access rights of all metadata in the target directory can also be normal. Access target metadata.
  • the processor 904 grants exclusive access rights of all metadata in the target directory to the first client, the processor 904 is also used to:
  • the target metadata server receives the request for the target metadata in the target directory sent by the first client, and then proceeds to the processor 904 before operating the target metadata in the target directory. Approve exclusive access rights for all metadata in the target directory. Accordingly, the processor 904 grants the exclusive operation authority of all metadata in the target directory to the target metadata server, so that there is always only one metadata server in all the metadata servers. It has the operation permission on the target directory, which ensures the persistence and preservation consistency of the metadata in the target directory.
  • the processor 904 will After the exclusive access rights of all the metadata in the target directory are granted to the first client, the processor 904 is further configured to:
  • processor 904 for performing the above operations may also refer to the corresponding description of the embodiment of the embodiment of the present invention, and details are not described herein again.
  • the processor 904 performs the above operation, and after the first client simultaneously functions as the client and the metadata server, the processor 904 grants exclusive access rights to all metadata in the target directory of the first client, first The client does not need to interact with the remote metadata server to request the metadata server to operate on the target metadata, which reduces the overhead between the nodes and improves the access efficiency of the access target metadata.
  • FIG. 10 is a schematic structural diagram of a distributed file system 100 according to an embodiment of the present invention.
  • the distributed file system 100 includes a first client 83, a directory lock server 84, and a second client 85. :
  • the first client 83 sends a first application message to the directory lock server 84 to request exclusive access rights to all metadata in the target directory when requesting access to the first target metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the The first client accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
  • the directory lock server 84 receives the first application message sent by the first client 83, and determines whether the exclusive access rights of all metadata in the target directory are granted to the second client 85;
  • the directory lock server 84 grants the exclusive access rights of all metadata in the target directory to the first a client 83, and sending a notification message to the first client to notify the first client of exclusive access rights of all metadata in the target directory;
  • the first client 83 receives the notification message and accesses the first target metadata according to the notification message.
  • the first client 83 in the distributed file system 100 can refer to the terminal 80 described in the embodiment shown in FIG. 8, and the directory lock server 84 can refer to the terminal 90 described in the embodiment shown in FIG. No longer repeat them.
  • the directory lock server 84 is applied for exclusive access rights of all metadata in the target directory at one time, so that The first client 83 can successfully access the first target metadata, and does not need to re-apply for access rights when accessing other metadata in the target directory, which saves the overhead of maintaining cache consistency.
  • FIG. 11 is a schematic structural diagram of still another terminal 110 according to an embodiment of the present invention.
  • the terminal 110 is a first client 110, which includes an application unit 1101 and a first access unit 1103.
  • the description of each unit is as follows:
  • the application unit 1101 is configured to, when requesting access to the first target metadata in the target directory, send a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the The first client accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
  • the first access unit 1103 is configured to access the first target metadata when the application unit 1101 applies for exclusive access rights to all metadata in the target directory.
  • the first access unit 1103 is further configured to: when the application unit 1101 does not apply for exclusive access to all metadata in the target directory, receive the directory lock server to send Node information of the second client, and sending a request message to the second client according to the node information to access metadata in the target directory by the second client, the second client A client that requests exclusive access to all metadata in the target directory to the directory lock server.
  • the application unit 1101 includes an acquisition subunit, a determination subunit, and an application subunit, where:
  • Obtaining a subunit configured to obtain metadata describing the target directory, where the metadata describing the target directory is provided with identifier information for identifying whether the target directory is a directory in which a plurality of metadata is stored;
  • a determining subunit configured to determine, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
  • the application subunit is configured to send a first application message to the directory lock server to request exclusive access rights to all metadata in the target directory when the judgment result of the subunit is YES.
  • the terminal 110 further includes:
  • a receiving unit configured to receive a request message sent by the third client, where the request message is used to request to access the first target metadata by using the first access unit 1103, where the request message is the third client Applying, to the directory lock server, exclusive access rights to all metadata in the target directory, and receiving node information of the first client returned by the directory lock server, according to the node of the first client a message sent by the information to the first client;
  • a sending unit configured to access the first target metadata according to the request message, and send an access result of accessing the first target metadata to the third client.
  • the terminal 110 further includes:
  • a second access unit configured to: when requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata in the target directory other than the first target metadata;
  • the first access unit 1103 includes:
  • a requesting subunit configured to request, by the metadata routing server, attribution information of the target directory, where the metadata routing server prestores the attribution information of the directory;
  • a first receiving subunit configured to receive the home information of the target directory sent by the metadata routing server, where the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
  • a first sending subunit configured to send an operation request message to the target metadata server, requesting the target metadata server to operate the first target according to the exclusive access right to all metadata in the target directory Metadata
  • a second receiving subunit configured to receive an operation result sent by the target metadata server; the operation result is that the target metadata server requests the directory lock server to queue all metadata in the target directory The result of the operation of the first target metadata after it operates the authority.
  • the first access unit 1103 includes:
  • a second sending subunit configured to send a switching message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client;
  • a third receiving subunit configured to receive a notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client;
  • An application subunit configured to send a second application message to the directory lock server to apply for exclusive operation authority on all metadata in the target directory
  • a fourth receiving subunit configured to receive a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for
  • the first client operates first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata
  • An operation subunit configured to operate the first target metadata according to an exclusive operation authority of all metadata in the target directory, to obtain an operation result.
  • the terminal 110 further includes:
  • a determining unit configured to determine whether a load generated by accessing metadata in the target directory meets a preset condition
  • a release unit configured to release exclusive access rights of all metadata in the target directory when the determination result of the determining unit is yes, so that the directory lock server will all metadata in the target directory Exclusive access is granted to the fourth client.
  • the first client (terminal) 110 accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time.
  • the first client can successfully access the first target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • FIG. 12 is still another terminal 120 according to an embodiment of the present invention.
  • the terminal 120 is a directory lock server 120, which includes a first receiving unit 1201, a determining unit 1202, and a first granting unit. 1203, wherein each unit is described as follows.
  • the first receiving unit 1201 is configured to receive, when the first client accesses the target metadata in the target directory, the first application message sent by the first client, where the first application message is used to apply for the target directory.
  • Exclusive access rights of all the metadata in the directory the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the determining unit 1202 is configured to determine, according to the first application message, whether an exclusive access right of all metadata in the target directory has been granted to the second client, and exclusive access rights of all metadata in the target directory are used for Providing, by the first client, target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
  • the first granting unit 1203 is configured to, when the determining result of the determining unit 1202 is negative, grant exclusive access rights of all metadata in the target directory to the first client; in the judgment of the determining unit 1202
  • the node information of the second client is sent to the first client, so that the first client passes the second client based on the node information where the second client is located. Access the target metadata.
  • the terminal 120 further includes:
  • a second receiving unit configured to receive the first application message sent by the third client when the third client accesses the target metadata
  • a first sending unit configured to send, to the third client, node information where the first client is located, so that the third client passes the first node based on node information where the first client is located
  • the client accesses the target metadata.
  • the terminal 120 further includes:
  • a third receiving unit configured to receive a second application message sent by the target metadata server, where the second application message is an operation sent by the target metadata server when the first client accesses the target metadata a message sent during the message to apply for exclusive operation rights to all metadata in the target directory;
  • a second granting unit configured to grant an exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server pairs the target metadata Performing an operation and feeding back an operation result to the first client, wherein an exclusive operation authority of all metadata in the target directory is used for the target metadata server to operate target metadata in the target directory, And operating metadata in the target directory other than the target metadata.
  • the terminal 120 further includes:
  • a fourth receiving unit configured to receive a second application message sent by the first client, where the second The application message is used to apply for exclusive operation rights on all metadata in the target directory;
  • a third granting unit configured to grant an exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client pairs the target metadata Performing operations, wherein exclusive operation rights of all metadata in the target directory are used by the first client to operate target metadata in the target directory, and operating the target directory except the target element Metadata outside of the data.
  • the directory lock server 120 For the principle of accessing the metadata of the terminal (the directory lock server) 120, reference may be made to the corresponding description of the embodiment of the present invention corresponding to FIG. 3 to FIG. 7 , and details are not described herein again.
  • the directory lock server is requested to access all the metadata in the target directory at a time, so that the first client The target metadata can be successfully accessed, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client accesses the first target metadata in the target directory
  • the first time accesses the directory lock server to the exclusive access permission of all the metadata in the target directory, so that the first client accesses the first target metadata in the target directory.
  • the first client can successfully access the first target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • the third client sends the access request to the The first client accesses the first target metadata through the first client, so that the client that does not apply for exclusive access rights of all metadata in the target directory can also normally access the first target metadata;
  • the metadata server After receiving the request for the first target metadata in the access target directory sent by the first client, the metadata server first applies to the directory lock server for all metadata rows in the target directory before performing the operation on the first target metadata.
  • the first client It has the right to operate, so that only one metadata server in all metadata servers has the operation authority to the target directory, which ensures the persistence and preservation consistency of the metadata in the target directory; further, the first client also acts as the client at the same time. End and metadata server functions, the first client gets exclusive access to all metadata in the target directory After the limitation, there is no need to interact with the remote metadata server to request the metadata server to operate on the first target metadata, thereby reducing the overhead between the nodes and improving the access efficiency of accessing the first target metadata; further Ground, when the load on the first client is too high, the exclusive access rights of all metadata in the target directory are automatically released, and the fourth client with lower load is obtained to obtain exclusive access rights of all metadata in the target directory.
  • Permissions allow clients with higher frequency of access to obtain exclusive access to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
  • the readable storage medium when executed, may include the flow of an embodiment of the methods as described above.
  • the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Storage Device Security (AREA)
  • Human Computer Interaction (AREA)

Abstract

Disclosed in an embodiment of the invention are a metadata access method, a related apparatus and a system. The method comprises: when requesting access to a first target metadata item in a target directory, transmitting, by a first client, a first application message to a directory lock server to apply for an exclusive access permission for all metadata items in the target directory, wherein the target directory is a directory storing multiple metadata items, and the exclusive access permission for all metadata items in the target directory is configured to allow the first client to access the first target metadata item as well as the metadata items other than the first target metadata item in the target directory; and upon grant of the permission, accessing the first target metadata item in the target directory. In the invention, acquiring the exclusive access permission once enables access to all metadata items in the target directory, thereby reducing overheads required for maintaining buffer consistency.

Description

一种元数据访问方法、相关设备及系统Metadata access method, related device and system 技术领域Technical field
本发明涉及计算机技术领域,尤其涉及一种元数据访问方法、相关设备及系统。The present invention relates to the field of computer technologies, and in particular, to a metadata access method, related device, and system.
背景技术Background technique
分布式文件系统(Distributed File System,GFS)采用了文件的元数据和数据访问分离的架构,通过独立的元数据服务子系统提供元数据访问功能,使分布式文件系统具备了一定的扩展能力,能够提供海量文件的存储和访问服务。请参见图1,GFS中元数据访问的部分包括三个子系统:元数据客户子系统10、元数据服务子系统20和集群服务子系统30,元数据客户子系统10包括多个客户端(Client),图1中以客户端101~103为例;元数据服务子系统20包括多个元数据服务器,图1中以元数据服务器201~203为例;集群服务子系统30主要包括集群管理服务器301、分布式一致性服务器302和元数据路由服务器303。Distributed File System (GFS) adopts the file metadata and data access separation architecture, provides metadata access function through independent metadata service subsystem, and makes the distributed file system have certain expansion capabilities. Ability to provide storage and access services for massive files. Referring to FIG. 1, the metadata access portion of the GFS includes three subsystems: a metadata client subsystem 10, a metadata service subsystem 20, and a cluster service subsystem 30. The metadata client subsystem 10 includes a plurality of clients (Client). The client 101-103 is taken as an example in FIG. 1; the metadata service subsystem 20 includes a plurality of metadata servers, and the metadata servers 201-203 are taken as an example in FIG. 1. The cluster service subsystem 30 mainly includes a cluster management server. 301. A distributed consistency server 302 and a metadata routing server 303.
图1中的客户端访问元数据的原理如图2所示,当客户端101需要访问目标元数据时,先向分布式一致性服务器302发送对该目标元数据的排它访问权限,如果申请到了该排它访问权限,那么该客户端101就是所有的客户端101~103中唯一可以直接请求元数据服务器201~203对该目标元数据进行操作的客户端,这样一来就保证了该目标元数据的缓存一致性。之后客户端101向元数据路由服务器303查询目标元数据的归属信息,即查询该目标元数据归属到哪个元数据服务器操作;元数据路由服务器303根据存储的元数据与元数据服务器的映射关系查询目标元数据的归属信息,并将查询到的归属信息发送给客户端101;该客户端101对该归属信息进行分析,当分析出目标元数据归属到元数据服务器201时,向该元数据服务器201发出请求以请求访问目标元数据,元数据服务器201接收客户端101的请求,并根据该请求对目标元数据进行操作,然后将操作结果反馈给客户端101,相应地,客户端101接收元数据服务器201发送的操作结果,并将该操作结果缓存到本地,至此,客户端101访问目标元数据结束。The principle of the client access metadata in FIG. 1 is as shown in FIG. 2. When the client 101 needs to access the target metadata, the distributed consistency server 302 is first sent exclusive access rights to the target metadata. When the exclusive access right is reached, the client 101 is the only client among all the clients 101-103 that can directly request the metadata server 201-203 to operate the target metadata, thus ensuring the target. Metadata cache coherency. Then, the client 101 queries the metadata routing server 303 for the attribution information of the target metadata, that is, queries which metadata server operation the target metadata belongs to; the metadata routing server 303 queries according to the mapping relationship between the stored metadata and the metadata server. The attribution information of the target metadata is sent to the client 101; the client 101 analyzes the attribution information, and when the target metadata is analyzed and belongs to the metadata server 201, the metadata server is sent to the metadata server. 201 issues a request to request access to the target metadata, the metadata server 201 receives the request of the client 101, and operates on the target metadata according to the request, and then feeds back the operation result to the client 101, and accordingly, the client 101 receives the element. The result of the operation sent by the data server 201 is cached locally, and the client 101 accesses the target metadata at this point.
上述分布式一致性服务器302授予客户端101对目标元数据的排它访问权限可以通过分布式锁、分布式协议等技术来实现,当通过分布式锁来时,分布式一致性服务器302和客户端101之间需要进行交互来执行锁申请、锁释放以及锁召回的操作,该过程中节点间的通信开销较大;尤其在海量小文件(Lot of Small Files,LoSF)的场景下,分布式文件系统(GFS)中的小文件的元数据非常多,客户端101访问其中每个小文件的元数据时都要与分布式一致性服务器进行锁申请、锁释放以及锁召回的操作,这个过程产生的巨大的通信开销降低了客户端访问元数据的效率。 The above-mentioned distributed consistency server 302 grants the client 101 exclusive access rights to the target metadata, which can be implemented by a distributed lock, a distributed protocol, etc., when distributed through the distributed lock, the distributed consistency server 302 and the client The interaction between the terminals 101 is required to perform the lock application, the lock release, and the lock recall operation. The communication overhead between the nodes is large in the process; especially in the scenario of Lot of Small Files (LoSF), distributed The metadata of the small files in the file system (GFS) is very large. When the client 101 accesses the metadata of each small file, the lock application, the lock release, and the lock recall operation are performed with the distributed consistency server. The huge communication overhead generated reduces the efficiency of client access to metadata.
发明内容Summary of the invention
本发明实施例公开了一种元数据的访问方法、相关设备及系统,能够减少客户端访问元数据的过程中维护元数据缓存一致性的开销,提高元数据的访问效率。The embodiment of the invention discloses a method for accessing metadata, a related device and a system, which can reduce the overhead of maintaining metadata cache consistency in the process of accessing metadata by the client, and improve the access efficiency of the metadata.
第一方面,本发明实施例提供了一种元数据的访问方法,该方法包括:In a first aspect, an embodiment of the present invention provides a method for accessing metadata, where the method includes:
第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述第一客户端当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target directory For storing a directory having a plurality of metadata, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the first a client accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据。The first client accesses the first target metadata when the first client applies for exclusive access to all metadata in the target directory.
通过执行上述步骤,第一客户端访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的排它访问权限,这样一来,第一客户端就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。By performing the above steps, when the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
结合第一方面,在第一方面的第一种可能的实现方式中,当所述第一客户端未申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端接收所述目录锁服务器发送的第二客户端所在的节点信息,并根据所述节点信息向所述第二客户端发送请求消息以通过所述第二客户端访问所述第一目标元数据,所述第二客户端为向所述目录锁服务器申请到了对所述目标目录中所有元数据的排它访问权限的客户端。With reference to the first aspect, in a first possible implementation manner of the first aspect, when the first client does not apply for exclusive access rights to all metadata in the target directory, the first client Receiving, by the terminal, the node information of the second client sent by the directory lock server, and sending a request message to the second client according to the node information, to access the first target metadata by using the second client The second client is a client that applies to the directory lock server for exclusive access to all metadata in the target directory.
通过执行上述步骤,第一客户端在申请不到目标目录中所有元数据的排它访问权限时,通过已经获取了目标目录中所有元数据的排它访问权限的第二客户端访问第一目标元数据,使得第一客户端在申请不到目标目录中所有元数据的排它访问权限时依然可以顺利访问到目标元数据。By performing the above steps, the first client accesses the first target through the second client that has obtained exclusive access rights of all metadata in the target directory when the application does not have exclusive access rights to all the metadata in the target directory. The metadata enables the first client to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
结合第一方面,或者第一方面的第一种可能的实现方式,在第一方面的第二种可能的实现方式中,所述第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限包括:With reference to the first aspect, or the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the first client is requesting access to the first target metadata in the target directory When the first application message is sent to the directory lock server to apply for exclusive access to all metadata in the target directory, the following:
第一客户端在请求访问目标目录中的第一目标元数据时,获取描述所述目标目录的元数据,所述描述所述目标目录的元数据设有用于标识所述目标目录是否为存储有多个元数据的目录的标识信息;The first client obtains metadata describing the target directory when requesting access to the first target metadata in the target directory, and the metadata describing the target directory is configured to identify whether the target directory is stored. Identification information of a directory of multiple metadata;
根据所述描述所述目标目录的元数据来判断所述目标目录是否为存储有多个元数据的目录;Determining, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
若是,则向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限。 If so, a first application message is sent to the directory lock server to request exclusive access to all metadata in the target directory.
结合第一方面,或者第一方面的第一种可能的实现方式,或者第一方面的第二种可能的实现方式,在第一方面的第三种可能的实现方式中,所述当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据之后,所述方法还包括:With reference to the first aspect, or the first possible implementation manner of the first aspect, or the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, After the first client requests the exclusive access to all the metadata in the target directory, after the first client accesses the first target metadata, the method further includes:
所述第一客户端接收第三客户端发送的请求消息,所述请求消息用于请求通过所述第一客户端访问所述第一目标元数据,所述请求消息为所述第三客户端向所述目录锁服务器申请对所述目标目录中所有元数据的排它访问权限,并接收到所述目录锁服务器返回的所述第一客户端的节点信息后,根据所述第一客户端的节点信息向所述第一客户端发送的消息;Receiving, by the first client, a request message sent by a third client, where the request message is used to request access to the first target metadata by using the first client, where the request message is the third client Applying, to the directory lock server, exclusive access rights to all metadata in the target directory, and receiving node information of the first client returned by the directory lock server, according to the node of the first client a message sent by the information to the first client;
所述第一客户端根据所述请求消息访问所述第一目标元数据,并将访问所述第一目标元数据的访问结果发送给所述第三客户端。The first client accesses the first target metadata according to the request message, and sends an access result of accessing the first target metadata to the third client.
通过执行上述步骤,第一客户端获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录中第一目标元数据的需求,则第三客户端将访问请求发送至第一客户端,通过第一客户端访问该第一目标元数据,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问第一目标元数据。After performing the above steps, after the first client obtains exclusive access rights to all metadata in the target directory, if the third client has a requirement to access the first target metadata in the target directory, the third client will access the request. Sending to the first client, accessing the first target metadata by the first client, ensuring that the client that does not apply for exclusive access rights of all metadata in the target directory can also access the first target metadata normally. .
结合第一方面,或者第一方面的第一种可能的实现方式,或者第一方面的第二种可能的实现方式,或者第一方面的第三种可能的实现方式,在第一方面的第四种可能的实现方式中,所述当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据之后,所述方法还包括:With reference to the first aspect, or the first possible implementation of the first aspect, or the second possible implementation of the first aspect, or the third possible implementation of the first aspect, in the first aspect In four possible implementation manners, when the first client applies exclusive access rights to all metadata in the target directory, after the first client accesses the first target metadata, The method further includes:
所述第一客户端在请求访问所述目标目录中的第二目标元数据时,如果第二目标元数据属于所述目标目录中除所述第一目标元数据之外的元数据;The first client, when requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata in the target directory other than the first target metadata;
则所述第一客户端按照已获得的对所述第二目标元数据的排它访问权限,对所述第二目标元数据进行访问。And then the first client accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
通过执行上述步骤,第一客户端在访问目标目录中的其他元数据(如第二目标元数据)时,可以直接对该第二目标元数据进行访问,无需再次申请访问访问权限,提高了第一客户端访问目标目录中的元数据的效率。By performing the above steps, when the first client accesses other metadata (such as the second target metadata) in the target directory, the second target metadata can be directly accessed, and the access permission is not required to be applied again. The efficiency of a client's access to metadata in the target directory.
结合第一方面,或者第一方面的第一种可能的实现方式,或者第一方面的第二种可能的实现方式,或者第一方面的第三种可能的实现方式,或者第一方面的第四种可能的实现方式,在第一方面的第五种可能的实现方式中,所述第一客户端访问所述第一目标元数据包括:With reference to the first aspect, or the first possible implementation of the first aspect, or the second possible implementation of the first aspect, or the third possible implementation of the first aspect, or the first aspect In a fourth possible implementation manner of the first aspect, the first client accessing the first target metadata includes:
所述第一客户端向元数据路由服务器请求所述目标目录的归属信息,所述元数据路由服务器预存有目录的归属信息;Determining, by the first client, the home information of the target directory to the metadata routing server, where the metadata routing server pre-stores the home information of the directory;
所述第一客户端接收所述元数据路由服务器发送的所述目标目录的归属信息,所述目标目录的归属信息用于指示所述目标目录的操作权限归属到目标元数据服务器;The first client receives the home information of the target directory sent by the metadata routing server, and the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
所述第一客户端向所述目标元数据服务器发送操作请求消息,请求所述目标元数据服务器根据所述对所述目标目录中所有元数据的排它访问权限操作 所述第一目标元数据;Sending, by the first client, an operation request message to the target metadata server, requesting the target metadata server to operate according to the exclusive access right to all metadata in the target directory The first target metadata;
所述第一客户端接收所述目标元数据服务器发送的操作结果;所述操作结果为所述目标元数据服务器向所述目录锁服务器申请到对所述目标目录中所有元数据的排它操作权限后,对所述第一目标元数据进行操作的结果。Receiving, by the first client, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server to perform exclusive operation on all metadata in the target directory. After the permission, the result of the operation of the first target metadata.
通过执行上述步骤,目标元数据服务器接收到第一客户端发送的访问目标目录下的第一目标元数据的请求后,在对目标目录中的第一目标元数据进行操作之前,先向目录锁服务器申请目标目录中所有元数据的排它操作权限,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性。After performing the above steps, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server first locks the directory before operating the first target metadata in the target directory. The server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata retention consistency of the target directory is ensured.
结合第一方面,或者第一方面的第一种可能的实现方式,或者第一方面的第二种可能的实现方式,或者第一方面的第三种可能的实现方式,或者第一方面的第四种可能的实现方式,在第一方面的第六种可能的实现方式中,所述第一客户端访问所述第一目标元数据包括:With reference to the first aspect, or the first possible implementation of the first aspect, or the second possible implementation of the first aspect, or the third possible implementation of the first aspect, or the first aspect In a fourth possible implementation manner of the first aspect, the first client accessing the first target metadata includes:
所述第一客户端向元数据路由服务器发送切换消息,指示所述元数据路由服务器将所述目标目录的操作权限归属到所述第一客户端;Sending, by the first client, a handover message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client;
所述第一客户端接收所述元数据路由服务器发送的通知消息,所述通知消息用于指示所述目标目录的操作权限归属到所述第一客户端;Receiving, by the first client, the notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client;
所述第一客户端向所述目录锁服务器发送第二申请消息以申请对所述目标目录中所有元数据的排它操作权限;The first client sends a second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory;
所述第一客户端接收所述目录锁服务器发送的授予所述目标目录中所有元数据的排它操作权限的授予消息,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的第一目标元数据,以及操作所述目标目录中除所述第一目标元数据之外的元数据;Receiving, by the first client, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for The first client operates the first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
所述第一客户端根据所述目标目录中所有元数据的排它操作权限对所述第一目标元数据进行操作,获得操作结果。The first client operates the first target metadata according to the exclusive operation authority of all metadata in the target directory, and obtains an operation result.
通过执行上述步骤,第一客户端同时充当客户端和元数据服务器的功能,第一客户端获取到目标目录中所有元数据的排它访问权限后,不需要与远端的元数据服务器进行交互来请求元数据服务器对第一目标元数据进行操作,减少了节点之间的开销,提高了访问第一目标元数据的访问效率。By performing the above steps, the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, the first client does not need to interact with the remote metadata server. To request the metadata server to operate on the first target metadata, the overhead between the nodes is reduced, and the access efficiency of accessing the first target metadata is improved.
结合第一方面,或者第一方面的第一种可能的实现方式,或者第一方面的第二种可能的实现方式,或者第一方面的第五种可能的实现方式,或者第一方面的第六种可能的实现方式,在第一方面的第七种可能的实现方式中,所述当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据之后,所述方法还包括:With reference to the first aspect, or the first possible implementation of the first aspect, or the second possible implementation of the first aspect, or the fifth possible implementation of the first aspect, or the first aspect In a seventh possible implementation manner of the first aspect, the first client, when the first client applies exclusive access rights to all metadata in the target directory, After the first client accesses the first target metadata, the method further includes:
所述第一客户端判断访问所述目标目录中的元数据所产生的负载是否满足预设条件;Determining, by the first client, whether a load generated by accessing metadata in the target directory meets a preset condition;
若满足,则所述第一客户端释放所述目标目录中的所有元数据的排它访问权限,以便所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予第四客户端。 If satisfied, the first client releases exclusive access rights of all metadata in the target directory, so that the directory lock server grants exclusive access rights of all metadata in the target directory to the fourth client end.
通过执行上述步骤,第一客户端上的负载过高时,自动释放目标目录中所有元数据的排它访问权限,让负载较低的第四客户端来获取该目标目录中所有元数据的排它访问权限以达到分担负载的目的;或者在第一客户端访问目标目录中的元数据的频率比其他客户端访问目标目录中的元数据的频率低时,自动释放目标目录中所有元数据的排它访问权限,让访问频率较高的客户端获取目标目录中所有元数据的排它访问权限,以减少转发访问所产生的负载,提高了访问目标目录中元数据的效率。By performing the above steps, when the load on the first client is too high, the exclusive access permission of all metadata in the target directory is automatically released, and the fourth client with lower load is obtained to obtain the row of all metadata in the target directory. It accesses the load to achieve the purpose of sharing the load; or automatically releases all metadata in the target directory when the first client accesses the metadata in the target directory less frequently than the metadata in other clients accessing the target directory. Exclusive access rights allow clients with higher frequency of access to obtain exclusive access rights to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
第二方面,本发明实施例提供一种元数据访问方法,该方法包括:In a second aspect, an embodiment of the present invention provides a metadata access method, where the method includes:
目录锁服务器在第一客户端访问目标目录中的目标元数据时,接收所述第一客户端发送的第一申请消息,所述第一申请消息用于申请对所述目标目录中所有元数据的排它访问权限,所述目标目录为存储有多个元数据的目录,所述目标元数据为所述第一客户端当前需要访问的元数据;The directory lock server receives the first application message sent by the first client when the first client accesses the target metadata in the target directory, and the first application message is used to apply for all metadata in the target directory. Exclusive access rights, the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
所述目录锁服务器根据所述第一申请消息判断所述目标目录中所有元数据的排它访问权限是否已经授予了第二客户端,所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的目标元数据,以及访问所述目标目录中除所述目标元数据之外的元数据;The directory lock server determines, according to the first application message, whether exclusive access rights of all metadata in the target directory have been granted to the second client, and exclusive access rights of all metadata in the target directory are used for Providing, by the first client, target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
若未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端;若授予了所述第二客户端,则所述目录锁服务器向所述第一客户端发送所述第二客户端所在的节点信息,以使所述第一客户端基于所述第二客户端所在的节点信息通过所述第二客户端访问所述目标元数据。If the second client is not granted, the directory lock server grants exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, Transmitting, by the directory lock server, node information of the second client to the first client, so that the first client accesses the second client by using node information of the second client The target metadata.
通过执行上述步骤,第一客户端访问目标目录中的目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的排它访问权限,这样一来,第一客户端就可以顺利访问目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。By performing the above steps, when the first client accesses the target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can successfully access the target. Metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
结合第二方面,在第二方面的第一种可能的实现方式中,所述若未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述方法还包括:With reference to the second aspect, in a first possible implementation manner of the second aspect, if the second client is not granted, the directory lock server performs exclusive access of all metadata in the target directory. After the rights are granted to the first client, the method further includes:
所述目录锁服务器在第三客户端访问所述目标元数据时,接收所述第三客户端发送的所述第一申请消息;Receiving, by the third client, the first application message sent by the third client when the third client accesses the target metadata;
所述目录锁服务器向所述第三客户端发送所述第一客户端所在的节点信息,以使所述第三客户端基于所述第一客户端所在的节点信息通过所述第一客户端访问所述目标元数据。The directory lock server sends the node information of the first client to the third client, so that the third client passes the first client based on the node information where the first client is located. Access the target metadata.
通过执行上述步骤,第一客户端获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录中目标元数据的需求,则第三客户端将访问请求发送至第一客户端,通过第一客户端访问该目标元数据,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问目标元数据。 After performing the above steps, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement to access the target metadata in the target directory, the third client sends the access request to the third client. The first client accesses the target metadata through the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the target metadata normally.
结合第二方面,或者第二方面的第一种可能的实现方式,在第二方面的第二种可能的实现方式中,所述若未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述方法还包括:With reference to the second aspect, or the first possible implementation manner of the second aspect, in the second possible implementation manner of the second aspect, if the second client is not granted, the directory lock server After the exclusive access rights of all the metadata in the target directory are granted to the first client, the method further includes:
所述目录锁服务器接收目标元数据服务器发送的第二申请消息,所述第二申请消息为所述目标元数据服务器接收到所述第一客户端访问所述目标元数据时发送的操作消息时发送的,用于申请对所述目标目录中所有元数据的排它操作权限的消息;Receiving, by the directory lock server, a second application message sent by the target metadata server, where the second application message is an operation message sent by the target metadata server when the first client accesses the target metadata a message sent for applying for exclusive operation rights to all metadata in the target directory;
所述目录锁服务器根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述目标元数据服务器,以使所述目标元数据服务器对所述目标元数据进行操作并将操作结果反馈给所述第一客户端,其中,所述目标目录中所有元数据的排它操作权限用于供所述目标元数据服务器操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。The directory lock server grants exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server operates the target metadata And feeding back the operation result to the first client, wherein the exclusive operation authority of all the metadata in the target directory is used for the target metadata server to operate the target metadata in the target directory, and the operation Metadata in the target directory other than the target metadata.
通过执行上述步骤,目标元数据服务器接收到第一客户端发送的访问目标目录下的目标元数据的请求后,在对目标目录中的目标元数据进行操作之前,先向目录锁服务器申请目标目录中所有元数据的排它操作权限,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性。After performing the above steps, after receiving the request for the target metadata in the target directory sent by the first client, the target metadata server requests the target directory from the directory lock server before performing the operation on the target metadata in the target directory. The exclusive operation permission of all the metadata in the metadata makes that only one metadata server in all the metadata servers has the operation authority for the target directory, which ensures the persistent preservation consistency of the metadata in the target directory.
结合第二方面,或者第二方面的第一种可能的实现方式,在第二方面的第三种可能的实现方式中,所述若未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述方法还包括:With reference to the second aspect, or the first possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, if the second client is not granted, the directory lock server After the exclusive access rights of all the metadata in the target directory are granted to the first client, the method further includes:
所述目录锁服务器接收所述第一客户端发送的第二申请消息,所述第二申请消息用于申请对所述目标目录中所有元数据的排它操作权限;The directory lock server receives a second application message sent by the first client, where the second application message is used to apply for exclusive operation rights on all metadata in the target directory;
所述目录锁服务器根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述第一客户端,以使所述第一客户端对所述目标元数据进行操作,其中,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。The directory lock server grants exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client operates the target metadata And an exclusive operation authority of all metadata in the target directory for the first client to operate target metadata in the target directory, and operating the target directory except the target metadata External metadata.
通过执行上述步骤,第一客户端同时充当客户端和元数据服务器的功能,第一客户端获取到目标目录中所有元数据的排它访问权限后,不需要与远端的元数据服务器进行交互来请求元数据服务器对目标元数据进行操作,减少了节点之间的开销,提高了访问目标元数据的访问效率。By performing the above steps, the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, the first client does not need to interact with the remote metadata server. To request the metadata server to operate on the target metadata, the overhead between the nodes is reduced, and the access efficiency of the access target metadata is improved.
第三方面,本发明实施例提供一种终端,其特征在于,所述终端为第一客户端,所述第一客户端包括发送装置、接收装置、存储器和处理器,所述处理器调用所述存储器中的访问元数据的程序,用于执行如下操作:In a third aspect, an embodiment of the present invention provides a terminal, where the terminal is a first client, and the first client includes a sending device, a receiving device, a memory, and a processor, where the processor invokes A program for accessing metadata in the memory for performing the following operations:
在请求访问目标目录中的第一目标元数据时,通过所述发送装置向目录锁 服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述处理器当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述处理器访问所述目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;Redirecting to the directory by the sending device when requesting access to the first target metadata in the target directory The server sends a first application message to apply for exclusive access rights to all metadata in the target directory, wherein the target directory is a directory in which a plurality of metadata is stored, and the first target metadata is the processing Metadata currently required to be accessed by the device; exclusive access rights for all metadata in the target directory for the processor to access first target metadata in the target directory, and access to the target directory Metadata other than the first target metadata;
当申请到对所述目标目录中所有元数据的排它访问权限时,访问所述第一目标元数据。The first target metadata is accessed when applying for exclusive access to all metadata in the target directory.
通过执行上述操作,第一客户端访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的排它访问权限,这样一来,第一客户端就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。By performing the above operation, when the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
结合第三方面,在第三方面的第一种可能的实现方式中,当未申请到对所述目标目录中所有元数据的排它访问权限时,通过所述接收装置接收所述目录锁服务器发送的第二客户端所在的节点信息,并根据所述节点信息通过所述发送装置向所述第二客户端发送请求消息以通过所述第二客户端访问所述目标目录中的元数据,所述第二客户端为向所述目录锁服务器申请到了对所述目标目录中所有元数据的排它访问权限的客户端。With reference to the third aspect, in a first possible implementation manner of the third aspect, when the exclusive access rights to all the metadata in the target directory are not applied, the directory lock server is received by the receiving device And sending, by the sending device, a request message to the second client, according to the node information, to access metadata in the target directory by using the second client, The second client is a client that applies to the directory lock server for exclusive access to all metadata in the target directory.
通过执行上述操作,第一客户端在申请不到目标目录中所有元数据的排它访问权限时,通过已经获取了目标目录中所有元数据的排它访问权限的第二客户端访问第一目标元数据,使得第一客户端在申请不到目标目录中所有元数据的排它访问权限时依然可以顺利访问到目标元数据。By performing the above operation, the first client accesses the first target through the second client that has acquired exclusive access rights of all metadata in the target directory when applying for exclusive access rights of all metadata in the target directory. The metadata enables the first client to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
结合第三方面,或者第三方面的第一种可能的实现方式,在第三方面的第二种可能的实现方式中,所述处理器向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限的方式具体为:With reference to the third aspect, or the first possible implementation manner of the third aspect, in a second possible implementation manner of the third aspect, the processor sends a first application message to the directory lock server to apply for the The exclusive access rights of all metadata in the target directory are as follows:
获取描述所述目标目录的元数据,所述描述所述目标目录的元数据设有用于标识所述目标目录是否为存储有多个元数据的目录的标识信息;Obtaining metadata describing the target directory, where the metadata describing the target directory is provided with identification information for identifying whether the target directory is a directory in which a plurality of metadata is stored;
根据所述描述所述目标目录的元数据来判断所述目标目录是否为存储有多个元数据的目录;Determining, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
若是,则通过发送装置向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限。If so, the first application message is sent to the directory lock server by the sending device to apply for exclusive access to all metadata in the target directory.
结合第三方面,或者第三方面的第一种可能的实现方式,或者第三方面的第二种可能的实现方式,在第三方面的第三种可能的实现方式中,当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理访问所述第一目标元数据之后,所述处理器还用于:With reference to the third aspect, or the first possible implementation manner of the third aspect, or the second possible implementation manner of the third aspect, in the third possible implementation manner of the third aspect, when applying to the opposite The processor is further configured to: after the process accesses the first target metadata, when the process accesses the first target metadata in the target directory,
通过所述接收装置接收第三客户端发送的请求消息,所述请求消息用于请求通过所述第一客户端访问所述第一目标元数据,所述请求消息为所述第三客户端向所述目录锁服务器申请对所述目标目录中所有元数据的排它访问权限,并接收到所述目录锁服务器返回的所述第一客户端的节点信息后,根据所述第 一客户端的节点信息向所述第一客户端发送的消息;Receiving, by the receiving device, a request message sent by a third client, where the request message is used to request access to the first target metadata by using the first client, where the request message is sent by the third client The directory lock server applies exclusive access rights to all metadata in the target directory, and after receiving the node information of the first client returned by the directory lock server, according to the a message sent by the node information of the client to the first client;
根据所述请求消息访问所述第一目标元数据,并将访问所述第一目标元数据的访问结果通过所述发送装置发送给所述第三客户端。And accessing the first target metadata according to the request message, and sending an access result of accessing the first target metadata to the third client by using the sending device.
通过执行上述操作,第一客户端获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录中第一目标元数据的需求,则第三客户端将访问请求发送至第一客户端,通过第一客户端访问该第一目标元数据,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问第一目标元数据。After performing the above operation, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement to access the first target metadata in the target directory, the third client will access the request. Sending to the first client, accessing the first target metadata by the first client, ensuring that the client that does not apply for exclusive access rights of all metadata in the target directory can also access the first target metadata normally. .
结合第三方面,或者第三方面的第一种可能的实现方式,或者第三方面的第二种可能的实现方式,或者第三方面的第三种可能的实现方式,在第三方面的第四种可能的实现方式中,所述当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理器访问所述第一目标元数据之后,所述处理器还用于:With reference to the third aspect, or the first possible implementation of the third aspect, or the second possible implementation of the third aspect, or the third possible implementation of the third aspect, in the third aspect In four possible implementation manners, after applying the exclusive access right to all metadata in the target directory, after the processor accesses the first target metadata, the processor is further used to :
在请求访问所述目标目录中的第二目标元数据时,如果第二目标元数据属于所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata other than the first target metadata in the target directory;
则所述处理器按照已获得的对所述第二目标元数据的排它访问权限,对所述第二目标元数据进行访问。And then the processor accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
通过执行上述操作,第一客户端在访问目标目录中的其他元数据(如第二目标元数据)时,可以直接对该第二目标元数据进行访问,无需再次申请访问访问权限,提高了第一客户端访问目标目录中的元数据的效率。By performing the above operation, when the first client accesses other metadata (such as the second target metadata) in the target directory, the second target metadata can be directly accessed, and the access permission is not required to be applied again. The efficiency of a client's access to metadata in the target directory.
结合第三方面,或者第三方面的第一种可能的实现方式,或者第三方面的第二种可能的实现方式,或者第三方面的第三种可能的实现方式,或者第三方面的第四种可能的实现方式,在第三方面的第五种可能的实现方式中,所述处理器访问所述第一目标元数据的方式具体为:With reference to the third aspect, or the first possible implementation manner of the third aspect, or the second possible implementation manner of the third aspect, or the third possible implementation manner of the third aspect, or the third aspect In a fourth possible implementation manner of the third aspect, the manner in which the processor accesses the first target metadata is specifically:
向元数据路由服务器请求所述目标目录的归属信息,所述元数据路由服务器预存有目录的归属信息;Requesting, to the metadata routing server, attribution information of the target directory, where the metadata routing server prestores the attribution information of the directory;
通过所述接收装置接收所述元数据路由服务器发送的所述目标目录的归属信息,所述目标目录的归属信息用于指示所述目标目录的操作权限归属到目标元数据服务器;Receiving, by the receiving device, the home information of the target directory that is sent by the metadata routing server, where the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
通过所述发送装置向所述目标元数据服务器发送操作请求消息,请求所述目标元数据服务器根据所述对所述目标目录中所有元数据的排它访问权限操作所述第一目标元数据;Sending, by the sending device, an operation request message to the target metadata server, requesting the target metadata server to operate the first target metadata according to the exclusive access right to all metadata in the target directory;
通过所述接收装置接收所述目标元数据服务器发送的操作结果;所述操作结果为所述目标元数据服务器向所述目录锁服务器申请到对所述目标目录中所有元数据的排它操作权限后,对所述第一目标元数据进行操作的结果。Receiving, by the receiving device, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server for exclusive operation permission on all metadata in the target directory. Thereafter, the result of operating the first target metadata.
通过执行上述操作,目标元数据服务器接收到第一客户端发送的访问目标目录下的第一目标元数据的请求后,在对目标目录中的第一目标元数据进行操作之前,先向目录锁服务器申请目标目录中所有元数据的排它操作权限,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性。 After performing the above operation, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server first locks the directory before operating the first target metadata in the target directory. The server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata retention consistency of the target directory is ensured.
结合第三方面,或者第三方面的第一种可能的实现方式,或者第三方面的第二种可能的实现方式,或者第三方面的第三种可能的实现方式,或者第三方面的第四种可能的实现方式,在第三方面的第六种可能的实现方式中,所述处理器访问所述第一目标元数据的方式具体为:With reference to the third aspect, or the first possible implementation manner of the third aspect, or the second possible implementation manner of the third aspect, or the third possible implementation manner of the third aspect, or the third aspect In a fourth possible implementation manner of the third aspect, the manner in which the processor accesses the first target metadata is specifically:
通过所述发送装置向元数据路由服务器发送切换消息,指示所述元数据路由服务器将所述目标目录的操作权限归属到所述第一客户端;Transmitting, by the sending device, a handover message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client;
通过所述接收装置接收所述元数据路由服务器发送的通知消息,所述通知消息用于指示所述目标目录的操作权限归属到所述第一客户端;Receiving, by the receiving device, the notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client;
通过所述发送装置向所述目录锁服务器发送第二申请消息以申请对所述目标目录中所有元数据的排它操作权限;Sending, by the sending device, a second application message to the directory lock server to apply for exclusive operation authority on all metadata in the target directory;
通过所述接收装置接收所述目录锁服务器发送的授予所述目标目录中所有元数据的排它操作权限的授予消息,所述目标目录中所有元数据的排它操作权限用于供所述处理器操作所述目标目录中的第一目标元数据,以及操作所述目标目录中除所述第一目标元数据之外的元数据;Receiving, by the receiving device, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for the processing The first target metadata in the target directory is operated, and metadata other than the first target metadata in the target directory is operated;
根据所述目标目录中所有元数据的排它操作权限对所述第一目标元数据进行操作,获得操作结果。The first target metadata is operated according to the exclusive operation authority of all the metadata in the target directory, and the operation result is obtained.
通过执行上述操作,第一客户端同时充当客户端和元数据服务器的功能,第一客户端获取到目标目录中所有元数据的排它访问权限后,不需要与远端的元数据服务器进行交互来请求元数据服务器对第一目标元数据进行操作,减少了节点之间的开销,提高了访问第一目标元数据的访问效率。By performing the above operations, the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, it does not need to interact with the remote metadata server. To request the metadata server to operate on the first target metadata, the overhead between the nodes is reduced, and the access efficiency of accessing the first target metadata is improved.
结合第三方面,或者第三方面的第一种可能的实现方式,或者第三方面的第二种可能的实现方式,或者第三方面的第五种可能的实现方式,或者第六方面的第四种可能的实现方式,在第三方面的第七种可能的实现方式中,当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理器访问所述第一目标元数据之后,所述处理器还用于:With reference to the third aspect, or the first possible implementation manner of the third aspect, or the second possible implementation manner of the third aspect, or the fifth possible implementation manner of the third aspect, or the sixth aspect In a seventh possible implementation manner of the third aspect, the processor accesses the first target when applying exclusive access rights to all metadata in the target directory After the metadata, the processor is also used to:
判断访问所述目标目录中的元数据所产生的负载是否满足预设条件;Determining whether the load generated by accessing the metadata in the target directory satisfies a preset condition;
若满足,则释放对所述目标目录中的所有元数据的排它访问权限,以便所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予第四客户端。If so, exclusive access to all metadata in the target directory is released, such that the directory lock server grants exclusive access to all metadata in the target directory to the fourth client.
通过执行上述操作,第一客户端上的负载过高时,自动释放目标目录中所有元数据的排它访问权限,让负载较低的第四客户端来获取该目标目录中所有元数据的排它访问权限以达到分担负载的目的;或者在第一客户端访问目标目录中的元数据的频率比其他客户端访问目标目录中的元数据的频率低时,自动释放目标目录中所有元数据的排它访问权限,让访问频率较高的客户端获取目标目录中所有元数据的排它访问权限,以减少转发访问所产生的负载,提高了访问目标目录中元数据的效率。By performing the above operation, when the load on the first client is too high, the exclusive access permission of all metadata in the target directory is automatically released, and the fourth client with lower load is obtained to obtain the row of all metadata in the target directory. It accesses the load to achieve the purpose of sharing the load; or automatically releases all metadata in the target directory when the first client accesses the metadata in the target directory less frequently than the metadata in other clients accessing the target directory. Exclusive access rights allow clients with higher frequency of access to obtain exclusive access rights to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
第四方面,本发明实施例提供一种终端,其特征在于,所述终端为目录锁服务器,所述目录锁服务器包括发送装置、接收装置、存储器和处理器,所述 处理器调用所述存储器中的访问元数据的程序,用于执行如下操作:In a fourth aspect, an embodiment of the present invention provides a terminal, where the terminal is a directory lock server, and the directory lock server includes a sending device, a receiving device, a memory, and a processor, where The processor calls a program for accessing metadata in the memory to perform the following operations:
在第一客户端访问目标目录中的目标元数据时,通过所述接收装置接收所述第一客户端发送的第一申请消息,所述第一申请消息用于申请对所述目标目录中所有元数据的排它访问权限,所述目标目录为存储有多个元数据的目录,所述目标元数据为所述第一客户端当前需要访问的元数据;Receiving, by the receiving device, a first application message sent by the first client, where the first client accesses the target metadata in the target directory, where the first application message is used to apply for all the target directories. Exclusive access rights of the metadata, the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
根据所述第一申请消息判断所述目标目录中所有元数据的排它访问权限是否已经授予了第二客户端,所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的目标元数据,以及访问所述目标目录中除所述目标元数据之外的元数据;Determining, according to the first application message, whether an exclusive access right of all metadata in the target directory has been granted to the second client, and exclusive access rights of all metadata in the target directory are used for the first Client accessing target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
若未授予所述第二客户端,则将所述目标目录中所有元数据的排它访问权限授予所述第一客户端;若授予了所述第二客户端,则通过所述发送装置向所述第一客户端发送所述第二客户端所在的节点信息,以使所述第一客户端基于所述第二客户端所在的节点信息通过所述第二客户端访问所述目标元数据。If the second client is not granted, grant exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, pass the sending device to Sending, by the first client, node information where the second client is located, so that the first client accesses the target metadata by using the second client according to node information of the second client .
通过执行上述操作,第一客户端访问目标目录中的目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的排它访问权限,这样一来,第一客户端就可以顺利访问目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。By performing the above operation, when the first client accesses the target metadata in the target directory, the application lock access authority of all the metadata in the target directory is requested to the directory lock server at one time, so that the first client can successfully access the target. Metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
结合第四方面,在第四方面的第一种可能的实现方式中,若未授予所述第二客户端,所述处理器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器还用于:With reference to the fourth aspect, in a first possible implementation manner of the fourth aspect, if the second client is not granted, the processor grants exclusive access rights of all metadata in the target directory to the After the first client, the processor is further configured to:
在第三客户端访问所述目标元数据时,通过接收装置接收所述第三客户端发送的所述第一申请消息;Receiving, by the receiving device, the first application message sent by the third client, when the third client accesses the target metadata;
通过所述发送装置向所述第三客户端发送所述第一客户端所在的节点信息,以使所述第三客户端基于所述第一客户端所在的节点信息通过所述第一客户端访问所述目标元数据。Sending, by the sending device, the node information of the first client to the third client, so that the third client passes the first client based on node information where the first client is located. Access the target metadata.
通过执行上述操作,第一客户端获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录中目标元数据的需求,则第三客户端将访问请求发送至第一客户端,通过第一客户端访问该目标元数据,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问目标元数据。After performing the above operation, after the first client obtains the exclusive access right of all the metadata in the target directory, if the third client has the requirement of accessing the target metadata in the target directory, the third client sends the access request to the third client. The first client accesses the target metadata through the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the target metadata normally.
结合第四方面,或者第四方面的第一种可能的实现方式,在第四方面的第二种可能的实现方式中,若未授予所述第二客户端,所述处理器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器还用于:With reference to the fourth aspect, or the first possible implementation manner of the fourth aspect, in a second possible implementation manner of the fourth aspect, if the second client is not granted, the processor: After the exclusive access rights of all metadata in the directory are granted to the first client, the processor is further configured to:
通过所述接收装置接收目标元数据服务器发送的第二申请消息,所述第二申请消息为所述目标元数据服务器接收到所述第一客户端访问所述目标元数据时发送的操作消息时发送的,用于申请对所述目标目录中所有元数据的排它操作权限的消息;Receiving, by the receiving device, a second application message sent by the target metadata server, where the second application message is when the target metadata server receives the operation message sent when the first client accesses the target metadata a message sent for applying for exclusive operation rights to all metadata in the target directory;
根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授 予所述目标元数据服务器,以使所述目标元数据服务器对所述目标元数据进行操作并将操作结果反馈给所述第一客户端,其中,所述目标目录中所有元数据的排它操作权限用于供所述目标元数据服务器操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。Authorizing the exclusive operation authority of all metadata in the target directory according to the second application message And the target metadata server is configured to cause the target metadata server to operate on the target metadata and feed back an operation result to the first client, wherein all metadata in the target directory is exclusive The operation authority is for the target metadata server to operate the target metadata in the target directory, and to operate metadata in the target directory other than the target metadata.
通过执行上述操作,目标元数据服务器接收到第一客户端发送的访问目标目录下的目标元数据的请求后,在对目标目录中的目标元数据进行操作之前,先向目录锁服务器申请目标目录中所有元数据的排它操作权限,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性。After performing the above operation, after receiving the request for the target metadata in the target directory sent by the first client, the target metadata server requests the target directory from the directory lock server before operating the target metadata in the target directory. The exclusive operation permission of all the metadata in the metadata makes that only one metadata server in all the metadata servers has the operation authority for the target directory, which ensures the persistent preservation consistency of the metadata in the target directory.
结合第四方面,或者第四方面的第一种可能的实现方式,在第四方面的第三种可能的实现方式中,若未授予所述第二客户端,所述处理器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器还用于:With reference to the fourth aspect, or the first possible implementation manner of the fourth aspect, in a third possible implementation manner of the fourth aspect, if the second client is not granted, the processor: After the exclusive access rights of all metadata in the directory are granted to the first client, the processor is further configured to:
通过所述接收装置接收所述第一客户端发送的第二申请消息,所述第二申请消息用于申请对所述目标目录中所有元数据的排它操作权限;Receiving, by the receiving device, a second application message sent by the first client, where the second application message is used to apply for exclusive operation permission on all metadata in the target directory;
根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述第一客户端,以使所述第一客户端对所述目标元数据进行操作,其中,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。Granting exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client operates on the target metadata, wherein the Exclusive operational rights for all metadata in the target directory are used by the first client to manipulate target metadata in the target directory and to manipulate metadata in the target directory other than the target metadata.
通过执行上述操作,第一客户端同时充当客户端和元数据服务器的功能,第一客户端获取到目标目录中所有元数据的排它访问权限后,不需要与远端的元数据服务器进行交互来请求元数据服务器对目标元数据进行操作,减少了节点之间的开销,提高了访问目标元数据的访问效率。By performing the above operations, the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, it does not need to interact with the remote metadata server. To request the metadata server to operate on the target metadata, the overhead between the nodes is reduced, and the access efficiency of the access target metadata is improved.
第五方面,本发明实施例提供一种分布式文件系统,该系统包括第一客户端、第二客户端和目录锁服务器,其中:In a fifth aspect, an embodiment of the present invention provides a distributed file system, where the system includes a first client, a second client, and a directory lock server, where:
第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述第一客户端当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target directory For storing a directory having a plurality of metadata, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the first a client accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
所述目录锁服务器接收所述第一客户端发送的第一申请消息,并判断所述目标目录中所有元数据的排它访问权限是否授予了第二客户端;Receiving, by the directory lock server, the first application message sent by the first client, and determining whether exclusive access rights of all metadata in the target directory are granted to the second client;
若所述目标目录中所有元数据的排它访问权限未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端,并向所述第一客户端发送通知消息以通知所述目标目录中所有元数据的排它访问权限授予所述第一客户端; If the exclusive access right of all metadata in the target directory is not granted to the second client, the directory lock server grants exclusive access rights of all metadata in the target directory to the first client And sending a notification message to the first client to notify the first client of exclusive access rights of all metadata in the target directory;
所述第一客户端接收所述通知消息,并根据所述通知消息访问所述第一目标元数据。The first client receives the notification message and accesses the first target metadata according to the notification message.
通过运行该系统,第一客户端访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的排它访问权限,这样一来,第一客户端就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。By running the system, when the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
结合第五方面,在第五方面的第一种可能的实现方式中:In conjunction with the fifth aspect, in a first possible implementation of the fifth aspect:
所述第一客户端为第三方面的任一种可能的实现方式所描述的第一客户端;The first client is the first client described in any possible implementation manner of the third aspect;
所述目录锁服务器为第四方面的任一种可能的实现方式所描述的目录锁服务器。The directory lock server is a directory lock server as described in any of the possible implementations of the fourth aspect.
在以上描述的一些方案中,所述存储有多个元数据的目录包括存储有用来描述海量小文件的多个元数据的目录。In some of the scenarios described above, the directory in which the plurality of metadata is stored includes a directory in which a plurality of metadata for describing a small number of files is stored.
在以上描述的一些方案中,所述向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限包括:In some aspects described above, the sending the first application message to the directory lock server to apply for exclusive access to all metadata in the target directory includes:
基于分布式锁服务向目录锁服务器发送第一申请消息,申请访问所述目标目录的锁以获取对所述目标目录中所有元数据的排它访问权限,其中,所述访问所述目标目录的锁为不可抢占的分布式锁。Sending a first application message to the directory lock server based on the distributed lock service, requesting access to the lock of the target directory to obtain exclusive access rights to all metadata in the target directory, wherein the accessing the target directory The lock is a non-preemptable distributed lock.
在以上描述的一些方案中,所述向所述目录锁服务器发送第二申请消息以申请对所述目标目录中所有元数据的排它操作权限包括:In some aspects described above, the sending the second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory includes:
基于分布式锁服务向目录锁服务器发送第二申请消息,申请操作所述目标目录的锁以获取对所述目标目录中所有元数据的排它操作权限,其中,所述操作所述目标目录的锁为可抢占的分布式锁。Sending a second application message to the directory lock server based on the distributed lock service, requesting operation of the lock of the target directory to obtain exclusive operation authority for all metadata in the target directory, wherein the operating the target directory The lock is a preemptible distributed lock.
在以上描述的一些方案中,所述第一客户端判断访问所述目标目录中的元数据所产生的负载是否满足预设条件包括:所述第一客户端记录自身主动访问所述目标目录下的元数据所产生的第一负载,以及根据其他客户端的请求消息访问所述目标目录中的元数据所产生的第二负载;端判断所述第一负载与所述第二负载的相对大小是否满足预设条件。In some aspects described above, the determining, by the first client, whether the load generated by accessing the metadata in the target directory meets a preset condition comprises: the first client records actively accessing the target directory by itself The first load generated by the metadata, and the second load generated by accessing the metadata in the target directory according to the request message of the other client; the terminal determines whether the relative size of the first load and the second load is Meet the preset conditions.
第六方面,本发明实施例提供一种访问元数据的装置,所述装置包括用于执行本发明实施例第一方面任一方式的部分或全部步骤的功能单元。In a sixth aspect, an embodiment of the present invention provides an apparatus for accessing metadata, where the apparatus includes a functional unit for performing some or all of the steps of any of the first aspects of the embodiments of the present invention.
第七方面,本发明实施例提供一种访问元数据的装置,所述装置包括用于执行本发明实施例第二方面任一方式的部分或全部步骤的功能单元。In a seventh aspect, an embodiment of the present invention provides an apparatus for accessing metadata, where the apparatus includes a functional unit for performing some or all of the steps of any of the second aspects of the embodiments of the present invention.
通过实施本发明实施例,第一客户端访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的排它访问权限,这样一来,第一客户端就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。When the first client accesses the first target metadata in the target directory, the first client requests the exclusive access permission of all the metadata in the target directory to the directory lock server, so that the first client The first target metadata can be successfully accessed, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所 需要使用的附图作简单地介绍。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following description will be made on the embodiments. The drawings to be used are briefly introduced.
图1是现有技术中的一种分布式文件系统的结构示意图;1 is a schematic structural diagram of a distributed file system in the prior art;
图2是图1所示的分布式文件系统的原理示意图;2 is a schematic diagram of the principle of the distributed file system shown in FIG. 1;
图3是本发明实施例提供的一种元数据访问方法的流程示意图;3 is a schematic flowchart of a metadata access method according to an embodiment of the present invention;
图4是本发明实施例提供的又一种元数据访问方法的流程示意图;4 is a schematic flowchart of still another metadata access method according to an embodiment of the present invention;
图5是本发明实施例提供的又一种元数据访问方法的流程示意图;FIG. 5 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention;
图5A是本发明实施例提供的一种元数据访问的原理示意图;FIG. 5A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention; FIG.
图5B是图5A中各个设备进行交互的流程示意图;FIG. 5B is a schematic flowchart of interaction between devices in FIG. 5A; FIG.
图6是本发明实施例提供的又一种元数据访问方法的流程示意图;FIG. 6 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention;
图6A是本发明实施例提供的一种元数据访问的原理示意图;FIG. 6 is a schematic diagram of a principle of metadata access according to an embodiment of the present invention; FIG.
图6B是图6A中各个设备进行交互的流程示意图;6B is a schematic flowchart of interaction between devices in FIG. 6A;
图7是本发明实施例提供的又一种元数据访问方法的流程示意图;FIG. 7 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention;
图8是本发明实施例提供的一种终端的结构示意图;FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present disclosure;
图9是本发明实施例提供的又一种终端的结构示意图;FIG. 9 is a schematic structural diagram of still another terminal according to an embodiment of the present disclosure;
图10是本发明实施例提供的一种分布式文件系统的结构示意图;FIG. 10 is a schematic structural diagram of a distributed file system according to an embodiment of the present disclosure;
图11是本发明实施例提供的又一种终端的结构示意图;FIG. 11 is a schematic structural diagram of still another terminal according to an embodiment of the present disclosure;
图12是本发明实施例提供的又一种终端的结构示意图。FIG. 12 is a schematic structural diagram of still another terminal according to an embodiment of the present invention.
具体实施方式detailed description
下面将结合本发明中的附图,对本发明实施例中的技术方案进行清楚地描述;其中,图3~7用来描述本发明实施例的方法。The technical solutions in the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the present invention. FIGS. 3-7 are used to describe the method of the embodiments of the present invention.
请参见图3,图3是本发明实施例提供的一种元数据访问方法的流程示意图,讲述第一客户端如何获取目标目录下所有元数据的排它访问权限,该方法包括但不限于如下步骤。Referring to FIG. 3, FIG. 3 is a schematic flowchart of a metadata access method according to an embodiment of the present invention, which describes how the first client obtains exclusive access rights of all metadata in the target directory, including but not limited to the following: step.
步骤S303:第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述第一客户端当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据。Step S303: When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, where The target directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for The first client accesses first target metadata in the target directory and accesses metadata in the target directory other than the first target metadata.
具体地,分布式文件系统中存在很多客户端,第一客户端为当前请求访问目标目录中的第一目标元数据的一个随机的客户端;该分布式文件系统中存在用来存放多个元数据的目标目录,该目标目录中的第一目标元数据为第一客户端当前需要访问的元数据。Specifically, there are many clients in the distributed file system, where the first client is a random client that requests access to the first target metadata in the target directory; the distributed file system exists to store multiple elements. The target directory of the data, the first target metadata in the target directory is the metadata that the first client currently needs to access.
进一步地,目录锁服务器能够提供分布式锁服务,如搭载zookeeper服务的服务器;目录锁服务器通过授予访问目标目录的锁来协调各个客户端对目标目录的访问权限,如果目录锁服务器将访问目标目录的锁授予第一客户端,那么第一客户端就具备了访问目标目录下所有元数据的权限。Further, the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the directory lock server coordinates the access rights of each client to the target directory by granting a lock to the target directory, if the directory lock server accesses the target directory The lock is granted to the first client, and the first client has access to all metadata in the target directory.
进一步地,分布式锁服务可以设置为可抢占分布式锁和不可抢占分布式 锁;若设置为可抢占的分布式锁,第一客户端申请访问目标目录的锁时,如果访问目标目录的锁已经授予了其他客户端,那么目录锁服务器会根据抢占机制释放访问目标目录的锁,并重新将访问目标目录的锁授予第一客户端,如果访问目标目录的锁未授予其他客户端,那么目录锁服务器则直接将访问目标目录的锁授予第一客户端;若设置为不可抢占的分布式锁,第一客户端申请访问目标目录的锁时,如果访问目标目录的锁已经授予了其他客户端,那么目录锁服务器会根据不可抢占的机制,不将访问目标目录的锁授予第一客户端,如果访问目标目录的锁未授予其他客户端,那么目录锁服务器则直接将访问目标目录的锁授予第一客户端。在本发明实施例中,访问目标目录的锁可以设置为不可抢占的分布式锁,这样一来,第一客户端申请到访问目标目录的锁后,就具备了对目标目录中所有元数据的排它访问权限,即除了第一客户端之外,其他的客户端没有访问该目标目录中元数据的权限(即访问第一目标元数据和除第一目标元数据之外的元数据的权限),除非第一客户端主动释放访问目标目录的锁。Further, the distributed lock service can be set to preemptive distributed locks and non-preemptive distributed Lock; if set to a preemptible distributed lock, when the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server will release the access target directory according to the preemption mechanism. Lock, and re-grant the lock to the target directory to the first client, if the lock accessing the target directory is not granted to other clients, then the directory lock server directly grants the lock to the target directory to the first client; if not set to Preemptive distributed lock. When the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server will not grant the lock to the target directory according to the non-preemptive mechanism. The first client, if the lock accessing the target directory is not granted to other clients, the directory lock server directly grants the lock accessing the target directory to the first client. In the embodiment of the present invention, the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the first client applies for the lock to access the target directory, it has all the metadata in the target directory. Exclusive access rights, that is, other clients except the first client do not have access to the metadata in the target directory (ie, access to the first target metadata and metadata other than the first target metadata) ) unless the first client actively releases the lock accessing the target directory.
在一种可选的方案中,当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据之后,所述方法还包括:所述第一客户端在请求访问所述目标目录中的第二目标元数据时,如果第二目标元数据属于所述目标目录中除所述第一目标元数据之外的元数据;则所述第一客户端按照已获得的对所述第二目标元数据的排它访问权限,对所述第二目标元数据进行访问。In an optional solution, after the first client applies exclusive access rights to all metadata in the target directory, after the first client accesses the first target metadata, The method further includes: when the first client requests access to the second target metadata in the target directory, if the second target metadata belongs to the target directory except the first target metadata Metadata; then the first client accesses the second target metadata according to the obtained exclusive access rights to the second target metadata.
也即是说,当第一客户端在请求访问目标目录中的第一目标元数据时,原本只需要申请第一目标元数据的排它访问权限,但是第一客户端实际申请了目标目录中所有元数据的排它访问权限,这样一来,当第一客户端请求访问目标目录中除第一目标元数据之外的元数据(如第二目标元数据)时,会判断出自身已经获取了访问该第二目标元数据的权限,因此不需要再申请第二目标元数据的排它访问权限,而是直接对第二目标元数据进行访问。That is to say, when the first client requests access to the first target metadata in the target directory, the original client only needs to apply exclusive access rights of the first target metadata, but the first client actually applies for the target directory. Exclusive access rights for all metadata, such that when the first client requests access to metadata other than the first target metadata (eg, second target metadata) in the target directory, it determines that it has acquired The right to access the second target metadata, so there is no need to apply for exclusive access rights of the second target metadata, but directly access the second target metadata.
在一种可选的方案中,所述第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限包括:第一客户端在请求访问目标目录中的第一目标元数据时,获取描述所述目标目录的元数据,所述描述所述目标目录的元数据设有用于标识所述目标目录是否为存储有多个元数据的目录的标识信息;根据所述描述所述目标目录的元数据来判断所述目标目录是否为存储有多个元数据的目录;若是,则向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限。In an optional solution, when the first client requests access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for all metadata in the target directory. The exclusive access permission includes: when the first client requests to access the first target metadata in the target directory, acquiring metadata describing the target directory, where the metadata describing the target directory is configured to identify the Whether the target directory is the identification information of the directory in which the plurality of metadata is stored; determining whether the target directory is a directory in which the plurality of metadata is stored according to the metadata of the target directory; if yes, the directory lock The server sends a first application message to request exclusive access to all metadata in the target directory.
具体地,该分布式文件系统中存在多种目录属性的目录,不同目录属性的目录中存储的内容可能存在区别,其中,有一种目录属性的目录中专门用来存储多个元数据,可选的,该多个元数据可以具体为描述海量小文件的元数据(每个小文件对应有自身的元数据)。第一客户端访问目标目录时先对该目标目录的目录属性进行分析,判断该目标目录是否为上述存储多个元数据的目录。该 目标目录也可以看作是一个文件,因此也对应有描述该目标目录的元数据,并且描述目标目录的元数据存放在目标目录的上一级目录中。Specifically, there are multiple directory attribute directories in the distributed file system, and contents stored in different directory attribute directories may be different. Among them, there is a directory attribute directory dedicated to storing multiple metadata, optionally The plurality of metadata may be specifically metadata describing a small number of small files (each small file corresponds to its own metadata). When the first client accesses the target directory, the directory attribute of the target directory is first analyzed, and it is determined whether the target directory is the above-mentioned directory storing multiple metadata. The The target directory can also be regarded as a file, so it also corresponds to the metadata describing the target directory, and the metadata describing the target directory is stored in the upper directory of the target directory.
当第一客户端请求访问目标目录下的第一目标元数据时,首先向元数据路由服务器发送该上一级目录的归属信息的查询请求;该元数据路由服务器接收第一客户端的查询请求,从存储的目录与元数据服务器的映射关系中查找该上一级目录归属的元数据服务器,其中,该上一级目录归属到哪个元数据服务器则该元数据服务器具备对该上一级目录的操作权限;然后将该上一级目录的归属信息反馈给第一客户端,第一客户端接收元数据路由服务器发送的归属信息,对该归属信息进行分析得出该上一级目录归属的元数据服务器,然后向该上一级目录归属的元数据服务器请求存储于该上一级目录中的用于描述目标目录的元数据,该上一级目录归属的元数据服务器读取存储于该上一级目录中的用于描述目标目录的元数据,并将读取到的描述目标目录的元数据发送给该第一客户端。When the first client requests to access the first target metadata in the target directory, first sending a query request for the home information of the upper-level directory to the metadata routing server; the metadata routing server receives the query request of the first client, Searching, from a mapping relationship between the stored directory and the metadata server, a metadata server to which the upper-level directory belongs, wherein the metadata server belongs to the metadata server, and the metadata server has the upper-level directory The operation authority; the attribution information of the upper-level directory is fed back to the first client, and the first client receives the home information sent by the metadata routing server, and analyzes the attribution information to obtain the element belonging to the upper-level directory. The data server then requests, from the metadata server to which the upper-level directory belongs, metadata stored in the upper-level directory for describing the target directory, where the metadata server to which the upper-level directory belongs is stored and stored. Metadata used to describe the target directory in the primary directory, and the metadata of the read target directory is read To the first client.
举例来说,假设存在一个路径…/A/B/f,其中,目录B为目标目录,目录A为目录B的上一级目录,目录B下存储有描述某个文件的第一目标元数据f,目录A下存储有描述目录B的元数据;当第一客户端要访问第一目标元数据f时,首先需要向元数据路由服务器查找目录A所归属的元数据服务器,然后向该元数据服务器请求目录A下描述目录B的元数据,相应地,该元数据服务器接收第一客户端的请求,读取目录A中描述目录B的元数据,并将读取到的描述目录B的元数据发送给第一客户端。For example, suppose there is a path.../A/B/f, where directory B is the target directory, directory A is the upper directory of directory B, and directory B stores the first target metadata describing a file. f, the directory A stores the metadata describing the directory B; when the first client wants to access the first target metadata f, first needs to find the metadata server to which the directory A belongs to the metadata routing server, and then to the element The data server requests the metadata describing the directory B under the directory A. Accordingly, the metadata server receives the request of the first client, reads the metadata describing the directory B in the directory A, and reads the element describing the directory B. The data is sent to the first client.
进一步地,第一客户端接收到目标目录的上一级目录所归属的元数据服务器发送的描述目标目录的元数据后,对描述目标目录的元数据进行分析,判断该目标目录是否存储多个元数据的目录,在本发明实施例中,描述每个目录的元数据都存在标识信息,用来定义各个目录的目录属性,因此第一客户端可以基于描述目标目录中的元数据中的“标识信息”得出目标目录的目录属性,进而得出该目标目录是否为存储多个元数据的目录。Further, after receiving the metadata describing the target directory sent by the metadata server to which the upper-level directory of the target directory belongs, the first client analyzes the metadata describing the target directory, and determines whether the target directory stores multiple In the embodiment of the present invention, the metadata describing each directory has identification information for defining the directory attribute of each directory, so the first client can be based on describing the metadata in the target directory. The identification information "derives the directory attribute of the target directory, and further determines whether the target directory is a directory in which a plurality of metadata is stored.
举例来说,描述目标目录的元数据可以包含“标识信息”来标识目标目录的目录属性,当该标识信息设置为“1”时,表明目标目录为上述存储多个元数据的目录,当该标识信息设置为“0”时,表明目标目录不为上述存储多个元数据的目录。“标识信息”可以由用户在建立目标目录时设置,也可以由用户在建立完目标目录后进行修改。当第一客户端接收到描述目标目录的元数据后,判断其中的“标识信息”是“1”还是“0”即可得出目标目录是否为存储多个元数据的目录。For example, the metadata describing the target directory may include "identification information" to identify the directory attribute of the target directory. When the identification information is set to "1", it indicates that the target directory is the above-mentioned directory storing a plurality of metadata, when When the identification information is set to "0", it indicates that the target directory is not the directory in which the plurality of metadata are stored as described above. The "identification information" may be set by the user when the target directory is created, or may be modified by the user after the target directory is created. After the first client receives the metadata describing the target directory, it is determined whether the "identification information" is "1" or "0", and whether the target directory is a directory storing a plurality of metadata.
步骤S304:当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据。Step S304: When the first client applies for exclusive access rights to all metadata in the target directory, the first client accesses the first target metadata.
具体地,目录锁服务器接收第一客户端发送的第一申请消息,对该第一申请消息进行分析,得出该第一申请消息用于申请目标目录中所有元数据的排它访问权限,因此该目录锁服务器判断目标目录中所有元数据的排它访问权限是否已经授予了其他客户端,若没有则目录锁服务器将目标目录中所有元数据的 排它访问权限授予第一客户端,这样一来,所有的客户端中只有第一客户端具备访问目标目录中元数据的权限,该第一客户端后续访问该目标目录中的任意元数据时,第一客户端不需要再申请访问该任意元数据的权限。获取到目标目录中所有元数据的排它访问权限后,该第一客户端访问该目标目录中的元数据。需要说明的是,元数据进行读、写等操作均属于访问元数据,其中,当对元数据进行读时,访问元数据的过程包括请求元数据服务器读取元数据并接收元数据服务器发送的读取的该元数据;当对元数据进行写时,访问元数据的过程包括请求元数据服务器向元数据中写入内容(包括修改)并接收元数据服务器发送的写入结果。除了读和写外,还存在其他的访问元数据的方式,此处不再对其他的方式一一举例。在现有技术中,元数据服务器为具备操作元数据权限的设备。Specifically, the directory lock server receives the first application message sent by the first client, analyzes the first application message, and obtains that the first application message is used to apply for exclusive access rights of all metadata in the target directory, so The directory lock server determines whether the exclusive access rights of all metadata in the target directory have been granted to other clients, and if not, the directory lock server will all metadata in the target directory. The exclusive access permission is granted to the first client, so that only the first client of all the clients has the right to access the metadata in the target directory, and the first client subsequently accesses any metadata in the target directory. The first client does not need to apply for access to the arbitrary metadata. After obtaining exclusive access to all metadata in the target directory, the first client accesses the metadata in the target directory. It should be noted that the operations of reading and writing metadata belong to the access metadata. When reading the metadata, the process of accessing the metadata includes requesting the metadata server to read the metadata and receiving the metadata server. The metadata is read; when the metadata is written, the process of accessing the metadata includes requesting the metadata server to write content (including modifications) to the metadata and receiving the write result sent by the metadata server. In addition to reading and writing, there are other ways to access metadata. No other way is given here. In the prior art, the metadata server is a device with operational metadata rights.
举例来说,假设目标目录为目录B,目录B的上一级目录为目录A,目录B下有文件的元数据d、e和f,当第一客户端访问路径…/A/B/下的第一目标元数据f时,向目录锁服务器申请目录B中所有元数据的排它访问权限,然后第一客户端就可以访问元数据f,并且在后续访问元数据e或者d时均可以直接访问,而无需再申请访问目录B的访问权限,也不需要再申请访问e和d的权限。For example, suppose the target directory is directory B, the upper directory of directory B is directory A, and the metadata of files is d, e, and f under directory B, when the first client access path.../A/B/ When the first target metadata f is applied to the directory lock server for exclusive access rights of all metadata in the directory B, the first client can access the metadata f, and can access the metadata e or d subsequently. Direct access without having to apply for access to directory B, and no need to apply for access to e and d.
通过执行步骤S303~S304,第一客户端访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的访问权限,这样一来,第一客户端就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。When the first client accesses the first target metadata in the target directory, the first client accesses the directory lock server for all the metadata access rights in the target directory, so that the first client can be smoothly performed by performing steps S303-S304. Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
在一种可选的方案中,当所述第一客户端未申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端接收所述目录锁服务器发送的第二客户端所在的节点信息,并根据所述节点信息向所述第二客户端发送请求消息以通过所述第二客户端访问所述第一目标元数据,所述第二客户端为向所述目录锁服务器申请到了对所述目标目录中所有元数据的排它访问权限的客户端。In an optional solution, when the first client does not apply for exclusive access to all metadata in the target directory, the first client receives the first sent by the directory lock server. And the node information where the client is located, and sending a request message to the second client according to the node information to access the first target metadata by using the second client, where the second client is a The directory lock server applies to the client for exclusive access to all metadata in the target directory.
具体地,当目录锁服务器判断出目标目录中所有元数据的排它访问权限已经授予了其他客户端(如第二客户端)时,目录锁服务器则不会将目标目录中所有元数据的排它访问权限授予第一客户端,但是,目录锁服务器会向第一客户端发送包含第二客户端所在的节点信息,相应地,第一客户端接收目录锁服务器发送的节点信息,并从该节点信息中解析出第二客户端的节点,然后基于该节点向第二客户端发送访问目标目录的请求,相应地,第二客户端接收第一客户端发送的访问目标目录的请求,根据该请求来访问目标目录下的第一目标元数据,并将访问结果发送给第一客户端。Specifically, when the directory lock server determines that the exclusive access rights of all metadata in the target directory have been granted to other clients (such as the second client), the directory lock server does not queue all the metadata in the target directory. The access permission is granted to the first client, but the directory lock server sends the node information including the second client to the first client, and correspondingly, the first client receives the node information sent by the directory lock server, and from the node Parsing out the node of the second client in the node information, and then sending a request to access the target directory to the second client based on the node, and correspondingly, the second client receives the request for accessing the target directory sent by the first client, according to the request The first target metadata under the target directory is accessed, and the access result is sent to the first client.
执行完上述步骤S303~S304之后,如果第一客户端获取了目标目录下所有元数据的权限,第一客户端还可执行步骤S305和S306来为其他无法获取目标目录的访问权限的第三客户端提供访问第一目标目录中元数据的服务,具体地流程图如图4所示。其中,步骤S305和S306的详细描述如下:After performing the above steps S303-S304, if the first client acquires the rights of all the metadata in the target directory, the first client may also perform steps S305 and S306 for other third clients that cannot obtain the access rights of the target directory. The terminal provides a service for accessing metadata in the first target directory, and the specific flowchart is as shown in FIG. 4. The detailed description of steps S305 and S306 is as follows:
步骤S305:所述第一客户端接收第三客户端发送的请求消息,所述请求消 息用于请求通过所述第一客户端访问所述第一目标元数据,所述请求消息为所述第三客户端向所述目录锁服务器申请对所述目标目录中所有元数据的排它访问权限,并接收到所述目录锁服务器返回的所述第一客户端的节点信息后,根据所述第一客户端的节点信息向所述第一客户端发送的消息。Step S305: The first client receives a request message sent by a third client, where the request is cancelled. The information is used to request access to the first target metadata by using the first client, and the request message is for the third client to apply to the directory lock server for exclusive use of all metadata in the target directory. After accessing the permission, and receiving the node information of the first client returned by the directory lock server, sending a message to the first client according to the node information of the first client.
具体地,当目录锁服务器将目标目录中所有元数据的排它访问权限授予了第一客户端后,目录锁服务器还会接收到第三客户端申请目标目录中所有元数据的排它访问权限的请求(该第三客户端泛指未获得该目标目录中所有元数据的排它访问权限的一类客户端),由于该排它访问权限已经授予了第一客户端,因此目录锁服务器不会将目标目录中所有元数据的排它访问权限再授予第三客户端,但是,为了保证第三客户端能够正常地访问目标目录中的元数据,目录锁服务器会向第三客户端发送第一客户端的节点信息,第三客户端接收目录锁服务器发送的节点信息,并根据该节点信息分析出需要通过第一客户端访问目标目录下的第一目标元数据,然后向第一客户端发送申请消息,该申请消息中包含第三客户端需要访问的目标目录下的第一目标元数据的信息,相应地,第一客户端接收第三客户端发送的申请消息。Specifically, when the directory lock server grants exclusive access rights of all metadata in the target directory to the first client, the directory lock server also receives exclusive access rights of all metadata in the target directory of the third client application. Request (the third client refers to a type of client that does not have exclusive access to all metadata in the target directory), since the exclusive access has been granted to the first client, the directory lock server does not The exclusive access to all metadata in the target directory is granted to the third client, but in order to ensure that the third client can normally access the metadata in the target directory, the directory lock server sends the third client to the third client. a node information of the client, the third client receives the node information sent by the directory lock server, and analyzes, according to the node information, the first target metadata that needs to be accessed through the first client, and then sends the first target metadata to the first client. An application message, where the application message includes information about the first target metadata in the target directory that the third client needs to access, Correspondingly, the first client application receives a third message sent by the client.
步骤S306:所述第一客户端根据所述请求消息访问所述第一目标元数据,并将访问所述第一目标元数据的访问结果发送给所述第三客户端。Step S306: The first client accesses the first target metadata according to the request message, and sends an access result of accessing the first target metadata to the third client.
具体地,第一客户端对申请消息进行分析,得出第三客户端需要访问的元数据为第一目标元数据,进一步判断该第一目标元数据是否在缓存中,若是则对缓存的第一目标元数据进行读、写等操作,并将操作结果反馈给第三客户端;若不在缓存中则请求目标目录所归属的元数据服务器对该第一目标元数据进行读、写等操作,然后将操作结果反馈给第三客户端。Specifically, the first client analyzes the application message, and obtains that the metadata that the third client needs to access is the first target metadata, and further determines whether the first target metadata is in the cache, and if yes, the cached A target metadata is read, written, etc., and the operation result is fed back to the third client; if not in the cache, the metadata server to which the target directory belongs is requested to read and write the first target metadata, The result of the operation is then fed back to the third client.
通过执行步骤S305和S306,第一客户端获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录中第一目标元数据的需求,则第三客户端将访问请求发送至第一客户端,通过第一客户端访问该第一目标元数据,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问第一目标元数据。After performing the steps S305 and S306, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement of accessing the first target metadata in the target directory, the third client will The access request is sent to the first client, and the first target metadata is accessed by the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the first target normally. Metadata.
步骤S304中第一客户端获取到目标目录中所有元数据的排它访问权限后,所述第一客户端访问所述第一目标元数据可以具体通过步骤S307~S310来实现;对应的流程图如图5所示。其中,步骤S307~S310的详细描述如下:After the first client obtains the exclusive access right of all the metadata in the target directory, the first client accesses the first target metadata by using the steps S307-S310; As shown in Figure 5. The detailed description of steps S307-S310 is as follows:
步骤S307:所述第一客户端向元数据路由服务器请求所述目标目录的归属信息,所述元数据路由服务器预存有目录的归属信息。Step S307: The first client requests the home information of the target directory from the metadata routing server, and the metadata routing server prestores the home information of the directory.
具体地,第一客户端向元数据路由服务器发送查询请求,以查询目标目录的归属信息;该元数据路由服务器接收第一客户端的查询请求,从存储的目录与元数据服务器的映射关系中查找该目标目录对应的目标元数据服务器,然后向第一客户端发送归属信息,指示目标目录归属到目标元数据服务器。Specifically, the first client sends a query request to the metadata routing server to query the home information of the target directory; the metadata routing server receives the query request of the first client, and searches from the mapping relationship between the stored directory and the metadata server. The target metadata server corresponding to the target directory, and then sending the home information to the first client, indicating that the target directory belongs to the target metadata server.
步骤S308:所述第一客户端接收所述元数据路由服务器发送的所述目标目录的归属信息,所述目标目录的归属信息用于指示所述目标目录的操作权限归属到目标元数据服务器。 Step S308: The first client receives the home information of the target directory sent by the metadata routing server, and the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server.
具体地,第一客户端接收元数据路由服务器发送的归属信息,对该归属信息进行分析得出该目标目录的操作权限归属到目标元数据服务器。Specifically, the first client receives the home information sent by the metadata routing server, and analyzes the attribution information to obtain that the operation authority of the target directory belongs to the target metadata server.
步骤S309:所述第一客户端向所述目标元数据服务器发送操作请求消息,请求所述目标元数据服务器根据所述对所述目标目录中所有元数据的排它访问权限操作所述第一目标元数据。Step S309: The first client sends an operation request message to the target metadata server, requesting the target metadata server to operate the first according to the exclusive access right to all metadata in the target directory. Target metadata.
具体地,由于第一客户端已经获取了目标目录中所有元数据的排他访问权限,因此第一客户端有权限向目标元数据服务器发送操作请求消息,请求目标元数据服务器对第一目标元数据进行操作,而其他未获取到目标目录中所有元数据的排它访问权限的客户端则无法请求目标元数据服务器对该第一目标元数据进行操作。Specifically, since the first client has obtained exclusive access rights of all metadata in the target directory, the first client has the right to send an operation request message to the target metadata server, requesting the target metadata server to the first target metadata. The operation is performed, and other clients that do not have exclusive access to all metadata in the target directory cannot request the target metadata server to operate on the first target metadata.
步骤S310:所述第一客户端接收所述目标元数据服务器发送的操作结果;所述操作结果为所述目标元数据服务器向所述目录锁服务器申请到对所述目标目录中所有元数据的排它操作权限后,对所述第一目标元数据进行操作的结果。Step S310: The first client receives an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server to all metadata in the target directory. The result of operating the first target metadata after exclusive operation permission.
具体地,目标元数据服务器接收第一客户端发送的操作请求消息,对该请求消息进行解析得出第一客户端请求操作的元数据为目标目录中的第一目标元数据,因此向目录锁服务器发送申请请求,申请目标目录中所有元数据的排它操作权限;相应地,目录锁服务器接收元数据服务器发送的申请请求,此时无论目标目录中的所有元数据的排它操作权限是否授予了其他客户端,目录锁服务器均会将目标目录中所有元数据的排它操作权限授予目标元数据服务器(假如目标目录的排它操作权限具体通过分布式锁来实现,那么当操作目标目录的锁已经授予了其他客户端时,目录锁服务器召回操作目标目录的锁,然后将操作目标目录的锁授予第一客户端;当操作目标目录的锁未授予其他客户端时,目录锁服务器直接将操作目标目录的锁授予第一客户端),使得所有的元数据服务器中只有目标元数据服务器能够对目标目录中的元数据进行操作。进一步地,目标元数据服务器对目标目录中的第一目标元数据进行操作,然后将操作该第一目标元数据的操作结果发送给第一客户端,相应地,第一客户端接收目标元数据服务器发送的操作该第一目标元数据的操作结果,至此,第一客户端完成了对目标目录中的第一目标元数据的访问。Specifically, the target metadata server receives the operation request message sent by the first client, and parses the request message to obtain that the metadata of the first client request operation is the first target metadata in the target directory, and thus the directory lock The server sends an application request to apply for exclusive operation permission of all metadata in the target directory; accordingly, the directory lock server receives the application request sent by the metadata server, and at this time, regardless of whether the exclusive operation permission of all metadata in the target directory is granted For other clients, the directory lock server grants the exclusive operation permission of all metadata in the target directory to the target metadata server (if the exclusive operation permission of the target directory is specifically implemented by distributed locks, then when the target directory is operated When the lock has been granted to other clients, the directory lock server recalls the lock of the operation target directory, and then grants the lock of the operation target directory to the first client; when the lock of the operation target directory is not granted to other clients, the directory lock server directly The lock of the operation target directory is granted to the first client), so that all Only the target metadata server capable of operating on metadata in the target directory metadata server. Further, the target metadata server operates the first target metadata in the target directory, and then sends an operation result of operating the first target metadata to the first client, and correspondingly, the first client receives the target metadata. The operation result of the operation of the first target metadata sent by the server, so far, the first client completes the access to the first target metadata in the target directory.
需要说明的是,申请目标目录中所有元数据的排它操作权限与申请目标目录中所有元数据的排它访问权限都可以通过基于目录粒度的分布式锁来实现,其中,申请目标目录中所有元数据的排它操作权限用到的是可抢占的分布式锁,申请目标目录中所有元数据的排它访问权限用到的是不可抢占的分布式锁。It should be noted that the exclusive operation permission of all metadata in the application target directory and the exclusive access permission of all metadata in the application target directory can be realized by the distributed lock based on the directory granularity, wherein all the application target directories are The exclusive operation permission of metadata uses a preemptible distributed lock. The exclusive access permission for all metadata in the target directory is a non-preemptable distributed lock.
通过执行步骤S307~S310,目标元数据服务器接收到第一客户端发送的访问目标目录下的第一目标元数据的请求后,在对目标目录中的第一目标元数据进行操作之前,先向目录锁服务器申请目标目录中所有元数据的排它操作权限,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性。 After performing the steps S307-S310, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server performs the operation on the first target metadata in the target directory. The directory lock server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata of the target directory is consistently saved. Sex.
步骤S304中第一客户端获取到目标目录中所有元数据的排它访问权限后,所述第一客户端访问所述第一目标元数据可以具体通过步骤S311~S315来实现;对应的流程图如图6所示。步骤S311~S315的详细描述如下:After the first client obtains the exclusive access rights of all the metadata in the target directory, the first client accesses the first target metadata by using the steps S311-S315; As shown in Figure 6. The detailed description of steps S311 to S315 is as follows:
步骤S311:所述第一客户端向元数据路由服务器发送切换消息,指示所述元数据路由服务器将所述目标目录的操作权限归属到所述第一客户端。Step S311: The first client sends a handover message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client.
具体地,第一客户端向元数据路由服务器发送切换消息,元数据路由服务器接收第一客户端发送的该切换消息,并对该切换消息进行分析,当分析出该切换消息用于指示将目标目录归属到第一客户端时,元数据路由服务器修改路由表中的映射关系,使目标目录对应到第一客户端,即,将目标目录的操作权限归属到第一客户端。Specifically, the first client sends a handover message to the metadata routing server, and the metadata routing server receives the handover message sent by the first client, and analyzes the handover message, and analyzes the handover message to indicate that the target is to be When the directory belongs to the first client, the metadata routing server modifies the mapping relationship in the routing table, so that the target directory corresponds to the first client, that is, the operation authority of the target directory is attributed to the first client.
步骤S312:所述第一客户端接收所述元数据路由服务器发送的通知消息,所述通知消息用于指示所述目标目录的操作权限归属到所述第一客户端。Step S312: The first client receives the notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client.
具体地,元数据路由服务器向第一客户端发送路由切换成功的通知消息,第一客户端接收元数据路由服务器发送的通知消息,对该通知消息进行分析可以获知目标目录已经归属到第一客户端。Specifically, the metadata routing server sends a notification message that the route switching succeeds to the first client, and the first client receives the notification message sent by the metadata routing server, and analyzes the notification message to know that the target directory has belonged to the first client. end.
需要说明的是,在第一客户端中可以设置客户端代理和元数据服务代理,客户端代理用于提供元数据的缓存和向目录锁服务器申请目标目录中所有元数据的排它访问权限;元数据服务代理用于存储目标目录和向目录锁服务器申请目标目录中所有元数据的排它操作权限。It should be noted that a client proxy and a metadata service proxy may be set in the first client, and the client proxy is used to provide a cache of metadata and an exclusive access right to apply for all metadata in the target directory to the directory lock server; The Metadata Service Agent is used to store the target directory and to apply exclusive access to the Directory Lock Server for all metadata in the target directory.
步骤S313:所述第一客户端向所述目录锁服务器发送第二申请消息以申请对所述目标目录中所有元数据的排它操作权限。Step S313: The first client sends a second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory.
具体地,当第一客户端通过通知消息获知目标目录归属到自身时,第一客户端向目录锁服务器发送第二申请消息,申请目标目录中所有元数据的排它操作权限;相应地,目录锁服务器接收第一客户端发送的第二申请消息,此时无论是否有其他的元数据服务器对目标目录进行操作,目录锁服务器均会将目标目录的操作权限授予第一客户端,并向第一客户端发送授予消息来通知目标目录中所有元数据的操作权限已经授予了该第一客户端。Specifically, when the first client knows that the target directory belongs to itself through the notification message, the first client sends a second application message to the directory lock server to apply for exclusive operation permission of all metadata in the target directory; correspondingly, the directory The lock server receives the second application message sent by the first client, and the directory lock server grants the operation authority of the target directory to the first client, regardless of whether another metadata server operates the target directory. A client sends a grant message to notify all the metadata in the target directory that the operational rights have been granted to the first client.
步骤S314:所述第一客户端接收所述目录锁服务器发送的授予所述目标目录中所有元数据的排它操作权限的授予消息,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的第一目标元数据,以及操作所述目标目录中除所述第一目标元数据之外的元数据。Step S314: The first client receives a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used. Providing, by the first client, first target metadata in the target directory, and operating metadata in the target directory other than the first target metadata.
具体地,第一客户端接收目录锁服务器发送的授予消息,对该授予消息分析之后可以获知自身已经获取了目标目录中所有元数据的排它操作权限(包括对第一目标元数据以及目标目录中除第一目标元数据以外的元数据的排它操作权限),至此,第一客户端就可以对目标目录中的元数据进行操作,而无需请求额外的元数据服务器对目标目录下的元数据进行操作。Specifically, the first client receives the grant message sent by the directory lock server, and after analyzing the grant message, can learn that it has acquired exclusive operation rights of all metadata in the target directory (including the first target metadata and the target directory). In this case, the exclusive operation permission of the metadata other than the first target metadata), so that the first client can operate on the metadata in the target directory without requesting an additional metadata server for the element in the target directory. The data is manipulated.
步骤S315:所述第一客户端根据所述目标目录中所有元数据的排它操作权限对所述第一目标元数据进行操作,获得操作结果。Step S315: The first client operates the first target metadata according to the exclusive operation authority of all the metadata in the target directory, and obtains an operation result.
具体地,第一客户端基于已经获取的目标目录中所有元数据的排它操作权 限对目标目录中的第一目标元数据进行操作,并获得操作结果,至此,第一客户端完成了对目标目录中的第一目标元数据的访问。当访问的形式是读时,该操作结果包括是否读取成功,当读取成功时还包括读取到的元数据;当访问的形式为写(或修改)时,该操作结果包括是否写入成功,其余形式的访问此处不一一举例。Specifically, the first client is based on the exclusive operation rights of all metadata in the target directory that has been acquired. The operation is performed on the first target metadata in the target directory, and the operation result is obtained. At this point, the first client completes the access to the first target metadata in the target directory. When the form of the access is read, the result of the operation includes whether the reading is successful, and the read metadata is included when the reading is successful; when the form of the access is written (or modified), the result of the operation includes whether to write or not Success, the rest of the forms of access here are not examples.
通过执行步骤S311~S315,第一客户端同时充当客户端和元数据服务器的功能,第一客户端获取到目标目录中所有元数据的排它访问权限后,不需要与远端的元数据服务器进行交互来请求元数据服务器对第一目标元数据进行操作,减少了节点之间的开销,提高了访问第一目标元数据的访问效率。By performing steps S311-S315, the first client functions as both a client and a metadata server, and the first client does not need to have a remote metadata server after obtaining exclusive access rights of all metadata in the target directory. Interacting to request the metadata server to operate on the first target metadata reduces the overhead between the nodes and improves the access efficiency of accessing the first target metadata.
执行完上述步骤S303~S304之后,如果目录锁服务器授予了第一客户端访问目标目录的权限,第一客户端还可执行步骤S316和S317来均衡各个客户端的负载,具体地流程图如图7所示。步骤S316和S317的详细描述如下:After performing the above steps S303-S304, if the directory lock server grants the first client access to the target directory, the first client may also perform steps S316 and S317 to balance the load of each client, specifically, the flowchart is as shown in FIG. 7. Shown. The detailed description of steps S316 and S317 is as follows:
步骤S316:所述第一客户端判断访问所述目标目录中的元数据所产生的负载是否满足预设条件。Step S316: The first client determines whether the load generated by accessing the metadata in the target directory satisfies a preset condition.
第一种情况,第一客户端在访问目标目录中的元数据之后,实时检测自身运行时的整体负载,并将该整体负载的负载值与预设阈值进行比较,判断第一客户端的整体负载是否超过了预设阈值,该预设阈值为根据实际需要设置的一个用来参考对比的负载值。In the first case, after accessing the metadata in the target directory, the first client detects the overall load of the runtime in real time, and compares the load value of the overall load with a preset threshold to determine the overall load of the first client. Whether the preset threshold is exceeded or not, and the preset threshold is a load value used for reference comparison according to actual needs.
第二种情况,所述第一客户端记录自身主动访问所述目标目录下的元数据所产生的第一负载,以及根据其他客户端的请求消息访问所述目标目录中的元数据所产生的第二负载;所述第一客户端进一步判断所述第一负载与所述第二负载的相对大小是否满足预设条件,例如,第一负载小于第二负载。In the second case, the first client records the first load generated by the user actively accessing the metadata in the target directory, and the first data generated by accessing the metadata in the target directory according to the request message of other clients. The second client further determines whether the relative size of the first load and the second load meets a preset condition, for example, the first load is smaller than the second load.
步骤S317:若满足,则所述第一客户端释放所述目标目录中的所有元数据的排它访问权限,以便所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予第四客户端。Step S317: if satisfied, the first client releases exclusive access rights of all metadata in the target directory, so that the directory lock server grants exclusive access rights of all metadata in the target directory. Fourth client.
具体地,当第一客户端判断访问所述目标目录中的元数据所产生的负载满足预设条件时,主动释放目标目录中所有元数据的排它访问权限,这时候,第四客户端则可以向目录锁服务器申请目标目录中所有元数据的排它访问权限,当第四客户端申请到了目标目录中所有元数据的排它访问权限后,第一客户端如果需要访问目标目录则需要通过该第四客户端来访问。Specifically, when the first client determines that the load generated by accessing the metadata in the target directory meets a preset condition, the exclusive access permission of all metadata in the target directory is actively released, and at this time, the fourth client is You can apply to the directory lock server for exclusive access to all metadata in the target directory. After the fourth client applies for exclusive access to all metadata in the target directory, the first client needs to access the target directory if it needs to access the target directory. The fourth client comes to visit.
需要说明的时,本发明实施例中的第三客户端和第四客户端可以为同一个客户端,也可以不为同一个客户端。The third client and the fourth client in the embodiment of the present invention may be the same client or may not be the same client.
通过执行步骤S316和S317,第一客户端上的负载过高时,自动释放目标目录中所有元数据的排它访问权限,让负载较低的第四客户端来获取该目标目录中所有元数据的排它访问权限以达到分担负载的目的;或者在第一客户端访问目标目录中的元数据的频率比其他客户端访问目标目录中的元数据的频率低时,自动释放目标目录中所有元数据的排它访问权限,让访问频率较高的客户端获取目标目录中所有元数据的排它访问权限,以减少转发访问所产生的负载,提高了访问目标目录中元数据的效率。 By performing steps S316 and S317, when the load on the first client is too high, the exclusive access permission of all metadata in the target directory is automatically released, and the fourth client with lower load is obtained to obtain all the metadata in the target directory. Exclude access rights to achieve the purpose of sharing the load; or automatically release all the elements in the target directory when the first client accesses the metadata in the target directory less frequently than the other clients access the target directory. Exclusive access to data allows clients with higher frequency of access to obtain exclusive access to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
进一步地,上述步骤S303~S317中的部分步骤可以进行组合形成其他方案例如步骤S303~S306与步骤S307~S310进行组合;步骤S303~S306与步骤S311~S315进行组合;步骤S303~S310与S316~S317进行组合;步骤S303~S306与步骤S311~S317进行组合等,组合产生的益效果可以参照上述各个步骤的相关描述,此处不再赘述。Further, some of the steps S303 to S317 may be combined to form another solution, for example, steps S303 to S306 are combined with steps S307 to S310; steps S303 to S306 are combined with steps S311 to S315; and steps S303 to S310 and S316 are combined. S317 is combined; steps S303-S306 are combined with steps S311-S317, etc., and the beneficial effects generated by the combination can be referred to the related descriptions of the above steps, and details are not described herein again.
以上对本发明的方案进行了详细介绍,以下结合具体场景从多侧对图5和图6所示的方案进行描述。The scheme of the present invention has been described in detail above. The schemes shown in FIG. 5 and FIG. 6 are described from multiple sides in combination with specific scenarios.
请参照图5A和5B,图5A是本发明实施例提供的一种元数据访问的原理示意图;图5B是图5A中各个设备进行交互的流程示意图。图5A中包括第一客户端101、第二客户端102、第三客户端103(可以根据实际需要来部署客户端的数量)、目录锁服务器305、元数据路由服务器303、第一元数据服务器201、第二元数据服务器202和第三元数据服务器203。5A and 5B, FIG. 5A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention; and FIG. 5B is a schematic flowchart of interaction between devices in FIG. 5A. FIG. 5A includes a first client 101, a second client 102, and a third client 103 (the number of clients can be deployed according to actual needs), a directory lock server 305, a metadata routing server 303, and a first metadata server 201. The second metadata server 202 and the third metadata server 203.
图5A和5B中还特地为各个设备示出了人为划分的功能单元,旨在突出各个设备能够执行哪些步骤,相应设备中可能存在与该功能单元相对应的物理实体,也可能不存在与该功能单元相对应的物理实体,以下对各个设备的功能单元进行简单的介绍:第一客户端101(其他客户端也一样)包含元数据缓存单元和访问目录的锁单元,元数据缓存单元用于对元数据服务器发送的元数据进行缓存,访问目录的锁单元用于向目录锁服务器305的访问目录的锁单元申请目录中所有元数据的排它访问权限;元数据路由服务器303包含路由服务单元,用于为第一客户端101提供目录的归属查询服务;第一元数据服务器201、第二元数据服务器202和第三元数据服务器203均包含元数据服务单元和操作目录的锁单元,元数据服务单元用于实现元数据的持久化保存一致,操作目录的锁单元用于向目录锁服务器305的操作目录的锁单元申请目录中所有元数据的排它操作权限。5A and 5B also specifically show the artificially divided functional units for each device, aiming to highlight which steps each device can perform, there may be physical entities corresponding to the functional units in the corresponding devices, or there may be no such The physical unit corresponding to the functional unit, the following briefly introduces the functional unit of each device: the first client 101 (the same is true for other clients) includes a metadata cache unit and a lock unit for accessing the directory, and the metadata cache unit is used for The metadata sent by the metadata server is cached, and the lock unit of the access directory is used to apply exclusive access rights to all metadata in the directory to the lock unit of the directory lock server 305; the metadata routing server 303 includes the routing service unit a home query service for providing a directory for the first client 101; the first metadata server 201, the second metadata server 202, and the third metadata server 203 each include a metadata service unit and a lock unit of the operation directory, and a meta The data service unit is used to implement consistent persistence of metadata, and the lock unit of the operation directory is used. Exclusive rights to all the metadata of the operating unit lock lock server application Catalog directory operation 305.
第一客户端101访问路径“…/A/B/f”时的流程如步骤S501~S517所示,其中,目录B为目录A下的目录,第一目标元数据f为目录B下的众多元数据中的一项。The flow when the first client 101 accesses the path ".../A/B/f" is as shown in steps S501 to S517, wherein the directory B is a directory under the directory A, and the first target metadata f is a plurality of directories B. An item in the metadata.
步骤S501:第一客户端101向元数据路由服务器303请求目录A的归属信息。Step S501: The first client 101 requests the metadata routing server 303 for the home information of the directory A.
步骤S502:元数据路由服务器303从存储的目录与元数据服务器的映射关系中,查找出目录A所归属的第一元数据服务器201,并告知第一客户端101。Step S502: The metadata routing server 303 searches the first metadata server 201 to which the directory A belongs from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
步骤S503:第一客户端101向第一元数据服务器201请求目录A下描述目录B的元数据。Step S503: The first client 101 requests the first metadata server 201 for metadata describing the directory B under the directory A.
步骤S504:第一元数据服务器201向目录锁服务器305发送申请请求,请求操作目录A的锁(为了描述方便可以称作“目录A的归属锁”)以获取目录A中所有元数据的排它访问权限。Step S504: The first metadata server 201 sends an application request to the directory lock server 305, requesting the operation of the lock of the directory A (for the convenience of description, may be referred to as "the home lock of the directory A") to obtain the exclusive information of all the metadata in the directory A. access permission.
步骤S505:目录锁服务器305将目录A的归属锁授予第一元数据服务器201,使得所有的元数据服务器中只有第一元数据服务器201具备对目录A中 所有元数据的操作权限。Step S505: the directory lock server 305 grants the home lock of the directory A to the first metadata server 201, so that only the first metadata server 201 of all the metadata servers is provided in the directory A. Operational authority for all metadata.
步骤S506:第一元数据服务器201从目录A中读取描述目录B的元数据。Step S506: The first metadata server 201 reads the metadata describing the directory B from the directory A.
步骤S507:第一元数据服务器201将读取到的描述目录B的元数据发送给第一客户端101。Step S507: The first metadata server 201 sends the read metadata describing the directory B to the first client 101.
步骤S508:第一客户端101对描述目录B的元数据进行分析,判断目录B是否为存储多个元数据的目录。Step S508: The first client 101 analyzes the metadata describing the directory B, and determines whether the directory B is a directory in which a plurality of metadata is stored.
步骤S509:若判断出目录B为存储多个元数据的目录,则第一客户端101向目录锁服务器305发送申请请求,请求访问目录B的锁(为了描述方便可以称作“目录B的加速锁”)以获取目录B中所有元数据的排它访问权限。Step S509: If it is determined that the directory B is a directory storing a plurality of metadata, the first client 101 sends an application request to the directory lock server 305, requesting access to the lock of the directory B (for the convenience of description, it may be referred to as "the acceleration of the directory B". Lock") to get exclusive access to all metadata in directory B.
步骤S510:当目录锁服务器305发现目录B的加速锁未授予其他客户端时,目录锁服务器305将目录B的加速锁授予第一客户端101,使得所有的客户端中只有第一客户端101具备访问目录B中所有元数据的权限。之后的步骤S511~S517为第一客户端访问目录B中的元数据的具体过程。Step S510: When the directory lock server 305 finds that the acceleration lock of the directory B is not granted to other clients, the directory lock server 305 grants the acceleration lock of the directory B to the first client 101, so that only the first client 101 among all the clients Have access to all metadata in directory B. Subsequent steps S511 to S517 are specific processes for the first client to access the metadata in the directory B.
步骤S511:第一客户端101向元数据路由服务器303请求目录B的归属信息。Step S511: The first client 101 requests the metadata routing server 303 for the home information of the directory B.
步骤S512:元数据路由服务器303从存储的目录与元数据服务器的映射关系中,查找出目录B所对应的(即归属的)第二元数据服务器202,并告知第一客户端101。Step S512: The metadata routing server 303 searches the second metadata server 202 corresponding to the directory B (that is, the home metadata server 202) from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
步骤S513:第一客户端101向第二元数据服务器202请求访问目录B下的第一目标元数据f。Step S513: The first client 101 requests the second metadata server 202 to access the first target metadata f under the directory B.
步骤S514:第二元数据服务器202向目录锁服务器305申请操作目录B的锁(即目录B的归属锁)以获取目录B中所有元数据的排它操作权限。Step S514: The second metadata server 202 applies to the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
步骤S515:基于抢占的分布式锁的规则,无论目录B的归属锁是否授予了其他元数据服务器,目录锁服务器305均会将目录B的归属锁授予第二元数据服务器202,并告知第二元数据服务器202,使得在所有的元数据服务器中只有第二元数据服务器202具备对目标目录中所有元数据的排它操作权限。Step S515: Based on the preemptive distributed lock rule, regardless of whether the home lock of the directory B is granted to other metadata servers, the directory lock server 305 grants the home lock of the directory B to the second metadata server 202, and informs the second The metadata server 202 enables only the second metadata server 202 to have exclusive operational rights to all metadata in the target directory among all metadata servers.
步骤S516:第二元数据服务器202对目录B中第一目标元数据f进行操作。Step S516: The second metadata server 202 operates on the first target metadata f in the directory B.
步骤S517:第二元数据服务器202将操作第一目标元数据f的操作结果发送给第一客户端101。Step S517: The second metadata server 202 transmits the operation result of operating the first target metadata f to the first client 101.
需要说明的是,步骤S501~S517执行的细节可以对应参照步骤S303~S304,以及步骤S307~S310的相关描述,此处不再赘述。It should be noted that the details of the steps S501 to S517 can be referred to the steps S303 to S304 and the related descriptions of the steps S307 to S310, and details are not described herein again.
请参见图6A和6B,图6A是本发明实施例提供的一种元数据访问的原理示意图;图6B是图6A中各个设备进行交互的流程示意图。图6A中包括第一客户端101、第二客户端102、第三客户端103(可以根据实际需要来部署客户端的数量)、目录锁服务器305和元数据路由服务器303。Referring to FIG. 6A and FIG. 6B, FIG. 6A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention; and FIG. 6B is a schematic flowchart of interaction between devices in FIG. 6A. FIG. 6A includes a first client 101, a second client 102, a third client 103 (the number of clients can be deployed according to actual needs), a directory lock server 305, and a metadata routing server 303.
图6A和6B中还特地为各个设备示出了人为划分的功能单元,以下对各个设备的功能单元进行简单的介绍:第一客户端101(其他客户端也一样)包含元数据缓存单元、访问目录的锁单元、元数据服务单元和操作目录的锁单元, 元数据缓存单元用于对元数据服务器发送的元数据进行缓存,访问目录的锁单元用于向目录锁服务器305的访问目录的锁单元申请该目录的排它访问权限,元数据服务单元用于实现元数据的读、写、持久化保存等操作,操作目录的锁单元用于向目录锁服务器305的操作目录的锁单元申请目录的排它操作权限;元数据路由服务器303包含路由服务单元和路由切换单元,路由服务单元用于为第一客户端101提供目录的归属查询服务,路由切换单元用于将目录归属到客户端上。6A and 6B also specifically show the functional units of artificial division for each device. The following briefly introduces the functional units of the respective devices: the first client 101 (the same is true for other clients) includes a metadata cache unit and access. The lock unit of the directory, the metadata service unit, and the lock unit of the operation directory, The metadata cache unit is configured to cache the metadata sent by the metadata server, and the lock unit of the access directory is used to apply for exclusive access rights of the directory to the lock unit of the directory lock server 305, and the metadata service unit is used for To implement operations such as reading, writing, and persisting metadata, the lock unit of the operation directory is used to apply the exclusive operation authority of the directory to the lock unit of the operation directory of the directory lock server 305; the metadata routing server 303 includes the routing service unit and The routing switching unit is configured to provide a first query service for the first client 101, and the routing switching unit is configured to attribute the directory to the client.
第一客户端101访问路径“…/A/B/f”时的流程如步骤S601~S615所示,其中,目录B为目录A下的目录,第一目标元数据f为目录B下的众多元数据中的一项。The flow when the first client 101 accesses the path ".../A/B/f" is as shown in steps S601-S615, wherein the directory B is a directory under the directory A, and the first target metadata f is a plurality of directories B. An item in the metadata.
步骤S601:第一客户端101向元数据路由服务303器请求目录A的归属信息。Step S601: The first client 101 requests the home information of the directory A from the metadata routing service 303.
步骤S602:元数据路由服务器303从存储的目录与元数据服务器的映射关系中,查找出目录A所归属的目标元数据服务器,并告知第一客户端101。Step S602: The metadata routing server 303 finds the target metadata server to which the directory A belongs from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
步骤S603:第一客户端101向目标元数据服务器请求目录A下描述目录B的元数据。Step S603: The first client 101 requests the target metadata server for the metadata describing the directory B under the directory A.
步骤S604:目标元数据服务器向目录锁服务器305发送申请请求,请求操作目录A的锁(为了描述方便可以称作“目录A的归属锁”)以获取目录A中所有元数据的排它访问权限。Step S604: The target metadata server sends an application request to the directory lock server 305, requesting the lock of the operation directory A (referred to as "home lock of directory A" for convenience of description) to obtain exclusive access rights of all metadata in the directory A. .
步骤S605:目录锁服务器305将目录A的归属锁授予目标元数据服务器,使得所有的元数据服务器中只有目标元数据服务器具备对目录A中所有元数据的操作权限。Step S605: The directory lock server 305 grants the home lock of the directory A to the target metadata server, so that only the target metadata server of all the metadata servers has the operation authority for all the metadata in the directory A.
步骤S606:目标元数据服务器从目录A中读取描述目录B的元数据。Step S606: The target metadata server reads the metadata describing the directory B from the directory A.
步骤S607:目标元数据服务器将读取到的描述目录B的元数据发送给第一客户端101。Step S607: The target metadata server sends the read metadata describing the directory B to the first client 101.
步骤S608:第一客户端101对描述目录B的元数据进行分析,判断目录B是否为存储多个元数据的目录。Step S608: The first client 101 analyzes the metadata describing the directory B, and determines whether the directory B is a directory in which a plurality of metadata is stored.
步骤S609:若判断出目录B为存储多个元数据的目录,则第一客户端101向目录锁服务器305发送申请请求,请求访问目录B的锁(为了描述方便可以称作“目录B的加速锁”)以获取目录B中所有元数据的排它访问权限。Step S609: If it is determined that the directory B is a directory storing a plurality of metadata, the first client 101 sends an application request to the directory lock server 305, requesting access to the lock of the directory B (for the convenience of description, it may be referred to as "the acceleration of the directory B". Lock") to get exclusive access to all metadata in directory B.
步骤S610:当目录锁服务器305发现目录B的加速锁未授予其他客户端时,目录锁服务器305将目录B的加速锁授予第一客户端101,使得所有的客户端中只有第一客户端101具备访问目录B中所有元数据的权限。之后的步骤S611~S615为第一客户端101访问目录B中的元数据的具体过程。Step S610: When the directory lock server 305 finds that the acceleration lock of the directory B is not granted to other clients, the directory lock server 305 grants the acceleration lock of the directory B to the first client 101, so that only the first client 101 among all the clients Have access to all metadata in directory B. Subsequent steps S611 to S615 are specific processes in which the first client 101 accesses the metadata in the directory B.
步骤S611:第一客户端101向元数据路由服务器303发送请求消息,请求元数据路由服务器修改目录与目标元数据服务器的对应关系,将目录B的归属对应到第一客户端101上。Step S611: The first client 101 sends a request message to the metadata routing server 303, requesting the metadata routing server to modify the correspondence between the directory and the target metadata server, and mapping the attribution of the directory B to the first client 101.
步骤S612:元数据路由服务器303修改目录与目标元数据服务器的对应关系,将目录B归属到第一客户端101上,并告知第一客户端101。 Step S612: The metadata routing server 303 modifies the correspondence between the directory and the target metadata server, attributes the directory B to the first client 101, and notifies the first client 101.
步骤S613:第一客户端101向目录锁服务器305申请操作目录B的锁(即目录B的归属锁)以获取目录B中所有元数据的排它操作权限。Step S613: The first client 101 applies to the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
步骤S614:基于抢占的分布式锁的规则,无论目录B的归属锁是否授予了其他目标元数据服务器,目录锁服务器305均会将目录B的归属锁授予第一客户端101,并告知第一客户端101,使得只有第一客户端101自身具备对目标目录中的元数据的操作权限。Step S614: Based on the preemptive distributed lock rule, regardless of whether the home lock of the directory B is granted to other target metadata servers, the directory lock server 305 grants the home lock of the directory B to the first client 101, and informs the first The client 101 is such that only the first client 101 itself has the operational authority to the metadata in the target directory.
步骤S615:第一客户端101对目录B中的第一目标元数据f进程操作,获得操作结果。Step S615: The first client 101 operates on the first target metadata f process in the directory B, and obtains an operation result.
需要说明的是,步骤S601~S615执行的细节可以对应参照步骤S302~S304,以及步骤S311~S315的相关描述,此处不再赘述。It should be noted that the details of the steps S601-S615 may be referred to the steps S302-S304 and the descriptions of the steps S311-S315, and details are not described herein again.
上述详细阐述了本发明实施例的方法,为了便于更好地实施本发明实施例的上述方案,相应地,下面提供了本发明实施例的装置。The method of the embodiments of the present invention is described in detail above. In order to facilitate the implementation of the above embodiments of the embodiments of the present invention, correspondingly, the apparatus of the embodiments of the present invention is provided below.
请参见图8,图8是本发明实施例提供的一种终端80的结构示意图,该终端80为以下描述的第一客户端80,该终端80可以包括接收装置801、发送装置802,存储器803和处理器804(处理器可以为一个也可以为多个,图8中以一个处理器为例),接收装置801、发送装置802,存储器803和处理器804分别连接总线805,其中,存储器803中存储一组元数据访问程序代码,且处理器804调用存储器803中存储的元数据访问程序代码,用于执行以下操作:Referring to FIG. 8, FIG. 8 is a schematic structural diagram of a terminal 80 according to an embodiment of the present invention. The terminal 80 is a first client 80, which may include a receiving device 801, a sending device 802, and a memory 803. And the processor 804 (the processor may be one or more, and one processor in FIG. 8 is taken as an example), the receiving device 801, the transmitting device 802, the memory 803 and the processor 804 are respectively connected to the bus 805, wherein the memory 803 A set of metadata access program code is stored, and the processor 804 calls the metadata access program code stored in the memory 803 for performing the following operations:
在请求访问目标目录中的第一目标元数据时通过所述发送装置802向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述处理器804当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述处理器804访问所述目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;Transmitting, by the sending device 802, a first application message to the directory lock server to request exclusive access rights to all metadata in the target directory when requesting access to the first target metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the processor 804 currently needs to access; exclusive access rights of all metadata in the target directory are used for the processing The device 804 accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
当申请到对所述目标目录中所有元数据的排它访问权限时,访问所述第一目标元数据。The first target metadata is accessed when applying for exclusive access to all metadata in the target directory.
在具体实现中,目录锁服务器能够提供分布式锁服务,如搭载zookeeper服务的服务器;目录锁服务器通过授予访问目标目录的锁来协调各个客户端对目标目录的访问权限,如果目录锁服务器将访问目标目录的锁授予处理器804,那么处理器804就具备了访问目标目录下所有元数据的权限;进一步地,分布式锁服务可以设置为可抢占分布式锁和不可抢占分布式锁;若设置为可抢占的分布式锁,处理器804申请访问目标目录的锁时,如果访问目标目录的锁已经授予了其他客户端,那么目录锁服务器会根据抢占机制释放访问目标目录的锁,并重新将访问目标目录的锁授予该处理器804,如果访问目标目录的锁未授予其他客户端,那么目录锁服务器则直接将访问目标目录的锁授予处理器804;若设置为不可抢占的分布式锁,处理器804申请访问目标目录的锁时,如果访问目标目录的锁已经授予了其他客户端,那么目录锁服务器会根据不可抢占的机制,不会将访问目标目录的锁授予第一客户端80,如果访问目标目录 的锁未授予其他客户端,那么目录锁服务器则直接将访问目标目录的锁授予处理器804。在本发明实施例中,访问目标目录的锁可以设置为不可抢占的分布式锁,这样一来,处理器804申请到访问目标目录的锁后,就具备了对目标目录中所有元数据的排它访问权限,即除了处理器804之外,其他的客户端没有访问该目标目录中元数据的权限,除非该处理器804主动释放访问目标目录的锁。In a specific implementation, the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the directory lock server coordinates the access rights of each client to the target directory by granting a lock to the target directory, if the directory lock server will access The lock of the target directory is granted to the processor 804, then the processor 804 has the right to access all the metadata in the target directory; further, the distributed lock service can be set to preemptive distributed locks and non-preemptable distributed locks; For a preemptible distributed lock, when the processor 804 requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server releases the lock accessing the target directory according to the preemption mechanism, and re- The lock accessing the target directory is granted to the processor 804. If the lock accessing the target directory is not granted to other clients, the directory lock server directly grants the lock accessing the target directory to the processor 804; if it is set to a non-preemptable distributed lock, When the processor 804 requests to access the lock of the target directory, if the access is Lock the target directory has been granted to other clients, the directory server will be based on the lock mechanism can not be seized, the lock access to the target directory does not grant the first client 80, if access to the target directory The lock is not granted to other clients, and the directory lock server directly grants the lock to the target directory to the processor 804. In the embodiment of the present invention, the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the processor 804 applies for the lock to access the target directory, it has the row of all the metadata in the target directory. It has access rights, that is, other clients than the processor 804 have no access to the metadata in the target directory unless the processor 804 actively releases the lock accessing the target directory.
处理器804通过执行上述操作,在访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中所有元数据的排它访问权限,这样一来,第一客户端80就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。The processor 804 performs the foregoing operation, and when accessing the first target metadata in the target directory, requests the directory lock server for exclusive access rights of all metadata in the target directory, so that the first client 80 can Successful access to the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply access rights, saving the overhead of maintaining cache coherency.
作为一种可选的实施方案,当未申请到对所述目标目录中所有元数据的排它访问权限时,通过所述接收装置801接收所述目录锁服务器发送的第二客户端所在的节点信息,并根据所述节点信息通过所述发送装置802向所述第二客户端发送请求消息以通过所述第二客户端访问所述目标目录中的元数据,所述第二客户端为向所述目录锁服务器申请到了对所述目标目录中所有元数据的排它访问权限的客户端。As an optional implementation, when the exclusive access right to all the metadata in the target directory is not applied, the receiving device 801 receives the node where the second client sent by the directory lock server is located. And sending, by the sending device 802, a request message to the second client according to the node information to access metadata in the target directory by using the second client, where the second client is The directory lock server applies to a client that has exclusive access to all metadata in the target directory.
处理器704通过执行上述操作,在申请不到目标目录中所有元数据的排它访问权限时,通过已经获取了目标目录中所有元数据的排它访问权限的第二客户端访问第一目标元数据,使得第一客户端80在申请不到目标目录中所有元数据的排它访问权限时依然可以顺利访问到目标元数据。The processor 704, by performing the above operation, accesses the first target element through the second client that has acquired exclusive access rights of all metadata in the target directory when the application does not obtain exclusive access rights of all metadata in the target directory. The data enables the first client 80 to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
作为一种可选的实施方案,所述处理器804通过所述发送装置802向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限的方式具体为:As an optional implementation manner, the manner in which the processor 804 sends the first application message to the directory lock server by using the sending device 802 to apply for exclusive access rights to all metadata in the target directory is specifically:
获取描述所述目标目录的元数据,所述描述所述目标目录的元数据设有用于标识所述目标目录是否为存储有多个元数据的目录的标识信息;Obtaining metadata describing the target directory, where the metadata describing the target directory is provided with identification information for identifying whether the target directory is a directory in which a plurality of metadata is stored;
根据所述描述所述目标目录的元数据来判断所述目标目录是否为存储有多个元数据的目录;Determining, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
若是,则通过所述发送装置802向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限。If so, the first application message is sent to the directory lock server by the sending device 802 to apply for exclusive access rights to all metadata in the target directory.
具体实现中,处理器804通过获取描述目标目录的元数据来判断目标目录中是否为存储多个元数据的目录,在现有的元数据的基础上,新增一项“目录属性”,该“目录属性”的值不同则代表目标目录为不同属性,其中,有一种属性用来表明该目标目录为存储多个元数据的目录,相应地,处理器804对该“目录属性”进行分析即可得出该目标目录是否为存储多个元数据的目录。作为一种可选的实施方式,当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理器804访问所述第一目标元数据之后,所述处理器804还用于:In a specific implementation, the processor 804 determines whether the target directory is a directory for storing multiple metadata by acquiring metadata describing the target directory, and adding a “directory attribute” based on the existing metadata. The value of the "directory attribute" indicates that the target directory is a different attribute, and one attribute is used to indicate that the target directory is a directory storing a plurality of metadata, and accordingly, the processor 804 analyzes the "directory attribute". It can be concluded whether the target directory is a directory that stores multiple metadata. As an optional implementation manner, after the processor 804 accesses the first target metadata, when the processor 804 accesses the exclusive access rights to all the metadata in the target directory, the processor 804 further uses to:
通过所述接收装置801接收第三客户端发送的请求消息,所述请求消息用于请求通过所述第一客户端80访问所述第一目标元数据,所述请求消息为所述第三客户端向所述目录锁服务器申请对所述目标目录中所有元数据的排它 访问权限,并接收到所述目录锁服务器返回的所述第一客户端80的节点信息后,根据所述第一客户端80的节点信息向所述第一客户端80发送的消息;Receiving, by the receiving device 801, a request message sent by a third client, where the request message is used to request access to the first target metadata by using the first client 80, where the request message is the third client Ending the exclusive use of the directory lock server for all metadata in the target directory After accessing the permission, and receiving the node information of the first client 80 returned by the directory lock server, the message sent to the first client 80 according to the node information of the first client 80;
根据所述请求消息访问所述第一目标元数据,并将访问所述第一目标元数据的访问结果通过所述发送装置802发送给所述第三客户端。And accessing the first target metadata according to the request message, and sending an access result of accessing the first target metadata to the third client by using the sending device 802.
处理器804通过执行上述操作,获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录中第一目标元数据的需求,则为第三客户端提供访问目标目录中的第一目标元数据的服务,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问第一目标元数据。After the processor 804 obtains the exclusive access permission of all the metadata in the target directory by performing the above operation, if the third client has the requirement of accessing the first target metadata in the target directory, the access target is provided for the third client. The service of the first target metadata in the directory ensures that the client that does not apply for exclusive access to all metadata in the target directory can also access the first target metadata normally.
作为一种可选的实施方式,所述当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理器804访问所述第一目标元数据之后,所述处理器804还用于:As an optional implementation manner, after the processor 804 accesses the first target metadata, the processor 804 is requested to apply exclusive access rights to all metadata in the target directory. Also used for:
在请求访问所述目标目录中的第二目标元数据时,如果第二目标元数据属于所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata other than the first target metadata in the target directory;
按照已获得的对所述第二目标元数据的排它访问权限,对所述第二目标元数据进行访问。The second target metadata is accessed in accordance with the obtained exclusive access rights to the second target metadata.
处理器804通过执行上述操作,在访问目标目录中的其他元数据(如第二目标元数据)时,可以直接对该第二目标元数据进行访问,无需再次申请访问访问权限,提高了处理器804访问目标目录中的元数据的效率。By performing the above operations, the processor 804 can directly access the second target metadata when accessing other metadata (such as the second target metadata) in the target directory, and does not need to apply for access permission again, thereby improving the processor. 804 access to metadata in the target directory is efficient.
作为一种可选的实施方式,所述处理器804访问所述第一目标元数据的方式具体为:As an optional implementation manner, the manner in which the processor 804 accesses the first target metadata is specifically:
向元数据路由服务器请求所述目标目录的归属信息,所述元数据路由服务器预存有目录的归属信息;Requesting, to the metadata routing server, attribution information of the target directory, where the metadata routing server prestores the attribution information of the directory;
通过所述接收装置801接收所述元数据路由服务器发送的所述目标目录的归属信息,所述目标目录的归属信息用于指示所述目标目录的操作权限归属到目标元数据服务器;Receiving, by the receiving device 801, the home information of the target directory that is sent by the metadata routing server, where the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
通过所述发送装置802向所述目标元数据服务器发送操作请求消息,请求所述目标元数据服务器根据所述对所述目标目录中所有元数据的排它访问权限操作所述第一目标元数据;Sending, by the sending device 802, an operation request message to the target metadata server, requesting the target metadata server to operate the first target metadata according to the exclusive access right to all metadata in the target directory. ;
通过所述接收装置801接收所述目标元数据服务器发送的操作结果;所述操作结果为所述目标元数据服务器向所述目录锁服务器申请到对所述目标目录中所有元数据的排它操作权限后,对所述第一目标元数据进行操作的结果。Receiving, by the receiving device 801, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server for exclusive operation on all metadata in the target directory. After the permission, the result of the operation of the first target metadata.
处理器804通过执行上述操作,目标元数据服务器接收到处理器804发送的访问目标目录下的第一目标元数据的请求后,在对目标目录中的第一目标元数据进行操作之前,先向目录锁服务器申请目标目录中所有元数据的排它操作权限,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性。After the processor 804 performs the above operation, after receiving the request of the first target metadata in the target directory sent by the processor 804, the target metadata server performs the operation on the first target metadata in the target directory. The directory lock server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata of the target directory is consistently saved. Sex.
作为一种可选的实施方式,所述处理器804访问所述第一目标元数据的方式具体为:As an optional implementation manner, the manner in which the processor 804 accesses the first target metadata is specifically:
通过所述发送装置802向元数据路由服务器发送切换消息,指示所述元数 据路由服务器将所述目标目录的操作权限归属到所述第一客户端80;Transmitting, by the sending device 802, a handover message to the metadata routing server, indicating the number of elements According to the routing server, the operation authority of the target directory is attributed to the first client 80;
通过所述接收装置801接收所述元数据路由服务器发送的通知消息,所述通知消息用于指示所述目标目录的操作权限归属到所述第一客户端80;Receiving, by the receiving device 801, the notification message sent by the metadata routing server, the notification message is used to indicate that the operation authority of the target directory belongs to the first client 80;
通过所述发送装置802向所述目录锁服务器发送第二申请消息申请对所述目标目录中所有元数据的排它操作权限;Sending, by the sending device 802, the second application message to the directory lock server to apply exclusive permission to all metadata in the target directory;
通过所述接收装置801接收所述目录锁服务器发送的授予所述目标目录中所有元数据的排它操作权限的授予消息,所述目标目录中所有元数据的排它操作权限用于供所述处理器804操作所述目标目录中的第一目标元数据,以及操作所述目标目录中除所述第一目标元数据之外的元数据;Receiving, by the receiving device 801, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for The processor 804 operates the first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
根据所述目标目录中所有元数据的排它操作权限对所述第一目标元数据进行操作,获得操作结果。The first target metadata is operated according to the exclusive operation authority of all the metadata in the target directory, and the operation result is obtained.
处理器804通过执行上述操作,使第一客户端80同时充当客户端和元数据服务器的功能,处理器804获取到目标目录中所有元数据的排它访问权限后,不需要与远端的元数据服务器进行交互来请求元数据服务器对第一目标元数据进行操作,减少了节点之间的开销,提高了访问第一目标元数据的访问效率The processor 804 performs the above operations to enable the first client 80 to function as both the client and the metadata server. After the processor 804 obtains exclusive access rights to all metadata in the target directory, the remote client does not need to be associated with the remote terminal. The data server interacts to request the metadata server to operate on the first target metadata, reducing the overhead between the nodes, and improving the access efficiency of accessing the first target metadata.
作为一种可选的实施方式,当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理器804访问所述第一目标元数据之后,所述处理器804还用于:As an optional implementation manner, after the processor 804 accesses the first target metadata, when the processor 804 accesses the exclusive access rights to all the metadata in the target directory, the processor 804 further uses to:
判断访问所述目标目录中的元数据所产生的负载是否满足预设条件;Determining whether the load generated by accessing the metadata in the target directory satisfies a preset condition;
若满足,则释放对所述目标目录中的所有元数据的排它访问权限,以便所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予第四客户端。If so, exclusive access to all metadata in the target directory is released, such that the directory lock server grants exclusive access to all metadata in the target directory to the fourth client.
具体实现中,处理器804会实时对访问目标目录中的元数据所产生的负载进行检测,检测的方式包括但不限于以下列举的两种情况:In a specific implementation, the processor 804 detects the load generated by accessing the metadata in the target directory in real time, and the detection manner includes but is not limited to the following two cases:
第一种情况,处理器804实时检测第一客户端80运行时的整体负载,并将该整体负载的负载值与预设阈值进行比较,判断第一客户端80的整体负载是否超过了预设阈值,该预设阈值为根据实际需要设置的一个用来参考对比的负载值。In the first case, the processor 804 detects the overall load of the first client 80 in real time, and compares the load value of the overall load with a preset threshold to determine whether the overall load of the first client 80 exceeds a preset. Threshold value, which is a load value used for reference comparison according to actual needs.
第二种情况,所述处理器804记录第一客户端80主动访问所述目标目录下的元数据所产生的第一负载,以及根据其他客户端的请求消息访问所述目标目录中的元数据所产生的第二负载;所述第一客户端80进一步判断所述第一负载与所述第二负载的相对大小是否满足预设条件,例如,第一负载小于第二负载。In the second case, the processor 804 records the first load generated by the first client 80 actively accessing the metadata in the target directory, and accesses the metadata in the target directory according to the request message of other clients. The generated second load; the first client 80 further determines whether the relative size of the first load and the second load meets a preset condition, for example, the first load is smaller than the second load.
处理器804执行上述操作的具体过程还可以参照图3~7对应的方发实施例的相应描述,此处不再赘述。For a specific process of the above-mentioned operations, the processor 804 can also refer to the corresponding description of the embodiment of the embodiment of the present invention, and details are not described herein again.
处理器804通过执行上述操作,在第一客户端80上的负载过高时,自动释放目标目录中所有元数据的排它访问权限,让负载较低的第四客户端来获取该目标目录中所有元数据的排它访问权限以达到分担负载的目的;或者在第一客户端访问目标目录中的元数据的频率比其他客户端访问目标目录中的元数 据的频率低时,自动释放目标目录中所有元数据的排它访问权限,让访问频率较高的客户端获取目标目录中所有元数据的排它访问权限,以减少转发访问所产生的负载,提高了访问目标目录中元数据的效率。The processor 804 automatically releases the exclusive access rights of all the metadata in the target directory when the load on the first client 80 is too high, and allows the fourth client with a lower load to obtain the target directory. Exclusive access to all metadata for load sharing purposes; or access to metadata in the target directory at the first client is more frequent than other clients accessing the target directory When the frequency is low, the exclusive access permission of all metadata in the target directory is automatically released, so that the client with higher access frequency obtains exclusive access rights of all metadata in the target directory, so as to reduce the load generated by forwarding access. Improves the efficiency of accessing metadata in the target directory.
请参见图9,图9是本发明实施例提供的又一种终端90的结构示意图,该终端90为以下描述的目录锁服务器90,该终端90可以包括接收装置901、发送装置902,存储器903和处理器904(处理器可以为一个也可以为多个,图9中以一个处理器为例),接收装置901、发送装置902,存储器903和处理器904分别连接总线905,其中,存储器903中存储一组元数据访问程序代码,且处理器904调用存储器903中存储的元数据访问程序代码,用于执行以下操作:Referring to FIG. 9, FIG. 9 is a schematic structural diagram of still another terminal 90 according to an embodiment of the present invention. The terminal 90 is a directory lock server 90, which may include a receiving device 901, a sending device 902, and a memory 903. And the processor 904 (the processor may be one or more, and one processor in FIG. 9 is taken as an example), the receiving device 901, the transmitting device 902, the memory 903 and the processor 904 are respectively connected to the bus 905, wherein the memory 903 A set of metadata access program code is stored, and the processor 904 calls the metadata access program code stored in the memory 903 for performing the following operations:
在第一客户端访问目标目录中的目标元数据时,通过所述接收装置901接收所述第一客户端发送的第一申请消息,所述第一申请消息用于申请对所述目标目录中所有元数据的排它访问权限,所述目标目录为存储有多个元数据的目录,所述目标元数据为所述第一客户端当前需要访问的元数据;Receiving, by the receiving device 901, a first application message sent by the first client, where the first client accesses the target metadata in the target directory, where the first application message is used to apply to the target directory. Exclusive access rights of all metadata, the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
根据所述第一申请消息判断所述目标目录中所有元数据的排它访问权限是否已经授予了第二客户端,所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的目标元数据,以及访问所述目标目录中除所述目标元数据之外的元数据;Determining, according to the first application message, whether an exclusive access right of all metadata in the target directory has been granted to the second client, and exclusive access rights of all metadata in the target directory are used for the first Client accessing target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
若未授予所述第二客户端,则将所述目标目录中所有元数据的排它访问权限授予所述第一客户端;若授予了所述第二客户端,则通过所述发送装置902向所述第一客户端发送所述第二客户端所在的节点信息,以使所述第一客户端基于所述第二客户端所在的节点信息通过所述第二客户端访问所述目标元数据。If the second client is not granted, grant exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, pass the sending device 902 Sending the node information of the second client to the first client, so that the first client accesses the target element by using the second client based on the node information of the second client data.
在具体实现中,目录锁服务器能够提供分布式锁服务,如搭载zookeeper服务的服务器;处理器904能够基于该分布式锁服务,通过授予访问目标目录的锁来协调各个客户端对目标目录的访问权限,如果处理器904将访问目标目录的锁授予第一客户端,那么第一客户端就具备了访问目标目录下所有元数据的权限;进一步地,分布式锁服务可以设置为可抢占分布式锁和不可抢占分布式锁;若设置为可抢占的分布式锁,第一客户端申请访问目标目录的锁时,如果访问目标目录的锁已经授予了其他客户端,那么处理器904会根据抢占机制释放访问目标目录的锁,并重新将访问目标目录的锁授予该第一客户端,如果访问目标目录的锁未授予其他客户端,那么处理器904则直接将访问目标目录的锁授予第一客户端;若设置为不可抢占的分布式锁,第一客户端申请访问目标目录的锁时,如果访问目标目录的锁已经授予了其他客户端,那么处理器904会根据不可抢占的机制,不会将访问目标目录的锁授予第一客户端,如果访问目标目录的锁未授予其他客户端,那么处理器904则直接将访问目标目录的锁授予第一客户端。在本发明实施例中,访问目标目录的锁可以设置为不可抢占的分布式锁,这样一来,第一客户端申请到访问目标目录的锁后,就具备了对目标目录中所有元数据的排它访问权限,即除了第一客户端之外,其他的客户 端没有访问该目标目录中元数据的权限,除非该第一客户端主动释放访问目标目录的锁。In a specific implementation, the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the processor 904 can coordinate the access of each client to the target directory by granting a lock accessing the target directory based on the distributed lock service. Permission, if the processor 904 grants the lock accessing the target directory to the first client, the first client has the right to access all the metadata in the target directory; further, the distributed lock service can be set to preemptively distribute Lock and non-preemptive distributed lock; if set to preemptible distributed lock, when the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the processor 904 will preempt according to the preemption The mechanism releases the lock accessing the target directory, and re-grants the lock accessing the target directory to the first client. If the lock accessing the target directory is not granted to other clients, the processor 904 directly grants the lock accessing the target directory to the first. Client; if set to a non-preemptable distributed lock, the first client requests access to the target When the lock is recorded, if the lock accessing the target directory has been granted to other clients, the processor 904 will not grant the lock to the target client to the first client according to the non-preemptive mechanism, if the lock of the target directory is not accessed. Granting to other clients, the processor 904 directly grants the lock accessing the target directory to the first client. In the embodiment of the present invention, the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the first client applies for the lock to access the target directory, it has all the metadata in the target directory. Exclusive access rights, that is, other customers besides the first client The terminal does not have access to the metadata in the target directory unless the first client actively releases the lock accessing the target directory.
处理器904通过执行上述操作,在第一客户端访问目标目录中的目标元数据时,处理器904一次性将目标目录中所有元数据的排它访问权全部授予第一客户端以维护缓存一致性,这样一来,第一客户端就可以顺利访问目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。The processor 904 performs the above operation, when the first client accesses the target metadata in the target directory, the processor 904 grants all the exclusive access rights of all the metadata in the target directory to the first client at one time to maintain the cache consistency. Sex, in this way, the first client can successfully access the target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
作为一种可选的方案,若未授予所述第二客户端,所述处理器904将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器904还用于:As an optional solution, if the second client is not granted, the processor 904 grants exclusive access rights of all metadata in the target directory to the first client, the processor 904 is also used to:
在第三客户端访问所述目标元数据时,通过接收装置901接收所述第三客户端发送的所述第一申请消息;Receiving, by the receiving device 901, the first application message sent by the third client, when the third client accesses the target metadata;
通过所述发送装置902向所述第三客户端发送所述第一客户端所在的节点信息,以使所述第三客户端基于所述第一客户端所在的节点信息通过所述第一客户端访问所述目标元数据。Sending, by the sending device 902, the node information where the first client is located to the third client, so that the third client passes the first client based on node information where the first client is located. The end accesses the target metadata.
处理器904通过执行上述操作,第一客户端获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录目标元数据的需求,则第三客户端可以基于处理器904发送的节点信息将访问请求发送至第一客户端,通过第一客户端访问该目标元数据,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问目标元数据。After the processor 904 performs the above operation, after the first client obtains the exclusive access right of all the metadata in the target directory, if the third client has the requirement to access the target directory target metadata, the third client may be based on the processing. The node information sent by the device 904 sends the access request to the first client, and accesses the target metadata through the first client, so that the client that does not apply for exclusive access rights of all metadata in the target directory can also be normal. Access target metadata.
作为一种可选的方案,若未授予所述第二客户端,所述处理器904将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器904还用于:As an optional solution, if the second client is not granted, the processor 904 grants exclusive access rights of all metadata in the target directory to the first client, the processor 904 is also used to:
通过所述接收装置901接收目标元数据服务器发送的第二申请消息,所述第二申请消息为所述目标元数据服务器接收到所述第一客户端访问所述目标元数据时发送的操作消息时发送的,用于申请对所述目标目录中所有元数据的排它操作权限的消息;Receiving, by the receiving device 901, a second application message sent by the target metadata server, where the second application message is an operation message sent by the target metadata server when the first client accesses the target metadata Message sent at the time to apply for exclusive operation rights to all metadata in the target directory;
根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述目标元数据服务器,以使所述目标元数据服务器对所述目标元数据进行操作并将操作结果反馈给所述第一客户端,其中,所述目标目录中所有元数据的排它操作权限用于供所述目标元数据服务器操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。Granting exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server operates on the target metadata and feedbacks operation results Giving the first client, wherein exclusive operation rights of all metadata in the target directory are used by the target metadata server to operate target metadata in the target directory, and operating the target directory Metadata other than the target metadata.
处理器904通过执行上述操作,目标元数据服务器接收到第一客户端发送的访问目标目录下的目标元数据的请求后,在对目标目录中的目标元数据进行操作之前,先向处理器904申请目标目录中所有元数据的排它操作权限,相应地,处理器904将目标目录中所有元数据的排它操作权限授予目标元数据服务器,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性。After the processor 904 performs the above operation, the target metadata server receives the request for the target metadata in the target directory sent by the first client, and then proceeds to the processor 904 before operating the target metadata in the target directory. Approve exclusive access rights for all metadata in the target directory. Accordingly, the processor 904 grants the exclusive operation authority of all metadata in the target directory to the target metadata server, so that there is always only one metadata server in all the metadata servers. It has the operation permission on the target directory, which ensures the persistence and preservation consistency of the metadata in the target directory.
作为一种可选的方案,若未授予所述第二客户端,所述处理器904将所述 目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器904还用于:As an alternative, if the second client is not granted, the processor 904 will After the exclusive access rights of all the metadata in the target directory are granted to the first client, the processor 904 is further configured to:
通过所述接收装置901接收所述第一客户端发送的第二申请消息,所述第二申请消息用于申请对所述目标目录中所有元数据的排它操作权限;Receiving, by the receiving device 901, a second application message sent by the first client, where the second application message is used to apply for exclusive operation permission on all metadata in the target directory;
根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述第一客户端,以使所述第一客户端对所述目标元数据进行操作。Granting exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client operates on the target metadata.
处理器904执行上述操作的具体过程还可以参照图3~7对应的方发实施例的相应描述,此处不再赘述。The specific process of the processor 904 for performing the above operations may also refer to the corresponding description of the embodiment of the embodiment of the present invention, and details are not described herein again.
处理器904通过执行上述操作,在第一客户端同时充当客户端和元数据服务器的功能的前提下,处理器904授予第一客户端目标目录中所有元数据的排它访问权限后,第一客户端不需要与远端的元数据服务器进行交互来请求元数据服务器对目标元数据进行操作,减少了节点之间的开销,提高了访问目标元数据的访问效率。The processor 904 performs the above operation, and after the first client simultaneously functions as the client and the metadata server, the processor 904 grants exclusive access rights to all metadata in the target directory of the first client, first The client does not need to interact with the remote metadata server to request the metadata server to operate on the target metadata, which reduces the overhead between the nodes and improves the access efficiency of the access target metadata.
请参见图10,图10是本发明实施例提供的一种分布式文件系统100的结构示意图,该分布式文件系统100包括第一客户端83、目录锁服务器84和第二客户端85,其中:Referring to FIG. 10, FIG. 10 is a schematic structural diagram of a distributed file system 100 according to an embodiment of the present invention. The distributed file system 100 includes a first client 83, a directory lock server 84, and a second client 85. :
第一客户端83在请求访问目标目录中的第一目标元数据时向目录锁服务器84发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述第一客户端当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;The first client 83 sends a first application message to the directory lock server 84 to request exclusive access rights to all metadata in the target directory when requesting access to the first target metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the The first client accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
所述目录锁服务器84接收所述第一客户端83发送的第一申请消息,并判断所述目标目录中所有元数据的排它访问权限是否授予了第二客户端85;The directory lock server 84 receives the first application message sent by the first client 83, and determines whether the exclusive access rights of all metadata in the target directory are granted to the second client 85;
若所述目标目录中所有元数据的排它访问权限未授予所述第二客户端85,则所述目录锁服务器84将所述目标目录中所有元数据的排它访问权限授予所述第一客户端83,并向所述第一客户端发送通知消息以通知所述目标目录中所有元数据的排它访问权限授予所述第一客户端;If the exclusive access rights of all the metadata in the target directory are not granted to the second client 85, the directory lock server 84 grants the exclusive access rights of all metadata in the target directory to the first a client 83, and sending a notification message to the first client to notify the first client of exclusive access rights of all metadata in the target directory;
所述第一客户端83接收所述通知消息,并根据所述通知消息访问所述第一目标元数据。The first client 83 receives the notification message and accesses the first target metadata according to the notification message.
进一步地,分布式文件系统100中的第一客户端83可以参照上述图8所示的实施例描述的终端80,目录锁服务器84可以参照图9所示实施例中描述的终端90,此处均不再赘述。Further, the first client 83 in the distributed file system 100 can refer to the terminal 80 described in the embodiment shown in FIG. 8, and the directory lock server 84 can refer to the terminal 90 described in the embodiment shown in FIG. No longer repeat them.
通过运行该分布式文件系统100,第一客户端83访问目标目录中的第一目标元数据时,一次性向目录锁服务器84申请目标目录中的所有元数据的排它访问权限,这样一来,第一客户端83就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。 When the first client 83 accesses the first target metadata in the target directory by running the distributed file system 100, the directory lock server 84 is applied for exclusive access rights of all metadata in the target directory at one time, so that The first client 83 can successfully access the first target metadata, and does not need to re-apply for access rights when accessing other metadata in the target directory, which saves the overhead of maintaining cache consistency.
请参见图11,图11是本发明实施例提供的又一种终端110的结构示意图;该终端110为以下描述的第一客户端110,该终端110包括申请单元1101和第一访问单元1103,各个单元的描述如下:Referring to FIG. 11, FIG. 11 is a schematic structural diagram of still another terminal 110 according to an embodiment of the present invention. The terminal 110 is a first client 110, which includes an application unit 1101 and a first access unit 1103. The description of each unit is as follows:
申请单元1101用于在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述第一客户端当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;The application unit 1101 is configured to, when requesting access to the first target metadata in the target directory, send a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the The first client accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
第一访问单元1103用于在所述申请单元1101申请到对所述目标目录中所有元数据的排它访问权限时,访问所述第一目标元数据。The first access unit 1103 is configured to access the first target metadata when the application unit 1101 applies for exclusive access rights to all metadata in the target directory.
作为一种可选的方案,所述第一访问单元1103还用于在所述申请单元1101未申请到对所述目标目录中所有元数据的排它访问权限时,接收所述目录锁服务器发送的第二客户端所在的节点信息,并根据所述节点信息向所述第二客户端发送请求消息以通过所述第二客户端访问所述目标目录中的元数据,所述第二客户端为向所述目录锁服务器申请到了对所述目标目录中所有元数据的排它访问权限的客户端。As an optional solution, the first access unit 1103 is further configured to: when the application unit 1101 does not apply for exclusive access to all metadata in the target directory, receive the directory lock server to send Node information of the second client, and sending a request message to the second client according to the node information to access metadata in the target directory by the second client, the second client A client that requests exclusive access to all metadata in the target directory to the directory lock server.
作为一种可选的方案,所述申请单元1101包括获取子单元、判断子单元和申请子单元,其中:As an optional solution, the application unit 1101 includes an acquisition subunit, a determination subunit, and an application subunit, where:
获取子单元,用于获取描述所述目标目录的元数据,所述描述所述目标目录的元数据设有用于标识所述目标目录是否为存储有多个元数据的目录的标识信息;Obtaining a subunit, configured to obtain metadata describing the target directory, where the metadata describing the target directory is provided with identifier information for identifying whether the target directory is a directory in which a plurality of metadata is stored;
判断子单元,用于根据所述描述所述目标目录的元数据来判断所述目标目录是否为存储有多个元数据的目录;a determining subunit, configured to determine, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
申请子单元,用于在判断子单元的判断结果为是时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限。The application subunit is configured to send a first application message to the directory lock server to request exclusive access rights to all metadata in the target directory when the judgment result of the subunit is YES.
作为一种可选的方案,所述终端110还包括:As an optional solution, the terminal 110 further includes:
接收单元,用于接收第三客户端发送的请求消息,所述请求消息用于请求通过所述第一访问单元1103访问所述第一目标元数据,所述请求消息为所述第三客户端向所述目录锁服务器申请对所述目标目录中所有元数据的排它访问权限,并接收到所述目录锁服务器返回的所述第一客户端的节点信息后,根据所述第一客户端的节点信息向所述第一客户端发送的消息;a receiving unit, configured to receive a request message sent by the third client, where the request message is used to request to access the first target metadata by using the first access unit 1103, where the request message is the third client Applying, to the directory lock server, exclusive access rights to all metadata in the target directory, and receiving node information of the first client returned by the directory lock server, according to the node of the first client a message sent by the information to the first client;
发送单元,用于根据所述请求消息访问所述第一目标元数据,并将访问所述第一目标元数据的访问结果发送给所述第三客户端。And a sending unit, configured to access the first target metadata according to the request message, and send an access result of accessing the first target metadata to the third client.
作为一种可选的方案,所述终端110还包括:As an optional solution, the terminal 110 further includes:
第二访问单元,用于在请求访问所述目标目录中的第二目标元数据时,如果第二目标元数据属于所述目标目录中除所述第一目标元数据之外的元数据;a second access unit, configured to: when requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata in the target directory other than the first target metadata;
则按照已获得的对所述第二目标元数据的排它访问权限,对所述第二目标 元数据进行访问。And then according to the obtained exclusive access right to the second target metadata, to the second target Metadata is accessed.
作为一种可选的方案,所述第一访问单元1103包括:As an optional solution, the first access unit 1103 includes:
请求子单元,用于向元数据路由服务器请求所述目标目录的归属信息,所述元数据路由服务器预存有目录的归属信息;a requesting subunit, configured to request, by the metadata routing server, attribution information of the target directory, where the metadata routing server prestores the attribution information of the directory;
第一接收子单元,用于接收所述元数据路由服务器发送的所述目标目录的归属信息,所述目标目录的归属信息用于指示所述目标目录的操作权限归属到目标元数据服务器;a first receiving subunit, configured to receive the home information of the target directory sent by the metadata routing server, where the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
第一发送子单元,用于向所述目标元数据服务器发送操作请求消息,请求所述目标元数据服务器根据所述对所述目标目录中所有元数据的排它访问权限操作所述第一目标元数据;a first sending subunit, configured to send an operation request message to the target metadata server, requesting the target metadata server to operate the first target according to the exclusive access right to all metadata in the target directory Metadata
第二接收子单元,用于接收所述目标元数据服务器发送的操作结果;所述操作结果为所述目标元数据服务器向所述目录锁服务器申请到对所述目标目录中所有元数据的排它操作权限后,对所述第一目标元数据进行操作的结果。a second receiving subunit, configured to receive an operation result sent by the target metadata server; the operation result is that the target metadata server requests the directory lock server to queue all metadata in the target directory The result of the operation of the first target metadata after it operates the authority.
作为一种可选的方案,所述第一访问单元1103包括:As an optional solution, the first access unit 1103 includes:
第二发送子单元,用于向元数据路由服务器发送切换消息,指示所述元数据路由服务器将所述目标目录的操作权限归属到所述第一客户端;a second sending subunit, configured to send a switching message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client;
第三接收子单元,用于接收所述元数据路由服务器发送的通知消息,所述通知消息用于指示所述目标目录的操作权限归属到所述第一客户端;a third receiving subunit, configured to receive a notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client;
申请子单元,用于向所述目录锁服务器发送第二申请消息以申请对所述目标目录中所有元数据的排它操作权限;An application subunit, configured to send a second application message to the directory lock server to apply for exclusive operation authority on all metadata in the target directory;
第四接收子单元,用于接收所述目录锁服务器发送的授予所述目标目录中所有元数据的排它操作权限的授予消息,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的第一目标元数据,以及操作所述目标目录中除所述第一目标元数据之外的元数据;a fourth receiving subunit, configured to receive a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for The first client operates first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
操作子单元,用于根据所述目标目录中所有元数据的排它操作权限对所述第一目标元数据进行操作,获得操作结果。An operation subunit, configured to operate the first target metadata according to an exclusive operation authority of all metadata in the target directory, to obtain an operation result.
作为一种可选的方案,所述终端110还包括:As an optional solution, the terminal 110 further includes:
判断单元,用于判断访问所述目标目录中的元数据所产生的负载是否满足预设条件;a determining unit, configured to determine whether a load generated by accessing metadata in the target directory meets a preset condition;
释放单元,用于在所述判断单元的判断结果为是时,端释放所述目标目录中的所有元数据的排它访问权限,以便所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予第四客户端。a release unit, configured to release exclusive access rights of all metadata in the target directory when the determination result of the determining unit is yes, so that the directory lock server will all metadata in the target directory Exclusive access is granted to the fourth client.
终端(第一客户端)110访问元数据的原理还可以参照图3~7对应的方发实施例的相应描述,此处不再赘述。For the principle of the terminal (the first client) 110 accessing the metadata, reference may be made to the corresponding description of the corresponding embodiment of the embodiment of the present invention, and details are not described herein again.
在图11所示的终端110中,第一客户端(终端)110访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中的所有元数据的排它访问权限这样一来,第一客户端就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。 In the terminal 110 shown in FIG. 11, when the first client (terminal) 110 accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time. The first client can successfully access the first target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
请参见图12,图12是本发明实施例提供的又一种终端120,该终端120为以下描述的目录锁服务器120,该终端120包括第一接收单元1201、判断单元1202和第一授予单元1203,其中,各个单元的描述如下。Referring to FIG. 12, FIG. 12 is still another terminal 120 according to an embodiment of the present invention. The terminal 120 is a directory lock server 120, which includes a first receiving unit 1201, a determining unit 1202, and a first granting unit. 1203, wherein each unit is described as follows.
第一接收单元1201用于在第一客户端访问目标目录中的目标元数据时,接收所述第一客户端发送的第一申请消息,所述第一申请消息用于申请对所述目标目录中所有元数据的排它访问权限,所述目标目录为存储有多个元数据的目录,所述目标元数据为所述第一客户端当前需要访问的元数据;The first receiving unit 1201 is configured to receive, when the first client accesses the target metadata in the target directory, the first application message sent by the first client, where the first application message is used to apply for the target directory. Exclusive access rights of all the metadata in the directory, the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
判断单元1202用于根据所述第一申请消息判断所述目标目录中所有元数据的排它访问权限是否已经授予了第二客户端,所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的目标元数据,以及访问所述目标目录中除所述目标元数据之外的元数据;The determining unit 1202 is configured to determine, according to the first application message, whether an exclusive access right of all metadata in the target directory has been granted to the second client, and exclusive access rights of all metadata in the target directory are used for Providing, by the first client, target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
第一授予单元1203用于在所述判断单元1202的判断结果为否时,将所述目标目录中所有元数据的排它访问权限授予所述第一客户端;在所述判断单元1202的判断结果为是时,向所述第一客户端发送所述第二客户端所在的节点信息,以使所述第一客户端基于所述第二客户端所在的节点信息通过所述第二客户端访问所述目标元数据。The first granting unit 1203 is configured to, when the determining result of the determining unit 1202 is negative, grant exclusive access rights of all metadata in the target directory to the first client; in the judgment of the determining unit 1202 When the result is YES, the node information of the second client is sent to the first client, so that the first client passes the second client based on the node information where the second client is located. Access the target metadata.
作为一种可选的方案,所述终端120还包括:As an optional solution, the terminal 120 further includes:
第二接收单元,用于在第三客户端访问所述目标元数据时,接收所述第三客户端发送的所述第一申请消息;a second receiving unit, configured to receive the first application message sent by the third client when the third client accesses the target metadata;
第一发送单元,用于向所述第三客户端发送所述第一客户端所在的节点信息,以使所述第三客户端基于所述第一客户端所在的节点信息通过所述第一客户端访问所述目标元数据。a first sending unit, configured to send, to the third client, node information where the first client is located, so that the third client passes the first node based on node information where the first client is located The client accesses the target metadata.
作为一种可选的方案,所述终端120还包括:As an optional solution, the terminal 120 further includes:
第三接收单元,用于接收目标元数据服务器发送的第二申请消息,所述第二申请消息为所述目标元数据服务器接收到所述第一客户端访问所述目标元数据时发送的操作消息时发送的,用于申请对所述目标目录中所有元数据的排它操作权限的消息;a third receiving unit, configured to receive a second application message sent by the target metadata server, where the second application message is an operation sent by the target metadata server when the first client accesses the target metadata a message sent during the message to apply for exclusive operation rights to all metadata in the target directory;
第二授予单元,用于根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述目标元数据服务器,以使所述目标元数据服务器对所述目标元数据进行操作并将操作结果反馈给所述第一客户端,其中,所述目标目录中所有元数据的排它操作权限用于供所述目标元数据服务器操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。a second granting unit, configured to grant an exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server pairs the target metadata Performing an operation and feeding back an operation result to the first client, wherein an exclusive operation authority of all metadata in the target directory is used for the target metadata server to operate target metadata in the target directory, And operating metadata in the target directory other than the target metadata.
作为一种可选的方案,所述第一授予单元1203在所述判断单元1202的判断结果为否时,将所述目标目录中所有元数据的排它访问权限授予所述第一客户端以使所述第一客户端访问所述目标目录中的元数据之后,所述终端120还包括:As an optional solution, the first granting unit 1203, when the determining result of the determining unit 1202 is negative, grant exclusive access rights of all metadata in the target directory to the first client. After the first client accesses the metadata in the target directory, the terminal 120 further includes:
第四接收单元,用于接收所述第一客户端发送的第二申请消息,所述第二 申请消息用于申请对所述目标目录中所有元数据的排它操作权限;a fourth receiving unit, configured to receive a second application message sent by the first client, where the second The application message is used to apply for exclusive operation rights on all metadata in the target directory;
第三授予单元,用于根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述第一客户端,以使所述第一客户端对所述目标元数据进行操作,其中,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。a third granting unit, configured to grant an exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client pairs the target metadata Performing operations, wherein exclusive operation rights of all metadata in the target directory are used by the first client to operate target metadata in the target directory, and operating the target directory except the target element Metadata outside of the data.
终端(目录锁服务器)120访问元数据的原理还可以参照图3~7对应的方发实施例的相应描述,此处不再赘述。For the principle of accessing the metadata of the terminal (the directory lock server) 120, reference may be made to the corresponding description of the embodiment of the present invention corresponding to FIG. 3 to FIG. 7 , and details are not described herein again.
在图12所示的终端120中,第一客户端访问目标目录中的目标元数据时,一次性向目录锁服务器申请目标目录中的所有元数据的访问权限,这样一来,第一客户端就可以顺利访问目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。In the terminal 120 shown in FIG. 12, when the first client accesses the target metadata in the target directory, the directory lock server is requested to access all the metadata in the target directory at a time, so that the first client The target metadata can be successfully accessed, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
综上所述,通过实施本发明实施例,第一客户端访问目标目录中的第一目标元数据时,一次性向目录锁服务器申请目标目录中的所有元数据的排它访问权限,这样一来,第一客户端就可以顺利访问第一目标元数据,并且后续访问目标目录中其他元数据时不需要再重新申请访问权限,节省了维护缓存一致性的开销。In summary, by implementing the embodiment of the present invention, when the first client accesses the first target metadata in the target directory, the first time accesses the directory lock server to the exclusive access permission of all the metadata in the target directory, so that the first client accesses the first target metadata in the target directory. The first client can successfully access the first target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
进一步地,第一客户端获取到目标目录中所有元数据的排它访问权限后,如果第三客户端有访问目标目录中第一目标元数据的需求,则第三客户端将访问请求发送至第一客户端,通过第一客户端访问该第一目标元数据,保证了申请不到目标目录中所有元数据的排它访问权限的客户端也都能正常地访问第一目标元数据;目标元数据服务器接收到第一客户端发送的访问目标目录下的第一目标元数据的请求后,在对第一目标元数据进行操作之前,先向目录锁服务器申请目标目录中所有元数据的排它操作权限,使得所有的元数据服务器中始终只有一个元数据服务器具备对目标目录的操作权限,保证了目标目录下的元数据的持久化保存一致性;进一步地,第一客户端同时充当客户端和元数据服务器的功能,第一客户端获取到目标目录中所有元数据的排它访问权限后,不需要与远端的元数据服务器进行交互来请求元数据服务器对第一目标元数据进行操作,减少了节点之间的开销,提高了访问第一目标元数据的访问效率;再进一步地,第一客户端上的负载过高时,自动释放目标目录中所有元数据的排它访问权限,让负载较低的第四客户端来获取该目标目录中所有元数据的排它访问权限以达到分担负载的目的;或者在第一客户端访问目标目录中的元数据的频率比其他客户端访问目标目录中的元数据的频率低时,自动释放目标目录中所有元数据的排它访问权限,让访问频率较高的客户端获取目标目录中所有元数据的排它访问权限,以减少转发访问所产生的负载,提高了访问目标目录中元数据的效率。Further, after the first client obtains the exclusive access right of all the metadata in the target directory, if the third client has the requirement to access the first target metadata in the target directory, the third client sends the access request to the The first client accesses the first target metadata through the first client, so that the client that does not apply for exclusive access rights of all metadata in the target directory can also normally access the first target metadata; After receiving the request for the first target metadata in the access target directory sent by the first client, the metadata server first applies to the directory lock server for all metadata rows in the target directory before performing the operation on the first target metadata. It has the right to operate, so that only one metadata server in all metadata servers has the operation authority to the target directory, which ensures the persistence and preservation consistency of the metadata in the target directory; further, the first client also acts as the client at the same time. End and metadata server functions, the first client gets exclusive access to all metadata in the target directory After the limitation, there is no need to interact with the remote metadata server to request the metadata server to operate on the first target metadata, thereby reducing the overhead between the nodes and improving the access efficiency of accessing the first target metadata; further Ground, when the load on the first client is too high, the exclusive access rights of all metadata in the target directory are automatically released, and the fourth client with lower load is obtained to obtain exclusive access rights of all metadata in the target directory. To achieve the purpose of sharing the load; or to automatically release the exclusive access of all metadata in the target directory when the first client accesses the metadata in the target directory less frequently than the metadata in other clients accessing the target directory. Permissions allow clients with higher frequency of access to obtain exclusive access to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机程序来指令相关的硬件来完成,所述的程序可存储于计算机 可读取存储介质中,该程序在执行时,可包括如上述各方法的实施例的流程。而前述的存储介质包括:ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。One of ordinary skill in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer. The readable storage medium, when executed, may include the flow of an embodiment of the methods as described above. The foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.
尽管结合具体特征及其实施例对本发明进行了描述,显而易见的,在不脱离本发明的精神和范围的情况下,可以对其进行修改和各种组合。相应地,本说明书和附图仅仅是所附权利要求书所界定的本发明的示例性说明,且视为已覆盖本发明范围内的任意和所有修改、变化、组合或等同物。 While the invention has been described in connection with the specific embodiments and embodiments thereof Accordingly, the description and drawings are to be regarded as

Claims (26)

  1. 一种元数据访问方法,其特征在于,包括:A metadata access method, comprising:
    第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述第一客户端当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target directory For storing a directory having a plurality of metadata, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the first a client accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
    当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据。The first client accesses the first target metadata when the first client applies for exclusive access to all metadata in the target directory.
  2. 根据权利要求1所述的方法,其特征在于,当所述第一客户端未申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端接收所述目录锁服务器发送的第二客户端所在的节点信息,并根据所述节点信息向所述第二客户端发送请求消息以通过所述第二客户端访问所述第一目标元数据,所述第二客户端为向所述目录锁服务器申请到了对所述目标目录中所有元数据的排它访问权限的客户端。The method according to claim 1, wherein said first client receives said directory lock when said first client does not apply for exclusive access to all metadata in said target directory Sending, by the server, the node information of the second client, and sending a request message to the second client according to the node information, to access the first target metadata by using the second client, the second client The client is a client that has applied to the directory lock server for exclusive access to all metadata in the target directory.
  3. 根据权利要求1或2所述的方法,其特征在于,所述第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限包括:The method according to claim 1 or 2, wherein the first client sends a first application message to the directory lock server to apply for the target when requesting access to the first target metadata in the target directory Exclusive access to all metadata in the directory includes:
    第一客户端在请求访问目标目录中的第一目标元数据时,获取描述所述目标目录的元数据,所述描述所述目标目录的元数据设有用于标识所述目标目录是否为存储有多个元数据的目录的标识信息;The first client obtains metadata describing the target directory when requesting access to the first target metadata in the target directory, and the metadata describing the target directory is configured to identify whether the target directory is stored. Identification information of a directory of multiple metadata;
    根据所述描述所述目标目录的元数据来判断所述目标目录是否为存储有多个元数据的目录;Determining, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
    若是,则向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限。If so, a first application message is sent to the directory lock server to request exclusive access to all metadata in the target directory.
  4. 根据权利要求1~3任一项所述的方法,其特征在于,所述当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据之后,所述方法还包括:The method according to any one of claims 1 to 3, wherein the first client when the first client applies exclusive access rights to all metadata in the target directory After the terminal accesses the first target metadata, the method further includes:
    所述第一客户端接收第三客户端发送的请求消息,所述请求消息用于请求通过所述第一客户端访问所述第一目标元数据,所述请求消息为所述第三客户端向所述目录锁服务器申请对所述目标目录中所有元数据的排它访问权限,并接收到所述目录锁服务器返回的所述第一客户端的节点信息后,根据所述第一客户端的节点信息向所述第一客户端发送的消息;Receiving, by the first client, a request message sent by a third client, where the request message is used to request access to the first target metadata by using the first client, where the request message is the third client Applying, to the directory lock server, exclusive access rights to all metadata in the target directory, and receiving node information of the first client returned by the directory lock server, according to the node of the first client a message sent by the information to the first client;
    所述第一客户端根据所述请求消息访问所述第一目标元数据,并将访问所述第一目标元数据的访问结果发送给所述第三客户端。The first client accesses the first target metadata according to the request message, and sends an access result of accessing the first target metadata to the third client.
  5. 根据权利要求1~4任一项所述的方法,其特征在于,所述当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第 一客户端访问所述第一目标元数据之后,所述方法还包括:The method according to any one of claims 1 to 4, wherein when the first client applies for exclusive access to all metadata in the target directory, the After a client accesses the first target metadata, the method further includes:
    所述第一客户端在请求访问所述目标目录中的第二目标元数据时,如果第二目标元数据属于所述目标目录中除所述第一目标元数据之外的元数据;The first client, when requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata in the target directory other than the first target metadata;
    则所述第一客户端按照已获得的对所述第二目标元数据的排它访问权限,对所述第二目标元数据进行访问。And then the first client accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
  6. 根据权利要求1~5任一项所述的方法,其特征在于,所述第一客户端访问所述第一目标元数据包括:The method according to any one of claims 1 to 5, wherein the accessing the first target metadata by the first client comprises:
    所述第一客户端向元数据路由服务器请求所述目标目录的归属信息,所述元数据路由服务器预存有目录的归属信息;Determining, by the first client, the home information of the target directory to the metadata routing server, where the metadata routing server pre-stores the home information of the directory;
    所述第一客户端接收所述元数据路由服务器发送的所述目标目录的归属信息,所述目标目录的归属信息用于指示所述目标目录的操作权限归属到目标元数据服务器;The first client receives the home information of the target directory sent by the metadata routing server, and the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
    所述第一客户端向所述目标元数据服务器发送操作请求消息,请求所述目标元数据服务器根据所述对所述目标目录中所有元数据的排它访问权限操作所述第一目标元数据;The first client sends an operation request message to the target metadata server, requesting the target metadata server to operate the first target metadata according to the exclusive access right to all metadata in the target directory. ;
    所述第一客户端接收所述目标元数据服务器发送的操作结果;所述操作结果为所述目标元数据服务器向所述目录锁服务器申请到对所述目标目录中所有元数据的排它操作权限后,对所述第一目标元数据进行操作的结果。Receiving, by the first client, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server to perform exclusive operation on all metadata in the target directory. After the permission, the result of the operation of the first target metadata.
  7. 根据权利要求1~5任一项所述的方法,其特征在于,所述第一客户端访问所述第一目标元数据包括:The method according to any one of claims 1 to 5, wherein the accessing the first target metadata by the first client comprises:
    所述第一客户端向元数据路由服务器发送切换消息,指示所述元数据路由服务器将所述目标目录的操作权限归属到所述第一客户端;Sending, by the first client, a handover message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client;
    所述第一客户端接收所述元数据路由服务器发送的通知消息,所述通知消息用于指示所述目标目录的操作权限归属到所述第一客户端;Receiving, by the first client, the notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client;
    所述第一客户端向所述目录锁服务器发送第二申请消息以申请对所述目标目录中所有元数据的排它操作权限;The first client sends a second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory;
    所述第一客户端接收所述目录锁服务器发送的授予所述目标目录中所有元数据的排它操作权限的授予消息,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的第一目标元数据,以及操作所述目标目录中除所述第一目标元数据之外的元数据;Receiving, by the first client, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for The first client operates the first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
    所述第一客户端根据所述目标目录中所有元数据的排它操作权限对所述第一目标元数据进行操作,获得操作结果。The first client operates the first target metadata according to the exclusive operation authority of all metadata in the target directory, and obtains an operation result.
  8. 根据权利要求1、2、3、6或7所述的方法,其特征在于,所述当所述第一客户端申请到对所述目标目录中所有元数据的排它访问权限时,所述第一客户端访问所述第一目标元数据之后,所述方法还包括:The method according to claim 1, 2, 3, 6 or 7, wherein said said first client applies for exclusive access to all metadata in said target directory, said After the first client accesses the first target metadata, the method further includes:
    所述第一客户端判断访问所述目标目录中的元数据所产生的负载是否满足预设条件; Determining, by the first client, whether a load generated by accessing metadata in the target directory meets a preset condition;
    若满足,则所述第一客户端释放所述目标目录中的所有元数据的排它访问权限,以便所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予第四客户端。If satisfied, the first client releases exclusive access rights of all metadata in the target directory, so that the directory lock server grants exclusive access rights of all metadata in the target directory to the fourth client end.
  9. 一种元数据访问方法,其特征在于,包括:A metadata access method, comprising:
    目录锁服务器在第一客户端访问目标目录中的目标元数据时,接收所述第一客户端发送的第一申请消息,所述第一申请消息用于申请对所述目标目录中所有元数据的排它访问权限,所述目标目录为存储有多个元数据的目录,所述目标元数据为所述第一客户端当前需要访问的元数据;The directory lock server receives the first application message sent by the first client when the first client accesses the target metadata in the target directory, and the first application message is used to apply for all metadata in the target directory. Exclusive access rights, the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
    所述目录锁服务器根据所述第一申请消息判断所述目标目录中所有元数据的排它访问权限是否已经授予了第二客户端,所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问所述目标目录中的目标元数据,以及访问所述目标目录中除所述目标元数据之外的元数据;The directory lock server determines, according to the first application message, whether exclusive access rights of all metadata in the target directory have been granted to the second client, and exclusive access rights of all metadata in the target directory are used for Providing, by the first client, target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
    若未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端;若授予了所述第二客户端,则所述目录锁服务器向所述第一客户端发送所述第二客户端所在的节点信息,以使所述第一客户端基于所述第二客户端所在的节点信息通过所述第二客户端访问所述目标元数据。If the second client is not granted, the directory lock server grants exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, Transmitting, by the directory lock server, node information of the second client to the first client, so that the first client accesses the second client by using node information of the second client The target metadata.
  10. 根据权利要求9所述的方法,其特征在于,所述若未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述方法还包括:The method according to claim 9, wherein if the second client is not granted, the directory lock server grants exclusive access rights of all metadata in the target directory to the first After the client, the method further includes:
    所述目录锁服务器在第三客户端访问所述目标元数据时,接收所述第三客户端发送的所述第一申请消息;Receiving, by the third client, the first application message sent by the third client when the third client accesses the target metadata;
    所述目录锁服务器向所述第三客户端发送所述第一客户端所在的节点信息,以使所述第三客户端基于所述第一客户端所在的节点信息通过所述第一客户端访问所述目标元数据。The directory lock server sends the node information of the first client to the third client, so that the third client passes the first client based on the node information where the first client is located. Access the target metadata.
  11. 根据权利要求9或10所述的方法,其特征在于,所述若未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述方法还包括:The method according to claim 9 or 10, wherein said directory lock server grants exclusive access rights of all metadata in said target directory to said said second client if said second client is not granted After the first client, the method further includes:
    所述目录锁服务器接收目标元数据服务器发送的第二申请消息,所述第二申请消息为所述目标元数据服务器接收到所述第一客户端访问所述目标元数据时发送的操作消息时发送的,用于申请对所述目标目录中所有元数据的排它操作权限的消息;Receiving, by the directory lock server, a second application message sent by the target metadata server, where the second application message is an operation message sent by the target metadata server when the first client accesses the target metadata a message sent for applying for exclusive operation rights to all metadata in the target directory;
    所述目录锁服务器根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述目标元数据服务器,以使所述目标元数据服务器对所述目标元数据进行操作并将操作结果反馈给所述第一客户端,其中,所述目标目录中所有元数据的排它操作权限用于供所述目标元数据服务器操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。The directory lock server grants exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server operates the target metadata And feeding back the operation result to the first client, wherein the exclusive operation authority of all the metadata in the target directory is used for the target metadata server to operate the target metadata in the target directory, and the operation Metadata in the target directory other than the target metadata.
  12. 根据权利要求9或10所述的方法,其特征在于,所述若未授予所 述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述方法还包括:The method according to claim 9 or 10, wherein said After the second client, the directory lock server grants exclusive access rights of all the metadata in the target directory to the first client, the method further includes:
    所述目录锁服务器接收所述第一客户端发送的第二申请消息,所述第二申请消息用于申请对所述目标目录中所有元数据的排它操作权限;The directory lock server receives a second application message sent by the first client, where the second application message is used to apply for exclusive operation rights on all metadata in the target directory;
    所述目录锁服务器根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述第一客户端,以使所述第一客户端对所述目标元数据进行操作,其中,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。The directory lock server grants exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client operates the target metadata And an exclusive operation authority of all metadata in the target directory for the first client to operate target metadata in the target directory, and operating the target directory except the target metadata External metadata.
  13. 一种终端,其特征在于,所述终端为第一客户端,所述第一客户端包括发送装置、接收装置、存储器和处理器,所述处理器调用所述存储器中的访问元数据的程序,用于来执行如下操作:A terminal, wherein the terminal is a first client, the first client includes a sending device, a receiving device, a memory, and a processor, and the processor calls a program for accessing metadata in the memory. , used to perform the following operations:
    在请求访问目标目录中的第一目标元数据时,通过所述发送装置向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述处理器当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述处理器访问所述目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the first target metadata in the target directory, sending, by the sending device, a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the processor currently needs to access; exclusive access rights of all metadata in the target directory are used for the processor Accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
    当申请到对所述目标目录中所有元数据的排它访问权限时,访问所述第一目标元数据。The first target metadata is accessed when applying for exclusive access to all metadata in the target directory.
  14. 根据权利要求13所述的终端,其特征在于,当未申请到对所述目标目录中所有元数据的排它访问权限时,通过所述接收装置接收所述目录锁服务器发送的第二客户端所在的节点信息,并根据所述节点信息通过所述发送装置向所述第二客户端发送请求消息以通过所述第二客户端访问所述第一目标元数据,所述第二客户端为向所述目录锁服务器申请到了对所述目标目录中所有元数据的排它访问权限的客户端。The terminal according to claim 13, wherein when the exclusive access right to all metadata in the target directory is not applied, the second client sent by the directory lock server is received by the receiving device a node information, and sending, by the sending device, a request message to the second client according to the node information to access the first target metadata by using the second client, where the second client is A client that has applied to the directory lock server for exclusive access to all metadata in the target directory.
  15. 根据权利要求13或14所述的终端,其特征在于,所述在请求访问目标目录中的第一目标元数据时,通过所述发送装置向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限的方式,具体为:The terminal according to claim 13 or 14, wherein, when requesting access to the first target metadata in the target directory, the sending device sends a first application message to the directory lock server to apply for the The exclusive access rights of all metadata in the target directory, specifically:
    获取描述所述目标目录的元数据,所述描述所述目标目录的元数据设有用于标识所述目标目录是否为存储有多个元数据的目录的标识信息;Obtaining metadata describing the target directory, where the metadata describing the target directory is provided with identification information for identifying whether the target directory is a directory in which a plurality of metadata is stored;
    根据所述描述所述目标目录的元数据来判断所述目标目录是否为存储有多个元数据的目录;Determining, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
    若是,则通过所述发送装置向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限。If so, the first application message is sent to the directory lock server by the sending device to apply for exclusive access rights to all metadata in the target directory.
  16. 根据权利要求13~15任一项所述的终端,其特征在于,当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理访问所述第一目标元数据之后,所述处理器还用于: The terminal according to any one of claims 13 to 15, wherein, when applying for exclusive access rights to all metadata in the target directory, after the processing accesses the first target metadata, The processor is further configured to:
    通过所述接收装置接收第三客户端发送的请求消息,所述请求消息用于请求通过所述第一客户端访问所述第一目标元数据,所述请求消息为所述第三客户端向所述目录锁服务器申请对所述目标目录中所有元数据的排它访问权限,并接收到所述目录锁服务器返回的所述第一客户端的节点信息后,根据所述第一客户端的节点信息向所述第一客户端发送的消息;Receiving, by the receiving device, a request message sent by a third client, where the request message is used to request access to the first target metadata by using the first client, where the request message is sent by the third client The directory lock server applies for exclusive access rights to all metadata in the target directory, and after receiving the node information of the first client returned by the directory lock server, according to the node information of the first client a message sent to the first client;
    根据所述请求消息访问所述第一目标元数据,并将访问所述第一目标元数据的访问结果通过所述发送装置发送给所述第三客户端。And accessing the first target metadata according to the request message, and sending an access result of accessing the first target metadata to the third client by using the sending device.
  17. 根据权利要求13~16任一项所述的终端,其特征在于,当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理器访问所述第一目标元数据之后,所述处理器还用于:The terminal according to any one of claims 13 to 16, characterized in that, when applying for exclusive access rights to all metadata in the target directory, the processor accesses the first target metadata The processor is further configured to:
    在请求访问所述目标目录中的第二目标元数据时,如果第二目标元数据属于所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata other than the first target metadata in the target directory;
    则所述处理器按照已获得的对所述第二目标元数据的排它访问权限,对所述第二目标元数据进行访问。And then the processor accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
  18. 根据权利要求13~17任一项所述的终端,其特征在于,所述处理器访问所述第一目标元数据的方式具体为:The terminal according to any one of claims 13 to 17, wherein the manner in which the processor accesses the first target metadata is specifically:
    向元数据路由服务器请求所述目标目录的归属信息,所述元数据路由服务器预存有目录的归属信息;Requesting, to the metadata routing server, attribution information of the target directory, where the metadata routing server prestores the attribution information of the directory;
    通过所述接收装置接收所述元数据路由服务器发送的所述目标目录的归属信息,所述目标目录的归属信息用于指示所述目标目录的操作权限归属到目标元数据服务器;Receiving, by the receiving device, the home information of the target directory that is sent by the metadata routing server, where the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
    通过所述发送装置向所述目标元数据服务器发送操作请求消息,请求所述目标元数据服务器根据所述对所述目标目录中所有元数据的排它访问权限操作所述第一目标元数据;Sending, by the sending device, an operation request message to the target metadata server, requesting the target metadata server to operate the first target metadata according to the exclusive access right to all metadata in the target directory;
    通过所述接收装置接收所述目标元数据服务器发送的操作结果;所述操作结果为所述目标元数据服务器向所述目录锁服务器申请到对所述目标目录中所有元数据的排它操作权限后,对所述第一目标元数据进行操作的结果。Receiving, by the receiving device, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server for exclusive operation permission on all metadata in the target directory. Thereafter, the result of operating the first target metadata.
  19. 根据权利要求13~17任一项所述的终端,其特征在于,所述处理器访问所述第一目标元数据的方式具体为:The terminal according to any one of claims 13 to 17, wherein the manner in which the processor accesses the first target metadata is specifically:
    通过所述发送装置向元数据路由服务器发送切换消息,指示所述元数据路由服务器将所述目标目录的操作权限归属到所述第一客户端;Transmitting, by the sending device, a handover message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client;
    通过所述接收装置接收所述元数据路由服务器发送的通知消息,所述通知消息用于指示所述目标目录的操作权限归属到所述第一客户端;Receiving, by the receiving device, the notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client;
    通过所述发送装置向所述目录锁服务器发送第二申请消息以申请对所述目标目录中所有元数据的排它操作权限;Sending, by the sending device, a second application message to the directory lock server to apply for exclusive operation authority on all metadata in the target directory;
    通过所述接收装置接收所述目录锁服务器发送的授予所述目标目录中所有元数据的排它操作权限的授予消息,所述目标目录中所有元数据的排它操作权限用于供第一客户端操作所述目标目录中的第一目标元数据,以 及操作所述目标目录中除所述第一目标元数据之外的元数据;Receiving, by the receiving device, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for the first client Endoperating the first target metadata in the target directory to And operating metadata in the target directory other than the first target metadata;
    根据所述目标目录中所有元数据的排它操作权限对所述第一目标元数据进行操作,获得操作结果。The first target metadata is operated according to the exclusive operation authority of all the metadata in the target directory, and the operation result is obtained.
  20. 根据权利要求13、14、15、18或19所述的终端,其特征在于,当申请到对所述目标目录中所有元数据的排它访问权限时,所述处理器访问所述第一目标元数据之后,所述处理器还用于:The terminal according to claim 13, 14, 15, 18 or 19, wherein said processor accesses said first target when applying exclusive access rights to all metadata in said target directory After the metadata, the processor is also used to:
    判断访问所述目标目录中的元数据所产生的负载是否满足预设条件;Determining whether the load generated by accessing the metadata in the target directory satisfies a preset condition;
    若满足,则释放对所述目标目录中的所有元数据的排它访问权限,以便所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予第四客户端。If so, exclusive access to all metadata in the target directory is released, such that the directory lock server grants exclusive access to all metadata in the target directory to the fourth client.
  21. 一种终端,其特征在于,所述终端为目录锁服务器,所述目录锁服务器包括发送装置、接收装置、存储器和处理器,所述处理器调用所述存储器中的访问元数据的程序,用于执行如下操作:A terminal, wherein the terminal is a directory lock server, the directory lock server includes a sending device, a receiving device, a memory, and a processor, and the processor calls a program for accessing metadata in the memory, Do the following:
    在第一客户端访问目标目录中的目标元数据时,通过所述接收装置接收所述第一客户端发送的第一申请消息,所述第一申请消息用于申请对所述目标目录中所有元数据的排它访问权限,所述目标目录为存储有多个元数据的目录,所述目标元数据为所述第一客户端当前需要访问的元数据;Receiving, by the receiving device, a first application message sent by the first client, where the first client accesses the target metadata in the target directory, where the first application message is used to apply for all the target directories. Exclusive access rights of the metadata, the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
    根据所述第一申请消息判断所述目标目录中所有元数据的排它访问权限是否已经授予了第二客户端,所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问目标目录中的目标元数据,以及访问所述目标目录中除所述目标元数据之外的元数据;Determining, according to the first application message, whether an exclusive access right of all metadata in the target directory has been granted to the second client, and exclusive access rights of all metadata in the target directory are used for the first The client accesses target metadata in the target directory, and accesses metadata in the target directory other than the target metadata;
    若未授予所述第二客户端,则将所述目标目录中所有元数据的排它访问权限授予所述第一客户端;若授予了所述第二客户端,则通过所述发送装置向所述第一客户端发送所述第二客户端所在的节点信息,以使所述第一客户端基于所述第二客户端所在的节点信息通过所述第二客户端访问所述目标元数据。If the second client is not granted, grant exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, pass the sending device to Sending, by the first client, node information where the second client is located, so that the first client accesses the target metadata by using the second client according to node information of the second client .
  22. 根据权利要求21所述的终端,其特征在于,若未授予所述第二客户端,所述处理器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器还用于:The terminal according to claim 21, wherein, if the second client is not granted, the processor grants exclusive access rights of all metadata in the target directory to the first client, The processor is further configured to:
    在第三客户端访问所述目标元数据时,通过接收装置接收所述第三客户端发送的所述第一申请消息;Receiving, by the receiving device, the first application message sent by the third client, when the third client accesses the target metadata;
    通过所述发送装置向所述第三客户端发送所述第一客户端所在的节点信息,以使所述第三客户端基于所述第一客户端所在的节点信息通过所述第一客户端访问所述目标元数据。Sending, by the sending device, the node information of the first client to the third client, so that the third client passes the first client based on node information where the first client is located. Access the target metadata.
  23. 根据权利要求21或22所述的终端,其特征在于,若未授予所述第二客户端,所述处理器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器还用于:The terminal according to claim 21 or 22, wherein if the second client is not granted, the processor grants exclusive access rights of all metadata in the target directory to the first client Thereafter, the processor is further configured to:
    通过所述接收装置接收目标元数据服务器发送的第二申请消息,所述第二申请消息为所述目标元数据服务器接收到所述第一客户端访问所述目 标元数据时发送的操作消息时发送的,用于申请对所述目标目录中所有元数据的排它操作权限的消息;Receiving, by the receiving device, a second application message sent by the target metadata server, where the second application message is that the target metadata server receives the first client accessing the target A message sent when an operation message is sent when the metadata is sent, for applying exclusive permission to all metadata in the target directory;
    根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述目标元数据服务器,以使所述目标元数据服务器对所述目标元数据进行操作并将操作结果反馈给所述第一客户端,其中,所述目标目录中所有元数据的排它操作权限用于供所述目标元数据服务器操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。Granting exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server operates on the target metadata and feedbacks operation results Giving the first client, wherein exclusive operation rights of all metadata in the target directory are used by the target metadata server to operate target metadata in the target directory, and operating the target directory Metadata other than the target metadata.
  24. 根据权利要求21或22所述的终端,其特征在于,若未授予所述第二客户端,所述处理器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端之后,所述处理器还用于:The terminal according to claim 21 or 22, wherein if the second client is not granted, the processor grants exclusive access rights of all metadata in the target directory to the first client Thereafter, the processor is further configured to:
    通过所述接收装置接收所述第一客户端发送的第二申请消息,所述第二申请消息用于申请对所述目标目录中所有元数据的排它操作权限;Receiving, by the receiving device, a second application message sent by the first client, where the second application message is used to apply for exclusive operation permission on all metadata in the target directory;
    根据所述第二申请消息将所述目标目录中所有元数据的排它操作权限授予所述第一客户端,以使所述第一客户端对所述目标元数据进行操作,其中,所述目标目录中所有元数据的排它操作权限用于供所述第一客户端操作所述目标目录中的目标元数据,以及操作所述目标目录中除所述目标元数据之外的元数据。Granting exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client operates on the target metadata, wherein the Exclusive operational rights for all metadata in the target directory are used by the first client to manipulate target metadata in the target directory and to manipulate metadata in the target directory other than the target metadata.
  25. 一种分布式文件系统,其特征在于,包括第一客户端、第二客户端和目录锁服务器,其中:A distributed file system, comprising: a first client, a second client, and a directory lock server, wherein:
    所述第一客户端在请求访问目标目录中的第一目标元数据时,向目录锁服务器发送第一申请消息以申请对所述目标目录中所有元数据的排它访问权限,其中,所述目标目录为存储有多个元数据的目录,所述第一目标元数据为所述第一客户端当前需要访问的元数据;所述目标目录中所有元数据的排它访问权限用于供所述第一客户端访问目标目录中的第一目标元数据,以及访问所述目标目录中除所述第一目标元数据之外的元数据;When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, where The target directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for a supply Decoding, by the first client, the first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
    所述目录锁服务器接收所述第一客户端发送的第一申请消息,并判断所述目标目录中所有元数据的排它访问权限是否授予了第二客户端;Receiving, by the directory lock server, the first application message sent by the first client, and determining whether exclusive access rights of all metadata in the target directory are granted to the second client;
    若所述目标目录中所有元数据的排它访问权限未授予所述第二客户端,则所述目录锁服务器将所述目标目录中所有元数据的排它访问权限授予所述第一客户端,并向所述第一客户端发送通知消息以通知所述目标目录中所有元数据的排它访问权限授予所述第一客户端;If the exclusive access right of all metadata in the target directory is not granted to the second client, the directory lock server grants exclusive access rights of all metadata in the target directory to the first client And sending a notification message to the first client to notify the first client of exclusive access rights of all metadata in the target directory;
    所述第一客户端接收所述通知消息,并根据所述通知消息访问所述第一目标元数据。The first client receives the notification message and accesses the first target metadata according to the notification message.
  26. 根据权利要求25所述的系统,其特征在于,The system of claim 25 wherein:
    所述第一客户端为权利要求13~20任一项所述的终端;The first client is the terminal according to any one of claims 13 to 20;
    所述目录锁服务器为权利要求21~24任一项所述的终端。 The directory lock server is the terminal according to any one of claims 21 to 24.
PCT/CN2016/107979 2015-12-04 2016-11-30 Metadata access method, related apparatus and system WO2017092673A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510888579.4 2015-12-04
CN201510888579.4A CN105512279B (en) 2015-12-04 2015-12-04 A kind of metadata access method, relevant device and system

Publications (1)

Publication Number Publication Date
WO2017092673A1 true WO2017092673A1 (en) 2017-06-08

Family

ID=55720261

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/107979 WO2017092673A1 (en) 2015-12-04 2016-11-30 Metadata access method, related apparatus and system

Country Status (2)

Country Link
CN (1) CN105512279B (en)
WO (1) WO2017092673A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112449018A (en) * 2020-11-25 2021-03-05 杭州视洞科技有限公司 Multi-computer-room data synchronization and deployment method based on end side

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105512279B (en) * 2015-12-04 2019-05-03 华为技术有限公司 A kind of metadata access method, relevant device and system
CN109510846B (en) * 2017-09-14 2020-11-03 北京金山云网络技术有限公司 API calling system, method and device, electronic equipment and storage medium
CN111049779B (en) * 2018-10-12 2022-04-26 伊姆西Ip控股有限责任公司 Information processing method, electronic device, and computer-readable medium
CN110347650B (en) * 2019-07-16 2021-06-01 北京明略软件系统有限公司 Metadata acquisition method and device
CN110941591A (en) * 2019-11-22 2020-03-31 浪潮电子信息产业股份有限公司 File deletion method, device and equipment and readable storage medium
CN111046001B (en) * 2019-12-28 2023-03-14 浪潮电子信息产业股份有限公司 Method, device and equipment for creating files in batch and storage medium
CN115250270B (en) * 2022-07-28 2024-02-23 中国银行股份有限公司 File application method based on 5G message and related equipment
CN117591038A (en) * 2024-01-18 2024-02-23 济南浪潮数据技术有限公司 Data access method, device, distributed storage system, equipment and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101375250A (en) * 2006-02-03 2009-02-25 甲骨文国际公司 Adaptive region locking
CN103501339A (en) * 2013-09-30 2014-01-08 华为技术有限公司 Metadata processing method and metadata server
US8850126B2 (en) * 2010-01-08 2014-09-30 Netapp, Inc. Exclusive access during a critical sub-operation to enable simultaneous operations
CN104113587A (en) * 2014-06-23 2014-10-22 华中科技大学 Client metadata buffer optimization method of distributed file system
CN105512279A (en) * 2015-12-04 2016-04-20 华为技术有限公司 Metadata access method, related equipment and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101375250A (en) * 2006-02-03 2009-02-25 甲骨文国际公司 Adaptive region locking
US8850126B2 (en) * 2010-01-08 2014-09-30 Netapp, Inc. Exclusive access during a critical sub-operation to enable simultaneous operations
CN103501339A (en) * 2013-09-30 2014-01-08 华为技术有限公司 Metadata processing method and metadata server
CN104113587A (en) * 2014-06-23 2014-10-22 华中科技大学 Client metadata buffer optimization method of distributed file system
CN105512279A (en) * 2015-12-04 2016-04-20 华为技术有限公司 Metadata access method, related equipment and system

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112449018A (en) * 2020-11-25 2021-03-05 杭州视洞科技有限公司 Multi-computer-room data synchronization and deployment method based on end side

Also Published As

Publication number Publication date
CN105512279A (en) 2016-04-20
CN105512279B (en) 2019-05-03

Similar Documents

Publication Publication Date Title
WO2017092673A1 (en) Metadata access method, related apparatus and system
US11388251B2 (en) Providing access to managed content
US10983868B2 (en) Epoch based snapshot summary
WO2019057212A1 (en) Method, apparatus and device for scheduling service within cdn node, and storage medium
US20120166403A1 (en) Distributed storage system having content-based deduplication function and object storing method
US20100088335A1 (en) Method and apparatus for improving file access performance of distributed storage system
US9952940B2 (en) Method of operating a shared nothing cluster system
US10742748B2 (en) System and method for supporting live addition of a tenant in a connection pool environment
US11221993B2 (en) Limited deduplication scope for distributed file systems
US20100161585A1 (en) Asymmetric cluster filesystem
US10545667B1 (en) Dynamic data partitioning for stateless request routing
US20200026690A1 (en) Global data deduplication across multiple distributed file systems
CN107493309B (en) File writing method and device in distributed system
US10516628B2 (en) Transfer device, transfer system, and transfer method
US9607072B2 (en) System and method for implementing nested relationships within a schemaless database
CN109844723B (en) Method and system for master control establishment using service-based statistics
US11580152B1 (en) Using path-based indexing to access media recordings stored in a media storage service
KR102214185B1 (en) Resource Retrieval Method
JP2017123040A (en) Server device, distribution file system, distribution file system control method, and program
KR102033383B1 (en) Method and system for managing data geographically distributed
US10621148B1 (en) Maintaining multiple object stores in a distributed file system
CN117075823B (en) Object searching method, system, electronic device and storage medium
US20230176908A1 (en) Systems, methods and computer program products for job management
AU2019208258B2 (en) Information processing apparatus, file management apparatus, file management system, and program
US20210326386A1 (en) Information processing system, information processing device, and non-transitory computer-readable storage medium for storing program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16869983

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16869983

Country of ref document: EP

Kind code of ref document: A1