WO2017092673A1 - Procédé d'accès à des métadonnées, appareil associé et système - Google Patents

Procédé d'accès à des métadonnées, appareil associé et système Download PDF

Info

Publication number
WO2017092673A1
WO2017092673A1 PCT/CN2016/107979 CN2016107979W WO2017092673A1 WO 2017092673 A1 WO2017092673 A1 WO 2017092673A1 CN 2016107979 W CN2016107979 W CN 2016107979W WO 2017092673 A1 WO2017092673 A1 WO 2017092673A1
Authority
WO
WIPO (PCT)
Prior art keywords
metadata
target
directory
client
server
Prior art date
Application number
PCT/CN2016/107979
Other languages
English (en)
Chinese (zh)
Inventor
王晓锋
方新
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2017092673A1 publication Critical patent/WO2017092673A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • G06F16/164—File meta data generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a metadata access method, related device, and system.
  • the metadata access portion of the GFS includes three subsystems: a metadata client subsystem 10, a metadata service subsystem 20, and a cluster service subsystem 30.
  • the metadata client subsystem 10 includes a plurality of clients (Client).
  • the client 101-103 is taken as an example in FIG. 1;
  • the metadata service subsystem 20 includes a plurality of metadata servers, and the metadata servers 201-203 are taken as an example in FIG. 1.
  • the cluster service subsystem 30 mainly includes a cluster management server. 301.
  • the principle of the client access metadata in FIG. 1 is as shown in FIG. 2.
  • the distributed consistency server 302 is first sent exclusive access rights to the target metadata.
  • the client 101 is the only client among all the clients 101-103 that can directly request the metadata server 201-203 to operate the target metadata, thus ensuring the target. Metadata cache coherency.
  • the client 101 queries the metadata routing server 303 for the attribution information of the target metadata, that is, queries which metadata server operation the target metadata belongs to; the metadata routing server 303 queries according to the mapping relationship between the stored metadata and the metadata server.
  • the attribution information of the target metadata is sent to the client 101; the client 101 analyzes the attribution information, and when the target metadata is analyzed and belongs to the metadata server 201, the metadata server is sent to the metadata server.
  • 201 issues a request to request access to the target metadata
  • the metadata server 201 receives the request of the client 101, and operates on the target metadata according to the request, and then feeds back the operation result to the client 101, and accordingly, the client 101 receives the element.
  • the result of the operation sent by the data server 201 is cached locally, and the client 101 accesses the target metadata at this point.
  • the above-mentioned distributed consistency server 302 grants the client 101 exclusive access rights to the target metadata, which can be implemented by a distributed lock, a distributed protocol, etc., when distributed through the distributed lock, the distributed consistency server 302 and the client
  • the interaction between the terminals 101 is required to perform the lock application, the lock release, and the lock recall operation.
  • the communication overhead between the nodes is large in the process; especially in the scenario of Lot of Small Files (LoSF), distributed
  • the metadata of the small files in the file system (GFS) is very large.
  • the client 101 accesses the metadata of each small file, the lock application, the lock release, and the lock recall operation are performed with the distributed consistency server.
  • the huge communication overhead generated reduces the efficiency of client access to metadata.
  • the embodiment of the invention discloses a method for accessing metadata, a related device and a system, which can reduce the overhead of maintaining metadata cache consistency in the process of accessing metadata by the client, and improve the access efficiency of the metadata.
  • an embodiment of the present invention provides a method for accessing metadata, where the method includes:
  • the first client When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target directory For storing a directory having a plurality of metadata, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the first a client accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
  • the first client accesses the first target metadata when the first client applies for exclusive access to all metadata in the target directory.
  • the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client when the first client does not apply for exclusive access rights to all metadata in the target directory, the first client Receiving, by the terminal, the node information of the second client sent by the directory lock server, and sending a request message to the second client according to the node information, to access the first target metadata by using the second client
  • the second client is a client that applies to the directory lock server for exclusive access to all metadata in the target directory.
  • the first client accesses the first target through the second client that has obtained exclusive access rights of all metadata in the target directory when the application does not have exclusive access rights to all the metadata in the target directory.
  • the metadata enables the first client to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
  • the first client is requesting access to the first target metadata in the target directory
  • the first application message is sent to the directory lock server to apply for exclusive access to all metadata in the target directory, the following:
  • the first client obtains metadata describing the target directory when requesting access to the first target metadata in the target directory, and the metadata describing the target directory is configured to identify whether the target directory is stored. Identification information of a directory of multiple metadata;
  • a first application message is sent to the directory lock server to request exclusive access to all metadata in the target directory.
  • the method further includes:
  • the first client accesses the first target metadata according to the request message, and sends an access result of accessing the first target metadata to the third client.
  • the third client After performing the above steps, after the first client obtains exclusive access rights to all metadata in the target directory, if the third client has a requirement to access the first target metadata in the target directory, the third client will access the request. Sending to the first client, accessing the first target metadata by the first client, ensuring that the client that does not apply for exclusive access rights of all metadata in the target directory can also access the first target metadata normally. .
  • the method when the first client applies exclusive access rights to all metadata in the target directory, after the first client accesses the first target metadata, the method further includes:
  • the first client when requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata in the target directory other than the first target metadata;
  • the first client accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
  • the second target metadata can be directly accessed, and the access permission is not required to be applied again.
  • the efficiency of a client's access to metadata in the target directory is not required to be applied again.
  • the first client accessing the first target metadata includes:
  • the first client receives the home information of the target directory sent by the metadata routing server, and the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
  • the operation result is that the target metadata server applies to the directory lock server to perform exclusive operation on all metadata in the target directory. After the permission, the result of the operation of the first target metadata.
  • the target metadata server After performing the above steps, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server first locks the directory before operating the first target metadata in the target directory.
  • the server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata retention consistency of the target directory is ensured.
  • the first client accessing the first target metadata includes:
  • the first client sends a second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory;
  • Receiving, by the first client, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for
  • the first client operates the first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
  • the first client operates the first target metadata according to the exclusive operation authority of all metadata in the target directory, and obtains an operation result.
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, the first client does not need to interact with the remote metadata server. To request the metadata server to operate on the first target metadata, the overhead between the nodes is reduced, and the access efficiency of accessing the first target metadata is improved.
  • the first client when the first client applies exclusive access rights to all metadata in the target directory, After the first client accesses the first target metadata, the method further includes:
  • the first client releases exclusive access rights of all metadata in the target directory, so that the directory lock server grants exclusive access rights of all metadata in the target directory to the fourth client end.
  • the exclusive access permission of all metadata in the target directory is automatically released, and the fourth client with lower load is obtained to obtain the row of all metadata in the target directory. It accesses the load to achieve the purpose of sharing the load; or automatically releases all metadata in the target directory when the first client accesses the metadata in the target directory less frequently than the metadata in other clients accessing the target directory.
  • Exclusive access rights allow clients with higher frequency of access to obtain exclusive access rights to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
  • an embodiment of the present invention provides a metadata access method, where the method includes:
  • the directory lock server receives the first application message sent by the first client when the first client accesses the target metadata in the target directory, and the first application message is used to apply for all metadata in the target directory.
  • Exclusive access rights the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the directory lock server determines, according to the first application message, whether exclusive access rights of all metadata in the target directory have been granted to the second client, and exclusive access rights of all metadata in the target directory are used for Providing, by the first client, target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
  • the directory lock server grants exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, Transmitting, by the directory lock server, node information of the second client to the first client, so that the first client accesses the second client by using node information of the second client The target metadata.
  • the first client accesses the target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can successfully access the target.
  • Metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the directory lock server performs exclusive access of all metadata in the target directory. After the rights are granted to the first client, the method further includes:
  • the directory lock server sends the node information of the first client to the third client, so that the third client passes the first client based on the node information where the first client is located. Access the target metadata.
  • the third client After performing the above steps, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement to access the target metadata in the target directory, the third client sends the access request to the third client.
  • the first client accesses the target metadata through the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the target metadata normally.
  • the directory lock server After the exclusive access rights of all the metadata in the target directory are granted to the first client, the method further includes:
  • the directory lock server grants exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server operates the target metadata And feeding back the operation result to the first client, wherein the exclusive operation authority of all the metadata in the target directory is used for the target metadata server to operate the target metadata in the target directory, and the operation Metadata in the target directory other than the target metadata.
  • the target metadata server After performing the above steps, after receiving the request for the target metadata in the target directory sent by the first client, the target metadata server requests the target directory from the directory lock server before performing the operation on the target metadata in the target directory.
  • the exclusive operation permission of all the metadata in the metadata makes that only one metadata server in all the metadata servers has the operation authority for the target directory, which ensures the persistent preservation consistency of the metadata in the target directory.
  • the directory lock server After the exclusive access rights of all the metadata in the target directory are granted to the first client, the method further includes:
  • the directory lock server receives a second application message sent by the first client, where the second application message is used to apply for exclusive operation rights on all metadata in the target directory;
  • the directory lock server grants exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client operates the target metadata And an exclusive operation authority of all metadata in the target directory for the first client to operate target metadata in the target directory, and operating the target directory except the target metadata External metadata.
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, the first client does not need to interact with the remote metadata server. To request the metadata server to operate on the target metadata, the overhead between the nodes is reduced, and the access efficiency of the access target metadata is improved.
  • an embodiment of the present invention provides a terminal, where the terminal is a first client, and the first client includes a sending device, a receiving device, a memory, and a processor, where the processor invokes A program for accessing metadata in the memory for performing the following operations:
  • the server sends a first application message to apply for exclusive access rights to all metadata in the target directory, wherein the target directory is a directory in which a plurality of metadata is stored, and the first target metadata is the processing Metadata currently required to be accessed by the device; exclusive access rights for all metadata in the target directory for the processor to access first target metadata in the target directory, and access to the target directory Metadata other than the first target metadata;
  • the first target metadata is accessed when applying for exclusive access to all metadata in the target directory.
  • the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the directory lock server is received by the receiving device And sending, by the sending device, a request message to the second client, according to the node information, to access metadata in the target directory by using the second client,
  • the second client is a client that applies to the directory lock server for exclusive access to all metadata in the target directory.
  • the first client accesses the first target through the second client that has acquired exclusive access rights of all metadata in the target directory when applying for exclusive access rights of all metadata in the target directory.
  • the metadata enables the first client to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
  • the processor sends a first application message to the directory lock server to apply for the
  • the exclusive access rights of all metadata in the target directory are as follows:
  • the first application message is sent to the directory lock server by the sending device to apply for exclusive access to all metadata in the target directory.
  • the processor when applying to the opposite The processor is further configured to: after the process accesses the first target metadata, when the process accesses the first target metadata in the target directory,
  • the directory lock server applies exclusive access rights to all metadata in the target directory, and after receiving the node information of the first client returned by the directory lock server, according to the a message sent by the node information of the client to the first client;
  • the third client After performing the above operation, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement to access the first target metadata in the target directory, the third client will access the request. Sending to the first client, accessing the first target metadata by the first client, ensuring that the client that does not apply for exclusive access rights of all metadata in the target directory can also access the first target metadata normally. .
  • the processor after applying the exclusive access right to all metadata in the target directory, after the processor accesses the first target metadata, the processor is further used to :
  • the processor accesses the second target metadata according to the obtained exclusive access right to the second target metadata.
  • the second target metadata can be directly accessed, and the access permission is not required to be applied again.
  • the efficiency of a client's access to metadata in the target directory is not required to be applied again.
  • the manner in which the processor accesses the first target metadata is specifically:
  • the receiving device Receiving, by the receiving device, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server for exclusive operation permission on all metadata in the target directory. Thereafter, the result of operating the first target metadata.
  • the target metadata server After performing the above operation, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server first locks the directory before operating the first target metadata in the target directory.
  • the server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata retention consistency of the target directory is ensured.
  • the manner in which the processor accesses the first target metadata is specifically:
  • Receiving, by the receiving device, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for the processing
  • the first target metadata in the target directory is operated, and metadata other than the first target metadata in the target directory is operated;
  • the first target metadata is operated according to the exclusive operation authority of all the metadata in the target directory, and the operation result is obtained.
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, it does not need to interact with the remote metadata server. To request the metadata server to operate on the first target metadata, the overhead between the nodes is reduced, and the access efficiency of accessing the first target metadata is improved.
  • the processor accesses the first target when applying exclusive access rights to all metadata in the target directory After the metadata, the processor is also used to:
  • the exclusive access permission of all metadata in the target directory is automatically released, and the fourth client with lower load is obtained to obtain the row of all metadata in the target directory. It accesses the load to achieve the purpose of sharing the load; or automatically releases all metadata in the target directory when the first client accesses the metadata in the target directory less frequently than the metadata in other clients accessing the target directory.
  • Exclusive access rights allow clients with higher frequency of access to obtain exclusive access rights to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
  • an embodiment of the present invention provides a terminal, where the terminal is a directory lock server, and the directory lock server includes a sending device, a receiving device, a memory, and a processor, where The processor calls a program for accessing metadata in the memory to perform the following operations:
  • the receiving device Receiving, by the receiving device, a first application message sent by the first client, where the first client accesses the target metadata in the target directory, where the first application message is used to apply for all the target directories.
  • Exclusive access rights of the metadata the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the second client If the second client is not granted, grant exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, pass the sending device to Sending, by the first client, node information where the second client is located, so that the first client accesses the target metadata by using the second client according to node information of the second client .
  • the application lock access authority of all the metadata in the target directory is requested to the directory lock server at one time, so that the first client can successfully access the target. Metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the processor grants exclusive access rights of all metadata in the target directory to the After the first client, the processor is further configured to:
  • the third client After performing the above operation, after the first client obtains the exclusive access right of all the metadata in the target directory, if the third client has the requirement of accessing the target metadata in the target directory, the third client sends the access request to the third client.
  • the first client accesses the target metadata through the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the target metadata normally.
  • the processor After the exclusive access rights of all metadata in the directory are granted to the first client, the processor is further configured to:
  • the target metadata server is configured to cause the target metadata server to operate on the target metadata and feed back an operation result to the first client, wherein all metadata in the target directory is exclusive
  • the operation authority is for the target metadata server to operate the target metadata in the target directory, and to operate metadata in the target directory other than the target metadata.
  • the target metadata server After performing the above operation, after receiving the request for the target metadata in the target directory sent by the first client, the target metadata server requests the target directory from the directory lock server before operating the target metadata in the target directory.
  • the exclusive operation permission of all the metadata in the metadata makes that only one metadata server in all the metadata servers has the operation authority for the target directory, which ensures the persistent preservation consistency of the metadata in the target directory.
  • the processor After the exclusive access rights of all metadata in the directory are granted to the first client, the processor is further configured to:
  • the first client functions as both a client and a metadata server. After the first client obtains exclusive access rights to all metadata in the target directory, it does not need to interact with the remote metadata server. To request the metadata server to operate on the target metadata, the overhead between the nodes is reduced, and the access efficiency of the access target metadata is improved.
  • an embodiment of the present invention provides a distributed file system, where the system includes a first client, a second client, and a directory lock server, where:
  • the first client When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target directory For storing a directory having a plurality of metadata, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the first a client accessing first target metadata in the target directory, and accessing metadata in the target directory other than the first target metadata;
  • the directory lock server grants exclusive access rights of all metadata in the target directory to the first client And sending a notification message to the first client to notify the first client of exclusive access rights of all metadata in the target directory;
  • the first client receives the notification message and accesses the first target metadata according to the notification message.
  • the first client accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time, so that the first client can smoothly Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client is the first client described in any possible implementation manner of the third aspect
  • the directory lock server is a directory lock server as described in any of the possible implementations of the fourth aspect.
  • the directory in which the plurality of metadata is stored includes a directory in which a plurality of metadata for describing a small number of files is stored.
  • the sending the first application message to the directory lock server to apply for exclusive access to all metadata in the target directory includes:
  • the lock is a non-preemptable distributed lock.
  • the sending the second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory includes:
  • the determining, by the first client, whether the load generated by accessing the metadata in the target directory meets a preset condition comprises: the first client records actively accessing the target directory by itself The first load generated by the metadata, and the second load generated by accessing the metadata in the target directory according to the request message of the other client; the terminal determines whether the relative size of the first load and the second load is Meet the preset conditions.
  • an embodiment of the present invention provides an apparatus for accessing metadata, where the apparatus includes a functional unit for performing some or all of the steps of any of the first aspects of the embodiments of the present invention.
  • an embodiment of the present invention provides an apparatus for accessing metadata, where the apparatus includes a functional unit for performing some or all of the steps of any of the second aspects of the embodiments of the present invention.
  • the first client When the first client accesses the first target metadata in the target directory, the first client requests the exclusive access permission of all the metadata in the target directory to the directory lock server, so that the first client The first target metadata can be successfully accessed, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • FIG. 1 is a schematic structural diagram of a distributed file system in the prior art
  • FIG. 2 is a schematic diagram of the principle of the distributed file system shown in FIG. 1;
  • FIG. 3 is a schematic flowchart of a metadata access method according to an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention.
  • FIG. 5A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention.
  • FIG. 5B is a schematic flowchart of interaction between devices in FIG. 5A; FIG.
  • FIG. 6 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a principle of metadata access according to an embodiment of the present invention.
  • FIG. 6B is a schematic flowchart of interaction between devices in FIG. 6A;
  • FIG. 7 is a schematic flowchart diagram of still another metadata access method according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of still another terminal according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of a distributed file system according to an embodiment of the present disclosure.
  • FIG. 11 is a schematic structural diagram of still another terminal according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic structural diagram of still another terminal according to an embodiment of the present invention.
  • FIGS. 3-7 are used to describe the method of the embodiments of the present invention.
  • FIG. 3 is a schematic flowchart of a metadata access method according to an embodiment of the present invention, which describes how the first client obtains exclusive access rights of all metadata in the target directory, including but not limited to the following: step.
  • Step S303 When requesting access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, where
  • the target directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for The first client accesses first target metadata in the target directory and accesses metadata in the target directory other than the first target metadata.
  • the distributed file system exists to store multiple elements.
  • the target directory of the data, the first target metadata in the target directory is the metadata that the first client currently needs to access.
  • the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the directory lock server coordinates the access rights of each client to the target directory by granting a lock to the target directory, if the directory lock server accesses the target directory The lock is granted to the first client, and the first client has access to all metadata in the target directory.
  • a distributed lock service such as a server carrying the zookeeper service
  • the distributed lock service can be set to preemptive distributed locks and non-preemptive distributed Lock; if set to a preemptible distributed lock, when the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server will release the access target directory according to the preemption mechanism. Lock, and re-grant the lock to the target directory to the first client, if the lock accessing the target directory is not granted to other clients, then the directory lock server directly grants the lock to the target directory to the first client; if not set to Preemptive distributed lock.
  • the directory lock server When the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server will not grant the lock to the target directory according to the non-preemptive mechanism.
  • the first client if the lock accessing the target directory is not granted to other clients, the directory lock server directly grants the lock accessing the target directory to the first client.
  • the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the first client applies for the lock to access the target directory, it has all the metadata in the target directory.
  • Exclusive access rights that is, other clients except the first client do not have access to the metadata in the target directory (ie, access to the first target metadata and metadata other than the first target metadata) ) unless the first client actively releases the lock accessing the target directory.
  • the method further includes: when the first client requests access to the second target metadata in the target directory, if the second target metadata belongs to the target directory except the first target metadata Metadata; then the first client accesses the second target metadata according to the obtained exclusive access rights to the second target metadata.
  • the original client when the first client requests access to the first target metadata in the target directory, the original client only needs to apply exclusive access rights of the first target metadata, but the first client actually applies for the target directory.
  • Exclusive access rights for all metadata such that when the first client requests access to metadata other than the first target metadata (eg, second target metadata) in the target directory, it determines that it has acquired The right to access the second target metadata, so there is no need to apply for exclusive access rights of the second target metadata, but directly access the second target metadata.
  • the first client when the first client requests access to the first target metadata in the target directory, the first client sends a first application message to the directory lock server to apply for all metadata in the target directory.
  • the exclusive access permission includes: when the first client requests to access the first target metadata in the target directory, acquiring metadata describing the target directory, where the metadata describing the target directory is configured to identify the Whether the target directory is the identification information of the directory in which the plurality of metadata is stored; determining whether the target directory is a directory in which the plurality of metadata is stored according to the metadata of the target directory; if yes, the directory lock The server sends a first application message to request exclusive access to all metadata in the target directory.
  • the plurality of metadata may be specifically metadata describing a small number of small files (each small file corresponds to its own metadata).
  • the target directory can also be regarded as a file, so it also corresponds to the metadata describing the target directory, and the metadata describing the target directory is stored in the upper directory of the target directory.
  • the metadata routing server receives the query request of the first client, Searching, from a mapping relationship between the stored directory and the metadata server, a metadata server to which the upper-level directory belongs, wherein the metadata server belongs to the metadata server, and the metadata server has the upper-level directory
  • the operation authority the attribution information of the upper-level directory is fed back to the first client, and the first client receives the home information sent by the metadata routing server, and analyzes the attribution information to obtain the element belonging to the upper-level directory.
  • the data server then requests, from the metadata server to which the upper-level directory belongs, metadata stored in the upper-level directory for describing the target directory, where the metadata server to which the upper-level directory belongs is stored and stored. Metadata used to describe the target directory in the primary directory, and the metadata of the read target directory is read To the first client.
  • directory B is the target directory
  • directory A is the upper directory of directory B
  • directory B stores the first target metadata describing a file.
  • the directory A stores the metadata describing the directory B; when the first client wants to access the first target metadata f, first needs to find the metadata server to which the directory A belongs to the metadata routing server, and then to the element
  • the data server requests the metadata describing the directory B under the directory A. Accordingly, the metadata server receives the request of the first client, reads the metadata describing the directory B in the directory A, and reads the element describing the directory B. The data is sent to the first client.
  • the first client analyzes the metadata describing the target directory, and determines whether the target directory stores multiple
  • the metadata describing each directory has identification information for defining the directory attribute of each directory, so the first client can be based on describing the metadata in the target directory.
  • the identification information "derives the directory attribute of the target directory, and further determines whether the target directory is a directory in which a plurality of metadata is stored.
  • the metadata describing the target directory may include "identification information" to identify the directory attribute of the target directory.
  • identification information When the identification information is set to "1", it indicates that the target directory is the above-mentioned directory storing a plurality of metadata, when When the identification information is set to "0", it indicates that the target directory is not the directory in which the plurality of metadata are stored as described above.
  • the "identification information” may be set by the user when the target directory is created, or may be modified by the user after the target directory is created. After the first client receives the metadata describing the target directory, it is determined whether the "identification information" is "1” or "0", and whether the target directory is a directory storing a plurality of metadata.
  • Step S304 When the first client applies for exclusive access rights to all metadata in the target directory, the first client accesses the first target metadata.
  • the directory lock server receives the first application message sent by the first client, analyzes the first application message, and obtains that the first application message is used to apply for exclusive access rights of all metadata in the target directory, so The directory lock server determines whether the exclusive access rights of all metadata in the target directory have been granted to other clients, and if not, the directory lock server will all metadata in the target directory.
  • the exclusive access permission is granted to the first client, so that only the first client of all the clients has the right to access the metadata in the target directory, and the first client subsequently accesses any metadata in the target directory.
  • the first client does not need to apply for access to the arbitrary metadata.
  • the first client accesses the metadata in the target directory.
  • the operations of reading and writing metadata belong to the access metadata.
  • the process of accessing the metadata includes requesting the metadata server to read the metadata and receiving the metadata server.
  • the metadata is read; when the metadata is written, the process of accessing the metadata includes requesting the metadata server to write content (including modifications) to the metadata and receiving the write result sent by the metadata server.
  • the metadata server is a device with operational metadata rights.
  • the target directory is directory B
  • the upper directory of directory B is directory A
  • the metadata of files is d, e, and f under directory B
  • the first client can access the metadata f, and can access the metadata e or d subsequently.
  • Direct access without having to apply for access to directory B, and no need to apply for access to e and d.
  • the first client accesses the first target metadata in the target directory
  • the first client accesses the directory lock server for all the metadata access rights in the target directory, so that the first client can be smoothly performed by performing steps S303-S304.
  • Accessing the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client when the first client does not apply for exclusive access to all metadata in the target directory, the first client receives the first sent by the directory lock server. And the node information where the client is located, and sending a request message to the second client according to the node information to access the first target metadata by using the second client, where the second client is a
  • the directory lock server applies to the client for exclusive access to all metadata in the target directory.
  • the directory lock server determines that the exclusive access rights of all metadata in the target directory have been granted to other clients (such as the second client)
  • the directory lock server does not queue all the metadata in the target directory.
  • the access permission is granted to the first client, but the directory lock server sends the node information including the second client to the first client, and correspondingly, the first client receives the node information sent by the directory lock server, and from the node Parsing out the node of the second client in the node information, and then sending a request to access the target directory to the second client based on the node, and correspondingly, the second client receives the request for accessing the target directory sent by the first client, according to the request The first target metadata under the target directory is accessed, and the access result is sent to the first client.
  • steps S303-S304 After performing the above steps S303-S304, if the first client acquires the rights of all the metadata in the target directory, the first client may also perform steps S305 and S306 for other third clients that cannot obtain the access rights of the target directory.
  • the terminal provides a service for accessing metadata in the first target directory, and the specific flowchart is as shown in FIG. 4. The detailed description of steps S305 and S306 is as follows:
  • Step S305 The first client receives a request message sent by a third client, where the request is cancelled.
  • the information is used to request access to the first target metadata by using the first client, and the request message is for the third client to apply to the directory lock server for exclusive use of all metadata in the target directory.
  • the directory lock server when the directory lock server grants exclusive access rights of all metadata in the target directory to the first client, the directory lock server also receives exclusive access rights of all metadata in the target directory of the third client application.
  • Request the third client refers to a type of client that does not have exclusive access to all metadata in the target directory
  • the directory lock server since the exclusive access has been granted to the first client, the directory lock server does not The exclusive access to all metadata in the target directory is granted to the third client, but in order to ensure that the third client can normally access the metadata in the target directory, the directory lock server sends the third client to the third client.
  • the third client receives the node information sent by the directory lock server, and analyzes, according to the node information, the first target metadata that needs to be accessed through the first client, and then sends the first target metadata to the first client.
  • An application message where the application message includes information about the first target metadata in the target directory that the third client needs to access,
  • the first client application receives a third message sent by the client.
  • Step S306 The first client accesses the first target metadata according to the request message, and sends an access result of accessing the first target metadata to the third client.
  • the first client analyzes the application message, and obtains that the metadata that the third client needs to access is the first target metadata, and further determines whether the first target metadata is in the cache, and if yes, the cached A target metadata is read, written, etc., and the operation result is fed back to the third client; if not in the cache, the metadata server to which the target directory belongs is requested to read and write the first target metadata, The result of the operation is then fed back to the third client.
  • the third client After performing the steps S305 and S306, after the first client obtains the exclusive access permission of all the metadata in the target directory, if the third client has the requirement of accessing the first target metadata in the target directory, the third client will The access request is sent to the first client, and the first target metadata is accessed by the first client, so that the client that does not apply for exclusive access to all metadata in the target directory can also access the first target normally. Metadata.
  • the first client After the first client obtains the exclusive access right of all the metadata in the target directory, the first client accesses the first target metadata by using the steps S307-S310; As shown in Figure 5.
  • the detailed description of steps S307-S310 is as follows:
  • Step S307 The first client requests the home information of the target directory from the metadata routing server, and the metadata routing server prestores the home information of the directory.
  • the first client sends a query request to the metadata routing server to query the home information of the target directory; the metadata routing server receives the query request of the first client, and searches from the mapping relationship between the stored directory and the metadata server.
  • the target metadata server corresponding to the target directory and then sending the home information to the first client, indicating that the target directory belongs to the target metadata server.
  • Step S308 The first client receives the home information of the target directory sent by the metadata routing server, and the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server.
  • the first client receives the home information sent by the metadata routing server, and analyzes the attribution information to obtain that the operation authority of the target directory belongs to the target metadata server.
  • Step S309 The first client sends an operation request message to the target metadata server, requesting the target metadata server to operate the first according to the exclusive access right to all metadata in the target directory.
  • Target metadata The first client sends an operation request message to the target metadata server, requesting the target metadata server to operate the first according to the exclusive access right to all metadata in the target directory.
  • the first client since the first client has obtained exclusive access rights of all metadata in the target directory, the first client has the right to send an operation request message to the target metadata server, requesting the target metadata server to the first target metadata. The operation is performed, and other clients that do not have exclusive access to all metadata in the target directory cannot request the target metadata server to operate on the first target metadata.
  • Step S310 The first client receives an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server to all metadata in the target directory. The result of operating the first target metadata after exclusive operation permission.
  • the target metadata server receives the operation request message sent by the first client, and parses the request message to obtain that the metadata of the first client request operation is the first target metadata in the target directory, and thus the directory lock
  • the server sends an application request to apply for exclusive operation permission of all metadata in the target directory; accordingly, the directory lock server receives the application request sent by the metadata server, and at this time, regardless of whether the exclusive operation permission of all metadata in the target directory is granted
  • the directory lock server grants the exclusive operation permission of all metadata in the target directory to the target metadata server (if the exclusive operation permission of the target directory is specifically implemented by distributed locks, then when the target directory is operated When the lock has been granted to other clients, the directory lock server recalls the lock of the operation target directory, and then grants the lock of the operation target directory to the first client; when the lock of the operation target directory is not granted to other clients, the directory lock server directly The lock of the operation target directory is granted to the first client), so that all Only the target metadata server capable of operating on metadata in the target directory metadata server.
  • the target metadata server operates the first target metadata in the target directory, and then sends an operation result of operating the first target metadata to the first client, and correspondingly, the first client receives the target metadata.
  • the exclusive operation permission of all metadata in the application target directory and the exclusive access permission of all metadata in the application target directory can be realized by the distributed lock based on the directory granularity, wherein all the application target directories are
  • the exclusive operation permission of metadata uses a preemptible distributed lock.
  • the exclusive access permission for all metadata in the target directory is a non-preemptable distributed lock.
  • the target metadata server After performing the steps S307-S310, after receiving the request for the first target metadata in the target directory sent by the first client, the target metadata server performs the operation on the first target metadata in the target directory.
  • the directory lock server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata of the target directory is consistently saved. Sex.
  • the first client After the first client obtains the exclusive access rights of all the metadata in the target directory, the first client accesses the first target metadata by using the steps S311-S315; As shown in Figure 6.
  • the detailed description of steps S311 to S315 is as follows:
  • Step S311 The first client sends a handover message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client.
  • the first client sends a handover message to the metadata routing server
  • the metadata routing server receives the handover message sent by the first client, and analyzes the handover message, and analyzes the handover message to indicate that the target is to be
  • the metadata routing server modifies the mapping relationship in the routing table, so that the target directory corresponds to the first client, that is, the operation authority of the target directory is attributed to the first client.
  • Step S312 The first client receives the notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client.
  • the metadata routing server sends a notification message that the route switching succeeds to the first client, and the first client receives the notification message sent by the metadata routing server, and analyzes the notification message to know that the target directory has belonged to the first client. end.
  • a client proxy and a metadata service proxy may be set in the first client, and the client proxy is used to provide a cache of metadata and an exclusive access right to apply for all metadata in the target directory to the directory lock server;
  • the Metadata Service Agent is used to store the target directory and to apply exclusive access to the Directory Lock Server for all metadata in the target directory.
  • Step S313 The first client sends a second application message to the directory lock server to apply for exclusive operation rights on all metadata in the target directory.
  • the first client when the first client knows that the target directory belongs to itself through the notification message, the first client sends a second application message to the directory lock server to apply for exclusive operation permission of all metadata in the target directory; correspondingly, the directory
  • the lock server receives the second application message sent by the first client, and the directory lock server grants the operation authority of the target directory to the first client, regardless of whether another metadata server operates the target directory.
  • a client sends a grant message to notify all the metadata in the target directory that the operational rights have been granted to the first client.
  • Step S314 The first client receives a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used. Providing, by the first client, first target metadata in the target directory, and operating metadata in the target directory other than the first target metadata.
  • the first client receives the grant message sent by the directory lock server, and after analyzing the grant message, can learn that it has acquired exclusive operation rights of all metadata in the target directory (including the first target metadata and the target directory). In this case, the exclusive operation permission of the metadata other than the first target metadata), so that the first client can operate on the metadata in the target directory without requesting an additional metadata server for the element in the target directory.
  • the data is manipulated.
  • Step S315 The first client operates the first target metadata according to the exclusive operation authority of all the metadata in the target directory, and obtains an operation result.
  • the first client is based on the exclusive operation rights of all metadata in the target directory that has been acquired.
  • the operation is performed on the first target metadata in the target directory, and the operation result is obtained.
  • the first client completes the access to the first target metadata in the target directory.
  • the result of the operation includes whether the reading is successful, and the read metadata is included when the reading is successful; when the form of the access is written (or modified), the result of the operation includes whether to write or not Success, the rest of the forms of access here are not examples.
  • the first client functions as both a client and a metadata server, and the first client does not need to have a remote metadata server after obtaining exclusive access rights of all metadata in the target directory.
  • Interacting to request the metadata server to operate on the first target metadata reduces the overhead between the nodes and improves the access efficiency of accessing the first target metadata.
  • the first client may also perform steps S316 and S317 to balance the load of each client, specifically, the flowchart is as shown in FIG. 7. Shown. The detailed description of steps S316 and S317 is as follows:
  • Step S316 The first client determines whether the load generated by accessing the metadata in the target directory satisfies a preset condition.
  • the first client after accessing the metadata in the target directory, the first client detects the overall load of the runtime in real time, and compares the load value of the overall load with a preset threshold to determine the overall load of the first client. Whether the preset threshold is exceeded or not, and the preset threshold is a load value used for reference comparison according to actual needs.
  • the first client records the first load generated by the user actively accessing the metadata in the target directory, and the first data generated by accessing the metadata in the target directory according to the request message of other clients.
  • the second client further determines whether the relative size of the first load and the second load meets a preset condition, for example, the first load is smaller than the second load.
  • Step S317 if satisfied, the first client releases exclusive access rights of all metadata in the target directory, so that the directory lock server grants exclusive access rights of all metadata in the target directory. Fourth client.
  • the fourth client is You can apply to the directory lock server for exclusive access to all metadata in the target directory. After the fourth client applies for exclusive access to all metadata in the target directory, the first client needs to access the target directory if it needs to access the target directory. The fourth client comes to visit.
  • the third client and the fourth client in the embodiment of the present invention may be the same client or may not be the same client.
  • steps S303 to S317 may be combined to form another solution, for example, steps S303 to S306 are combined with steps S307 to S310; steps S303 to S306 are combined with steps S311 to S315; and steps S303 to S310 and S316 are combined.
  • S317 is combined; steps S303-S306 are combined with steps S311-S317, etc., and the beneficial effects generated by the combination can be referred to the related descriptions of the above steps, and details are not described herein again.
  • FIG. 5A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention
  • FIG. 5B is a schematic flowchart of interaction between devices in FIG. 5A.
  • FIG. 5A includes a first client 101, a second client 102, and a third client 103 (the number of clients can be deployed according to actual needs), a directory lock server 305, a metadata routing server 303, and a first metadata server 201.
  • the second metadata server 202 and the third metadata server 203 are examples of the metadata server 203.
  • the first client 101 includes a metadata cache unit and a lock unit for accessing the directory, and the metadata cache unit is used for The metadata sent by the metadata server is cached, and the lock unit of the access directory is used to apply exclusive access rights to all metadata in the directory to the lock unit of the directory lock server 305;
  • the metadata routing server 303 includes the routing service unit a home query service for providing a directory for the first client 101;
  • the first metadata server 201, the second metadata server 202, and the third metadata server 203 each include a metadata service unit and a lock unit of the operation directory, and a meta
  • the data service unit is used to implement consistent persistence of metadata, and the lock unit of the operation directory is used.
  • the flow when the first client 101 accesses the path ".../A/B/f" is as shown in steps S501 to S517, wherein the directory B is a directory under the directory A, and the first target metadata f is a plurality of directories B. An item in the metadata.
  • Step S501 The first client 101 requests the metadata routing server 303 for the home information of the directory A.
  • Step S502 The metadata routing server 303 searches the first metadata server 201 to which the directory A belongs from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
  • Step S503 The first client 101 requests the first metadata server 201 for metadata describing the directory B under the directory A.
  • Step S504 The first metadata server 201 sends an application request to the directory lock server 305, requesting the operation of the lock of the directory A (for the convenience of description, may be referred to as "the home lock of the directory A") to obtain the exclusive information of all the metadata in the directory A. access permission.
  • Step S505 the directory lock server 305 grants the home lock of the directory A to the first metadata server 201, so that only the first metadata server 201 of all the metadata servers is provided in the directory A. Operational authority for all metadata.
  • Step S506 The first metadata server 201 reads the metadata describing the directory B from the directory A.
  • Step S507 The first metadata server 201 sends the read metadata describing the directory B to the first client 101.
  • Step S508 The first client 101 analyzes the metadata describing the directory B, and determines whether the directory B is a directory in which a plurality of metadata is stored.
  • Step S509 If it is determined that the directory B is a directory storing a plurality of metadata, the first client 101 sends an application request to the directory lock server 305, requesting access to the lock of the directory B (for the convenience of description, it may be referred to as "the acceleration of the directory B”. Lock”) to get exclusive access to all metadata in directory B.
  • Step S510 When the directory lock server 305 finds that the acceleration lock of the directory B is not granted to other clients, the directory lock server 305 grants the acceleration lock of the directory B to the first client 101, so that only the first client 101 among all the clients Have access to all metadata in directory B. Subsequent steps S511 to S517 are specific processes for the first client to access the metadata in the directory B.
  • Step S511 The first client 101 requests the metadata routing server 303 for the home information of the directory B.
  • Step S512 The metadata routing server 303 searches the second metadata server 202 corresponding to the directory B (that is, the home metadata server 202) from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
  • Step S513 The first client 101 requests the second metadata server 202 to access the first target metadata f under the directory B.
  • Step S514 The second metadata server 202 applies to the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • Step S515 Based on the preemptive distributed lock rule, regardless of whether the home lock of the directory B is granted to other metadata servers, the directory lock server 305 grants the home lock of the directory B to the second metadata server 202, and informs the second The metadata server 202 enables only the second metadata server 202 to have exclusive operational rights to all metadata in the target directory among all metadata servers.
  • Step S516 The second metadata server 202 operates on the first target metadata f in the directory B.
  • Step S517 The second metadata server 202 transmits the operation result of operating the first target metadata f to the first client 101.
  • FIG. 6A is a schematic diagram of a principle of metadata access according to an embodiment of the present invention
  • FIG. 6B is a schematic flowchart of interaction between devices in FIG. 6A.
  • FIG. 6A includes a first client 101, a second client 102, a third client 103 (the number of clients can be deployed according to actual needs), a directory lock server 305, and a metadata routing server 303.
  • the first client 101 includes a metadata cache unit and access.
  • the lock unit of the directory, the metadata service unit, and the lock unit of the operation directory The metadata cache unit is configured to cache the metadata sent by the metadata server, and the lock unit of the access directory is used to apply for exclusive access rights of the directory to the lock unit of the directory lock server 305, and the metadata service unit is used for To implement operations such as reading, writing, and persisting metadata, the lock unit of the operation directory is used to apply the exclusive operation authority of the directory to the lock unit of the operation directory of the directory lock server 305;
  • the metadata routing server 303 includes the routing service unit and The routing switching unit is configured to provide a first query service for the first client 101, and the routing switching unit is configured to attribute the directory to the client.
  • the flow when the first client 101 accesses the path ".../A/B/f" is as shown in steps S601-S615, wherein the directory B is a directory under the directory A, and the first target metadata f is a plurality of directories B. An item in the metadata.
  • Step S601 The first client 101 requests the home information of the directory A from the metadata routing service 303.
  • Step S602 The metadata routing server 303 finds the target metadata server to which the directory A belongs from the mapping relationship between the stored directory and the metadata server, and notifies the first client 101.
  • Step S603 The first client 101 requests the target metadata server for the metadata describing the directory B under the directory A.
  • Step S604 The target metadata server sends an application request to the directory lock server 305, requesting the lock of the operation directory A (referred to as "home lock of directory A" for convenience of description) to obtain exclusive access rights of all metadata in the directory A. .
  • Step S605 The directory lock server 305 grants the home lock of the directory A to the target metadata server, so that only the target metadata server of all the metadata servers has the operation authority for all the metadata in the directory A.
  • Step S606 The target metadata server reads the metadata describing the directory B from the directory A.
  • Step S607 The target metadata server sends the read metadata describing the directory B to the first client 101.
  • Step S608 The first client 101 analyzes the metadata describing the directory B, and determines whether the directory B is a directory in which a plurality of metadata is stored.
  • Step S609 If it is determined that the directory B is a directory storing a plurality of metadata, the first client 101 sends an application request to the directory lock server 305, requesting access to the lock of the directory B (for the convenience of description, it may be referred to as "the acceleration of the directory B”. Lock”) to get exclusive access to all metadata in directory B.
  • Step S610 When the directory lock server 305 finds that the acceleration lock of the directory B is not granted to other clients, the directory lock server 305 grants the acceleration lock of the directory B to the first client 101, so that only the first client 101 among all the clients Have access to all metadata in directory B. Subsequent steps S611 to S615 are specific processes in which the first client 101 accesses the metadata in the directory B.
  • Step S611 The first client 101 sends a request message to the metadata routing server 303, requesting the metadata routing server to modify the correspondence between the directory and the target metadata server, and mapping the attribution of the directory B to the first client 101.
  • Step S612 The metadata routing server 303 modifies the correspondence between the directory and the target metadata server, attributes the directory B to the first client 101, and notifies the first client 101.
  • Step S613 The first client 101 applies to the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • the directory lock server 305 for the lock of the operation directory B (ie, the home lock of the directory B) to obtain the exclusive operation authority of all the metadata in the directory B.
  • Step S614 Based on the preemptive distributed lock rule, regardless of whether the home lock of the directory B is granted to other target metadata servers, the directory lock server 305 grants the home lock of the directory B to the first client 101, and informs the first The client 101 is such that only the first client 101 itself has the operational authority to the metadata in the target directory.
  • Step S615 The first client 101 operates on the first target metadata f process in the directory B, and obtains an operation result.
  • steps S601-S615 may be referred to the steps S302-S304 and the descriptions of the steps S311-S315, and details are not described herein again.
  • FIG. 8 is a schematic structural diagram of a terminal 80 according to an embodiment of the present invention.
  • the terminal 80 is a first client 80, which may include a receiving device 801, a sending device 802, and a memory 803.
  • the processor 804 (the processor may be one or more, and one processor in FIG. 8 is taken as an example), the receiving device 801, the transmitting device 802, the memory 803 and the processor 804 are respectively connected to the bus 805, wherein the memory 803 A set of metadata access program code is stored, and the processor 804 calls the metadata access program code stored in the memory 803 for performing the following operations:
  • the directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the processor 804 currently needs to access; exclusive access rights of all metadata in the target directory are used for the processing
  • the device 804 accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
  • the first target metadata is accessed when applying for exclusive access to all metadata in the target directory.
  • the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the directory lock server coordinates the access rights of each client to the target directory by granting a lock to the target directory, if the directory lock server will access The lock of the target directory is granted to the processor 804, then the processor 804 has the right to access all the metadata in the target directory; further, the distributed lock service can be set to preemptive distributed locks and non-preemptable distributed locks; For a preemptible distributed lock, when the processor 804 requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the directory lock server releases the lock accessing the target directory according to the preemption mechanism, and re- The lock accessing the target directory is granted to the processor 804.
  • a distributed lock service such as a server carrying the zookeeper service
  • the directory lock server directly grants the lock accessing the target directory to the processor 804; if it is set to a non-preemptable distributed lock,
  • the directory server will be based on the lock mechanism can not be seized, the lock access to the target directory does not grant the first client 80, if access to the target directory The lock is not granted to other clients, and the directory lock server directly grants the lock to the target directory to the processor 804.
  • the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the processor 804 applies for the lock to access the target directory, it has the row of all the metadata in the target directory. It has access rights, that is, other clients than the processor 804 have no access to the metadata in the target directory unless the processor 804 actively releases the lock accessing the target directory.
  • the processor 804 performs the foregoing operation, and when accessing the first target metadata in the target directory, requests the directory lock server for exclusive access rights of all metadata in the target directory, so that the first client 80 can Successful access to the first target metadata, and subsequent access to other metadata in the target directory does not need to re-apply access rights, saving the overhead of maintaining cache coherency.
  • the receiving device 801 receives the node where the second client sent by the directory lock server is located. And sending, by the sending device 802, a request message to the second client according to the node information to access metadata in the target directory by using the second client, where the second client is The directory lock server applies to a client that has exclusive access to all metadata in the target directory.
  • the processor 704 by performing the above operation, accesses the first target element through the second client that has acquired exclusive access rights of all metadata in the target directory when the application does not obtain exclusive access rights of all metadata in the target directory.
  • the data enables the first client 80 to successfully access the target metadata when the application does not have exclusive access to all metadata in the target directory.
  • the manner in which the processor 804 sends the first application message to the directory lock server by using the sending device 802 to apply for exclusive access rights to all metadata in the target directory is specifically:
  • the first application message is sent to the directory lock server by the sending device 802 to apply for exclusive access rights to all metadata in the target directory.
  • the processor 804 determines whether the target directory is a directory for storing multiple metadata by acquiring metadata describing the target directory, and adding a “directory attribute” based on the existing metadata.
  • the value of the "directory attribute" indicates that the target directory is a different attribute, and one attribute is used to indicate that the target directory is a directory storing a plurality of metadata, and accordingly, the processor 804 analyzes the "directory attribute". It can be concluded whether the target directory is a directory that stores multiple metadata.
  • the processor 804 further uses to:
  • the processor 804 obtains the exclusive access permission of all the metadata in the target directory by performing the above operation, if the third client has the requirement of accessing the first target metadata in the target directory, the access target is provided for the third client.
  • the service of the first target metadata in the directory ensures that the client that does not apply for exclusive access to all metadata in the target directory can also access the first target metadata normally.
  • the processor 804 accesses the first target metadata
  • the processor 804 is requested to apply exclusive access rights to all metadata in the target directory. Also used for:
  • the second target metadata is accessed in accordance with the obtained exclusive access rights to the second target metadata.
  • the processor 804 can directly access the second target metadata when accessing other metadata (such as the second target metadata) in the target directory, and does not need to apply for access permission again, thereby improving the processor. 804 access to metadata in the target directory is efficient.
  • the manner in which the processor 804 accesses the first target metadata is specifically:
  • the receiving device 801 Receiving, by the receiving device 801, an operation result sent by the target metadata server; the operation result is that the target metadata server applies to the directory lock server for exclusive operation on all metadata in the target directory. After the permission, the result of the operation of the first target metadata.
  • the target metadata server After the processor 804 performs the above operation, after receiving the request of the first target metadata in the target directory sent by the processor 804, the target metadata server performs the operation on the first target metadata in the target directory.
  • the directory lock server applies for the exclusive operation permission of all the metadata in the target directory, so that only one metadata server in all the metadata servers has the operation authority for the target directory, and the metadata of the target directory is consistently saved. Sex.
  • the manner in which the processor 804 accesses the first target metadata is specifically:
  • the notification message sent by the metadata routing server is used to indicate that the operation authority of the target directory belongs to the first client 80;
  • Receiving, by the receiving device 801, a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for
  • the processor 804 operates the first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata;
  • the first target metadata is operated according to the exclusive operation authority of all the metadata in the target directory, and the operation result is obtained.
  • the processor 804 performs the above operations to enable the first client 80 to function as both the client and the metadata server. After the processor 804 obtains exclusive access rights to all metadata in the target directory, the remote client does not need to be associated with the remote terminal.
  • the data server interacts to request the metadata server to operate on the first target metadata, reducing the overhead between the nodes, and improving the access efficiency of accessing the first target metadata.
  • the processor 804 after the processor 804 accesses the first target metadata, when the processor 804 accesses the exclusive access rights to all the metadata in the target directory, the processor 804 further uses to:
  • the processor 804 detects the load generated by accessing the metadata in the target directory in real time, and the detection manner includes but is not limited to the following two cases:
  • the processor 804 detects the overall load of the first client 80 in real time, and compares the load value of the overall load with a preset threshold to determine whether the overall load of the first client 80 exceeds a preset.
  • Threshold value which is a load value used for reference comparison according to actual needs.
  • the processor 804 records the first load generated by the first client 80 actively accessing the metadata in the target directory, and accesses the metadata in the target directory according to the request message of other clients.
  • the processor 804 can also refer to the corresponding description of the embodiment of the embodiment of the present invention, and details are not described herein again.
  • the processor 804 automatically releases the exclusive access rights of all the metadata in the target directory when the load on the first client 80 is too high, and allows the fourth client with a lower load to obtain the target directory.
  • Exclusive access to all metadata for load sharing purposes; or access to metadata in the target directory at the first client is more frequent than other clients accessing the target directory
  • the exclusive access permission of all metadata in the target directory is automatically released, so that the client with higher access frequency obtains exclusive access rights of all metadata in the target directory, so as to reduce the load generated by forwarding access. Improves the efficiency of accessing metadata in the target directory.
  • FIG. 9 is a schematic structural diagram of still another terminal 90 according to an embodiment of the present invention.
  • the terminal 90 is a directory lock server 90, which may include a receiving device 901, a sending device 902, and a memory 903.
  • the processor 904 (the processor may be one or more, and one processor in FIG. 9 is taken as an example), the receiving device 901, the transmitting device 902, the memory 903 and the processor 904 are respectively connected to the bus 905, wherein the memory 903 A set of metadata access program code is stored, and the processor 904 calls the metadata access program code stored in the memory 903 for performing the following operations:
  • a first application message sent by the first client where the first client accesses the target metadata in the target directory, where the first application message is used to apply to the target directory.
  • Exclusive access rights of all metadata the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the second client If the second client is not granted, grant exclusive access rights of all metadata in the target directory to the first client; if the second client is granted, pass the sending device 902 Sending the node information of the second client to the first client, so that the first client accesses the target element by using the second client based on the node information of the second client data.
  • the directory lock server can provide a distributed lock service, such as a server carrying the zookeeper service; the processor 904 can coordinate the access of each client to the target directory by granting a lock accessing the target directory based on the distributed lock service.
  • a distributed lock service such as a server carrying the zookeeper service
  • the processor 904 Permission, if the processor 904 grants the lock accessing the target directory to the first client, the first client has the right to access all the metadata in the target directory; further, the distributed lock service can be set to preemptively distribute Lock and non-preemptive distributed lock; if set to preemptible distributed lock, when the first client requests to access the lock of the target directory, if the lock accessing the target directory has been granted to other clients, the processor 904 will preempt according to the preemption The mechanism releases the lock accessing the target directory, and re-grants the lock accessing the target directory to the first client. If the lock accessing the target directory is not granted to other clients, the processor 904 directly grants the lock accessing the target directory to the first.
  • the processor 904 if set to a non-preemptable distributed lock, the first client requests access to the target When the lock is recorded, if the lock accessing the target directory has been granted to other clients, the processor 904 will not grant the lock to the target client to the first client according to the non-preemptive mechanism, if the lock of the target directory is not accessed. Granting to other clients, the processor 904 directly grants the lock accessing the target directory to the first client.
  • the lock of the access target directory may be set as a non-preemptable distributed lock, so that after the first client applies for the lock to access the target directory, it has all the metadata in the target directory. Exclusive access rights, that is, other customers besides the first client The terminal does not have access to the metadata in the target directory unless the first client actively releases the lock accessing the target directory.
  • the processor 904 performs the above operation, when the first client accesses the target metadata in the target directory, the processor 904 grants all the exclusive access rights of all the metadata in the target directory to the first client at one time to maintain the cache consistency. Sex, in this way, the first client can successfully access the target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • the processor 904 grants exclusive access rights of all metadata in the target directory to the first client, the processor 904 is also used to:
  • the end accesses the target metadata.
  • the processor 904 After the processor 904 performs the above operation, after the first client obtains the exclusive access right of all the metadata in the target directory, if the third client has the requirement to access the target directory target metadata, the third client may be based on the processing.
  • the node information sent by the device 904 sends the access request to the first client, and accesses the target metadata through the first client, so that the client that does not apply for exclusive access rights of all metadata in the target directory can also be normal. Access target metadata.
  • the processor 904 grants exclusive access rights of all metadata in the target directory to the first client, the processor 904 is also used to:
  • the target metadata server receives the request for the target metadata in the target directory sent by the first client, and then proceeds to the processor 904 before operating the target metadata in the target directory. Approve exclusive access rights for all metadata in the target directory. Accordingly, the processor 904 grants the exclusive operation authority of all metadata in the target directory to the target metadata server, so that there is always only one metadata server in all the metadata servers. It has the operation permission on the target directory, which ensures the persistence and preservation consistency of the metadata in the target directory.
  • the processor 904 will After the exclusive access rights of all the metadata in the target directory are granted to the first client, the processor 904 is further configured to:
  • processor 904 for performing the above operations may also refer to the corresponding description of the embodiment of the embodiment of the present invention, and details are not described herein again.
  • the processor 904 performs the above operation, and after the first client simultaneously functions as the client and the metadata server, the processor 904 grants exclusive access rights to all metadata in the target directory of the first client, first The client does not need to interact with the remote metadata server to request the metadata server to operate on the target metadata, which reduces the overhead between the nodes and improves the access efficiency of the access target metadata.
  • FIG. 10 is a schematic structural diagram of a distributed file system 100 according to an embodiment of the present invention.
  • the distributed file system 100 includes a first client 83, a directory lock server 84, and a second client 85. :
  • the first client 83 sends a first application message to the directory lock server 84 to request exclusive access rights to all metadata in the target directory when requesting access to the first target metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the The first client accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
  • the directory lock server 84 receives the first application message sent by the first client 83, and determines whether the exclusive access rights of all metadata in the target directory are granted to the second client 85;
  • the directory lock server 84 grants the exclusive access rights of all metadata in the target directory to the first a client 83, and sending a notification message to the first client to notify the first client of exclusive access rights of all metadata in the target directory;
  • the first client 83 receives the notification message and accesses the first target metadata according to the notification message.
  • the first client 83 in the distributed file system 100 can refer to the terminal 80 described in the embodiment shown in FIG. 8, and the directory lock server 84 can refer to the terminal 90 described in the embodiment shown in FIG. No longer repeat them.
  • the directory lock server 84 is applied for exclusive access rights of all metadata in the target directory at one time, so that The first client 83 can successfully access the first target metadata, and does not need to re-apply for access rights when accessing other metadata in the target directory, which saves the overhead of maintaining cache consistency.
  • FIG. 11 is a schematic structural diagram of still another terminal 110 according to an embodiment of the present invention.
  • the terminal 110 is a first client 110, which includes an application unit 1101 and a first access unit 1103.
  • the description of each unit is as follows:
  • the application unit 1101 is configured to, when requesting access to the first target metadata in the target directory, send a first application message to the directory lock server to apply for exclusive access rights to all metadata in the target directory, wherein the target The directory is a directory in which a plurality of metadata is stored, the first target metadata is metadata that the first client currently needs to access; exclusive access rights of all metadata in the target directory are used for the The first client accesses the first target metadata in the target directory, and accesses metadata in the target directory other than the first target metadata;
  • the first access unit 1103 is configured to access the first target metadata when the application unit 1101 applies for exclusive access rights to all metadata in the target directory.
  • the first access unit 1103 is further configured to: when the application unit 1101 does not apply for exclusive access to all metadata in the target directory, receive the directory lock server to send Node information of the second client, and sending a request message to the second client according to the node information to access metadata in the target directory by the second client, the second client A client that requests exclusive access to all metadata in the target directory to the directory lock server.
  • the application unit 1101 includes an acquisition subunit, a determination subunit, and an application subunit, where:
  • Obtaining a subunit configured to obtain metadata describing the target directory, where the metadata describing the target directory is provided with identifier information for identifying whether the target directory is a directory in which a plurality of metadata is stored;
  • a determining subunit configured to determine, according to the metadata describing the target directory, whether the target directory is a directory in which a plurality of metadata is stored;
  • the application subunit is configured to send a first application message to the directory lock server to request exclusive access rights to all metadata in the target directory when the judgment result of the subunit is YES.
  • the terminal 110 further includes:
  • a receiving unit configured to receive a request message sent by the third client, where the request message is used to request to access the first target metadata by using the first access unit 1103, where the request message is the third client Applying, to the directory lock server, exclusive access rights to all metadata in the target directory, and receiving node information of the first client returned by the directory lock server, according to the node of the first client a message sent by the information to the first client;
  • a sending unit configured to access the first target metadata according to the request message, and send an access result of accessing the first target metadata to the third client.
  • the terminal 110 further includes:
  • a second access unit configured to: when requesting access to the second target metadata in the target directory, if the second target metadata belongs to metadata in the target directory other than the first target metadata;
  • the first access unit 1103 includes:
  • a requesting subunit configured to request, by the metadata routing server, attribution information of the target directory, where the metadata routing server prestores the attribution information of the directory;
  • a first receiving subunit configured to receive the home information of the target directory sent by the metadata routing server, where the home information of the target directory is used to indicate that the operation authority of the target directory belongs to the target metadata server;
  • a first sending subunit configured to send an operation request message to the target metadata server, requesting the target metadata server to operate the first target according to the exclusive access right to all metadata in the target directory Metadata
  • a second receiving subunit configured to receive an operation result sent by the target metadata server; the operation result is that the target metadata server requests the directory lock server to queue all metadata in the target directory The result of the operation of the first target metadata after it operates the authority.
  • the first access unit 1103 includes:
  • a second sending subunit configured to send a switching message to the metadata routing server, instructing the metadata routing server to attribute the operation authority of the target directory to the first client;
  • a third receiving subunit configured to receive a notification message sent by the metadata routing server, where the notification message is used to indicate that the operation authority of the target directory belongs to the first client;
  • An application subunit configured to send a second application message to the directory lock server to apply for exclusive operation authority on all metadata in the target directory
  • a fourth receiving subunit configured to receive a grant message sent by the directory lock server to grant exclusive operation rights of all metadata in the target directory, and exclusive operation rights of all metadata in the target directory are used for
  • the first client operates first target metadata in the target directory, and operates metadata in the target directory other than the first target metadata
  • An operation subunit configured to operate the first target metadata according to an exclusive operation authority of all metadata in the target directory, to obtain an operation result.
  • the terminal 110 further includes:
  • a determining unit configured to determine whether a load generated by accessing metadata in the target directory meets a preset condition
  • a release unit configured to release exclusive access rights of all metadata in the target directory when the determination result of the determining unit is yes, so that the directory lock server will all metadata in the target directory Exclusive access is granted to the fourth client.
  • the first client (terminal) 110 accesses the first target metadata in the target directory, it applies to the directory lock server for exclusive access rights of all metadata in the target directory at one time.
  • the first client can successfully access the first target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • FIG. 12 is still another terminal 120 according to an embodiment of the present invention.
  • the terminal 120 is a directory lock server 120, which includes a first receiving unit 1201, a determining unit 1202, and a first granting unit. 1203, wherein each unit is described as follows.
  • the first receiving unit 1201 is configured to receive, when the first client accesses the target metadata in the target directory, the first application message sent by the first client, where the first application message is used to apply for the target directory.
  • Exclusive access rights of all the metadata in the directory the target directory is a directory in which a plurality of metadata is stored, and the target metadata is metadata that the first client currently needs to access;
  • the determining unit 1202 is configured to determine, according to the first application message, whether an exclusive access right of all metadata in the target directory has been granted to the second client, and exclusive access rights of all metadata in the target directory are used for Providing, by the first client, target metadata in the target directory, and accessing metadata in the target directory other than the target metadata;
  • the first granting unit 1203 is configured to, when the determining result of the determining unit 1202 is negative, grant exclusive access rights of all metadata in the target directory to the first client; in the judgment of the determining unit 1202
  • the node information of the second client is sent to the first client, so that the first client passes the second client based on the node information where the second client is located. Access the target metadata.
  • the terminal 120 further includes:
  • a second receiving unit configured to receive the first application message sent by the third client when the third client accesses the target metadata
  • a first sending unit configured to send, to the third client, node information where the first client is located, so that the third client passes the first node based on node information where the first client is located
  • the client accesses the target metadata.
  • the terminal 120 further includes:
  • a third receiving unit configured to receive a second application message sent by the target metadata server, where the second application message is an operation sent by the target metadata server when the first client accesses the target metadata a message sent during the message to apply for exclusive operation rights to all metadata in the target directory;
  • a second granting unit configured to grant an exclusive operation authority of all metadata in the target directory to the target metadata server according to the second application message, so that the target metadata server pairs the target metadata Performing an operation and feeding back an operation result to the first client, wherein an exclusive operation authority of all metadata in the target directory is used for the target metadata server to operate target metadata in the target directory, And operating metadata in the target directory other than the target metadata.
  • the terminal 120 further includes:
  • a fourth receiving unit configured to receive a second application message sent by the first client, where the second The application message is used to apply for exclusive operation rights on all metadata in the target directory;
  • a third granting unit configured to grant an exclusive operation authority of all metadata in the target directory to the first client according to the second application message, so that the first client pairs the target metadata Performing operations, wherein exclusive operation rights of all metadata in the target directory are used by the first client to operate target metadata in the target directory, and operating the target directory except the target element Metadata outside of the data.
  • the directory lock server 120 For the principle of accessing the metadata of the terminal (the directory lock server) 120, reference may be made to the corresponding description of the embodiment of the present invention corresponding to FIG. 3 to FIG. 7 , and details are not described herein again.
  • the directory lock server is requested to access all the metadata in the target directory at a time, so that the first client The target metadata can be successfully accessed, and subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache coherency.
  • the first client accesses the first target metadata in the target directory
  • the first time accesses the directory lock server to the exclusive access permission of all the metadata in the target directory, so that the first client accesses the first target metadata in the target directory.
  • the first client can successfully access the first target metadata, and the subsequent access to other metadata in the target directory does not need to re-apply for access rights, saving the overhead of maintaining cache consistency.
  • the third client sends the access request to the The first client accesses the first target metadata through the first client, so that the client that does not apply for exclusive access rights of all metadata in the target directory can also normally access the first target metadata;
  • the metadata server After receiving the request for the first target metadata in the access target directory sent by the first client, the metadata server first applies to the directory lock server for all metadata rows in the target directory before performing the operation on the first target metadata.
  • the first client It has the right to operate, so that only one metadata server in all metadata servers has the operation authority to the target directory, which ensures the persistence and preservation consistency of the metadata in the target directory; further, the first client also acts as the client at the same time. End and metadata server functions, the first client gets exclusive access to all metadata in the target directory After the limitation, there is no need to interact with the remote metadata server to request the metadata server to operate on the first target metadata, thereby reducing the overhead between the nodes and improving the access efficiency of accessing the first target metadata; further Ground, when the load on the first client is too high, the exclusive access rights of all metadata in the target directory are automatically released, and the fourth client with lower load is obtained to obtain exclusive access rights of all metadata in the target directory.
  • Permissions allow clients with higher frequency of access to obtain exclusive access to all metadata in the target directory to reduce the load generated by forwarding access and improve the efficiency of accessing metadata in the target directory.
  • the readable storage medium when executed, may include the flow of an embodiment of the methods as described above.
  • the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Storage Device Security (AREA)
  • Human Computer Interaction (AREA)

Abstract

Un mode de réalisation de l'invention concerne un procédé d'accès à des métadonnées, un appareil associé et un système. Le procédé consiste : lors d'une demande d'accès à un premier élément de métadonnées cible dans un répertoire cible, à transmettre, par l'intermédiaire d'un premier client, un premier message d'application à un serveur de verrouillage de répertoire pour demander une autorisation d'accès exclusif à tous les éléments de métadonnées dans le répertoire cible, le répertoire cible étant un répertoire stockant de multiples éléments de métadonnées, et l'autorisation d'accès exclusif à tous les éléments de métadonnées dans le répertoire cible étant configurée pour permettre au premier client d'accéder au premier élément de métadonnées cible, ainsi qu'aux éléments de métadonnées autres que le premier élément de métadonnées cible dans le répertoire cible ; et, une fois l'autorisation accordée, à accéder au premier élément de métadonnées cible dans le répertoire cible. Selon l'invention, le fait d'obtenir l'autorisation d'accès exclusif une fois donne accès à tous les éléments de métadonnées dans le répertoire cible, ce qui permet de réduire les temps système requis pour maintenir une cohérence de mémoire tampon.
PCT/CN2016/107979 2015-12-04 2016-11-30 Procédé d'accès à des métadonnées, appareil associé et système WO2017092673A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510888579.4 2015-12-04
CN201510888579.4A CN105512279B (zh) 2015-12-04 2015-12-04 一种元数据访问方法、相关设备及系统

Publications (1)

Publication Number Publication Date
WO2017092673A1 true WO2017092673A1 (fr) 2017-06-08

Family

ID=55720261

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/107979 WO2017092673A1 (fr) 2015-12-04 2016-11-30 Procédé d'accès à des métadonnées, appareil associé et système

Country Status (2)

Country Link
CN (1) CN105512279B (fr)
WO (1) WO2017092673A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112449018A (zh) * 2020-11-25 2021-03-05 杭州视洞科技有限公司 一种基于端侧的多机房数据同步与部署方法

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105512279B (zh) * 2015-12-04 2019-05-03 华为技术有限公司 一种元数据访问方法、相关设备及系统
CN109510846B (zh) * 2017-09-14 2020-11-03 北京金山云网络技术有限公司 Api调用系统、方法、装置、电子设备及存储介质
CN111049779B (zh) * 2018-10-12 2022-04-26 伊姆西Ip控股有限责任公司 信息处理方法、电子设备以及计算机可读介质
CN110347650B (zh) * 2019-07-16 2021-06-01 北京明略软件系统有限公司 一种元数据采集方法及装置
CN110941591A (zh) * 2019-11-22 2020-03-31 浪潮电子信息产业股份有限公司 一种文件删除方法、装置、设备及可读存储介质
CN111046001B (zh) * 2019-12-28 2023-03-14 浪潮电子信息产业股份有限公司 一种批量创建文件的方法、装置、设备及存储介质
CN115250270B (zh) * 2022-07-28 2024-02-23 中国银行股份有限公司 基于5g消息的文件申请方法及相关设备
CN117591038B (zh) * 2024-01-18 2024-06-11 济南浪潮数据技术有限公司 一种数据访问方法、装置、分布式存储系统及设备和介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101375250A (zh) * 2006-02-03 2009-02-25 甲骨文国际公司 自适应区域锁定
CN103501339A (zh) * 2013-09-30 2014-01-08 华为技术有限公司 元数据处理方法及元数据服务器
US8850126B2 (en) * 2010-01-08 2014-09-30 Netapp, Inc. Exclusive access during a critical sub-operation to enable simultaneous operations
CN104113587A (zh) * 2014-06-23 2014-10-22 华中科技大学 一种分布式文件系统客户端元数据缓存优化方法
CN105512279A (zh) * 2015-12-04 2016-04-20 华为技术有限公司 一种元数据访问方法、相关设备及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101375250A (zh) * 2006-02-03 2009-02-25 甲骨文国际公司 自适应区域锁定
US8850126B2 (en) * 2010-01-08 2014-09-30 Netapp, Inc. Exclusive access during a critical sub-operation to enable simultaneous operations
CN103501339A (zh) * 2013-09-30 2014-01-08 华为技术有限公司 元数据处理方法及元数据服务器
CN104113587A (zh) * 2014-06-23 2014-10-22 华中科技大学 一种分布式文件系统客户端元数据缓存优化方法
CN105512279A (zh) * 2015-12-04 2016-04-20 华为技术有限公司 一种元数据访问方法、相关设备及系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112449018A (zh) * 2020-11-25 2021-03-05 杭州视洞科技有限公司 一种基于端侧的多机房数据同步与部署方法

Also Published As

Publication number Publication date
CN105512279A (zh) 2016-04-20
CN105512279B (zh) 2019-05-03

Similar Documents

Publication Publication Date Title
WO2017092673A1 (fr) Procédé d'accès à des métadonnées, appareil associé et système
US10958752B2 (en) Providing access to managed content
US10983868B2 (en) Epoch based snapshot summary
US8086634B2 (en) Method and apparatus for improving file access performance of distributed storage system
WO2019057212A1 (fr) Procédé, appareil et dispositif permettant de planifier un service dans un nœud cdn, et support de stockage
US20160048430A1 (en) Method of operating a shared nothing cluster system
KR20120072909A (ko) 내용 기반 중복 방지 기능을 가지는 분산 저장 시스템 및 그 오브젝트 저장 방법 및 컴퓨터에 의하여 독출가능한 저장 매체
US10742748B2 (en) System and method for supporting live addition of a tenant in a connection pool environment
US10262024B1 (en) Providing consistent access to data objects transcending storage limitations in a non-relational data store
US20100161585A1 (en) Asymmetric cluster filesystem
US20200026689A1 (en) Limited deduplication scope for distributed file systems
US11809377B2 (en) Global data deduplication across multiple distributed file systems
CN107493309B (zh) 一种分布式系统中的文件写入方法及装置
CN107992340B (zh) 用于分布式集群系统的系统退出方法及分布式集群系统
US9607072B2 (en) System and method for implementing nested relationships within a schemaless database
CN109844723B (zh) 使用基于服务的统计信息进行主控建立的方法和系统
US11580152B1 (en) Using path-based indexing to access media recordings stored in a media storage service
US10516628B2 (en) Transfer device, transfer system, and transfer method
KR102214185B1 (ko) 자원 획득 방법
JP2017123040A (ja) サーバー装置、分散ファイルシステム、分散ファイルシステム制御方法、および、プログラム
KR102033383B1 (ko) 분산데이터 환경에서의 데이터 관리방법 및 시스템
US10621148B1 (en) Maintaining multiple object stores in a distributed file system
CN117075823B (zh) 对象查找方法、系统、电子设备及存储介质
US20230176908A1 (en) Systems, methods and computer program products for job management
US11169864B2 (en) Systems, methods and computer program products for application environment synchronization between remote devices and on-premise devices

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16869983

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16869983

Country of ref document: EP

Kind code of ref document: A1