WO2023093245A1 - 基于分布式文件系统的元数据查询方法、设备和存储介质 - Google Patents

基于分布式文件系统的元数据查询方法、设备和存储介质 Download PDF

Info

Publication number
WO2023093245A1
WO2023093245A1 PCT/CN2022/120227 CN2022120227W WO2023093245A1 WO 2023093245 A1 WO2023093245 A1 WO 2023093245A1 CN 2022120227 W CN2022120227 W CN 2022120227W WO 2023093245 A1 WO2023093245 A1 WO 2023093245A1
Authority
WO
WIPO (PCT)
Prior art keywords
directory
modification operation
metadata
path
metadata management
Prior art date
Application number
PCT/CN2022/120227
Other languages
English (en)
French (fr)
Inventor
黎海兵
Original Assignee
北京百度网讯科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京百度网讯科技有限公司 filed Critical 北京百度网讯科技有限公司
Publication of WO2023093245A1 publication Critical patent/WO2023093245A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/14—Details of searching files based on file metadata
    • G06F16/148—File search processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G06F16/134—Distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/172—Caching, prefetching or hoarding of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems

Definitions

  • the present disclosure relates to the field of distributed file system technology and the field of data storage technology, for example, to a metadata query method, device, electronic device and computer-readable storage medium based on a distributed file system.
  • Metadata refers to data that describes data, mainly information that describes data attributes, and is used to support functions such as indicating storage locations, searching historical data and resources, and document records.
  • metadata is an important unit to describe the file system.
  • metadata query requests account for more than half of all requests. Therefore, improving the metadata query efficiency in the distributed file system and reducing the metadata query delay have an extremely important impact on the performance of the entire file system.
  • the present disclosure provides a metadata query method, device, device and storage medium based on a distributed file system.
  • the present disclosure provides a metadata query method based on a distributed file system, including:
  • the page replacement cache is adjusted according to the directory modification operation of the client.
  • the present disclosure provides a metadata query device based on a distributed file system, including:
  • the metadata query request acquisition module is configured to obtain the metadata query request of the file to be queried by the client;
  • the parent index node identification determination module is configured to, in response to the metadata query request, determine whether the parent index node identification of the parent directory path to which the file to be queried belongs exists in the page replacement cache of the metadata management node;
  • the metadata query module is configured to query the file to be queried from the metadata database according to the parent index node ID and the file name of the file to be queried when the parent index node identifier exists in the page replacement cache metadata;
  • the page replacement cache is adjusted according to the directory modification operation of the client.
  • the present disclosure provides an electronic device, including:
  • the at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to Enabling the at least one processor to execute the distributed file system-based metadata query method provided by any embodiment of the present disclosure.
  • the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the distributed file system-based metadata query method provided by any embodiment of the present disclosure .
  • the present disclosure provides a computer program product, including a computer program.
  • the computer program When the computer program is executed by a processor, the distributed file system-based metadata query method provided by any embodiment of the present disclosure is implemented.
  • FIG. 1 is a schematic diagram of a metadata query method based on a distributed file system provided according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of another metadata query method based on a distributed file system according to an embodiment of the present disclosure
  • Fig. 3 is a schematic diagram of a metadata query device based on a distributed file system provided according to an embodiment of the present disclosure
  • Fig. 4 is a block diagram of an electronic device for implementing the method for querying metadata based on a distributed file system according to an embodiment of the present disclosure.
  • the architecture of a distributed file system generally includes the following parts: client, metadata management cluster, metadata database, and storage cluster.
  • the client is compiled into the business process in the form of a library, and the client provides related access interfaces of the file system to the business, such as: creating folder/file interface, opening/closing file interface, subdirectory/subfile enumeration interface under the directory , file data reading interface and file data writing interface, etc.
  • the metadata management cluster is composed of multiple metadata management nodes (master), each metadata management node is completely equal, there is no distinction between master and slave, metadata management Nodes can be expanded horizontally.
  • the metadata management node is used to manage directory tree, index node (inode) information of folder/file, file data block information and mapping information of data block to storage node.
  • the metadata management node provides functions such as metadata query and modification.
  • the metadata management node does not persist metadata such as directory trees locally, and all metadata is persisted in the external metadata database.
  • the metadata database is used to persist all metadata of the distributed file system, such as directory trees, file index node information, file data block information, and data block-to-storage node mapping information.
  • a storage cluster consists of multiple storage nodes and is used to store file data in a distributed file system. The data of each file is horizontally divided into multiple data blocks (blocks), and each storage node stores data blocks of multiple files.
  • the metadata management node queries the index node ID (inodeid) of a file with a path depth of N.
  • index node ID inodeid
  • N path depth
  • Cache acceleration query method set up a cache on the metadata management node.
  • the inodeid of a file needs to be queried, first obtain the parent directory index node ID of the file directory path from the cache, and then execute N times of primary key queries concurrently to the metadata database. If all sub-requests return success, the query is successful. If a sub-request fails, the query fails. For ease of understanding, continue the above example.
  • querying the index node identifier of a file with a path depth of N requires querying the metadata database N times, and the query process is complicated. Based on this, the present disclosure provides a metadata query solution based on a distributed file system.
  • Fig. 1 is a schematic diagram of a method for querying metadata based on a distributed file system according to an embodiment of the present disclosure.
  • the embodiment of the present disclosure is applicable to the case of querying metadata in a distributed file system.
  • the method can be executed by a metadata query device based on a distributed file system, and the device can be implemented in hardware and/or software, and can be configured in electronic equipment. With reference to Fig. 1, this method comprises the steps:
  • Metadata refers to data that describes data, mainly information that describes data attributes, and is used to support functions such as indicating storage locations, searching historical data and resources, and document records.
  • the metadata of the file to be queried is information used to describe the attributes of the file to be queried.
  • the metadata query request is generated by the client, and is used to request metadata of the file to be queried from the metadata management node.
  • the metadata query request of the file to be queried includes an identifier of the file to be queried, such as a file name.
  • the file to be queried may be a file or a folder.
  • Obtain the metadata query request of the file to be queried by the client for example, the metadata management node obtains the metadata query request of the file to be queried by the client.
  • a page replacement cache refers to a cache structure designed using a page replacement cache algorithm, where the page replacement cache algorithm can be The Least Recently Used (LRU).
  • the page replacement cache is set in the metadata management node and maintained by the metadata management node.
  • the parent index node identifier of the parent directory path to which the file most recently accessed by the metadata management node belongs is stored in the page replacement cache.
  • the metadata management node determines whether the parent index node identifier of the parent directory path to which the file to be queried belongs exists in the page replacement cache maintained by itself.
  • the page replacement cache is adjusted according to the directory modification operation of the client. For example, directory paths included in the page replacement cache are adjusted based on client-side directory modification operations.
  • the directory modification operation of the client refers to the modification operation performed by the client on a non-empty directory.
  • the directory modification operation includes a rename operation or a deletion operation.
  • the metadata management node adjusts the index node identifier of the directory path in the page replacement cache maintained by itself according to the directory modification operation of the client, ensuring that the index node identifier of the directory path in the page replacement cache of the metadata management node is consistent with the directory modification of the client.
  • the consistency of the operation ensures the accuracy of the index node identification of the directory path in the page replacement cache of the metadata management node.
  • the metadata of each file is stored in the metadata database in the form of key-value pairs.
  • the key value is composed of the parent index node identifier and the file identifier of the parent directory path of the file.
  • the value value contains the metadata of the file such as the inode ID and creation time.
  • the index node identifier (inodeid) is globally unique identifier information allocated for each file by the distributed file system. The index node identifier is used to distinguish different files in the distributed system, and the index node identifiers of different files are different.
  • the metadata management node When querying the metadata of a file in the distributed file system, the metadata management node will obtain the parent index node ID of the parent directory path of the file to be queried from the page replacement cache maintained by itself, and then according to the parent index node ID of the parent directory path and the identifier of the file to be queried to query the inode identifier of the file to be queried from the metadata database. If the page replacement cache maintained by the metadata management node itself has the parent index node identifier of the parent directory path to which the file to be queried belongs, the metadata management node directly selects the parent index node identifier from the metadata database according to the parent index node identifier and the file name of the file to be queried. Query the metadata of the file to be queried.
  • the metadata database is used to persist all metadata of the distributed file system, such as directory trees, file index node information, file data block information, and data block-to-storage node mapping information.
  • the metadata database is a NewSQL database.
  • the metadata management node needs to use the serial step-by-step query method to query the metadata of the file to be queried in the metadata database.
  • the metadata query request of the file to be queried by the client is obtained; in response to the metadata query request, it is determined whether there is a parent index of the parent directory path to which the file to be queried belongs in the page replacement cache maintained by the metadata management node node identification; in the case where the parent index node identification exists in the page replacement cache, according to the parent index node identification and the file name of the file to be queried, query the metadata of the file to be queried from the metadata database; Wherein, the page replacement cache is adjusted according to the directory modification operation of the client, which ensures the accuracy of the index node identification of the directory path in the page replacement cache, simplifies the process of metadata query, and reduces the query delay.
  • This embodiment is an optional solution proposed on the basis of the above embodiments, and describes how to adjust the index node identifier of the target path included in the page replacement cache according to the directory modification operation of the client.
  • deleting the index node of the first directory path from the page replacement cache maintained by the metadata management node that detected the directory modification operation The identifier and the index node identifier of the subdirectory path of the first directory path, and control other metadata management nodes to delete the second metadata management node from the page replacement cache maintained by each metadata management node in the other metadata management nodes
  • the first directory path refers to a directory path stored in the metadata management node's own page replacement cache, and the directory path is used to indicate a file storage location.
  • the metadata management node detects that the client modifies the directory of the first directory path, the metadata management node will delete the first directory path from the page replacement cache maintained by the metadata management node in order to ensure data consistency in the page replacement cache maintained by itself.
  • the first directory path /a, the subdirectory path /a/b of the first directory path, and the index node identifiers of /a and /a/b are stored in the replacement cache maintained by the metadata management node itself.
  • the client performs a renaming operation on the first directory path, renaming /a/b to /f/b, and then creates directory paths /a/b and /a/b/c.
  • the metadata management node does not delete the inode identifier of the first directory path and the inode identifier of the subdirectory path of the first directory path from the page replacement cache maintained by itself, at this time, the page replacement cache maintained by the metadata management node itself
  • the index node ID of /a/b is inaccurate and inconsistent with the actual index node ID of /a/b.
  • the metadata management node queries the index node ID of /a/b/c from the replacement cache maintained by itself. , which is actually the inode ID of /f/b/c.
  • the metadata management node after detecting the directory modification operation, deletes the index node identifier of the first directory path and the index node identifier of the subdirectory path of the first directory path from the page replacement cache maintained by itself, and then It will control other metadata management nodes to delete the index node identifier of the first directory path and the index node identifier of the subdirectory path of the first directory path from the page replacement cache maintained by the other metadata management nodes, ensuring that the entire distributed file System data consistency.
  • the metadata management node that detects the directory modification operation of the client on the first directory path sends a page replacement cache adjustment request to other metadata management nodes in the distributed file
  • the cache adjustment request deletes the index node identifier of the first directory path and the index node identifier of the subdirectory path of the first directory path in the page replacement caches maintained respectively.
  • the directory modification operation of the client on the first directory path is detected, the directory modification operation on the first directory path is added to the modification operation record, and the modification operation record is used for The Metadata Management node that tunes the Page Replacement Cache provides directory modification operations.
  • the modification operation record is used to record the directory modification operation of the client.
  • the modification operation record includes: at least one of the modified directory identifier, modification time, and modification content.
  • the metadata database may be a NewSQL database.
  • the metadata management node that detects the directory modification operation of the client on the first directory path adds the directory modification operation of the first directory path to the modification operation record, so that each metadata management node in the distributed file system, Adjust the page replacement cache maintained by each metadata management node according to the directory modification operation in the modification operation record.
  • Adding the directory modification operation to the modification operation record ensures the queryability and sharing of the directory modification operation.
  • the page replacement maintained by the metadata management node can be improved Cache adjustment success rate. Even if a metadata management node is abnormal and cannot receive cache adjustment requests sent by other metadata management nodes, it can delete the first page from the page replacement cache maintained by itself within a certain period of time according to the directory modification operation in the modification operation record.
  • the metadata management node that detects the directory modification operation of the client on the first directory path sends a page replacement cache adjustment request to other metadata management nodes in the distributed file system, it receives the page replacement cache
  • the metadata management node that adjusts the request deletes the index node identifier of the first directory path and the index node identifier of the subdirectory path of the first directory path from the page replacement cache maintained by itself according to the page replacement cache adjustment request, and sends the page to
  • the metadata management node that replaces the cache adjustment request feedbacks the cache adjustment completion message, so that the metadata management node that sends the page replacement cache adjustment request, after receiving the cache adjustment completion message fed back by all other metadata management nodes, feedbacks the directory to the user A message that the modification operation was successful.
  • the metadata management node that sends the page replacement cache adjustment request needs to wait for a period of time before returning to the user a message that the directory modification operation is successful. During this period, the abnormal metadata management node can delete the index node identifier of the first directory path and the subdirectories of the first directory path from the page replacement cache maintained by itself according to the directory modification operation recorded in the metadata database. The i-node ID of the path.
  • after adding the directory modification operation on the first directory path to the modification operation record it further includes: after all metadata management nodes have performed the operation on the first directory path In the case of adjusting the page replacement cache maintained by the directory modification operation, the directory modification operation on the first directory path is deleted from the modification operation record.
  • the metadata management node that detects the client’s directory modification operation on the first directory path has adjusted the page replacement cache maintained by itself according to the directory modification operation on the first directory path, and other metadata management nodes in the distributed file system
  • the node has adjusted the page replacement caches maintained by the other metadata management nodes according to the directory modification operation on the first directory path, that is to say, the page replacement caches maintained by multiple metadata management nodes in the distributed file system
  • the relevant content of the first directory path in .
  • the directory modification operation on the first directory path may be deleted from the modification operation record. Doing so can improve data management efficiency and improve resource utilization.
  • the directory modification operation on the first directory path is deleted from the persistently stored modification operation records in the metabase.
  • the technical solution of the embodiment of the present disclosure adjusts the page replacement cache maintained by the metadata management node that detects the directory modification operation on the first directory path by the client and controls other metadata
  • the data management nodes adjust the page replacement caches maintained by them; and add the directory modification operation on the first directory path to the modification operation record, so that each metadata management node adjusts its own directory modification operation according to the modification operation record Maintained page replacement cache.
  • the disclosure ensures the consistency of data in the page replacement cache maintained by each metadata management node in the distributed file system, and ensures the accuracy of the data in the page replacement cache.
  • Fig. 2 is a schematic diagram of another metadata query method based on a distributed file system according to an embodiment of the present disclosure. This embodiment is an optional solution proposed on the basis of the foregoing embodiments.
  • the metadata query method based on the distributed file system provided in this embodiment includes:
  • the second directory path refers to the target path recorded in the modification operation record.
  • the second directory path includes but is not limited to the first directory path.
  • the second directory path can be added to the modification operation by other metadata management nodes in the distributed file system In the record, for example, it may be a metadata management node that detects the directory modification operation of the client on the second directory path.
  • the metadata management node pulls the directory modification operation on the second directory path from the modification operation record, and adjusts the page replacement cache maintained by itself according to the directory modification operation on the second target path. For example, each metadata management node can periodically pull the modification operation record from the metadata database, and clear the page replacement cache maintained by itself according to the directory modification operation on the second directory path in the modification operation record, so that the metadata management node is abnormal. In the case that no page replacement cache adjustment request sent by other metadata management nodes is received, the changed directory can be deleted in the page replacement cache maintained by itself in a short period of time, ensuring the cache consistency of multiple metadata management nodes .
  • pulling the directory modification operation on the second directory path from the modification operation record includes: obtaining the identifier of the directory modification operation pulled by each metadata management node; The obtained directory modification operation identifier is used to extract the directory modification operation on the second directory path from the modification operation record.
  • the directory modification operation identifier is used to distinguish different directory modification operations. Considering resource utilization, the directory modification operations that have been pulled by the metadata management node do not need to be pulled again. Each metadata management node obtains the directory modification operation identifier that it has pulled, and pulls the directory modification operations that have not been pulled from the modification operation record according to the directory modification operation identifier that has been pulled.
  • the directory modification operation identifier is the number of the directory modification operation.
  • the metadata management node can pull the directory modification operation from the modification operation record according to a certain numbering sequence, such as the order from small to large. Exemplarily, if the metadata management node has fetched directory modification operations numbered 1-198, the metadata management node will directly start pulling directory modification operations numbered 199 when pulling the directory modification operation again. Doing so improves resource utilization.
  • the page replacement cache maintained by the metadata management node performing the pulling operations is cleared.
  • N and the preset time are preset by relevant technical personnel according to actual business requirements, and the specific values of N and the preset time are not limited here, and are determined according to actual conditions. Exemplarily, the value of N may be 3, and the preset time may be several seconds.
  • the metadata management node fails to pull the directory modification operations for N consecutive times from the modification operation record, it indicates that the metadata management node is abnormal and may be down. In order to ensure data accuracy, the metadata management node maintains its own The data in the page replacement cache is no longer trusted, and the metadata management node needs to clear the page replacement cache maintained by itself.
  • the metadata management node deletes the index node identifier of the second directory path and the index node identifier of the subdirectory path of the second directory path from the page replacement cache maintained by itself according to the directory modification operation of the fetched second directory path.
  • the technical solution of the embodiment of the present disclosure pulls the directory modification operation on the second directory path from the modification operation record, and deletes the index of the second directory path from the page replacement cache maintained by the metadata management node that performs the pulling operation
  • the node ID and the index node ID of the subdirectory path of the second directory path ensure the consistency and accuracy of the data in the page replacement cache maintained by the metadata management node itself, and provide accurate and powerful data support for metadata query.
  • Fig. 3 is a schematic diagram of a metadata query device based on a distributed file system provided according to an embodiment of the present disclosure. This embodiment is applicable to the situation of querying metadata in a distributed file system, and the device is configured in an electronic device , the method for querying metadata based on a distributed file system described in any embodiment of the present disclosure may be implemented. Referring to FIG.
  • the metadata query device 300 based on a distributed file system includes the following modules: a metadata query request acquisition module 310 configured to obtain a metadata query request of a file to be queried by a client; a parent index node identification determination module 320, Set to respond to the metadata query request, determine whether there is a parent index node identifier of the parent directory path to which the file to be queried belongs to in the page replacement cache maintained by the metadata management node; the metadata query module 330 is set to If the parent index node identifier exists in the page replacement cache, query the metadata of the file to be queried from the metadata database according to the parent index node identifier and the file name of the file to be queried; wherein, the page The replacement cache adjusts based on the client's directory modification operations.
  • the device further includes: an index node identification processing module configured to, in the case of detecting a directory modification operation of the client on the first directory path, from the detected directory modification operation Delete the index node identifier of the first directory path and the index node identifier of the subdirectory path of the first directory path from the page replacement cache maintained by the metadata management node, and control other metadata management nodes to obtain from the other metadata
  • the index node identifier of the first directory path and the index node identifier of the subdirectory path of the first directory path are deleted from the page replacement cache maintained by each metadata management node in the management nodes.
  • the device further includes: a directory modification operation adding module, configured to detect the directory modification operation of the client on the first directory path In this case, the directory modification operation on the first directory path is added to the modification operation record, and the modification operation record is used to provide the directory modification operation for the metadata management node that needs to adjust the page replacement cache.
  • a directory modification operation adding module configured to detect the directory modification operation of the client on the first directory path
  • the directory modification operation on the first directory path is added to the modification operation record
  • the modification operation record is used to provide the directory modification operation for the metadata management node that needs to adjust the page replacement cache.
  • the device further includes: a directory modification operation pulling module configured to pull the directory modification operation on the second directory path from the modification operation record; an index node identifier deletion module configured to deleting the index node identifier of the second directory path and the index node identifier of the subdirectory path of the second directory path from the page replacement cache maintained by the metadata management node performing the pulling operation.
  • a directory modification operation pulling module configured to pull the directory modification operation on the second directory path from the modification operation record
  • an index node identifier deletion module configured to deleting the index node identifier of the second directory path and the index node identifier of the subdirectory path of the second directory path from the page replacement cache maintained by the metadata management node performing the pulling operation.
  • the directory modification operation pulling module includes: a pulled directory modification operation identifier acquisition submodule, configured to obtain the directory modification operation identifier that has been pulled by each metadata management node;
  • the modification operation pulling sub-module is configured to pull the directory modification operation on the second directory path from the modification operation record according to the extracted directory modification operation identifier.
  • the device further includes: its own page replacement cache clearing module, which is configured to clear and execute the pull operation record in the case of N consecutive failures to pull the directory modification operation from the modification operation record.
  • the page replacement cache maintained by the metadata management node for fetch operations.
  • the device further includes: a directory modification operation deletion module, configured to, after adding the directory modification operation on the first directory path to the modification operation record, all metadata management nodes If the maintained page replacement cache has been adjusted according to the directory modification operation on the first directory path, delete the directory modification operation on the first directory path from the modification operation record.
  • a directory modification operation deletion module configured to, after adding the directory modification operation on the first directory path to the modification operation record, all metadata management nodes If the maintained page replacement cache has been adjusted according to the directory modification operation on the first directory path, delete the directory modification operation on the first directory path from the modification operation record.
  • the metadata query request of the file to be queried by the client is obtained; in response to the metadata query request, it is determined whether there is a parent to which the file to be queried belongs in the page replacement cache maintained by the metadata management node.
  • the parent index node identifier of the directory path in the case where the parent index node identifier exists in the page replacement cache, query the query from the metabase according to the parent index node identifier and the file name of the file to be queried.
  • the metadata of the file wherein, the page replacement cache is adjusted according to the directory modification operation of the client, which ensures the accuracy of the index node identification of the directory path in the page replacement cache, simplifies the process of metadata query, and reduces the query delay.
  • the acquisition, storage and application of the user's personal information involved are in compliance with relevant laws and regulations, and do not violate public order and good customs.
  • the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
  • FIG. 4 shows a schematic block diagram of an example electronic device 400 that may be used to implement embodiments of the present disclosure.
  • Electronic devices may be various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other computers.
  • Electronic devices may also be various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices.
  • the components, connection relationships between the components, and functions of the components shown herein are merely examples.
  • the device 400 includes a computing unit 401 that can be loaded into a random access memory (Random Access Memory, RAM) according to a computer program stored in a read-only memory (Read-Only Memory, ROM) 402 or from a storage unit 408. ) 403 to perform various actions and processes. In the RAM 403, various programs and data necessary for the operation of the device 400 can also be stored.
  • the computing unit 401, ROM 402, and RAM 403 are connected to each other through a bus 404.
  • An input/output (Input/Output, I/O) interface 405 is also connected to the bus 404 .
  • Multiple components in the device 400 are connected to the I/O interface 405, and the multiple components include: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk , an optical disc, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, and the like.
  • the communication unit 409 allows the device 400 to exchange information/data with other devices over a computer network such as the Internet and/or various telecommunication networks.
  • Computing unit 401 may be various general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a variety of dedicated artificial intelligence (Artificial Intelligence, AI) computing chips, a variety of execution Computing unit of machine learning model algorithm, digital signal processor (Digital Signal Processor, DSP), and processor, controller, microcontroller, etc.
  • the computing unit 401 executes multiple methods and processes described above, for example, a metadata query method based on a distributed file system.
  • the distributed file system-based metadata query method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 408 .
  • part or all of the computer program may be loaded and/or installed on the device 400 via the ROM 402 and/or the communication unit 409.
  • the computing unit 401 may be configured in other ways (for example, by means of firmware) to execute the metadata query method based on the distributed file system.
  • programmable processor can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.
  • Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes can be provided to processors or controllers of general-purpose computers, special-purpose computers, or other programmable data processing devices, so that when the program codes are executed by the processors or controllers, the functions/operations specified in the flowcharts and/or block diagrams be implemented.
  • the program code may execute entirely or partially on the machine, or as a stand-alone software package that executes partly on the machine and partly on a remote machine or entirely on a remote machine or server.
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared semiconductor systems, devices, or devices, or any combination of the foregoing.
  • the machine-readable storage medium may include one or more wire-based electrical connections, a portable computer disk, hard disk, RAM, ROM, Electrically Erasable Programmable read only memory (EPROM) or flash memory, Fiber optics, Compact Disc Read-Only Memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination of the foregoing.
  • the systems and techniques described herein can be implemented on a computer having a display device (e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD)) to display information to the user.
  • a display device e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD)
  • LCD Liquid Crystal Display
  • keyboard and pointing device such as a mouse or trackball
  • Other types of devices may also be configured to provide interaction with the user, for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and may be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.
  • the systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system.
  • the components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (Wide Area Network, WAN), blockchain networks, and the Internet.
  • a computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by executing a computer program on a computer that has a client-server relationship.
  • the server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system, which solves the defects of difficult management and weak business scalability of traditional physical hosts.
  • steps may be reordered, added or deleted using the various forms of the flow shown above.
  • steps described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

一种基于分布式文件系统的元数据查询方法、装置、设备和存储介质。实现方案为：获取客户端对待查询文件的元数据查询请求；响应于所述元数据查询请求，确定元数据管理节点维护的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识；在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名称，从元数据库中查询所述待查询文件的元数据；其中，所述页面置换缓存根据客户端的目录修改操作进行调整。

Description

基于分布式文件系统的元数据查询方法、设备和存储介质
本申请要求在2021年11月26日提交中国专利局、申请号为202111423724.3的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开涉及分布式文件系统技术领域和数据存储技术领域，例如涉及一种基于分布式文件系统的元数据查询方法、装置、电子设备和计算机可读存储介质。
背景技术
元数据是指描述数据的数据，主要是描述数据属性的信息，用来支持如指示存储位置、历史数据和资源的查找以及文件记录等功能。在分布式文件系统中，元数据是描述文件系统的重要单元。大部分分布式文件系统的应用中，如互联网，科学计算等，元数据查询请求占到所有请求一半以上的比例。因此，提高分布式文件系统中元数据的查询效率，降低元数据查询延迟，对整个文件系统的性能有着极其重要的影响。
发明内容
本公开提供了一种用于基于分布式文件系统的元数据查询方法、装置、设备和存储介质。
本公开提供了一种基于分布式文件系统的元数据查询方法，包括：
获取客户端对待查询文件的元数据查询请求；
响应于所述元数据查询请求，确定元数据管理节点的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识；
在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据；
其中，所述页面置换缓存根据客户端的目录修改操作进行调整。
本公开提供了一种基于分布式文件系统的元数据查询装置，包括：
元数据查询请求获取模块，设置为获取客户端对待查询文件的元数据查询请求；
父索引节点标识确定模块，设置为响应于所述元数据查询请求，确定元数 据管理节点的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识；
元数据查询模块，设置为在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据；
其中，所述页面置换缓存根据客户端的目录修改操作进行调整。
本公开提供了一种电子设备，包括：
至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行本公开任意实施例所提供的基于分布式文件系统的元数据查询方法。
本公开提供了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行本公开任意实施例所提供的基于分布式文件系统的元数据查询方法。
本公开提供了一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现本公开任意实施例所提供的基于分布式文件系统的元数据查询方法。
附图说明
图1是根据本公开实施例提供的一种基于分布式文件系统的元数据查询方法的示意图；
图2是根据本公开实施例提供的另一种基于分布式文件系统的元数据查询方法的示意图；
图3是根据本公开实施例提供的一种基于分布式文件系统的元数据查询装置的示意图；
图4是用来实现本公开实施例的基于分布式文件系统的元数据查询方法的电子设备的框图。
具体实施方式
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的多种细节，应当将它们认为仅仅是示范性的。以下的描述中省略了对公知功能和结构的描述。
以下结合附图，对本公开实施例提供的该方案进行说明。
分布式文件系统的架构一般包括如下几个部分：客户端、元数据管理集群、元数据库和存储集群。其中，客户端是以库的形式编译进业务进程中的，客户端向业务提供文件系统的相关访问接口如：创建文件夹/文件接口、打开/关闭文件接口、目录下子目录/子文件列举接口、文件数据读取接口和文件数据写入接口等；元数据管理集群由多个元数据管理节点(master)组成，每个元数据管理节点完全对等，没有主、从之分，元数据管理节点可以水平扩容。
元数据管理节点用于管理目录树、文件夹/文件的索引节点(inode)信息、文件数据块信息和数据块到存储节点的映射信息。元数据管理节点对外提供元数据查询与修改等功能。元数据管理节点本地不持久化目录树等元数据，所有的元数据都持久化到外部的元数据库中。
元数据库用于持久化分布式文件系统的所有元数据，如目录树、文件的索引节点信息、文件的数据块信息和数据块到存储节点的映射信息等。存储集群由多个存储节点组成，用来存储分布式文件系统的文件数据。每个文件的数据被水平切割成多个数据块(block)，每个存储节点存储了多个文件的数据块。
在上述分布式文件系统中，元数据管理节点查询一个路径深度为N的文件的索引节点标识(inodeid)，目前主要有如下两个方法来实现：1)串行逐级查询法：假设一个文件目录路径为/a/b/c，需要查询文件c的inodeid；首先通过根节点inodeid+a去数据库查询a的inodeid,然后通过a的inodeid+b查询到b的inodeid，最后通过b的inodeid+c查询到文件c的inodeid。利用串行逐级查询法，查询一个路径深度为N文件的inodeid需要串行查询数据库N次。2)缓存加速查询法：在元数据管理节点设置缓存，当需要查询文件的inodeid时，首先从缓存获取该文件目录路径的父目录索引节点标识，然后向元数据库并发执行N次主键查询，当所有子请求都返回成功，就代表查询成功，如果某个子请求查询失败，则代表查询失败。为了便于理解，继续上例进行说明，假设本地分别缓存了/a、/a/b、/a/b/c的inodeid，那么为了查询c的实际inodeid，向数据库并发执行3次查询：根目录inodeid+a，查询a的inodeid；a的inodeid+b，查询b的inodeid；b的inodeid+c,查询c的inodeid；如果3次查询均返回成功，则查询成功。如果3次查询有某个子请求查询失败，则查询失败。
上述两种元数据查询方法，查询一个路径深度为N的文件的索引节点标识均需要查询元数据库N次，查询流程复杂。基于此，本公开提供了一种基于分布式文件系统的元数据查询方案。
图1是根据本公开实施例提供的一种基于分布式文件系统的元数据查询方法的示意图，本公开实施例可适用于在分布式文件系统中查询元数据的情况。该方法可由一种基于分布式文件系统的元数据查询装置来执行，该装置可采用硬件和/或软件的方式来实现，可配置于电子设备中。参考图1，该方法包括如下步骤：
S110、获取客户端对待查询文件的元数据查询请求。
元数据是指描述数据的数据，主要是描述数据属性的信息，用来支持如指示存储位置、历史数据和资源的查找以及文件记录等功能。待查询文件的元数据是用于描述待查询文件属性的信息。元数据查询请求由客户端产生，用于向元数据管理节点请求待查询文件的元数据。可选的，待查询文件的元数据查询请求中包括待查询文件标识，如文件名称。可选的，待查询文件可以是文件或者文件夹。获取客户端对待查询文件的元数据查询请求，例如由元数据管理节点获取客户端对待查询文件的元数据查询请求。
S120、响应于所述元数据查询请求，确定元数据管理节点维护的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识。
页面置换缓存是指利用页面置换缓存算法设计的缓存结构，其中，页面置换缓存算法可以为最近最久未使用算法(The Least Recently Used，LRU)。页面置换缓存设置在元数据管理节点中，由元数据管理节点维护。页面置换缓存中存储有元数据管理节点最近访过的文件所属父目录路径的父索引节点标识。
元数据管理节点响应于接收到元数据查询请求，确定自身维护的页面置换缓存中是否存在有待查询文件所属父目录路径的父索引节点标识。
S130、在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据。
所述页面置换缓存根据客户端的目录修改操作进行调整。例如，页面置换缓存中所包括的目录路径根据客户端的目录修改操作进行调整。
客户端的目录修改操作是指客户端对非空目录进行的修改操作，可选的，目录修改操作包括重命名操作或者删除操作。
元数据管理节点根据客户端的目录修改操作对自身维护的页面置换缓存中目录路径的索引节点标识进行调整，保证了元数据管理节点的页面置换缓存中的目录路径的索引节点标识与客户端的目录修改操作的一致性，进而保证了元数据管理节点的页面置换缓存中的目录路径的索引节点标识的准确性。
在分布式文件系统中，每个文件的元数据均以键值对(key-value)的形式 存储在元数据库中，key值是由文件的父目录路径的父索引节点标识和文件标识组成，value值包含文件的元数据如索引节点标识和创建时间等。其中，索引节点标识(inodeid)是分布式文件系统为每个文件分配的全局唯一标识信息。索引节点标识用于区分分布式系统中的不同文件，不同文件的索引节点标识不同。
在分布式文件系统中查询文件的元数据时，元数据管理节点会在自身维护的页面置换缓存中获取待查询文件的父目录路径的父索引节点标识，然后依据父目录路径的父索引节点标识和待查询文件的标识从元数据库中查询待查询文件的索引节点标识。若元数据管理节点自身维护的页面置换缓存中存在有待查询文件所属父目录路径的父索引节点标识，元数据管理节点则根据该父索引节点标识和待查询文件的文件名，从元数据库中直接查询待查询文件的元数据。
元数据库用于持久化分布式文件系统的所有元数据，如目录树、文件的索引节点信息、文件的数据块信息和数据块到存储节点的映射信息等。可选的，元数据库为NewSQL数据库。
若元数据管理节点在自身页面置换缓存未找到待查询文件所属父目录路径的父索引节点标识，则元数据管理节点需要采用串行逐级查询法在元数据库中查询待查询文件的元数据。
采用本公开实施例提供的方案，无需采用串行逐级查询法和缓存加速查询法查询元数据库N次去查询一个路径深度为N的文件的索引节点标识在元数据管理节点自身维护的页面置换缓存中存在有待查询文件所属父目录路径的父索引节点标识的情况下，仅需要在元数据库中查询一次即可找到待查询文件的元数据，减少了查询次数，简化了元数据查询流程，降低了查询延迟。
本公开实施例中，获取客户端对待查询文件的元数据查询请求；响应于所述元数据查询请求，确定元数据管理节点维护的页面置换缓存中是否存在有待查询文件所属父目录路径的父索引节点标识；在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据；其中，所述页面置换缓存根据客户端的目录修改操作进行调整，保证了页面置换缓存中目录路径的索引节点标识的准确性，简化了元数据查询的流程，降低了查询延迟。
本实施例是在上述实施例的基础上提出的一种可选方案，对根据客户端的目录修改操作，调整页面置换缓存所包括的目标路径的索引节点标识进行了说明。
一方面，在检测到客户端对第一目录路径的目录修改操作的情况下，从检 测到所述目录修改操作的元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识，且控制其他元数据管理节点从所述其他元数据管理节点中的每个元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识。
第一目录路径是指存储在元数据管理节点自身页面置换缓存中的目录路径，目录路径用于指示文件存储位置。元数据管理节点在检测到客户端对第一目录路径的目录修改操作的情况下，元数据管理节点为了保证自身维护的页面置换缓存中数据一致性，会从自身维护的页面置换缓存中删除第一目录路径的索引节点标识以及第一目录路径的子目录路径的索引节点标识。
示例性的，元数据管理节点自身维护的置换缓存中存储了第一目录路径/a和第一目录路径的子目录路径/a/b，以及/a和/a/b的索引节点标识。客户端执行了对第一目录路径的重命名操作，将/a/b重命名为/f/b，后又创建了目录路径/a/b和/a/b/c。若元数据管理节点不从自身维护的页面置换缓存中删除第一目录路径的索引节点标识和第一目录路径的子目录路径的索引节点标识，此时，元数据管理节点自身维护的页面置换缓存中/a/b的索引节点标识是不准确的，和实际/a/b的索引节点标识并不一致，元数据管理节点从自身维护的置换缓存查询到的/a/b/c的索引节点标识，实际上为/f/b/c的索引节点标识。
本公开中，在检测到所述目录修改操作的元数据管理节点从自身维护的页面置换缓存中删除第一目录路径的索引节点标识和第一目录路径的子目录路径的索引节点标识以后，还会控制其他元数据管理节点从所述其他元数据管理节点维护的页面置换缓存中删除第一目录路径的索引节点标识和第一目录路径的子目录路径的索引节点标识，保证了整个分布式文件系统的数据一致性。例如，检测到客户端对第一目录路径的目录修改操作的元数据管理节点，向分布式文件系统中的其他的元数据管理节点发送页面置换缓存调整请求，使得其他元数据管理节点根据页面置换缓存调整请求，在各自维护的页面置换缓存中删除第一目录路径的索引节点标识和第一目录路径的子目录路径的索引节点标识。
另一方面，在检测到客户端对第一目录路径的目录修改操作的情况下，将所述对第一目录路径的目录修改操作添加到修改操作记录中，所述修改操作记录用于为需要调整页面置换缓存的元数据管理节点提供目录修改操作。
修改操作记录用于记录客户端的目录修改操作，可选的，修改操作记录包括：已修改目录标识、修改时间以及修改内容中的至少一项。
可选的，将修改操作记录持久化到元数据库中。示例性的，元数据库可以为NewSQL数据库。
检测到客户端对第一目录路径的目录修改操作的元数据管理节点，将第一目录路径的目录修改操作添加至修改操作记录中，可以使得分布式文件系统中的每个元数据管理节点，根据修改操作记录中目录修改操作调整所述每个元数据管理节点维护的页面置换缓存。将目录修改操作添加至修改操作记录，保证了目录修改操作的可查询性以及共享性，通过将第一目录路径的目录修改操作添加至修改操作记录中，可以提高元数据管理节点维护的页面置换缓存调整成功率。即使一个元数据管理节点出现异常，无法接收到其他元数据管理节点发送过来的缓存调整请求，也能根据修改操作记录中的目录修改操作，在一定时间内从自身维护的页面置换缓存中删除第一目录路径的索引节点标识和第一目录路径的子目录路径的索引节点标识。
可选的，在检测到客户端对第一目录路径的目录修改操作的元数据管理节点，向分布式文件系统中的其他的元数据管理节点发送页面置换缓存调整请求以后，收到页面置换缓存调整请求的元数据管理节点，根据页面置换缓存调整请求，从自身维护的页面置换缓存中删除第一目录路径的索引节点标识和第一目录路径的子目录路径的索引节点标识，并向发送页面置换缓存调整请求的元数据管理节点反馈缓存调整完毕消息，以使发送页面置换缓存调整请求的元数据管理节点，在接收到所有其他元数据管理节点反馈的缓存调整完毕消息后，向用户反馈目录修改操作成功的消息。若由于网络时延等问题，导致一个元数据管理节点缓存调整完毕消息反馈超时，则发送页面置换缓存调整请求的元数据管理节点需要等待一段时间后再向用户返回目录修改操作成功的消息，在这段时间内，发生异常的元数据管理节点可以根据元数据库中修改操作记录的目录修改操作，从自身维护的页面置换缓存中删除第一目录路径的索引节点标识和第一目录路径的子目录路径的索引节点标识。
在一个可选的实施例中，所述将所述对第一目录路径的目录修改操作添加到修改操作记录中之后，还包括：在所有的元数据管理节点已根据所述对第一目录路径的目录修改操作调整维护的页面置换缓存的情况下，从所述修改操作记录中删除所述对第一目录路径的目录修改操作。
检测到客户端对第一目录路径的目录修改操作的元数据管理节点，在自身已根据对第一目录路径的目录修改操作调整自身维护的页面置换缓存，且分布式文件系统中其他元数据管理节点已根据对第一目录路径的目录修改操作调整所述其他元数据管理节点维护的页面置换缓存的情况下，也就是说分布式文件系统中的多个元数据管理节点各自维护的页面置换缓存中第一目录路径的相关内容均已调整完毕。此时，可以从修改操作记录中删除对第一目录路径的目录修改操作。这样做可以提高数据管理效率，提高资源利用率。可选的，从元数据库已持久化保存的修改操作记录中删除对第一目录路径的目录修改操作。
本公开实施例的技术方案通过在检测到客户端对第一目录路径的目录修改操作的情况下，对检测到所述目录修改操作的元数据管理节点维护的页面置换缓存进行调整且控制其他元数据管理节点对各自维护的页面置换缓存进行调整；还将对第一目录路径的目录修改操作添加到修改操作记录中，以使每个元数据管理节点根据修改操作记录中的目录修改操作调整各自维护的页面置换缓存。本公开保证了分布式文件系统中每个元数据管理节点维护的页面置换缓存中数据的一致性，保证了页面置换缓存中数据的准确性。
图2是根据本公开实施例提供的另一种基于分布式文件系统的元数据查询方法的示意图。本实施例是在上述实施例的基础上提出的一种可选方案。参见图2，本实施例提供的基于分布式文件系统的元数据查询方法包括：
S210、从所述修改操作记录中拉取对第二目录路径的目录修改操作。
第二目录路径是指记录在修改操作记录中的目标路径，第二目录路径包括但不限于第一目录路径，第二目录路径可以是由分布式文件系统中其他元数据管理节点添加至修改操作记录中，例如可以是检测到客户端对第二目录路径的目录修改操作的元数据管理节点。
元数据管理节点从修改操作记录中拉取对第二目录路径的目录修改操作，根据第二目标路径的目录修改操作对自身维护的页面置换缓存进行调整。例如，每个元数据管理节点可以定期从元数据库中拉取修改操作记录，根据修改操作记录中对第二目录路径的目录修改操作，清除自身维护的页面置换缓存，使得元数据管理节点在异常情况下未收到其他元数据管理节点发送过来的页面置换缓存调整请求，也能在短时间内在自身维护的页面置换缓存中删除发生变化的目录，保证了多个元数据管理节点的缓存一致性。
在一个可选实施例中，从所述修改操作记录中拉取对第二目录路径的目录修改操作，包括：获取每个元数据管理节点已拉取的目录修改操作标识；根据所述已拉取的目录修改操作标识，从所述修改操作记录中拉取对第二目录路径的目录修改操作。
目录修改操作标识用于区分不同的目录修改操作。考虑到资源利用率，元数据管理节点已拉取的目录修改操作不必再次拉取。每个元数据管理节点获取自身已拉取的目录修改操作标识，根据已拉取的目录修改操作标识，从修改操作记录中拉取尚未拉取的目录修改操作。
可选的，目录修改操作标识为目录修改操作的编号。元数据管理节点可以按照一定的编号顺序，如从小到大的顺序，从修改操作记录中拉取目录修改操 作。示例性的，若元数据管理节点已拉取的编号为1-198的目录修改操作，则元数据管理节点再次拉取目录修改操作时直接从编号为199的目录修改操作开始拉取。这样做可以提高资源利用率。
在一个可选的实施例中，在从所述修改操作记录中连续N次拉取目录修改操作失败的情况下，清空执行所述拉取操作的元数据管理节点维护的页面置换缓存。
由于网络时延等原因，可能造成元数据管理节点不能成功拉取目录修改操作。为了提高页面置换缓存调整成功率，若元数据管理节点拉取目录修改操作失败，可以等待预设时间，再次拉取目录修改操作，直到该元数据管理节点连续N次拉取目录修改操作均失败。其中，N和预设时间均是由相关技术人员根据实际业务需求预先设定的，N和预设时间的具体数值在这里不作限定，根据实际情况确定。示例性的，N的取值可以为3，预设时间可以为几秒。
若元数据管理节点从修改操作记录中连续N次拉取目录修改操作均失败，则表明该元数据管理节点异常，可能出现了宕机情况，为了保证数据准确性，该元数据管理节点自身维护的页面置换缓存中数据不再被信任，该元数据管理节点需要清空自身维护的页面置换缓存。
S220、从执行所述拉取操作的元数据管理节点维护的页面置换缓存中删除第二目录路径的索引节点标识和第二目录路径的子目录路径的索引节点标识。
元数据管理节点根据拉取到的第二目录路径的目录修改操作，从自身维护的页面置换缓存中删除第二目录路径的索引节点标识和第二目录路径的子目录路径的索引节点标识。
本公开实施例的技术方案通过从修改操作记录中拉取对第二目录路径的目录修改操作，从执行所述拉取操作的元数据管理节点维护的页面置换缓存中删除第二目录路径的索引节点标识和第二目录路径的子目录路径的索引节点标识，保证了元数据管理节点自身维护的页面置换缓存中数据的一致性和准确性，为元数据查询提供了准确有力的数据支持。
图3是根据本公开实施例提供的一种基于分布式文件系统的元数据查询装置的示意图，本实施例可适用于在分布式文件系统中查询元数据的情况，该装置配置于电子设备中，可实现本公开任意实施例所述的基于分布式文件系统的元数据查询方法。参考图3，该基于分布式文件系统的元数据查询装置300包括如下模块：元数据查询请求获取模块310，设置为获取客户端对待查询文件的元数据查询请求；父索引节点标识确定模块320，设置为响应于所述元数据查询请 求，确定元数据管理节点维护的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识；元数据查询模块330，设置为在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据；其中，所述页面置换缓存根据客户端的目录修改操作进行调整。
在一种可选实施方式中，所述装置还包括：索引节点标识处理模块，设置为在检测到客户端对第一目录路径的目录修改操作的情况下，从检测到所述目录修改操作的元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识，且控制其他元数据管理节点从所述其他元数据管理节点中的每个元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识。
在一种可选实施方式中，所述元数据管理节点的数量为多个，所述装置还包括：目录修改操作添加模块，设置为在检测到客户端对第一目录路径的目录修改操作的情况下，将所述对第一目录路径的目录修改操作添加到修改操作记录中，所述修改操作记录用于为需要调整页面置换缓存的元数据管理节点提供目录修改操作。
在一种可选实施方式中，所述装置还包括：目录修改操作拉取模块，设置为从所述修改操作记录中拉取对第二目录路径的目录修改操作；索引节点标识删除模块，设置为从执行所述拉取操作的元数据管理节点维护的页面置换缓存中删除所述第二目录路径的索引节点标识和所述第二目录路径的子目录路径的索引节点标识。
在一种可选实施方式中，所述目录修改操作拉取模块，包括：已拉取目录修改操作标识获取子模块，设置为获取每个元数据管理节点已拉取的目录修改操作标识；目录修改操作拉取子模块，设置为根据所述已拉取的目录修改操作标识，从所述修改操作记录中拉取对第二目录路径的目录修改操作。
在一种可选实施方式中，所述装置还包括：自身页面置换缓存清空模块，设置为在从所述修改操作记录中连续N次拉取目录修改操作失败的情况下，清空执行所述拉取操作的元数据管理节点维护的页面置换缓存。
在一种可选实施方式中，所述装置还包括：目录修改操作删除模块，设置为将所述对第一目录路径的目录修改操作添加到修改操作记录中之后，在所有的元数据管理节点已根据所述对第一目录路径的目录修改操作调整维护的页面置换缓存的情况下，从所述修改操作记录中删除所述对第一目录路径的目录修改操作。
本公开实施例的技术方案，获取客户端对待查询文件的元数据查询请求；响应于所述元数据查询请求，确定元数据管理节点维护的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识；在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据；其中，所述页面置换缓存根据客户端的目录修改操作进行调整，保证了页面置换缓存中目录路径的索引节点标识的准确性，简化了元数据查询的流程，降低了查询延迟。
本公开的技术方案中，所涉及的用户个人信息的获取，存储和应用等，均符合相关法律法规的规定，且不违背公序良俗。
根据本公开的实施例，本公开还提供了一种电子设备、一种可读存储介质和一种计算机程序产品。
图4示出了可以用来实施本公开的实施例的示例电子设备400的示意性框图。电子设备可以是多种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它计算机。电子设备还可以使多种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、部件之间的连接关系、以及部件的功能仅仅作为示例。
如图4所示，设备400包括计算单元401，其可以根据存储在只读存储器(Read-Only Memory，ROM)402中的计算机程序或者从存储单元408加载到随机访问存储器(Random Access Memory，RAM)403中的计算机程序，来执行多种动作和处理。在RAM 403中，还可存储设备400操作所需的多种程序和数据。计算单元401、ROM 402以及RAM 403通过总线404彼此相连。输入/输出(Input/Output，I/O)接口405也连接至总线404。
设备400中的多个部件连接至I/O接口405，多个部件包括：输入单元406，例如键盘、鼠标等；输出单元407，例如多种类型的显示器、扬声器等；存储单元408，例如磁盘、光盘等；以及通信单元409，例如网卡、调制解调器、无线通信收发机等。通信单元409允许设备400通过诸如因特网的计算机网络和/或多种电信网络与其他设备交换信息/数据。
计算单元401可以是多种具有处理和计算能力的通用和/或专用处理组件。计算单元401的一些示例包括但不限于中央处理单元(Central Processing Unit，CPU)、图形处理单元(Graphics Processing Unit，GPU)、多种专用的人工智能(Artificial Intelligence，AI)计算芯片、多种执行机器学习模型算法的计算单 元、数字信息处理器(Digital Signal Processor，DSP)、以及处理器、控制器、微控制器等。计算单元401执行上文所描述的多个方法和处理，例如基于分布式文件系统的元数据查询方法。例如，在一些实施例中，基于分布式文件系统的元数据查询方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元408。在一些实施例中，计算机程序的部分或者全部可以经由ROM 402和/或通信单元409而被载入和/或安装到设备400上。当计算机程序加载到RAM 403并由计算单元401执行时，可以执行上文描述的基于分布式文件系统的元数据查询方法的一个或多个步骤。备选地，在其他实施例中，计算单元401可以通过其他方式(例如，借助于固件)而被配置为执行基于分布式文件系统的元数据查询方法。
本文中以上描述的系统和技术的多种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(Field Programmable Gate Array，FPGA)、专用集成电路(Application Specific Integrated Circuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、芯片上系统的系统(System on Chip，SOC)、负载可编程逻辑设备(Complex Programmable Logic Device，CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些多种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码由处理器或控制器执行时，流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行或部分地在机器上执行，程序代码也可以作为独立软件包一部分地在机器上执行且另一部分在远程机器上执行或完全在远程机器或服务器上执行。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的半导体系统、装置或设备，或者上述内容的任何组合。机器可读存储介质可以包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、RAM、ROM、可擦除可编程只读存储器(Electrically Erasable Programmable read only memory，EPROM)或快闪存储器、光纤、便捷式紧凑盘只读存储器(Compact Disc  Read-Only Memory，CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何组合。
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：向用户显示信息的显示装置(例如，阴极射线管(Cathode Ray Tube，CRT)或者液晶显示器(Liquid Crystal Display，LCD)监视器)，以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以设置为提供与用户的交互，例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)，并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(Local Area Network，LAN)、广域网(Wide Area Network，WAN)、区块链网络和互联网。
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在计算机上执行具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，解决了传统物理主机存在的管理难度大、业务扩展性弱的缺陷。
应该理解，可以使用上面所示的多种形式的流程，重新排序、增加或删除步骤。例如，本发公开中记载的多步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。

Claims (17)

  1. 一种基于分布式文件系统的元数据查询方法，包括：
    获取客户端对待查询文件的元数据查询请求；
    响应于所述元数据查询请求，确定元数据管理节点维护的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识；
    在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据；其中，所述页面置换缓存根据客户端的目录修改操作进行调整。
  2. 根据权利要求1所述的方法，还包括：
    在检测到客户端对第一目录路径的目录修改操作的情况下，从检测到所述目录修改操作的元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识，且控制其他元数据管理节点从所述其他元数据管理节点中的每个元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识。
  3. 根据权利要求1所述的方法，其中，所述元数据管理节点的数量为多个，所述方法还包括：
    在检测到客户端对第一目录路径的目录修改操作的情况下，将所述对第一目录路径的目录修改操作添加到修改操作记录中，所述修改操作记录用于为需要调整页面置换缓存的元数据管理节点提供目录修改操作。
  4. 根据权利要求3所述的方法，还包括：
    从所述修改操作记录中拉取对第二目录路径的目录修改操作；
    从执行所述拉取操作的元数据管理节点维护的页面置换缓存中删除所述第二目录路径的索引节点标识和所述第二目录路径的子目录路径的索引节点标 识。
  5. 根据权利要求4所述的方法，其中，所述从所述修改操作记录中拉取对第二目录路径的目录修改操作，包括：
    获取每个元数据管理节点已拉取的目录修改操作标识；
    根据所述已拉取的目录修改操作标识，从所述修改操作记录中拉取对第二目录路径的目录修改操作。
  6. 根据权利要求3所述的方法，还包括：
    在从所述修改操作记录中连续N次拉取目录修改操作失败的情况下，清空执行所述拉取操作的元数据管理节点维护的页面置换缓存。
  7. 根据权利要求3所述的方法，所述将所述对第一目录路径的目录修改操作添加到修改操作记录中之后，还包括：
    在所有的元数据管理节点已根据所述对第一目录路径的目录修改操作调整维护的页面置换缓存的情况下，从所述修改操作记录中删除所述对第一目录路径的目录修改操作。
  8. 一种基于分布式文件系统的元数据查询装置，包括：
    元数据查询请求获取模块，设置为获取客户端对待查询文件的元数据查询请求；
    父索引节点标识确定模块，设置为响应于所述元数据查询请求，确定元数据管理节点维护的页面置换缓存中是否存在有所述待查询文件所属父目录路径的父索引节点标识；
    元数据查询模块，设置为在所述页面置换缓存中存在所述父索引节点标识的情况下，根据所述父索引节点标识和待查询文件的文件名，从元数据库中查询所述待查询文件的元数据；
    其中，所述页面置换缓存根据客户端的目录修改操作进行调整。
  9. 根据权利要求8所述的装置，还包括：
    索引节点标识处理模块，设置为在检测到客户端对第一目录路径的目录修改操作的情况下，从检测到所述目录修改操作的元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识，且控制其他元数据管理节点从所述其他元数据管理节点中的每个元数据管理节点维护的页面置换缓存中删除所述第一目录路径的索引节点标识和所述第一目录路径的子目录路径的索引节点标识。
  10. 根据权利要求8所述的装置，其中，所述元数据管理节点的数量为多个，所述装置还包括：
    目录修改操作添加模块，设置为在检测到客户端对第一目录路径的目录修改操作的情况下，将所述对第一目录路径的目录修改操作添加到修改操作记录中，所述修改操作记录用于为需要调整页面置换缓存的元数据管理节点提供目录修改操作。
  11. 根据权利要求10所述的装置，还包括：
    目录修改操作拉取模块，设置为从所述修改操作记录中拉取对第二目录路径的目录修改操作；
    索引节点标识删除模块，设置为从执行所述拉取操作的元数据管理节点维护的页面置换缓存中删除所述第二目录路径的索引节点标识和所述第二目录路径的子目录路径的索引节点标识。
  12. 根据权利要求11所述的装置，其中，所述目录修改操作拉取模块，包括：
    已拉取目录修改操作标识获取子模块，设置为获取每个元数据管理节点已 拉取的目录修改操作标识；
    目录修改操作拉取子模块，设置为根据所述已拉取的目录修改操作标识，从所述修改操作记录中拉取对第二目录路径的目录修改操作。
  13. 根据权利要求10所述的装置，还包括：
    自身页面置换缓存清空模块，设置为在从所述修改操作记录中连续N次拉取目录修改操作失败的情况下，清空执行所述拉取操作的元数据管理节点维护的页面置换缓存。
  14. 根据权利要求10所述的装置，还包括：
    目录修改操作删除模块，设置为将所述对第一目录路径的目录修改操作添加到修改操作记录中之后，在所有的元数据管理节点已根据所述对第一目录路径的目录修改操作调整维护的页面置换缓存的情况下，从所述修改操作记录中删除所述对第一目录路径的目录修改操作。
  15. 一种电子设备，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-7中任一项所述的方法。
  16. 一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行权利要求1-7中任一项所述的方法。
  17. 一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现根据权利要求1-7中任一项所述的方法。
PCT/CN2022/120227 2021-11-26 2022-09-21 基于分布式文件系统的元数据查询方法、设备和存储介质 WO2023093245A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111423724.3 2021-11-26
CN202111423724.3A CN114116613A (zh) 2021-11-26 2021-11-26 基于分布式文件系统的元数据查询方法、设备和存储介质

Publications (1)

Publication Number Publication Date
WO2023093245A1 true WO2023093245A1 (zh) 2023-06-01

Family

ID=80370255

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/120227 WO2023093245A1 (zh) 2021-11-26 2022-09-21 基于分布式文件系统的元数据查询方法、设备和存储介质

Country Status (2)

Country Link
CN (1) CN114116613A (zh)
WO (1) WO2023093245A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117056379A (zh) * 2023-10-11 2023-11-14 宁波银行股份有限公司 一种元数据缓存方法、装置、电子设备及可读存储介质
CN117312297A (zh) * 2023-11-30 2023-12-29 中国光大银行股份有限公司 数据管理方法及装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114116613A (zh) * 2021-11-26 2022-03-01 北京百度网讯科技有限公司 基于分布式文件系统的元数据查询方法、设备和存储介质
CN114840487A (zh) * 2022-03-25 2022-08-02 阿里巴巴（中国）有限公司 分布式文件系统的元数据管理方法和装置
CN114756509B (zh) * 2022-05-19 2023-03-24 北京百度网讯科技有限公司 文件系统的操作方法、系统、设备以及存储介质
CN116303267A (zh) * 2023-03-06 2023-06-23 北京百度网讯科技有限公司 数据访问方法、装置、设备以及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10649852B1 (en) * 2017-07-14 2020-05-12 EMC IP Holding Company LLC Index metadata for inode based backups
CN113010486A (zh) * 2021-02-05 2021-06-22 湖南国科亿存信息科技有限公司 用于无中心分布式文件系统的元数据分层缓存方法及装置
CN113010476A (zh) * 2021-03-15 2021-06-22 腾讯科技（深圳）有限公司 元数据查找方法、装置、设备及计算机可读存储介质
CN114116613A (zh) * 2021-11-26 2022-03-01 北京百度网讯科技有限公司 基于分布式文件系统的元数据查询方法、设备和存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101697168B (zh) * 2009-10-22 2011-10-19 中国科学技术大学 一种分布式文件系统动态元数据管理方法及系统
CN103019960B (zh) * 2012-12-03 2016-03-30 华为技术有限公司 分布式缓存方法及系统
CN103020315B (zh) * 2013-01-10 2015-08-19 中国人民解放军国防科学技术大学 一种基于主从分布式文件系统的海量小文件存储方法
CN103150394B (zh) * 2013-03-25 2014-07-23 中国人民解放军国防科学技术大学 面向高性能计算的分布式文件系统元数据管理方法
US10140304B1 (en) * 2015-12-10 2018-11-27 EMC IP Holding Company LLC Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata
CN106874481B (zh) * 2017-02-20 2020-02-07 郑州云海信息技术有限公司 一种分布式文件系统元数据信息读取方法及系统
CN109241021A (zh) * 2018-09-04 2019-01-18 郑州云海信息技术有限公司 一种文件查询方法、装置、设备及计算机可读存储介质
CN109684282B (zh) * 2018-11-23 2021-01-01 华为技术有限公司 一种构建元数据缓存的方法及装置
US11586586B2 (en) * 2019-06-03 2023-02-21 EMC IP Holding Company LLC Indexes and queries for files by indexing file directories
CN113127420B (zh) * 2021-03-30 2023-03-14 山东英信计算机技术有限公司 一种元数据请求处理方法、装置、设备、介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10649852B1 (en) * 2017-07-14 2020-05-12 EMC IP Holding Company LLC Index metadata for inode based backups
CN113010486A (zh) * 2021-02-05 2021-06-22 湖南国科亿存信息科技有限公司 用于无中心分布式文件系统的元数据分层缓存方法及装置
CN113010476A (zh) * 2021-03-15 2021-06-22 腾讯科技（深圳）有限公司 元数据查找方法、装置、设备及计算机可读存储介质
CN114116613A (zh) * 2021-11-26 2022-03-01 北京百度网讯科技有限公司 基于分布式文件系统的元数据查询方法、设备和存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117056379A (zh) * 2023-10-11 2023-11-14 宁波银行股份有限公司 一种元数据缓存方法、装置、电子设备及可读存储介质
CN117056379B (zh) * 2023-10-11 2024-01-26 宁波银行股份有限公司 一种元数据缓存方法、装置、电子设备及可读存储介质
CN117312297A (zh) * 2023-11-30 2023-12-29 中国光大银行股份有限公司 数据管理方法及装置
CN117312297B (zh) * 2023-11-30 2024-04-02 中国光大银行股份有限公司 数据管理方法及装置

Also Published As

Publication number Publication date
CN114116613A (zh) 2022-03-01

Similar Documents

Publication Publication Date Title
WO2023093245A1 (zh) 基于分布式文件系统的元数据查询方法、设备和存储介质
CN110502507B (zh) 一种分布式数据库的管理系统、方法、设备和存储介质
EP3170106B1 (en) High throughput data modifications using blind update operations
US10191932B2 (en) Dependency-aware transaction batching for data replication
EP2874077B1 (en) Stateless database cache
EP2653986B1 (en) Client-side caching of a database transaction token.
JP4794571B2 (ja) データベースに対するアクセスを効率化するシステム、および、その方法
US9148329B1 (en) Resource constraints for request processing
US10515078B2 (en) Database management apparatus, database management method, and storage medium
US9734177B2 (en) Index merge ordering
WO2022063059A1 (zh) 键值存储系统的数据管理方法及其装置
US11799961B2 (en) File directory traversal method, apparatus, device, and medium
EP4141687A1 (en) Global secondary index method and apparatus for distributed database
US20230012642A1 (en) Method and device for snapshotting metadata, and storage medium
CN117120998A (zh) 用于读取树数据结构中保存的数据的方法和装置
US10558636B2 (en) Index page with latch-free access
WO2020192663A1 (zh) 一种数据管理方法及相关设备
WO2024087426A1 (zh) 一种全量快照快速生成方法、装置、电子设备及存储介质
CN113051244A (zh) 数据访问方法和装置、数据获取方法和装置
WO2018075009A1 (en) Response caching
CN107423375B (zh) 应用程序搜索方法和装置
US10685046B2 (en) Data processing system and data processing method
US20220107949A1 (en) Method of optimizing search system
US20230132173A1 (en) Data reading method, device and storage medium
US20240143594A1 (en) Offloading graph components to persistent storage for reducing resident memory in distributed graph processing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22897331

Country of ref document: EP

Kind code of ref document: A1