CN116756096A

CN116756096A - Metadata processing method and device, electronic equipment and storage medium

Info

Publication number: CN116756096A
Application number: CN202311064170.1A
Authority: CN
Inventors: 汪正洋; 张在贵; 张凯
Original assignee: Suzhou Inspur Intelligent Technology Co Ltd
Current assignee: Suzhou Inspur Intelligent Technology Co Ltd
Priority date: 2023-08-23
Filing date: 2023-08-23
Publication date: 2023-09-15
Anticipated expiration: 2043-08-23
Also published as: CN116756096B

Abstract

The invention relates to the technical field of servers, and provides a metadata processing method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: receiving a first metadata processing request sent by a client; analyzing the first metadata processing request, and determining index node information of an upper-level catalog of a file to be queried carried in the first metadata processing request; determining attribution metadata service information of the file to be queried based on the index node information; and acquiring target metadata of the file to be queried based on the home metadata service information of the file to be queried, and sending the target metadata to the client. The invention does not need to carry out layer-by-layer directory inquiry according to the request sent by the client, thereby simplifying the overall flow of metadata processing, shortening the processing time delay of the metadata and improving the metadata processing efficiency.

Description

Metadata processing method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of server technologies, and in particular, to a metadata processing method, a metadata processing device, an electronic device, and a storage medium.

Background

Under the scene of multi-protocol fusion and intercommunication, object storage adopts a flattened structure to manage all buckets (buckets) and objects, each Bucket and each object have a globally unique ID, and object searching and data access can be rapidly realized according to the IDs.

Currently, for Metadata processing requests, such as HEAD/a/B/file 1 requests, a query is required according to the order of a-B-file1, i.e. a layer-by-layer directory query, so as to obtain Metadata services (mds) of an object, which is complicated to process, and mds of different levels needs to be queried multiple times, so that Metadata processing time delay is longer.

Disclosure of Invention

The invention provides a metadata processing method, a device, electronic equipment and a storage medium, which are used for solving the defect of longer metadata processing time delay in the prior art and realizing the improvement of metadata processing efficiency.

The invention provides a metadata processing method, which is applied to a storage access node, and comprises the following steps:

receiving a first metadata processing request sent by a client;

analyzing the first metadata processing request, and determining index node information of an upper-level catalog of a file to be queried carried in the first metadata processing request;

Determining attribution metadata service information of the file to be queried based on the index node information;

and acquiring target metadata of the file to be queried based on the home metadata service information of the file to be queried, and sending the target metadata to the client.

According to the metadata processing method provided by the invention, the method further comprises the following steps:

receiving a metadata query request sent by the client, wherein the metadata query request is a request which is sent by the client for the first time and is used for requesting to query metadata of the file to be queried;

performing hierarchical directory query based on the metadata query request to obtain home metadata service information of each level and metadata of each level corresponding to the file to be queried;

determining index node information of an upper-level directory of the file to be queried based on attribution metadata service information of each level corresponding to the file to be queried;

and sending the index node information of the upper-level catalogue of the file to be queried to the client so that the client caches the index node information of the upper-level catalogue of the file to be queried.

According to the metadata processing method provided by the invention, the determining the attribution metadata service information of the file to be queried based on the index node information comprises the following steps:

Determining a first hash value based on the inode information;

mapping the first hash value to first metadata service information based on a consistent hash algorithm, and taking the first metadata service information as the attribution metadata service information of the file to be queried.

determining a second hash value based on the inode information and the file name of the file to be queried;

mapping the second hash value to second metadata service information based on a consistent hash algorithm, and taking the second metadata service information as the attribution metadata service information of the file to be queried.

receiving a second metadata processing request sent by a client, wherein the second metadata processing request is used for requesting to query metadata of the file to be queried, and the second metadata processing request does not carry index node information of an upper directory of the file to be queried;

performing layer-by-layer directory query on the second metadata processing request to obtain home metadata service information of each level corresponding to the file to be queried and metadata of each level;

And returning the metadata of the last hierarchy to the client.

According to the metadata processing method provided by the invention, the hierarchical directory query is performed based on the metadata query request to obtain the home metadata service information of each level and the metadata of each level corresponding to the file to be queried, including:

determining a root directory and at least two layers of father directories of the file to be queried based on the metadata query request, wherein the at least two layers of father directories comprise a first layer of father directory and a second layer of father directory, and the second layer of father directory is an upper layer directory of the file to be queried;

and sequentially acquiring first attribution metadata service information of the first-layer parent directory, second attribution metadata service information of the second-layer parent directory and third attribution metadata service information of the file to be queried according to the hierarchical relationship among the root directory, the first-layer parent directory and the second-layer parent directory.

According to the metadata processing method provided by the invention, according to the hierarchical relationship among the root directory, the first layer parent directory and the second layer parent directory, the first attribution metadata service information of the first layer parent directory, the second attribution metadata service information of the second layer parent directory and the third attribution metadata service information of the file to be queried are sequentially acquired, and the metadata processing method comprises the following steps:

Determining index node information of the root directory, determining first attribution metadata service information of the first-layer parent directory based on the index node information of the root directory, and querying first metadata of the first-layer parent directory based on the first attribution metadata service information;

determining index node information of the first-layer father directory based on the first metadata, determining second attribution metadata service information of the second-layer father directory based on the index node information of the first-layer father directory, and inquiring second metadata of the second-layer father directory based on the second attribution metadata service information;

and determining the index node information of the second-layer father directory based on the second metadata, and determining the third attribution metadata service information of the file to be queried based on the index node information of the second-layer father directory.

According to the metadata processing method provided by the invention, the determining the first attribution metadata service information of the first layer parent directory based on the index node information of the root directory comprises the following steps:

and determining a first hash value based on the index node information of the root directory, mapping the first hash value to first target metadata service information based on a consistent hash algorithm, and taking the first target metadata service information as first attribution metadata service information of the first-layer parent directory.

According to the metadata processing method provided by the invention, the determining the second attribution metadata service information of the second-layer parent directory based on the index node information of the first-layer parent directory comprises the following steps:

and determining a second hash value based on the index node information of the first-layer father directory, mapping the second hash value to second target metadata service information based on a consistent hash algorithm, and taking the second target metadata service information as second attribution metadata service information of the second-layer father directory.

According to the metadata processing method provided by the invention, the determining of the third attribution metadata service information of the file to be queried based on the index node information of the second-layer parent directory includes:

and determining a third hash value based on the index node information of the second-layer father directory, mapping the third hash value to third target metadata service information based on a consistent hash algorithm, and taking the third target metadata service information as third attribution metadata service information of the file to be queried.

receiving a third metadata processing request sent by a client, wherein the third metadata processing request is used for requesting to query metadata of the file to be queried, and the third metadata processing request carries routing information of home metadata service of the file to be queried;

Determining the attribution metadata service information of the file to be queried based on the routing information of the attribution metadata service of the file to be queried;

According to the metadata processing method provided by the invention, the metadata processing request is a head request sent by the client through a target communication protocol;

the target communication protocol is a hypertext transfer security protocol or a hypertext transfer protocol.

According to the metadata processing method provided by the invention, the home metadata service information comprises at least one of the following items:

inode information of the parent directory;

inode information of a file or a target;

inode information of a directory, names of subdirectories or files in the directory;

logical identification of the home metadata service.

The invention also provides a metadata processing method applied to the client, which comprises the following steps:

sending a first metadata processing request to a storage access node, wherein the first metadata processing request carries index node information of an upper-level catalogue of a file to be queried;

Receiving target metadata sent by the storage access node; wherein the target metadata is obtained by the storage access node based on the inode information.

sending a metadata query request to the storage access node, wherein the metadata query request is metadata which is sent by the client for the first time and is used for requesting to query the file to be queried;

receiving index node information of an upper catalog of the file to be queried, which is sent by the storage access node;

caching index node information of a superior directory of the file to be queried;

the index node information of the upper-level catalogue of the file to be queried is determined by the storage access node through layer-by-layer catalogue query.

sending a second metadata processing request to a storage access node, wherein the second metadata processing request is used for requesting to query metadata of the file to be queried, and the second metadata processing request does not carry index node information of an upper directory of the file to be queried;

And receiving the metadata of the last hierarchy returned by the storage access node.

sending a third metadata processing request to a storage access node, wherein the third metadata processing request is used for requesting to query metadata of the file to be queried, and the third metadata processing request carries routing information of home metadata service of the file to be queried;

and receiving target metadata of the file to be queried returned by the storage access node, wherein the target metadata are obtained by the storage access node based on routing information of home metadata service of the file to be queried.

The invention also provides a metadata processing apparatus applied to a storage access node, the apparatus comprising:

the first receiving module is used for receiving a first metadata processing request sent by the client;

the analysis module is used for analyzing the first metadata processing request and determining index node information of an upper-level catalogue of the file to be queried carried in the first metadata processing request;

the determining module is used for determining the attribution metadata service information of the file to be queried based on the index node information;

The first sending module is used for obtaining target metadata of the file to be queried based on the home metadata service information of the file to be queried and sending the target metadata to the client.

The invention also provides a metadata processing device applied to a client, which comprises:

the second sending module is used for sending a first metadata processing request to the storage access node, wherein the first metadata processing request carries index node information of an upper-level directory of a file to be queried;

the second receiving module is used for receiving the target metadata sent by the storage access node; wherein the target metadata is obtained by the storage access node based on the inode information.

The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing any of the metadata processing methods described above when executing the program.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a metadata processing method as described in any of the above.

The invention also provides a computer program product comprising a computer program which when executed by a processor implements a method of metadata processing as described in any of the above.

According to the metadata processing method, the metadata processing device, the electronic equipment and the storage medium, through receiving the first metadata processing request sent by the client, the index node information of the upper-level catalogue of the file to be queried is determined, and the attribution metadata service of the file to be queried can be directly obtained according to the index node, so that the target metadata of the file to be queried can be queried according to the attribution metadata service. The invention does not need to carry out layer-by-layer directory inquiry according to the request sent by the client, thereby simplifying the overall flow of metadata processing, shortening the processing time delay of the metadata and improving the metadata processing efficiency.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a diagram of a fusion architecture of a distributed storage system;

FIG. 2 is one of the flow diagrams of the prior art;

FIG. 3 is a second flow chart of the prior art;

FIG. 4 is a schematic flow chart of a metadata processing method according to the present invention;

FIG. 5 is a second flowchart of a metadata processing method according to the present invention;

FIG. 6 is a third flow chart of the metadata processing method according to the present invention;

FIG. 7 is a schematic diagram of a metadata processing apparatus according to the present invention;

FIG. 8 is a schematic diagram of a metadata processing apparatus according to a second embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In the prior art, when the distributed storage system supports fusion and intercommunication, a set of distributed storage support file/object/big data three unstructured services are used for supporting read-write access of other protocols by using a file written in one protocol.

Distributed storage supporting unstructured data fusion interworking is generally based on a file storage system, on which object, big data services are supported.

Referring to fig. 1, fig. 1 is a diagram of a fusion architecture of a distributed storage system, the logic architecture including an access layer110, an MDS120, and a storage pool 130; the access layer comprises a file protocol, an object protocol and a big data protocol; the access layer supports processing file/object/big data protocols, metadata of each protocol is unified, metadata is managed by mds, and data is written into a storage pool.

In order to achieve semantic intercommunication, conceptual models of various unstructured services are unified. The distributed storage unstructured service provides a unified multiple Namespace (Namespace), the file service is called a file system, and the object service is called a Bucket. The user data of each Namespace are isolated from each other, and one Namespace can simultaneously support big data/file/object service.

Referring to fig. 2, in order to implement unstructured service data interworking, a naspace is mapped into a socket of an object, and a directory structure of a file and a URL of the object are mapped, where a mapping relationship is: namespace is mapped to bucket1, file1 under directory/A/B is mapped to object/A/B/file 1 under bucket1, file2 under root directory is mapped to object file2 under bucket bucekt 1.

Conversely, the object created by the user from the S3 object side will be mapped to the corresponding directory structure, if an empty Namespace (mapped to bucket 1), the user creates an/a/B/file 1 object under bucket1 from the S3 object side, and the corresponding distributed storage will create a corresponding directory A, B for this object, and file1 under directory B. Because "/" in the file store represents a directory separator, the object interface creates an object name that contains/when a corresponding directory needs to be created. This results in the object interface reading and writing requiring a layer-by-layer lookup of directory metadata if included/when in the object name.

Referring to fig. 3, fig. 3 is a flow chart illustrating a metadata processing method in the prior art:

and sending the HEAD/A/B/file 1 request to the storage access node by the object client, and inquiring the metadata of the/A/B/file 1 by the storage access node and returning the metadata to the object client.

In the prior art, the HEAD/A/B/file 1 request needs to be queried layer by layer according to the catalogue or the file, the whole flow is complex, and the metadata query time delay is long.

Referring to fig. 4, the metadata processing method provided by the present invention is applied to a storage access node, and the method includes:

Step 410, receiving a first metadata processing request sent by a client;

step 420, analyzing the first metadata processing request, and determining inode information of an upper directory carrying a file to be queried in the first metadata processing request;

step 430, determining attribution metadata service information of the file to be queried based on the index node information;

step 440, acquiring target metadata of the file to be queried based on the home metadata service information of the file to be queried, and sending the target metadata to the client.

The following describes the above steps in detail, with the storage access node of the distributed system as the execution body. It should be noted that, the storage access node in this embodiment belongs to a distributed file system. The storage access node is mainly used for realizing interaction with the client.

In step 410, a first metadata processing request sent by a client is received.

The client is an object client for sending metadata query requests, and the first metadata processing request is a metadata query request sent by the client, for example, the request is a HEAD/A/B/file 1 request.

Optionally, the metadata processing request is a HEAD request sent by the client through a target communication protocol; the target communication protocol is hypertext transfer security protocol (http) or hypertext transfer protocol (http).

In the first metadata processing request, the HEAD is a HEAD request of the request, and in the HTTP request, the HEAD is a request method for acquiring response HEAD information of a specified URL without returning specific response contents. Similar to the GET method, but does not return the actual response body. The HEAD request is typically used to examine metadata of the resource, such as Last-Modified (Last-Modified), content-Length (Content-Length), etc. By sending the HEAD request, the basic information about the resource can be obtained without transmitting the entire response content.

Accordingly, a in the first metadata processing request represents a primary directory of a file, B represents a secondary directory of the file, and file1 represents a file to be queried.

Alternatively, the first metadata processing request in this embodiment may include multiple layers of directories, for example, file1 to be queried is divided according to a hierarchy, and the upper layer of directories is A, B, C, D from top to bottom, that is, the upper layer of directories of file1 is a D directory.

In step 420, the first metadata processing request is parsed, and it is determined that the first metadata processing request carries inode information of an upper directory of the file to be queried.

Analyzing a first metadata processing request HEAD/A/B/file 1, firstly determining the file name of a file to be queried to be processed, namely file1, then determining that the upper-level directory of the file to be queried is a B target and the upper-level directory is an A directory, and finally determining that the first metadata processing request carries index node information of the upper-level directory of the file to be queried, namely the B directory.

The region storing the file meta information is called an inode, and each inode has a number, and the operating system identifies a different file by the inode number.

It may be understood that the first metadata processing request carries the inode information of the upper directory of the file to be queried, which may represent the inode information of the upper directory (B directory) of the file to be queried cached in the client, so that when the client sends the first metadata processing request, the inode containing the B directory.

Then, through the above step 430, the home metadata service information of the file to be queried is determined according to the inode information of the B directory.

The home metadata service is a service for managing and maintaining data home information. During data analysis and processing, the source and attribution of the data is very important for the attribution metadata service to help track and record the owners, creation dates, modification dates, and other relevant attributes of the data. The user can trace back the source of the data and know the credibility, integrity and accuracy of the data. In addition, the home metadata service may also provide rights management and access control of the data, ensuring that only authorized personnel can access and modify the data.

It should be noted that, in the embodiment, the access node of the distributed storage system determines that the transmitted first metadata processing request, that is, the HEAD request, carries the routing information of the home metadata service of the file1 to be queried, that is, the inode information inode of the B directory, and does not query the directory metadata layer by layer, but directly calculates the home metadata service mds of the file1 according to the inode of the B directory, and queries the home metadata service mds of the file1, so that the metadata service mds of the a and the B directory does not need to be queried, thereby simplifying the overall flow of metadata query, improving the processing efficiency of metadata, and reducing the metadata query delay.

Finally, through the step 440, the target metadata of the file to be queried is obtained according to the home metadata service information of the file to be queried, and the queried target metadata is returned to the client.

According to the metadata processing method provided by the invention, the index node information of the upper-level catalogue of the file to be queried is determined by receiving the first metadata processing request sent by the client, and the home metadata service of the file to be queried can be directly acquired according to the index node, so that the target metadata of the file to be queried can be queried according to the home metadata service. The invention does not need to carry out layer-by-layer directory inquiry according to the request sent by the client, thereby simplifying the overall flow of metadata processing, shortening the processing time delay of the metadata and improving the metadata processing efficiency.

In some embodiments, the method further comprises:

It will be appreciated that this embodiment is a preparation process prior to metadata querying.

Firstly, a metadata query request sent by a client is received, and the metadata query request is a request sent by the client for the first time, wherein the request is used for querying metadata of a file to be queried. And then carrying out layer-by-layer directory inquiry according to the metadata inquiry request to obtain the attribution metadata service information of each layer corresponding to the file to be inquired and the metadata of each layer.

If the query requests HEAD/a/B/file 1, in this embodiment, the home metadata service information of the a directory and the metadata of the a directory, the home metadata service information of the B directory and the metadata of the B directory need to be determined sequentially, and the home metadata service information of the file1 to be queried and the metadata of the file to be queried need to be queried.

Further, according to the acquired home metadata service information of each level of the file to be queried, determining the index node information of the upper-level catalogue of the file to be queried, namely the index node of the B catalogue.

And finally, the index node information of the upper-level catalogue of the file to be queried is sent to the client, and the client caches the index node information after receiving the index node information. After the client caches the index node information, the client can carry the index node information when sending a request to the storage access node of the distributed service system again.

According to the metadata processing method provided by the invention, the metadata inquiry requests sent by the client side for the first time are subjected to the layer-by-layer catalog inquiry to respectively obtain the metadata attribution service information of each layer of the file to be inquired so as to determine the index node information of the upper catalog of the file to be inquired, and the client side caches the index node information to enable the metadata processing requests sent subsequently to carry the index node information, so that the metadata inquiry can be directly carried out according to the index node information, and the metadata processing efficiency is improved.

In some embodiments, the determining, based on the inode information, home metadata service information for the file to be queried includes:

determining a first hash value based on the inode information;

Or alternatively, the first and second heat exchangers may be,

The present embodiment provides different ways of determining the home metadata service information.

Firstly, it should be noted that, in this embodiment, the query mode of the home metadata service information may be according to a directory query or according to a file query.

Under the condition of inquiring according to the catalogue, for the attribution metadata service information of the file to be inquired, a first hash value can be calculated according to index node information inode of the B catalogue by a storage access node, and then attribution metadata service mds of the file to be inquired can be calculated according to a consistent hash algorithm.

Correspondingly, under the condition of inquiring according to the file, for the attribution metadata service information of the file to be inquired, a first hash value can be calculated by the storage access node according to the index node information inode of the B directory and the file name of the file to be inquired, namely file1, and then the attribution metadata service mds of the file to be inquired can be calculated according to a consistent hash algorithm.

According to the metadata processing method provided by the invention, the attribution metadata service information of the file to be queried is calculated by utilizing a consistent hash algorithm according to two modes of directory query or file query, and the attribution metadata service information can be realized by different query modes.

In some embodiments, the method further comprises:

And returning the metadata of the last hierarchy to the client.

It can be understood that, in this embodiment, the process of performing the hierarchical directory query according to the metadata processing request may be performed on the premise that the metadata processing request sent by the client is received for the first time, or the metadata processing request sent by the client is received for the nth time (N is a positive integer greater than 1), and when the request is received for the nth time, it is judged that the metadata processing request does not carry the inode information of the upper directory of the file to be queried, which may be caused by failure of the client to cache the inode information of the upper directory, and at this time, metadata of each level needs to be obtained for the metadata processing request still according to the hierarchical directory query mode, and metadata of the last level is returned to the client.

According to the metadata processing method provided by the invention, the index node information of the upper-level catalogue of the file to be queried is not carried in the second metadata processing request, the layer-by-layer catalogue query is carried out to obtain the metadata of the file to be queried, and the metadata obtained by the query is sent to the client, so that the subsequent query can be directly carried out through the index node when the metadata processing request is sent again, and the metadata processing efficiency is improved.

In some embodiments, the step of performing a hierarchical directory query based on the metadata query request, to obtain home metadata service information of each level and metadata of each level corresponding to the file to be queried, includes:

The embodiment is a layer-by-layer directory query process performed when a metadata query request sent by a client is received for the first time.

In this embodiment, a root directory and at least two layers of parent directories of a file to be queried are determined according to a metadata query request. For example, in the HEAD/a/B/file 1 request, the root directory is typically the root directory of a storage access node of the distributed service system, and the two layers of parent directories are a and B, where a is a first layer of parent directory, and B is a second layer of parent directory, i.e. an upper layer directory of the file to be queried.

Alternatively, the parent directory may be multiple levels, i.e., starting from the root directory, and multiple levels of parent directories may be extended down, such as HEAD/A/B/C/D/E/F/file 1, where the upper directory of file1 is the F directory.

And then sequentially acquiring the first attribution metadata service information of the first-layer parent directory, the second attribution metadata service information of the second-layer parent directory and the third attribution metadata service information of the file to be queried according to the hierarchical relationship between the root directory and each layer of parent directory.

Further, the sequentially obtaining, according to the hierarchical relationships of the root directory, the first-layer parent directory and the second-layer parent directory, the first home metadata service information of the first-layer parent directory, the second home metadata service information of the second-layer parent directory, and the third home metadata service information of the file to be queried, includes:

The embodiment is a specific process of acquiring the home metadata service information of each layer of directory or file.

First, determining index node information of a root directory, determining first attribution metadata service information of a first-layer parent directory according to the index node information of the root directory, and inquiring first metadata of the first-layer parent directory according to the first attribution metadata service information. That is, the storage access node calculates the home mds of the A directory according to the root directory inode, and queries the metadata of the A directory.

And then, determining the index node information of the first-layer father catalog, namely the A catalog, according to the first metadata, determining the second attribution metadata service information of the second-layer father catalog according to the index node information of the A catalog, and inquiring the second metadata of the second-layer father catalog according to the second attribution metadata service information. That is, the storage access node calculates the home mds of the B directory according to the a directory inode, and queries the metadata of the B directory.

And finally, determining the index node information of the second-layer parent directory according to the second metadata, and determining the third attribution metadata service information of the file to be queried based on the index node information of the second-layer parent directory. I.e. the storage access node calculates the home mds of file1 from the B-directory inode. And the metadata of the file1 can be queried after the attribution mds of the file1 to be queried is acquired.

According to the metadata processing method provided by the invention, the metadata inquiry requests are inquired in a layer-by-layer manner to respectively obtain the metadata attribution service information of each layer of the file to be inquired so as to determine the index node information of the upper-level directory of the file to be inquired, and the client caches the index node information to enable the index node information to be carried by the metadata processing requests which are subsequently sent, so that the metadata inquiry can be directly carried out according to the index node information, and the metadata processing efficiency is improved.

In some embodiments, the determining the first home metadata service information for the first tier parent directory based on the root directory's inode information comprises:

Further, the determining the second home metadata service information of the second layer parent directory based on the inode information of the first layer parent directory includes:

Still further, the determining the third home metadata service information of the file to be queried based on the inode information of the second-layer parent directory includes:

It will be appreciated that this embodiment is a specific process of calculating home metadata service information for a directory or file according to a hash algorithm.

When the attribution mds is selected according to the catalogs, the attribution mds of one catalogs stores metadata of the subdirectories and the files in the catalogs, the storage access node calculates a hash value according to the catalogue inode, and then calculates the attribution mds of the catalogs, namely the attribution mds of the subdirectories and the files in the catalogs according to a consistency hash algorithm.

For the home metadata service information of the a directory, the second hash value can be calculated by storing the index node information inode of the access node according to the root directory, and then the home metadata service mds of the B directory can be calculated according to the consistent hash algorithm.

For the home metadata service information of the B directory, the access node may calculate a second hash value according to the inode information inode of the a directory, and then calculate the home metadata service mds of the B directory according to the consistent hash algorithm.

For the home metadata service information of the file1 to be queried, a third hash value can be calculated according to the inode information inode of the B directory by the storage access node, and then the home metadata service mds of the file1 to be queried is calculated according to a consistent hash algorithm.

In some embodiments, the method further comprises:

In this embodiment, a third metadata processing request sent by a client is received, where the third metadata processing request is a request carrying routing information of a home metadata service of a file to be queried, and generally, the routing information of the home metadata service of the file to be queried is index node information of an upper directory of the file to be queried. That is, when determining that the third metadata processing request carries the routing information of the home metadata service of the file to be queried, that is, determining the index node of the upper directory of the file to be queried in the request, the home metadata service information of the file to be queried can be queried directly according to the index node, so as to obtain the target metadata of the file to be queried.

According to the metadata processing method provided by the invention, the routing information of the attribution original data service of the file to be queried is determined in the third metadata processing request, so that the index node of the upper-level catalog of the file to be queried is determined, the attribution metadata service information of the file to be queried is directly queried according to the index node, and the metadata processing efficiency is improved.

In some embodiments, the home metadata service information comprises at least one of:

inode information of the parent directory;

inode information of a file or a target;

logical identification of the home metadata service.

Referring to fig. 5, the present invention further provides a metadata processing method, applied to a client, the method including:

step 510, a first metadata processing request is sent to a storage access node, wherein the first metadata processing request carries index node information of an upper-level directory of a file to be queried;

step 520, receiving target metadata sent by the storage access node; wherein the target metadata is obtained by the storage access node based on the inode information.

In step 510, a first metadata processing request is sent to a storage access node of the distributed service system by a client, where the client is an object client that sends a metadata query request, and the first metadata processing request is a metadata query request sent by the client, for example, the request is a HEAD/a/B/file 1 request.

It should be noted that, the first metadata processing request carries inode information of the upper directory of the file to be queried.

In step 520, target metadata sent by the storage access node is received, where the target metadata is obtained by the storage access node based on the index node information of the file to be queried, i.e. obtained by the index node of the B-directory.

According to the metadata processing method provided by the invention, the client side sends the first metadata processing request to the storage access node, the index node information of the upper-level catalogue of the file to be queried is determined to be carried in the request, and the home metadata service of the file to be queried can be directly acquired according to the index node, so that the target metadata of the file to be queried can be queried according to the home metadata service. The invention does not need to carry out layer-by-layer directory inquiry according to the request sent by the client, thereby simplifying the overall flow of metadata processing, shortening the processing time delay of the metadata and improving the metadata processing efficiency.

In some embodiments, the method further comprises:

Firstly, a client sends a metadata query request to a storage access node, and the metadata query request is a request sent by the client for the first time, wherein the request is used for querying metadata of a file to be queried. And then carrying out layer-by-layer directory inquiry according to the metadata inquiry request to obtain the attribution metadata service information of each layer corresponding to the file to be inquired and the metadata of each layer.

Further, receiving and storing the index node information of the upper-level catalogue of the file to be queried, namely the index node of the B catalogue, sent by the access node, and caching the index node information.

After the client caches the index node information, the client can carry the index node information when sending a request to the storage access node of the distributed service system again.

In some embodiments, the method further comprises:

1. sending a second metadata processing request to a storage access node, wherein the second metadata processing request is used for requesting to query metadata of the file to be queried, and the second metadata processing request does not carry index node information of an upper directory of the file to be queried;

2. Sending a third metadata processing request to a storage access node, wherein the third metadata processing request is used for requesting to query metadata of the file to be queried, and the third metadata processing request carries routing information of home metadata service of the file to be queried;

The embodiment is two different processing modes when the client sends the metadata processing request to the storage access node.

If the metadata processing request carries the routing information of the attribution metadata service of the file to be queried, the routing information can be determined to be the index node information of the upper-level catalogue of the file to be queried, the attribution metadata service information of the file to be queried is directly queried according to the index node information, and then the target metadata of the file to be queried is determined.

If it is determined that the metadata processing request does not carry the inode information of the upper directory of the file to be queried, the request can be used as a request sent by the client for the first time, and the request is used for querying the attribution metadata service information of the file to be queried layer by layer.

According to the metadata processing method provided by the invention, whether the index node of the upper-level catalogue of the file to be queried is carried in the metadata processing request is judged, so that the metadata of the file to be queried is queried in different modes, when the index node of the upper-level catalogue of the file to be queried is carried, the query of the catalogue layer by layer is not needed, the metadata query is directly carried out according to the index node information, and the metadata processing efficiency is improved.

Referring to fig. 6, fig. 6 is an interaction schematic diagram of a metadata processing method provided by the present invention;

1) When the object client side sends HEAD/A/B/file 1 for the first time, the distributed storage still needs to inquire mds in a layer-by-layer directory;

2) The storage access node calculates the attribution mds of the A directory according to the root directory inode (index node), and inquires the metadata of the A directory;

3) The storage access node calculates the attribution mds of the B directory according to the A directory inode and inquires the metadata of the B directory;

4) The storage access node calculates the attribution mds of the file1 according to the B directory inode, and inquires the metadata of the file 1;

5) After receiving the response, the object client caches the home mds route information corresponding to the A/B/file1, namely the inode of the B directory;

6) When the object client sends HEAD/A/B/file 1 again, carrying an inode of the B directory;

7) The distributed stored access node judges that the HEAD request carries the route information of the home mds, namely the inode of the B directory, directory metadata is not queried layer by layer, the home mds of the file1 is directly calculated according to the inode of the B directory, and the home mds of the file1 is queried, so that the mds of the A directory and the B directory do not need to be queried.

The metadata processing apparatus provided by the present invention will be described below, and the metadata processing apparatus described below and the metadata processing method described above may be referred to correspondingly to each other.

Referring to fig. 7, the present invention provides a metadata processing apparatus applied to a storage access node, the apparatus comprising:

a first receiving module 710, configured to receive a first metadata processing request sent by a client;

the parsing module 720 is configured to parse the first metadata processing request, and determine that the first metadata processing request carries inode information of an upper directory of a file to be queried;

a determining module 730, configured to determine, based on the inode information, home metadata service information of the file to be queried;

the first sending module 740 is configured to obtain target metadata of the file to be queried based on the home metadata service information of the file to be queried, and send the target metadata to the client.

According to the metadata processing device provided by the invention, the index node information of the upper-level catalogue of the file to be queried is determined by receiving the first metadata processing request sent by the client, and the home metadata service of the file to be queried can be directly acquired according to the index node, so that the target metadata of the file to be queried can be queried according to the home metadata service. The invention does not need to carry out layer-by-layer directory inquiry according to the request sent by the client, thereby simplifying the overall flow of metadata processing, shortening the processing time delay of the metadata and improving the metadata processing efficiency.

In some embodiments, the apparatus further comprises a layer-by-layer query module to:

determining a first hash value based on the inode information;

In some embodiments, the apparatus further comprises a first determination module configured to:

and returning the metadata of the last hierarchy to the client.

In some embodiments, the sequentially obtaining the first home metadata service information of the first layer parent directory, the second home metadata service information of the second layer parent directory, and the third home metadata service information of the file to be queried according to the hierarchical relationship of the root directory, the first layer parent directory, and the second layer parent directory includes:

In some embodiments, the determining the second home metadata service information for the second tier parent directory based on the inode information for the first tier parent directory includes:

In some embodiments, the determining third home metadata service information of the file to be queried based on the inode information of the second-tier parent directory includes:

In some embodiments, the apparatus further comprises a second determination module for:

In some embodiments, the metadata processing request is a head request sent by the client over a target communication protocol;

inode information of the parent directory;

inode information of a file or a target;

logical identification of the home metadata service.

Referring to fig. 8, the present invention also provides a metadata processing apparatus applied to a client, the apparatus comprising:

a second sending module 810, configured to send a first metadata processing request to a storage access node, where the first metadata processing request carries index node information of an upper directory of a file to be queried;

A second receiving module 820, configured to receive target metadata sent by the storage access node; wherein the target metadata is obtained by the storage access node based on the inode information.

According to the metadata processing device provided by the invention, the client side sends the first metadata processing request to the storage access node, the index node information of the upper-level catalogue of the file to be queried is determined to be carried in the request, and the home metadata service of the file to be queried can be directly acquired according to the index node, so that the target metadata of the file to be queried can be queried according to the home metadata service. The invention does not need to carry out layer-by-layer directory inquiry according to the request sent by the client, thereby simplifying the overall flow of metadata processing, shortening the processing time delay of the metadata and improving the metadata processing efficiency.

In some embodiments, the apparatus further includes a caching module, where the caching module is specifically configured to:

In some embodiments, the apparatus further includes a third determining module, where the third determining module is specifically configured to:

In some embodiments, the apparatus further includes a fourth determination module, where the fourth determination module is specifically configured to:

Fig. 9 illustrates a physical schematic diagram of an electronic device, as shown in fig. 9, which may include: processor 910, communication interface (Communications Interface), memory 930, and communication bus 940, wherein processor 910, communication interface 920, and memory 930 communicate with each other via communication bus 940. Processor 910 can invoke logic instructions in memory 930 to perform a metadata processing method comprising:

receiving a first metadata processing request sent by a client;

acquiring target metadata of the file to be queried based on the home metadata service information of the file to be queried, and sending the target metadata to the client;

or alternatively, the first and second heat exchangers may be,

Further, the logic instructions in the memory 930 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the metadata processing method provided by the above methods, the method comprising:

Receiving a first metadata processing request sent by a client;

or alternatively, the first and second heat exchangers may be,

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the metadata processing method provided by the above methods, the method comprising:

receiving a first metadata processing request sent by a client;

or alternatively, the first and second heat exchangers may be,

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of metadata processing applied to a storage access node, the method comprising:

receiving a first metadata processing request sent by a client;

2. The metadata processing method according to claim 1, wherein the method further comprises:

3. The metadata processing method according to claim 1, wherein determining the home metadata service information of the file to be queried based on the inode information comprises:

determining a first hash value based on the inode information;

4. The metadata processing method according to claim 1, wherein determining the home metadata service information of the file to be queried based on the inode information comprises:

5. The metadata processing method according to claim 1, wherein the method further comprises:

and returning the metadata of the last hierarchy to the client.

6. The method for processing metadata according to claim 2, wherein the step-by-step directory query based on the metadata query request obtains home metadata service information of each level and metadata of each level corresponding to the file to be queried, including:

7. The method according to claim 6, wherein sequentially obtaining the first home metadata service information of the first layer parent directory, the second home metadata service information of the second layer parent directory, and the third home metadata service information of the file to be queried according to the hierarchical relationship among the root directory, the first layer parent directory, and the second layer parent directory, comprises:

8. The method according to claim 7, wherein the determining the first home metadata service information of the first-layer parent directory based on the inode information of the root directory includes:

9. The method of claim 7, wherein the determining the second home metadata service information of the second level parent directory based on the inode information of the first level parent directory comprises:

10. The method according to claim 7, wherein determining third home metadata service information of the file to be queried based on the inode information of the second-layer parent directory comprises:

11. The metadata processing method according to claim 1, wherein the method further comprises:

12. The metadata processing method according to any one of claims 1 to 11, wherein the metadata processing request is a HEAD request sent by the client through a target communication protocol;

13. The metadata processing method according to any one of claims 1 to 11, wherein the home metadata service information includes at least one of:

inode information of the parent directory;

inode information of a file or a target;

logical identification of the home metadata service.

14. A metadata processing method, applied to a client, the method comprising:

15. The method of metadata processing according to claim 14, wherein the method further comprises:

16. The method of metadata processing according to claim 14, wherein the method further comprises:

17. The method of metadata processing according to claim 14, wherein the method further comprises:

18. A metadata processing apparatus for use in a storage access node, the apparatus comprising:

19. A metadata processing apparatus, for application to a client, the apparatus comprising:

20. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the metadata processing method according to any one of claims 1 to 13 or the metadata processing method according to any one of claims 14 to 17 when executing the program.

21. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the metadata processing method of any of claims 1 to 13 or implements the metadata processing method of any of claims 14 to 17.