CN112035413A - Metadata information query method and device and storage medium - Google Patents
Metadata information query method and device and storage medium Download PDFInfo
- Publication number
- CN112035413A CN112035413A CN202010917087.4A CN202010917087A CN112035413A CN 112035413 A CN112035413 A CN 112035413A CN 202010917087 A CN202010917087 A CN 202010917087A CN 112035413 A CN112035413 A CN 112035413A
- Authority
- CN
- China
- Prior art keywords
- node
- path information
- metadata
- information
- metadata information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 29
- 230000006870 function Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 10
- 238000007726 management method Methods 0.000 description 82
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/1827—Management specifically adapted to NAS
Abstract
The embodiment of the application discloses a metadata information query method, a metadata information query device and a storage medium, and belongs to the technical field of file storage. In this embodiment of the present application, the bloom filter includes path information corresponding to the metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be inquired, and then whether the metadata information corresponding to the path information is inquired through the second node is determined. Therefore, compared with the blind query through the second node, the processing pressure of the second node is reduced, and the query efficiency of the metadata information is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of file storage, in particular to a metadata information query method, a metadata information query device and a storage medium.
Background
A NAS (Network Attached Storage) cluster system typically includes NAS clients, NAS gateway nodes, cluster management nodes, and file metadata management nodes. The NAS client is used for receiving file operation of a user and sending a corresponding request to the NAS gateway node according to the received file operation. For example, when the file operation is a download operation, the NAS client sends a read request to the NAS gateway node. The read request carries the path information of the file or folder to be read. Since the cluster management node manages the metadata information of the folder, the file metadata management node manages the metadata information of the file, and the NAS gateway node cannot distinguish whether the read request is for the folder or the file according to the path information. Therefore, the NAS gateway node cannot determine whether to acquire the metadata information corresponding to the path information by the cluster management node or the file metadata management node.
Based on this, currently, the NAS gateway node usually performs query through the cluster management node according to the path information. And if the metadata information corresponding to the path information cannot be inquired, inquiring the corresponding metadata information through the file metadata management node according to the path information. Or the NAS gateway node firstly queries through the file metadata management node, and if the NAS gateway node cannot query, the NAS gateway node queries through the cluster management node. Thus, for the node that is queried first, since each query of metadata information needs to pass through the node, the processing pressure of the node is large. In addition, since it is uncertain which node manages metadata information to be queried, it is necessary to perform a test on a node-by-node basis, which affects the query rate of the metadata information.
Disclosure of Invention
The embodiment of the application provides a metadata information query method, a metadata information query device and a storage medium, which can reduce the processing pressure of nodes and improve the metadata information query rate of an NAS cluster system. The technical scheme is as follows:
in one aspect, a metadata information query method is provided, and the method includes:
acquiring path information to be inquired;
and if the bloom filter does not contain the path information, inquiring metadata information corresponding to the path information through a first node, wherein the bloom filter comprises the path information corresponding to the metadata information managed by a second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In a possible implementation manner of the embodiment of the present application, the method further includes:
processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and if the numerical value at any one of a plurality of positions corresponding to the plurality of hash values in a one-to-one manner in the bit array contained in the bloom filter is not the first numerical value, determining that the bloom filter does not contain the path information.
In a possible implementation manner of the embodiment of the present application, the method further includes:
if the numerical values at the positions corresponding to the hash values in a bit array contained in the bloom filter are the first numerical values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not inquired through the second node, inquiring the metadata information corresponding to the path information through the first node.
In a possible implementation manner of the embodiment of the present application, the first node is a file metadata management node in a network attached storage NAS cluster system, the second node is a cluster management node in the NAS cluster system, metadata information of a file is managed in the file metadata management node, and metadata information of a folder is managed in the cluster management node.
In a possible implementation manner of the embodiment of the present application, the method further includes:
receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created;
adding the path information of the folder to be created to the bloom filter, and sending the metadata information of the folder to be created to the second node.
In a possible implementation manner of the embodiment of the present application, the first node is a cluster management node in an NAS cluster system, the second node is a file metadata management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In another aspect, there is provided a metadata information query apparatus, the apparatus including:
the acquisition module is used for acquiring the path information to be inquired;
and the query module is used for querying the metadata information corresponding to the path information through the first node if the bloom filter does not contain the path information, the bloom filter comprises the path information corresponding to the metadata information managed by the second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In a possible implementation manner of the embodiment of the present application, the apparatus further includes:
the processing module is used for processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
a determining module, configured to determine that the bloom filter does not include the path information if a value at any one of a plurality of positions in a bit array included in the bloom filter, which correspond to the plurality of hash values one to one, is not a first value.
In a possible implementation manner of the embodiment of the present application, the query module is further configured to:
if the numerical values at the positions corresponding to the hash values in a bit array contained in the bloom filter are the first numerical values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not inquired from the second node, inquiring the metadata information corresponding to the path information through the first node.
In a possible implementation manner of the embodiment of the present application, the first node is a file metadata management node in a network attached storage NAS cluster system, the second node is a cluster management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In a possible implementation manner of the embodiment of the present application, the apparatus further includes:
the device comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a folder creation request which carries path information of a folder to be created;
the adding module is used for adding the path information of the folder to be created to the bloom filter;
and the storage module is used for sending the metadata information of the folder to be created to the second node.
In a possible implementation manner of the embodiment of the present application, the first node is a cluster management node in an NAS cluster system, the second node is a file metadata management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In another aspect, a computer device is provided, where the computer device includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus, the memory is used to store a computer program, and the processor is used to execute the program stored in the memory to implement the steps of the metadata information query method.
In another aspect, a computer-readable storage medium is provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the metadata information query method described above.
In another aspect, a computer program product containing instructions is provided, which when run on a computer, causes the computer to perform the steps of the above-described metadata information query method.
The technical scheme provided by the embodiment of the application can at least bring the following beneficial effects:
in this embodiment of the present application, the bloom filter includes path information corresponding to the metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be inquired or not, and then whether the metadata information corresponding to the path information is inquired through the second node or not is determined. Therefore, compared with the blind query through the second node, the processing pressure of the second node is reduced, and the query efficiency of the metadata information is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a diagram of an architecture of a NAS cluster system according to an embodiment of the present application;
fig. 2 is a flowchart of a metadata information query method provided in an embodiment of the present application;
FIG. 3 is a flowchart of a method for adding path information to a bloom filter according to an embodiment of the present application;
FIG. 4 is a schematic diagram illustrating a principle of adding path information to a bloom filter according to an embodiment of the present application;
FIG. 5 is a flow diagram of another metadata information query provided by an embodiment of the present application;
fig. 6 is a schematic structural diagram of a metadata information query apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.
Before explaining the metadata information query method provided by the embodiment of the present application in detail, a system architecture related to the embodiment of the present application is introduced.
Fig. 1 is a diagram of an architecture of a NAS cluster system according to an embodiment of the present application. As shown in fig. 1, the NAS cluster system includes a NAS client 101, a NAS gateway node 102, a cluster management node 103, a file metadata management node 104, and a cloud storage node 105.
The NAS client 101 and the NAS gateway node 102 establish communication connection, the NAS gateway node 102 establishes communication connection with the cluster management node 103, the file metadata management node 104 and the cloud storage node 105, and the cluster management node 103 and the file metadata management node 104 establish communication connection. The number of the NAS gateway nodes 102 is one or more, and the number of the file metadata management nodes 104 is one or more.
It should be noted that the cluster management node 103 is used for managing metadata information of folders, and besides, the cluster management node 103 is also used for managing configuration information of the NAS cluster system and one or more file metadata management nodes 104. The file metadata management node 104 is used to manage metadata information of a file.
In a possible implementation manner, databases are respectively disposed on the cluster management node 103 and the file metadata management node 104, metadata information of a folder managed by the cluster management node 103 is stored in the database disposed on the cluster management node 103, and metadata information of a file managed by the file metadata management node 104 is stored in the database disposed on the file metadata management node 104. Alternatively, the metadata information of the folders managed by the cluster management node 103 and the metadata information of the files managed by the file metadata management node 104 are stored in one database. The embodiment of the present application does not limit this.
The cloud storage node 105 provides an underlying data storage function, that is, data for storing files and the like.
The NAS client 101 is configured to receive a file operation of a user, such as an access operation of a folder, a read-write operation of a file, and the like. After receiving the file operation of the user, the NAS client 101 generates a corresponding request and sends the corresponding request to the NAS gateway node 102 through the NAS protocol.
After receiving the request sent by the NAS client 101, the NAS gateway node 102 determines whether to query the metadata information corresponding to the path information according to the type of the request. If the path information is queried, the path information carried in the request is acquired, and the metadata information corresponding to the path information is queried through the cluster management node 103 or the file metadata management node 104 by using the metadata information query method provided by the embodiment of the application. After obtaining the metadata information corresponding to the path information, the NAS gateway node 102 operates the data stored in the cloud storage node 105 according to the metadata information.
It should be noted that the above nodes may be distributed and deployed on multiple devices, that is, each node is a separate device. Or, in some possible implementations, some of the nodes except for the NAS client may be deployed on the same device, which is not limited in this embodiment of the present application.
The following explains the metadata information query method provided in the embodiments of the present application in detail.
Fig. 2 is a flowchart of a metadata information query method provided in an embodiment of the present application, where the method is applied to any NAS gateway node in the NAS cluster system shown in fig. 1. Referring to fig. 2, the method includes the following steps.
Step 201: and acquiring path information to be inquired.
In the embodiment of the application, when receiving a file operation of a user, the NAS client generates a corresponding request according to the file operation and sends the request to the NAS gateway node. The file operation may include an operation on an existing folder and/or file in the NAS cluster system, for example, a download operation for an existing folder or file in the NAS cluster system. For the requests corresponding to the file operation, the requests carry path information of the files or folders to be operated.
After receiving the type of request sent by the NAS client, the NAS gateway node cannot distinguish whether the request needs to be operated by a folder or a file, so that the NAS gateway node obtains path information carried in the request, so as to distinguish whether the request needs to be operated by a folder or a file by the method provided by the embodiment of the present application, and further obtains metadata information of the folder or the file from a corresponding node. At this time, the path information acquired by the NAS gateway node from the request is the path information to be queried.
Step 202: if the bloom filter does not contain the path information, the metadata information corresponding to the path information is inquired through the first node, the bloom filter comprises the path information corresponding to the metadata information managed by the second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In the embodiment of the present application, the NAS gateway node includes a bloom filter. The bloom filter comprises a plurality of hash functions and a bit array, and comprises path information corresponding to metadata information managed by the second node.
It should be noted that, in one possible implementation manner, the second node is a cluster management node in the NAS cluster system, and correspondingly, the first node is a file metadata management node in the NAS cluster system. In this case, since the cluster management node is used to manage the metadata information of the folders in the NAS cluster system, the bloom filter includes path information of the folders in the NAS cluster system.
In another possible implementation manner, the second node refers to a file metadata management node in the NAS cluster system, and the first node refers to a cluster management node in the NAS cluster system. In this case, since the file metadata management node is used to manage metadata information of the files in the NAS cluster system, the bloom filter includes path information of the files in the NAS cluster system.
Since the bloom filter includes path information corresponding to the metadata information managed by the second node, that is, the bloom filter includes path information of a folder or path information of a file in the NAS cluster system. Therefore, before performing this step, the NAS gateway node needs to collect the path information of the folder or the file, and then add the path information of the folder or the file to the bloom filter. Based on this, before introducing the implementation process of this step, the NAS gateway node first collects the path information of the folder, and introduces the process of adding the path information of the folder to the bloom filter. It should be noted that, for an implementation manner in which the NAS gateway node collects path information of a file and adds the path information of the file in the bloom filter, the following implementation process may be referred to, and this is not described in detail in this embodiment of the present application.
Illustratively, FIG. 3 shows a flow diagram for adding path information for a folder in a bloom filter. Referring to fig. 3, the process includes the steps of:
301: receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created.
When a user wants to create a folder, a folder creation operation is performed on the NAS client. After receiving the operation, the NAS client generates a folder creation request and sends the folder creation request to the NAS gateway node, and accordingly, the NAS gateway node receives the folder creation request. Wherein the folder creation request carries path information of a folder to be created.
302: and adding the path information of the folder to be created to a bloom filter.
After receiving the folder creation request, the NAS gateway node acquires the path information of the folder to be created, which is carried in the folder creation request, and then, the NAS gateway node adds the path information of the folder to be created to the bloom filter.
As can be seen from the foregoing description, the bloom filter is composed of a plurality of hash functions and a bit array, wherein the initial value of the value at each position in the bit array is the second value. And the NAS gateway node performs hash operation on the path information of the folder to be created through the plurality of hash functions so as to obtain a plurality of hash values. And then, the NAS gateway node sets the numerical values on a plurality of positions in the bit array, which are in one-to-one correspondence with the plurality of hash values, as first numerical values. In this way, the path information of the folder to be created is added to the bloom filter. Wherein the first value is 1 and the second value is 0. Alternatively, the first value is 0 and the second value is 1.
Fig. 4 is a schematic diagram illustrating a principle of adding path information to a bloom filter according to an embodiment of the present application. As shown in fig. 4, assuming that the path information of the folder to be created is m, the hash functions in the bloom filter are H1、H2And H3The bit array is n in length. Suppose the NAS gateway node is according to H1And m, the hash operation is performed to obtain a hash value k1 of 2. According to H2And m, the hash operation is performed to obtain a hash value k2 of 4. According to H3And m, performing hash operation to obtain a hash value k3 being 5, and setting the numerical values of the 2 nd bit, the 4 th bit and the 5 th bit in the bit array to be 1 by the NAS gateway node, so that the path information m is added to the bloom filter.
303: and sending the metadata information of the folder to be created to the cluster management node.
And when the path information of the folder to be created is added to the bloom filter, the NAS gateway node creates the folder according to the path information and sends the metadata information of the folder to the cluster management node. And after receiving the metadata information of the folder, the cluster management node stores the metadata information of the folder into a database.
Through the above step 301 and 303, the NAS gateway node adds the path information of the folder to the bloom filter in the process of creating the folder, and sends the metadata information of the folder to the cluster management node for management. In this way, the bloom filter includes path information corresponding to the metadata information of the folder managed by the cluster management node.
The foregoing is an implementation manner of adding path information of a folder to a bloom filter according to an embodiment of the present application. In some possible embodiments, the NAS gateway node may also obtain, at regular intervals, path information of newly added folders in the cluster management node according to metadata information of the folders, and further add the path information to the bloom filter. The embodiments of the present application do not limit this.
Based on the above description, after adding the path information of the folder or the path information of the file to the bloom filter, when the NAS gateway node acquires the path information to be queried, the NAS gateway node can query the path information in the bloom filter, and distinguish whether the path information is the path information of the folder or the path information of the file according to whether the path information is queried in the bloom filter.
Exemplarily, the NAS gateway node processes the path information to be queried according to a plurality of hash functions included in the bloom filter to obtain a plurality of hash values corresponding to the path information; and if the numerical value at any one of the positions in the bit array contained in the bloom filter, which correspond to the hash values one by one, is not the first numerical value, determining that the bloom filter does not contain the path information.
It should be noted that, since the path information corresponding to the metadata information managed by the second node has been added to the bloom filter by the method described above, if the path information to be queried is the path information corresponding to the metadata information managed by the second node, the hash values obtained by processing the path information to be queried by the hash functions and the hash values obtained by adding the path information to the bloom filter are necessarily equal to each other. In this way, the values at the positions in the bit array corresponding to the hash values one to one are also necessarily all the first values. Based on this, if the value at any position of the plurality of positions corresponding to the hash values corresponding to the path information to be queried in the bit array is not the first value, it can be determined that the path information to be queried is not included in the bloom filter. That is, the path information to be queried is not the path information corresponding to the metadata information managed by the second node.
Under the condition that it is determined that the path information to be queried is not the path information corresponding to the metadata information managed by the second node, the NAS gateway node does not need to query the metadata information corresponding to the path information in the second node, but can directly query the metadata information corresponding to the path information through the first node. Therefore, compared with the method that the metadata information is inquired through the second node and the first node in sequence, the processing pressure of the second node is reduced, and the inquiry efficiency of the metadata information is improved.
When the NAS gateway node queries the metadata information corresponding to the path information to be queried through the first node, the NAS gateway node may send a query request to the first node, where the query request carries the path information to be queried. After receiving the query request, the first node acquires corresponding metadata information according to the path information, and then feeds the metadata information back to the NAS gateway node.
Optionally, if the numerical values at the positions corresponding to the hash values corresponding to the path information to be queried in the bit array included in the bloom filter are all first numerical values, querying metadata information corresponding to the path information through a second node; and if the metadata information corresponding to the path information to be inquired is not inquired through the second node, inquiring the metadata information corresponding to the path information through the first node.
It should be noted that, in the process of adding the path information to the bloom filter, different path information corresponds to different hash values, so that the value at the position corresponding to the different hash values is the first value. However, for the path information that does not exist in the bloom filter, the numerical values at the positions corresponding to the hash values processed by the hash functions may be the first numerical value. For example, the bloom filter includes path information a and b. Assuming that the hash values corresponding to the path information a are 1, 2, and 8, and the hash values corresponding to the path information b are 3, 5, and 9, the values at the 1 st, 2 nd, 3 nd, 5 th, 8 th, and 9 th bits in the bit array will all be the first value. Assuming that the path information c does not exist in the bloom filter, but the hash values corresponding to the path information c obtained by the hash function processing are 2, 5, and 8, in this case, although the numerical values at the positions corresponding to the hash values in the bit array will all be the first numerical value, the path information c does not actually exist in the bloom filter.
Therefore, when the numerical values at the positions corresponding to the hash values corresponding to the path information to be queried are all the first numerical values, it is indicated that the bloom filter has a high probability of possibly including the path information, but it cannot be indicated that the bloom filter necessarily includes the path information. In this case, the NAS gateway node first queries the second node for metadata information corresponding to the path information. If not, the false identification of the bloom filter is illustrated. At this time, the first node queries the metadata information corresponding to the path information. Therefore, compared with completely blind metadata information which is not distinguished and directly passes through the nodes in sequence and corresponds to the query path information, the processing pressure of the nodes queried firstly can be reduced to a certain extent, and the query efficiency is improved.
In this embodiment of the present application, the bloom filter includes path information corresponding to the metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be inquired, and then whether the metadata information corresponding to the path information is inquired through the second node is determined. Therefore, compared with the blind query through the second node, the processing pressure of the second node is reduced, and the query efficiency of the metadata information is improved. Especially, when the second node is a cluster management node, because the number of the cluster management nodes included in the NAS cluster system is usually small and the processing pressure of the cluster management node itself is large, the method provided by the embodiment of the present application can better alleviate the processing pressure of the cluster management node, avoid the performance bottleneck of the cluster management node called as the NAS cluster system, and improve the metadata processing efficiency.
Optionally, in the above embodiment, the NAS gateway node comprises a bloom filter. In other possible embodiments, the bloom filter is deployed separately in a separate node. For example, the bloom filter is a bloom filter on a redis source node. Therefore, for different gateway nodes, the scheme is realized through the bloom filter, so that the global uniqueness of the data can be ensured.
Next, an implementation flow of the metadata query method provided in the embodiment of the present application is illustrated by an example, where in this example, the first node is a file metadata management node, and the second node is a cluster management node, that is, a bloom filter includes metadata information of a folder managed by the cluster management node.
Referring to fig. 5, firstly, the NAS gateway node receives path information to be queried, and then determines whether the bloom filter includes the path information to be queried. And if the bloom filter does not contain the path information to be inquired, the NAS gateway node acquires the attribute information corresponding to the path information from the file metadata management node and returns the acquired attribute information to the NAS client. And if the bloom filter contains the path information to be inquired, the NAS gateway node requests the cluster management node to acquire the attribute information corresponding to the path information, and judges whether the attribute information is acquired from the cluster management node. And if the attribute information is acquired from the cluster management node, returning the acquired attribute information to the NAS client. And if the attribute information cannot be acquired from the cluster management node, acquiring the attribute information corresponding to the path information from the file metadata management node, and returning the acquired attribute information to the NAS client.
Fig. 6 is a schematic structural diagram of a metadata information query apparatus 600 according to an embodiment of the present application, where the metadata information query apparatus may be implemented by software, hardware, or a combination of the two to be part or all of a NAS gateway node, and the NAS gateway node may be the NAS gateway node shown in fig. 1. Referring to fig. 6, the apparatus 600 includes: an acquisition module 601 and a query module 602.
An obtaining module 601, configured to obtain path information to be queried;
the querying module 602 is configured to query, by a first node, metadata information corresponding to path information if a bloom filter does not include the path information, where the bloom filter includes path information corresponding to metadata information managed by a second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In a possible implementation manner of the embodiment of the present application, the apparatus 600 further includes:
the processing module is used for processing the path information according to the plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and the determining module is used for determining that the bloom filter does not contain the path information if the numerical value at any position in the plurality of positions corresponding to the plurality of hash values in a one-to-one manner in the bit array contained in the bloom filter is not the first numerical value.
In a possible implementation manner of this embodiment of the present application, the query module 602 is further configured to:
if the numerical values at the positions corresponding to the hash values in a bit array contained in the bloom filter are all first numerical values, inquiring metadata information corresponding to the path information through a second node;
and if the metadata information corresponding to the path information is not inquired from the second node, inquiring the metadata information corresponding to the path information through the first node.
In a possible implementation manner of the embodiment of the present application, the first node is a file metadata management node in a network attached storage NAS cluster system, the second node is a cluster management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In a possible implementation manner of the embodiment of the present application, the apparatus 600 further includes:
the receiving module is used for receiving a folder creation request, and the folder creation request carries path information of a folder to be created;
the adding module is used for adding the path information of the folder to be created to the bloom filter;
and the storage module is used for sending the metadata information of the folder to be created to the second node.
In a possible implementation manner of the embodiment of the present application, the first node is a cluster management node in an NAS cluster system, the second node is a file metadata management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In this embodiment of the present application, the bloom filter includes path information corresponding to the metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be inquired or not, and then whether the metadata information corresponding to the path information is inquired through the second node or not is determined. Therefore, compared with the blind query through the second node, the processing pressure of the second node is reduced, and the query efficiency of the metadata information is improved.
It should be noted that: in the metadata information query apparatus provided in the above embodiment, when querying metadata information, only the division of the above functional modules is taken as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules, so as to complete all or part of the above described functions. In addition, the metadata information query apparatus provided in the foregoing embodiment and the metadata information query method embodiment belong to the same concept, and specific implementation processes thereof are described in the method embodiment and are not described herein again.
Fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application. The NAS gateway node in the foregoing embodiment may be implemented by the server. The server 700 includes a Central Processing Unit (CPU) 701, a system Memory 704 including a Random Access Memory (RAM) 702 and a Read-Only Memory (ROM) 703, and a system bus 705 connecting the system Memory 704 and the CPU 701. The server 700 also includes an I/O (basic Input/Output) system 706 that facilitates transfer of information between devices within the computer, and a mass storage device 707 for storing an operating system 713, application programs 714, and other program modules 715.
The basic input/output system 706 includes a display 708 for displaying information and an input device 709, such as a mouse, keyboard, etc., for a user to input information. Wherein the display 708 and the input device 709 are connected to the central processing unit 701 through an input output controller 710 connected to the system bus 705. The basic input/output system 706 may also include an input/output controller 710 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 710 may also provide output to a display screen, a printer, or other type of output device.
The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer-readable media provide non-volatile storage for the server 700. That is, the mass storage device 707 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM (Compact disk Read-Only Memory) drive.
Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash Memory or other solid state Memory technology, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 704 and mass storage device 707 described above may be collectively referred to as memory.
According to various embodiments of the present application, server 700 may also operate as a remote computer connected to a network via a network, such as the Internet. That is, the server 700 may be connected to the network 712 through a network interface unit 711 connected to the system bus 705, or the network interface unit 711 may be used to connect to other types of networks or remote computer systems (not shown).
The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.
In some embodiments, a computer-readable storage medium is also provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the metadata information query method in the above embodiments. For example, the computer readable storage medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
It is noted that the computer-readable storage medium referred to in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transitory storage medium.
It should be understood that all or part of the steps for implementing the above embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions may be stored in the computer-readable storage medium described above.
That is, in some embodiments, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of the above-described metadata information query method.
The above description should not be taken as limiting the embodiments of the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the embodiments of the present application.
Claims (13)
1. A metadata information query method, characterized in that the method comprises:
acquiring path information to be inquired;
and if the bloom filter does not contain the path information, inquiring metadata information corresponding to the path information through a first node, wherein the bloom filter comprises the path information corresponding to the metadata information managed by a second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
2. The method of claim 1, further comprising:
processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and if the numerical value at any one of a plurality of positions corresponding to the plurality of hash values in a one-to-one manner in the bit array contained in the bloom filter is not the first numerical value, determining that the bloom filter does not contain the path information.
3. The method of claim 2, further comprising:
if the numerical values at the positions corresponding to the hash values in a bit array contained in the bloom filter are the first numerical values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not inquired through the second node, inquiring the metadata information corresponding to the path information through the first node.
4. The method according to any of claims 1-3, wherein the first node is a file metadata management node in a Network Attached Storage (NAS) cluster system, the second node is a cluster management node in the NAS cluster system, the file metadata management node is used for managing metadata information of files, and the cluster management node is used for managing metadata information of folders.
5. The method of claim 4, further comprising:
receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created;
adding the path information of the folder to be created to the bloom filter, and sending the metadata information of the folder to be created to the second node.
6. The method according to any of claims 1-3, wherein the first node is a cluster management node in a NAS cluster system, the second node is a file metadata management node in the NAS cluster system, the file metadata management node is used for managing metadata information of files, and the cluster management node is used for managing metadata information of folders.
7. An apparatus for querying metadata information, the apparatus comprising:
the acquisition module is used for acquiring the path information to be inquired;
and the query module is used for querying the metadata information corresponding to the path information through the first node if the bloom filter does not contain the path information, the bloom filter comprises the path information corresponding to the metadata information managed by the second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
8. The apparatus of claim 7, further comprising:
the processing module is used for processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
a determining module, configured to determine that the bloom filter does not include the path information if a value at any one of a plurality of positions in a bit array included in the bloom filter, which correspond to the plurality of hash values one to one, is not a first value.
9. The apparatus of claim 8, wherein the query module is further configured to:
if the numerical values at the positions corresponding to the hash values in a bit array contained in the bloom filter are the first numerical values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not inquired from the second node, inquiring the metadata information corresponding to the path information through the first node.
10. The apparatus of any of claims 7-9, wherein the first node is a file metadata management node in a Network Attached Storage (NAS) cluster system, wherein the second node is a cluster management node in the NAS cluster system, wherein the file metadata management node is configured to manage metadata information of files, and wherein the cluster management node is configured to manage metadata information of folders.
11. The apparatus of claim 10, further comprising:
the device comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a folder creation request which carries path information of a folder to be created;
the adding module is used for adding the path information of the folder to be created to the bloom filter;
and the storage module is used for sending the metadata information of the folder to be created to the second node.
12. The apparatus of any of claims 7-9, wherein the first node is a cluster management node in a NAS cluster system, wherein the second node is a file metadata management node in the NAS cluster system, wherein the file metadata management node is configured to manage metadata information of a file, and wherein the cluster management node is configured to manage metadata information of a folder.
13. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010917087.4A CN112035413B (en) | 2020-09-03 | 2020-09-03 | Metadata information query method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010917087.4A CN112035413B (en) | 2020-09-03 | 2020-09-03 | Metadata information query method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112035413A true CN112035413A (en) | 2020-12-04 |
| CN112035413B CN112035413B (en) | 2024-02-20 |
Family
ID=73592133
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010917087.4A Active CN112035413B (en) | 2020-09-03 | 2020-09-03 | Metadata information query method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112035413B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114443913A (en) * | 2022-04-06 | 2022-05-06 | 创智和宇信息技术股份有限公司 | Metadata multi-function multi-condition based user-defined query method, system and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150483A (en) * | 2007-11-02 | 2008-03-26 | 华为技术有限公司 | Route table adjustment method, route query method and device and route table storage device |
| US20140036908A1 (en) * | 2012-07-31 | 2014-02-06 | Cisco Technology, Inc. | Recording packet routes using bloom filters |
| CN105408857A (en) * | 2013-07-29 | 2016-03-16 | 亚马逊科技公司 | Generating a multi-column index for relational databases by interleaving data bits for selectivity |
| CN109495525A (en) * | 2017-09-12 | 2019-03-19 | 华为技术有限公司 | Networking component, the method and computer readable storage medium for parsing content identification |
-
2020
- 2020-09-03 CN CN202010917087.4A patent/CN112035413B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150483A (en) * | 2007-11-02 | 2008-03-26 | 华为技术有限公司 | Route table adjustment method, route query method and device and route table storage device |
| US20140036908A1 (en) * | 2012-07-31 | 2014-02-06 | Cisco Technology, Inc. | Recording packet routes using bloom filters |
| CN105408857A (en) * | 2013-07-29 | 2016-03-16 | 亚马逊科技公司 | Generating a multi-column index for relational databases by interleaving data bits for selectivity |
| CN109495525A (en) * | 2017-09-12 | 2019-03-19 | 华为技术有限公司 | Networking component, the method and computer readable storage medium for parsing content identification |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114443913A (en) * | 2022-04-06 | 2022-05-06 | 创智和宇信息技术股份有限公司 | Metadata multi-function multi-condition based user-defined query method, system and medium |
| CN114443913B (en) * | 2022-04-06 | 2022-06-07 | 创智和宇信息技术股份有限公司 | Metadata multi-function multi-condition based user-defined query method, system and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112035413B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2020038623A (en) | Method, device, and system for storing data | |
| WO2021082157A1 (en) | Methods, systems and devices for data sharing, and data and metadata storage | |
| US10860604B1 (en) | Scalable tracking for database udpates according to a secondary index | |
| US20170031948A1 (en) | File synchronization method, server, and terminal | |
| US11429566B2 (en) | Approach for a controllable trade-off between cost and availability of indexed data in a cloud log aggregation solution such as splunk or sumo | |
| CN108614837B (en) | File storage and retrieval method and device | |
| CN111258978B (en) | Data storage method | |
| US20180248977A1 (en) | Selective distribution of messages in a publish-subscribe system | |
| CN110958300B (en) | Data uploading method, system, device, electronic equipment and computer readable medium | |
| EP3285186A1 (en) | Methods and procedures for timestamp-based indexing of items in real-time storage | |
| CN111490908B (en) | Network speed measurement method, device, equipment, medium and speed measurement system | |
| CN111694793A (en) | Log storage method and device and log query method and device | |
| US11934466B2 (en) | Direct cloud storage intake and upload architecture | |
| CN112035413B (en) | Metadata information query method, device and storage medium | |
| CN103905512A (en) | Data processing method and equipment | |
| US10956373B1 (en) | Page-based metadata system for distributed filesystem | |
| CN111147226A (en) | Data storage method, device and storage medium | |
| EP3572951A1 (en) | A method for handling requests in a storage system and a storage node for a storage system | |
| EP4030312A1 (en) | Method and apparatus for querying data, computing device, and storage medium | |
| CN109960695B (en) | Management method and device for database in cloud computing system | |
| CN111212138B (en) | Cross-site storage system and data information access method | |
| CN110990394B (en) | Method, device and storage medium for counting number of rows of distributed column database table | |
| CN108989245B (en) | User data storage method and device | |
| CN116594848B (en) | Task monitoring method, device, equipment, terminal equipment and storage medium | |
| CN116775570A (en) | Information query method and device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |