CN112035413B - Metadata information query method, device and storage medium - Google Patents
Metadata information query method, device and storage medium Download PDFInfo
- Publication number
- CN112035413B CN112035413B CN202010917087.4A CN202010917087A CN112035413B CN 112035413 B CN112035413 B CN 112035413B CN 202010917087 A CN202010917087 A CN 202010917087A CN 112035413 B CN112035413 B CN 112035413B
- Authority
- CN
- China
- Prior art keywords
- node
- path information
- metadata
- information
- bloom filter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012545 processing Methods 0.000 claims abstract description 27
- 230000006870 function Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 9
- 238000007726 management method Methods 0.000 description 82
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/1827—Management specifically adapted to NAS
Abstract
The embodiment of the application discloses a metadata information query method, a metadata information query device and a storage medium, and belongs to the technical field of file storage. In the embodiment of the present application, the bloom filter includes path information corresponding to metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be queried or not, and whether the metadata information corresponding to the path information is queried by the second node is further determined. Therefore, compared with blind inquiry through the second node, the method reduces the processing pressure of the second node and improves the inquiry efficiency of the metadata information.
Description
Technical Field
The embodiment of the application relates to the technical field of file storage, in particular to a metadata information query method, a metadata information query device and a storage medium.
Background
NAS (Network Attached Storage ) cluster systems typically include NAS clients, NAS gateway nodes, cluster management nodes, and file metadata management nodes. The NAS client is used for receiving file operation of a user and sending a corresponding request to the NAS gateway node according to the received file operation. For example, when the file operation is a download operation, the NAS client sends a read request to the NAS gateway node. The read request will carry path information of the file or folder to be read. Since the cluster management node manages metadata information of the folder, the file metadata management node manages metadata information of the file, and the NAS gateway node cannot distinguish whether the read request needs to read the folder or the file according to the path information. Therefore, the NAS gateway node cannot determine whether to obtain metadata information corresponding to the path information through the cluster management node or the file metadata management node.
Based on this, at present, NAS gateway nodes generally query through cluster management nodes according to the path information. If the metadata information corresponding to the path information cannot be queried, querying the corresponding metadata information through the file metadata management node according to the path information. Or the NAS gateway node firstly queries through the file metadata management node, and then queries through the cluster management node if the query is not completed. Thus, for the node that is queried first, the processing pressure of the node is greater because each time the metadata information is queried through the node. Further, since it is not determined which node the metadata information to be queried is managed at, it is necessary to perform a trial on a node-by-node basis, and the query rate of the metadata information is affected.
Disclosure of Invention
The embodiment of the application provides a metadata information query method, a metadata information query device and a storage medium, which can reduce the processing pressure of nodes and improve the metadata information query rate of an NAS cluster system. The technical scheme is as follows:
in one aspect, a metadata information query method is provided, the method including:
acquiring path information to be queried;
and if the path information is not contained in the bloom filter, inquiring metadata information corresponding to the path information through a first node, wherein the bloom filter comprises path information corresponding to metadata information managed by a second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In one possible implementation manner of the embodiment of the present application, the method further includes:
processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and if the numerical value in any one of a plurality of positions corresponding to the hash values in one-to-one correspondence in the bit array contained in the bloom filter is not the first numerical value, determining that the bloom filter does not contain the path information.
In one possible implementation manner of the embodiment of the present application, the method further includes:
if the values at a plurality of positions, which are in one-to-one correspondence with the hash values, in the bit array contained in the bloom filter are all the first values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not queried through the second node, querying the metadata information corresponding to the path information through the first node.
In a possible implementation manner of the embodiments of the present application, the first node is a file metadata management node in a network attached storage NAS cluster system, the second node is a cluster management node in the NAS cluster system, metadata information of a file is managed in the file metadata management node, and metadata information of a folder is managed in the cluster management node.
In one possible implementation manner of the embodiment of the present application, the method further includes:
receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created;
and adding the path information of the folder to be created to the bloom filter, and sending the metadata information of the folder to be created to the second node.
In a possible implementation manner of the embodiments of the present application, the first node is a cluster management node in a NAS cluster system, the second node is a file metadata management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In another aspect, there is provided a metadata information query apparatus, the apparatus including:
the acquisition module is used for acquiring path information to be queried;
and the query module is used for querying metadata information corresponding to the path information through a first node if the path information is not contained in a bloom filter, wherein the bloom filter comprises path information corresponding to metadata information managed by a second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In one possible implementation manner of the embodiment of the present application, the apparatus further includes:
the processing module is used for processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and the determining module is used for determining that the bloom filter does not contain the path information if the numerical value at any one of a plurality of positions corresponding to the hash values in one-to-one correspondence in the bit array contained in the bloom filter is not a first numerical value.
In a possible implementation manner of the embodiment of the present application, the query module is further configured to:
if the values at a plurality of positions, which are in one-to-one correspondence with the hash values, in the bit array contained in the bloom filter are all the first values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not queried from the second node, querying the metadata information corresponding to the path information through the first node.
In a possible implementation manner of the embodiments of the present application, the first node is a file metadata management node in a network attached storage NAS cluster system, the second node is a cluster management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In one possible implementation manner of the embodiment of the present application, the apparatus further includes:
the receiving module is used for receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created;
an adding module, configured to add path information of the folder to be created to the bloom filter;
and the storage module is used for sending the metadata information of the folder to be created to the second node.
In a possible implementation manner of the embodiments of the present application, the first node is a cluster management node in a NAS cluster system, the second node is a file metadata management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In another aspect, a computer device is provided, where the computer device includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus, where the memory is used to store a computer program, and where the processor is used to execute the program stored on the memory, so as to implement the steps of the metadata information query method described above.
In another aspect, a computer readable storage medium is provided, in which a computer program is stored, the computer program implementing the steps of the metadata information query method described above when executed by a processor.
In another aspect, a computer program product is provided comprising instructions which, when run on a computer, cause the computer to perform the steps of the metadata information query method described above.
The technical scheme provided by the embodiment of the application at least has the following beneficial effects:
in the embodiment of the present application, the bloom filter includes path information corresponding to metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be queried or not, and whether the metadata information corresponding to the path information is queried by the second node is further determined. Therefore, compared with blind inquiry through the second node, the method reduces the processing pressure of the second node and improves the inquiry efficiency of the metadata information.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a NAS cluster system architecture diagram provided in an embodiment of the present application;
fig. 2 is a flowchart of a metadata information query method provided in an embodiment of the present application;
FIG. 3 is a flowchart of a method for adding path information to a bloom filter provided by an embodiment of the present application;
FIG. 4 is a schematic diagram of adding path information to a bloom filter according to an embodiment of the present application;
FIG. 5 is a flow chart of another metadata information query provided by an embodiment of the present application;
fig. 6 is a schematic structural diagram of a metadata information query device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Before explaining the metadata information query method provided in the embodiment of the present application in detail, a system architecture related to the embodiment of the present application is first described.
Fig. 1 is a schematic diagram of a NAS cluster system according to an embodiment of the present application. As shown in fig. 1, the NAS cluster system includes a NAS client 101, a NAS gateway node 102, a cluster management node 103, a file metadata management node 104, and a cloud storage node 105.
The NAS client 101 establishes communication connection with the NAS gateway node 102, the NAS gateway node 102 establishes communication connection with the cluster management node 103, the file metadata management node 104, and the cloud storage node 105, and the cluster management node 103 and the file metadata management node 104 establish communication connection. Wherein the number of NAS gateway nodes 102 is one or more and the number of file metadata management nodes 104 is one or more.
Note that, the cluster management node 103 is configured to manage metadata information of folders, and in addition, the cluster management node 103 is configured to manage configuration information of the NAS cluster system and one or more file metadata management nodes 104. The file metadata management node 104 is used to manage metadata information of files.
In one possible implementation, databases are disposed on the cluster management node 103 and the file metadata management node 104, respectively, metadata information of a folder managed by the cluster management node 103 is stored in the database disposed on the cluster management node 103, and metadata information of a file managed by the file metadata management node 104 is stored in the database disposed on the file metadata management node 104. Alternatively, the metadata information of the folders managed by the cluster management node 103 and the metadata information of the files managed by the file metadata management node 104 are stored in one database. The embodiments of the present application are not limited in this regard.
Cloud storage nodes 105 provide underlying data storage functions, i.e., data for storing files, etc.
The NAS client 101 is configured to receive a file operation of a user, such as an access operation of a folder, a read/write operation of a file, and the like. After receiving the file operation of the user, the NAS client 101 generates a corresponding request and sends the corresponding request to the NAS gateway node 102 through the NAS protocol.
After receiving the request sent by the NAS client 101, the NAS gateway node 102 decides whether to query the metadata information corresponding to the path information according to the type of the request. If so, acquiring path information carried in the request, and adopting the metadata information query method provided by the embodiment of the application, querying metadata information corresponding to the path information through the cluster management node 103 or the file metadata management node 104. After acquiring the metadata information corresponding to the path information, the NAS gateway node 102 operates on the data stored in the cloud storage node 105 according to the metadata information.
It should be noted that, each node may be distributed and deployed on multiple devices, that is, each node is a separate device. Alternatively, in some possible implementations, some of the nodes other than the NAS client may be deployed on the same device, which is not limited in the embodiments of the present application.
The metadata information query method provided in the embodiment of the present application is explained in detail below.
Fig. 2 is a flowchart of a metadata information query method provided in an embodiment of the present application, where the method is applied to any NAS gateway node in the NAS cluster system shown in fig. 1. Referring to fig. 2, the method includes the following steps.
Step 201: and acquiring path information to be queried.
In the embodiment of the application, when receiving the file operation of the user, the NAS client generates a corresponding request according to the file operation and sends the request to the NAS gateway node. The file operations may include operations on folders and/or files already present in the NAS cluster system, such as download operations for folders or files already present in the NAS cluster system. And for the request corresponding to the file operation, carrying path information of the file or the folder to be operated in the request.
After receiving the request sent by the NAS client, the NAS gateway node cannot distinguish whether the request needs to operate a folder or a file, so that the NAS gateway node obtains path information carried in the request, so that whether the request needs to operate the folder or the file is distinguished by the method provided by the embodiment of the present application, and further obtains metadata information of the folder or the file from the corresponding node. At this time, the path information obtained from the request by the NAS gateway node is the path information to be queried.
Step 202: if the path information is not contained in the bloom filter, the metadata information corresponding to the path information is queried through the first node, the bloom filter comprises the path information corresponding to the metadata information managed by the second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In the embodiment of the application, the NAS gateway node comprises a bloom filter. The bloom filter consists of a plurality of hash functions and a bit array, and comprises path information corresponding to metadata information managed by the second node.
It should be noted that, in one possible implementation manner, the second node refers to a cluster management node in the NAS cluster system, and correspondingly, the first node refers to a file metadata management node in the NAS cluster system. In this case, since the cluster management node is used to manage metadata information of folders in the NAS cluster system, the bloom filter includes path information of the folders in the NAS cluster system.
In another possible implementation, the second node refers to a file metadata management node in the NAS cluster system, and the first node refers to a cluster management node in the NAS cluster system. In this case, since the file metadata management node is used to manage metadata information of files in the NAS cluster system, the bloom filter includes path information of files in the NAS cluster system.
Since the bloom filter includes path information corresponding to metadata information managed by the second node, that is, the bloom filter includes path information of folders or path information of files in the NAS cluster system. Therefore, before performing this step, the NAS gateway node needs to collect path information of the folder or file, and then add the path information of the folder or file to the bloom filter. Based on this, before the implementation process of this step is described, the NAS gateway node is first described as a process of collecting path information of a folder, and the path information of the folder is added to a bloom filter. It should be noted that, for an implementation manner of collecting path information of a file for a NAS gateway node and adding the path information of the file in a bloom filter, reference may be made to the following implementation process, which is not described in detail in this embodiment of the present application.
Illustratively, FIG. 3 shows a flow chart of adding path information for folders in a bloom filter. Referring to fig. 3, the process includes the steps of:
301: and receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created.
When a user wants to create a folder, a folder creation operation is performed on the NAS client. After receiving the operation, the NAS client generates a folder creation request and sends the folder creation request to the NAS gateway node, and accordingly, the NAS gateway node receives the folder creation request. Wherein the folder creation request carries path information of the folder to be created.
302: and adding the path information of the folder to be created to the bloom filter.
After receiving the folder creation request, the NAS gateway node acquires path information of a folder to be created, which is carried in the folder creation request, and then adds the path information of the folder to be created to a bloom filter.
As can be seen from the foregoing description, the bloom filter is composed of a plurality of hash functions and a bit array, wherein the initial value of the value at each position in the bit array is the second value. And the NAS gateway node performs hash operation on the path information of the folder to be created through the plurality of hash functions, so that a plurality of hash values are obtained. And the NAS gateway node sets the numerical values at a plurality of positions corresponding to the hash values one by one in the bit array as first numerical values. Thus, the path information of the folder to be created is added to the bloom filter. Wherein the first value is 1 and the second value is 0. Alternatively, the first value is 0 and the second value is 1.
Fig. 4 is a schematic diagram illustrating a principle of adding path information to a bloom filter according to an embodiment of the present application. As shown in fig. 4, assuming that path information of a folder to be created is m, a plurality of hash functions in the bloom filter are H, respectively 1 、H 2 And H 3 The bit array has a length of n. Assume a NAS gateway nodeAccording to H 1 And m to obtain a hash value k1=2. According to H 2 And m to obtain a hash value k2=4. According to H 3 And m to obtain a hash value k3=5, the NAS gateway node sets the values on the 2 nd bit, the 4 th bit and the 5 th bit in the bit array to 1, so that the path information m is added to the bloom filter.
303: and sending the metadata information of the folder to be created to the cluster management node.
The NAS gateway node creates the folder according to the path information and sends the metadata information of the folder to the cluster management node while adding the path information of the folder to be created to the bloom filter. After receiving the metadata information of the folder, the cluster management node stores the metadata information of the folder into a database.
Through the steps 301-303, the nas gateway node adds path information of the folder to the bloom filter in the process of creating the folder, and sends metadata information of the folder to the cluster management node for management. Thus, the bloom filter includes path information corresponding to metadata information of folders managed by the cluster management node.
It should be noted that the foregoing is an implementation manner of adding path information of a folder to a bloom filter according to an embodiment of the present application. In some possible embodiments, the NAS gateway node may also acquire path information of the folders according to metadata information of the folders newly added in the cluster management node at regular intervals, and then add the path information to the bloom filter. The embodiments of the present application are not limited in this regard.
Based on the above description, after the path information of the folder or the path information of the file is added to the bloom filter, when the NAS gateway node acquires the path information to be queried, the path information can be queried in the bloom filter, and whether the path information is the path information of the folder or the path information of the file is resolved according to whether the path information is queried in the bloom filter.
Illustratively, the NAS gateway node processes path information to be queried according to a plurality of hash functions included in the bloom filter, and obtains a plurality of hash values corresponding to the path information; and if the numerical value in any one of a plurality of positions corresponding to the hash values in one-to-one correspondence in the bit array contained in the bloom filter is not the first numerical value, determining that the bloom filter does not contain the path information.
It should be noted that, since the path information corresponding to the metadata information managed by the second node has been added to the bloom filter by the method described above, if the path information to be queried is the path information corresponding to the metadata information managed by the second node, the hash values obtained after processing the path information to be queried by the hash functions and the hash values obtained when adding the path information to the Xiang Bulong filter are necessarily equal. Thus, the values at the positions in the bit array corresponding to the hash values are necessarily the first values. Based on the above, if the value at any one of the positions corresponding to the hash values in the bit array is not the first value, it can be determined that the bloom filter does not include the path information to be queried. That is, the path information to be queried is not path information corresponding to metadata information managed by the second node.
In the case that the path information to be queried is determined not to be the path information corresponding to the metadata information managed by the second node, the NAS gateway node does not need to query the metadata information corresponding to the path information in the second node any more, but can query the metadata information corresponding to the path information directly through the first node. Therefore, compared with the blind method that metadata information is queried through the second node and the first node in sequence, the processing pressure of the second node is reduced, and the query efficiency of the metadata information is improved.
When the NAS gateway node queries metadata information corresponding to the path information to be queried through the first node, a query request may be sent to the first node, where the query request carries the path information to be queried. After receiving the query request, the first node acquires corresponding metadata information according to the path information, and then feeds the metadata information back to the NAS gateway node.
Optionally, if values at a plurality of positions corresponding to a plurality of hash values one by one to be queried in a bit array contained in the bloom filter are all first values, querying metadata information corresponding to the path information through a second node; and if the metadata information corresponding to the path information to be queried is not queried through the second node, querying the metadata information corresponding to the path information through the first node.
It should be noted that, in the process of adding the path information to the bloom filter, different path information corresponds to different hash values, so that the value at the position corresponding to the different hash values is the first value. However, for the path information that does not exist in the bloom filter, the values at the positions corresponding to the hash values processed by the hash functions may be the first values. For example, the bloom filter includes path information a and b. Assuming that the hash values corresponding to the path information a are 1, 2, 8 and the hash values corresponding to the path information b are 3, 5, 9, the values on the 1 st, 2 nd, 3 rd, 5 th, 8 th, and 9 th bits in the bit array will be the first values. It is assumed that the path information c does not exist in the bloom filter, but the hash values corresponding to the path information c obtained by the hash function processing are 2, 5, and 8, and at this time, although the values at the positions corresponding to the hash values in the bit array will all be the first values, the path information c does not actually exist in the bloom filter.
Therefore, when the values at the positions corresponding to the hash values to be queried are all the first values, it is indicated that the bloom filter has a high probability of possibly including the path information, but the bloom filter cannot be indicated to necessarily include the path information. In this case, the NAS gateway node first queries metadata information corresponding to the path information through the second node. If not, a false recognition of the bloom filter is indicated. At this time, the first node is used to query the metadata information corresponding to the path information. Therefore, compared with the metadata information which is not distinguished by the complete blind purpose and directly and sequentially passes through the query path information in each node, the processing pressure of the node which is queried first can be reduced to a certain extent, and the query efficiency is improved.
In the embodiment of the present application, the bloom filter includes path information corresponding to metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be queried or not, and whether the metadata information corresponding to the path information is queried by the second node is further determined. Therefore, compared with blind inquiry through the second node, the method reduces the processing pressure of the second node and improves the inquiry efficiency of the metadata information. Especially when the second node is a cluster management node, the number of the cluster management nodes included in the NAS cluster system is usually smaller, and the processing pressure of the cluster management node is larger, so that the processing pressure of the cluster management node can be better relieved by the method provided by the embodiment of the application, the performance bottleneck of the cluster management node called as the NAS cluster system is avoided, and the metadata processing efficiency is improved.
Optionally, in the above embodiment, the NAS gateway node includes a bloom filter. In other possible embodiments, the bloom filter is deployed separately in a separate node. For example, the bloom filter is a bloom filter on a redis open source node. In this way, the scheme is realized by the bloom filter for different gateway nodes, so that the global uniqueness of the data can be ensured.
Next, an implementation flow of the metadata query method provided in the embodiment of the present application is illustrated by an example, where in the example, the first node is a file metadata management node, and the second node is a cluster management node, that is, the bloom filter includes metadata information of folders managed by the cluster management node.
Referring to fig. 5, first, the NAS gateway node receives path information to be queried, and then determines whether the bloom filter includes the path information to be queried. If the bloom filter does not contain the path information to be queried, the NAS gateway node acquires attribute information corresponding to the path information from the file metadata management node, and returns the acquired attribute information to the NAS client. If the bloom filter contains the path information to be queried, the NAS gateway node requests to acquire attribute information corresponding to the path information from the cluster management node and judges whether the attribute information is acquired from the cluster management node. And if the attribute information is acquired from the cluster management node, returning the acquired attribute information to the NAS client. If the attribute information cannot be acquired from the cluster management node, acquiring the attribute information corresponding to the path information from the file metadata management node, and returning the acquired attribute information to the NAS client.
Fig. 6 is a schematic structural diagram of a metadata information query apparatus 600 provided in the embodiment of the present application, where the metadata information query apparatus may be implemented by software, hardware, or a combination of both as part or all of a NAS gateway node, and the NAS gateway node may be the NAS gateway node shown in fig. 1. Referring to fig. 6, the apparatus 600 includes: an acquisition module 601 and a query module 602.
An obtaining module 601, configured to obtain path information to be queried;
and a query module 602, configured to query, if the bloom filter does not include path information, metadata information corresponding to the path information by a first node, where the bloom filter includes path information corresponding to metadata information managed by a second node, and the metadata information managed by the first node is different from the metadata information managed by the second node.
In one possible implementation manner of the embodiment of the present application, the apparatus 600 further includes:
the processing module is used for processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and the determining module is used for determining that the bloom filter does not contain path information if the numerical value at any one of a plurality of positions corresponding to the hash values in the bit array contained in the bloom filter is not the first numerical value.
In one possible implementation of the embodiment of the present application, the query module 602 is further configured to:
if the values at a plurality of positions, which are in one-to-one correspondence with a plurality of hash values, in the bit array contained in the bloom filter are all first values, inquiring metadata information corresponding to the path information through a second node;
and if the metadata information corresponding to the path information is not queried from the second node, querying the metadata information corresponding to the path information through the first node.
In one possible implementation manner of the embodiment of the present application, the first node is a file metadata management node in a network attached storage NAS cluster system, the second node is a cluster management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In one possible implementation manner of the embodiment of the present application, the apparatus 600 further includes:
the receiving module is used for receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created;
the adding module is used for adding the path information of the folder to be created to the bloom filter;
and the storage module is used for sending the metadata information of the folder to be created to the second node.
In one possible implementation manner of the embodiment of the present application, the first node is a cluster management node in the NAS cluster system, the second node is a file metadata management node in the NAS cluster system, the file metadata management node manages metadata information of a file, and the cluster management node manages metadata information of a folder.
In the embodiment of the present application, the bloom filter includes path information corresponding to metadata information stored in the second node. Whether the metadata information corresponding to the path information is possibly managed by the second node is determined by judging whether the bloom filter contains the path information to be queried or not, and whether the metadata information corresponding to the path information is queried by the second node is further determined. Therefore, compared with blind inquiry through the second node, the method reduces the processing pressure of the second node and improves the inquiry efficiency of the metadata information.
It should be noted that: in the metadata information query device provided in the above embodiment, when querying metadata information, only the division of the above functional modules is used for illustration, in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the metadata information query device and the metadata information query method provided in the foregoing embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.
Fig. 7 is a schematic structural diagram of a server according to an embodiment of the present application. The NAS gateway node in the foregoing embodiment may be implemented by the server. The server 700 includes a central processing unit CPU (Central Processing Unit ) 701, a system Memory 704 including a RAM (Random Access Memory ) 702 and a ROM (Read-Only Memory) 703, and a system bus 705 connecting the system Memory 704 and the central processing unit 701. The server 700 also includes an Input/Output (I/O) system 706 that facilitates the transfer of information between various devices within the computer, and a mass storage device 707 for storing an operating system 713, application programs 714 and other program modules 715.
The basic input/output system 706 includes a display 708 for displaying information and an input device 709, such as a mouse, keyboard, or the like, for a user to input information. Wherein both the display 708 and the input device 709 are coupled to the central processing unit 701 through an input output controller 710 coupled to the system bus 705. The basic input/output system 706 may also include an input/output controller 710 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input output controller 710 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer readable media provide non-volatile storage for the server 700. That is, the mass storage device 707 may include a computer readable medium (not shown) such as a hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.
Computer readable media may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable read only memory, charged erasable programmable read-only memory), flash memory or other solid state memory technology, CD-ROM, DVD (Digital Video Disc, digital versatile disk) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the ones described above. The system memory 704 and mass storage device 707 described above may be collectively referred to as memory.
According to various embodiments of the present application, server 700 may also operate by a remote computer connected to the network through a network, such as the Internet. I.e., server 700 may be connected to network 712 through a network interface unit 711 coupled to system bus 705, or other types of networks or remote computer systems (not shown) may be coupled using network interface unit 711.
The memory also includes one or more programs, one or more programs stored in the memory and configured to be executed by the CPU.
In some embodiments, there is also provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of the metadata information query method of the above embodiments. For example, the computer readable storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It is noted that the computer readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, may be a non-transitory storage medium.
It should be understood that all or part of the steps to implement the above-described embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions may be stored in the computer-readable storage medium described above.
That is, in some embodiments, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of the metadata information query method described above.
The foregoing description is not intended to limit the embodiments of the present application, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the embodiments of the present application are intended to be included within the scope of the embodiments of the present application.
Claims (9)
1. A method for querying metadata information, which is applied to NAS gateway nodes of a network attached storage NAS cluster system, the method comprising:
acquiring path information of a folder or a file requested to be operated by a user;
if the bloom filter does not contain the path information, inquiring metadata information corresponding to the path information through a first node, wherein the bloom filter comprises the path information corresponding to the metadata information managed by a second node;
operating the data in the file or the folder according to the metadata information corresponding to the path information;
the first node is a file metadata management node in the NAS cluster system, the second node is a cluster management node in the NAS cluster system, or the first node is a cluster management node in the NAS cluster system, and the second node is a file metadata management node in the NAS cluster system; the file metadata management node is used for managing metadata information of files, and the cluster management node is used for managing metadata information of folders.
2. The method according to claim 1, wherein the method further comprises:
processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and if the numerical value in any one of a plurality of positions corresponding to the hash values in one-to-one correspondence in the bit array contained in the bloom filter is not the first numerical value, determining that the bloom filter does not contain the path information.
3. The method according to claim 2, wherein the method further comprises:
if the values at a plurality of positions, which are in one-to-one correspondence with the hash values, in the bit array contained in the bloom filter are all the first values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not queried through the second node, querying the metadata information corresponding to the path information through the first node.
4. A method according to any of claims 1-3, wherein if the first node is a file metadata management node in the NAS cluster system and the second node is a cluster management node in the NAS cluster system, the method further comprises:
receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created;
and adding the path information of the folder to be created to the bloom filter, and sending the metadata information of the folder to be created to the second node.
5. A metadata information query apparatus, comprised in a NAS gateway node of a network attached storage NAS cluster system, the apparatus comprising:
the acquisition module is used for acquiring path information of folders or files which are requested to be operated by a user;
the query module is used for querying metadata information corresponding to the path information through the first node if the path information is not contained in the bloom filter, wherein the bloom filter comprises the path information corresponding to the metadata information managed by the second node; operating the data in the file or the folder according to the metadata information corresponding to the path information;
the first node is a file metadata management node in the NAS cluster system, the second node is a cluster management node in the NAS cluster system, or the first node is a cluster management node in the NAS cluster system, and the second node is a file metadata management node in the NAS cluster system; the file metadata management node is used for managing metadata information of files, and the cluster management node is used for managing metadata information of folders.
6. The apparatus of claim 5, wherein the apparatus further comprises:
the processing module is used for processing the path information according to a plurality of hash functions contained in the bloom filter to obtain a plurality of hash values corresponding to the path information;
and the determining module is used for determining that the bloom filter does not contain the path information if the numerical value at any one of a plurality of positions corresponding to the hash values in one-to-one correspondence in the bit array contained in the bloom filter is not a first numerical value.
7. The apparatus of claim 6, wherein the query module is further configured to:
if the values at a plurality of positions, which are in one-to-one correspondence with the hash values, in the bit array contained in the bloom filter are all the first values, inquiring metadata information corresponding to the path information through the second node;
and if the metadata information corresponding to the path information is not queried from the second node, querying the metadata information corresponding to the path information through the first node.
8. The apparatus of any of claims 5-7, wherein if the first node is a file metadata management node in the NAS cluster system and the second node is a cluster management node in the NAS cluster system, the apparatus further comprises:
the receiving module is used for receiving a folder creation request, wherein the folder creation request carries path information of a folder to be created;
an adding module, configured to add path information of the folder to be created to the bloom filter;
and the storage module is used for sending the metadata information of the folder to be created to the second node.
9. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the steps of the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010917087.4A CN112035413B (en) | 2020-09-03 | 2020-09-03 | Metadata information query method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010917087.4A CN112035413B (en) | 2020-09-03 | 2020-09-03 | Metadata information query method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112035413A CN112035413A (en) | 2020-12-04 |
| CN112035413B true CN112035413B (en) | 2024-02-20 |
Family
ID=73592133
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010917087.4A Active CN112035413B (en) | 2020-09-03 | 2020-09-03 | Metadata information query method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112035413B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114443913B (en) * | 2022-04-06 | 2022-06-07 | 创智和宇信息技术股份有限公司 | Metadata multi-function multi-condition based user-defined query method, system and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150483A (en) * | 2007-11-02 | 2008-03-26 | 华为技术有限公司 | Route table adjustment method, route query method and device and route table storage device |
| CN105408857A (en) * | 2013-07-29 | 2016-03-16 | 亚马逊科技公司 | Generating a multi-column index for relational databases by interleaving data bits for selectivity |
| CN109495525A (en) * | 2017-09-12 | 2019-03-19 | 华为技术有限公司 | Networking component, the method and computer readable storage medium for parsing content identification |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9455903B2 (en) * | 2012-07-31 | 2016-09-27 | Cisco Technology, Inc. | Recording packet routes using bloom filters |
-
2020
- 2020-09-03 CN CN202010917087.4A patent/CN112035413B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150483A (en) * | 2007-11-02 | 2008-03-26 | 华为技术有限公司 | Route table adjustment method, route query method and device and route table storage device |
| CN105408857A (en) * | 2013-07-29 | 2016-03-16 | 亚马逊科技公司 | Generating a multi-column index for relational databases by interleaving data bits for selectivity |
| CN109495525A (en) * | 2017-09-12 | 2019-03-19 | 华为技术有限公司 | Networking component, the method and computer readable storage medium for parsing content identification |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112035413A (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2020038623A (en) | Method, device, and system for storing data | |
| CN107562757B (en) | Query and access method, device and system based on distributed file system | |
| WO2021082157A1 (en) | Methods, systems and devices for data sharing, and data and metadata storage | |
| CN108874803B (en) | Data storage method, device and storage medium | |
| CN108614837B (en) | File storage and retrieval method and device | |
| CN110413650B (en) | Method, device, equipment and storage medium for processing service data | |
| CN112287182A (en) | Graph data storage and processing method and device and computer storage medium | |
| CN111159219B (en) | Data management method, device, server and storage medium | |
| US20180248977A1 (en) | Selective distribution of messages in a publish-subscribe system | |
| CN110958300A (en) | Data uploading method, system, device, electronic equipment and computer readable medium | |
| CN112035413B (en) | Metadata information query method, device and storage medium | |
| CN103905512A (en) | Data processing method and equipment | |
| US9626378B2 (en) | Method for handling requests in a storage system and a storage node for a storage system | |
| US20220215021A1 (en) | Data Query Method and Apparatus, Computing Device, and Storage Medium | |
| CN107977381B (en) | Data configuration method, index management method, related device and computing equipment | |
| CN114466031B (en) | CDN system node configuration method, device, equipment and storage medium | |
| CN106777230B (en) | Partition system, partition method and device | |
| CN108023920B (en) | Data packet transmission method, equipment and application interface | |
| CN112835896B (en) | Real-time database data hotspot balancing method, device, equipment and medium | |
| CN107526530B (en) | Data processing method and device | |
| CN110990394B (en) | Method, device and storage medium for counting number of rows of distributed column database table | |
| CN108989245B (en) | User data storage method and device | |
| CN116069788B (en) | Data processing method, database system, computer device, and storage medium | |
| CN112463879B (en) | Data volume synchronization method and equipment | |
| CN111104787B (en) | Method, apparatus and computer program product for comparing files |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |