CN112351038A

CN112351038A - File request method, system and medium based on network file system

Info

Publication number: CN112351038A
Application number: CN202011239132.1A
Authority: CN
Inventors: 臧林劼
Original assignee: Suzhou Inspur Intelligent Technology Co Ltd
Current assignee: Suzhou Inspur Intelligent Technology Co Ltd
Priority date: 2020-11-09
Filing date: 2020-11-09
Publication date: 2021-02-09

Abstract

The invention provides a file request method, a system and a medium based on a network file system, wherein the method comprises the following steps: receiving an OPEN request of a first client; distributing the first secret key to the first client; if the OPEN requests of other clients are received, the first secret key is recovered to the first client; and establishing an RPC-B link with other clients and distributing a marking key to the other clients, wherein the RPC-B link is used for binding the marking key and a file lock of the file requested by the OPEN. The invention ensures that the client does not need to send a GetAttr request by directly providing a proper secret key for the OPEN request provided by the client by the server in the network file system, namely determining that the metadata cached locally is effective, thereby reducing the blockage of IO read-write operation of the network file system.

Description

File request method, system and medium based on network file system

Technical Field

The present invention relates to the field of network file system technology, and in particular, to a file request method, system and medium based on a network file system.

Background

In a distributed File storage System, an NFS (Network File System) protocol is widely applied, and based on Network remote procedure call, the NFS can provide a transparent File access mechanism and File IO (input output) transmission, so that the limitation of single machine service is eliminated, and remote data sharing is realized; the NFS protocol belongs to a presentation layer protocol (UNIX) of the UNIX protocol, and is implemented mainly by using a Remote Procedure Call (RPC) mechanism, which provides a set of operations for accessing remote files that are not related to a machine, an operating system, and an underlying transport protocol.

The network file system uses the metadata cache to improve the reading efficiency of the hot data and reduce the request pressure for the bottom storage, and generally, the NFS client caches the file to improve the performance. To determine whether the cached data is valid and whether the cache is consistent, the NFS client sends a GETATTR (function, return a long, which is an attribute of a file, directory, or folder) request to update the cache information.

In the prior art, methods for ensuring cache consistency of a network file system all obtain metadata information by performing a GetAttr request when a request is initiated at an NFS client, so as to determine whether a local metadata cache is valid. In a massive distributed cluster file system, the number of NFS clients is large, the concurrency is large, and a large number of GetAttr requests can block normal IO read-write operations, so that the performance of the network file system is influenced.

Disclosure of Invention

In view of this, an object of the present invention is to provide a file request method, system and medium based on a network file system, in which a server in the network file system directly provides a proper key to an OPEN request provided by a client, that is, it is determined that metadata cached locally is valid, so that the client does not need to send a GetAttr request, and blocking of IO read-write operations of the network file system is reduced.

Based on the above object, one aspect of the present invention provides a file request method based on a network file system, which includes the following steps:

receiving an OPEN request of a first client;

distributing the first secret key to the first client;

if the OPEN requests of other clients are received, the first secret key is recovered to the first client;

and establishing an RPC-B link with other clients and distributing the marking key to the other clients, wherein the RPC-B link is used for binding the marking key and a file lock of the file requested by the OPEN.

In some embodiments, the marker key comprises a valid key and an invalid key, the file lock comprises a count 1 for binding the valid key and a count 0 for binding the invalid key; count 1 and count 0 are superimposed to record the number of times the file is opened.

In some embodiments, the invalid key comprises a key to be updated.

In some embodiments, if an OPEN request from another client is received, reclaiming the first secret key from the first client includes: and after the first key is recovered to the first client, a file lock is created for the file requested by the OPEN.

In some embodiments, if an OPEN request of another client is received, reclaiming the first key from the first client further includes: if the OPEN requests of other clients are received, a notification request is sent to the first client; and in response to the first client receiving the notification request and updating the attribute information of the metadata cached by the first client, recovering the first key.

In some embodiments, the attribute information includes an expiration time for determining a cache miss.

In some embodiments, receiving the OPEN request of the first client comprises: the network file system calls the OPEN request to the first client through the VFS virtual file system layer and sends the OPEN request to the RPC layer, and the server receives the OPEN request of the RPC layer.

In some embodiments, receiving an OPEN request of the RPC layer comprises: and receiving an OPEN request of an RPC layer through an RPC channel, wherein the RPC channel comprises a request message, and the request message comprises an Ethernet header, an IP header, a TCP header, an RPC header and network file system data.

In another aspect of the present invention, a file request system based on a network file system is further provided, including:

the receiving request module is configured to receive an OPEN request of a first client;

the key distribution module is configured to distribute a first secret key to the first client;

the key recovering module is configured to recover the first key to the first client if an OPEN request of another client is received; and

and the link processing module is configured to establish an RPC-B link with other clients and distribute the marking key to the other clients, wherein the RPC-B link is used for binding the marking key and the file lock of the file requested by the OPEN.

In yet another aspect of the present invention, there is also provided a computer storage medium storing computer program instructions which, when executed, implement any one of the methods described above.

The invention has at least the following beneficial technical effects:

according to the method, the server in the user mode network file system directly provides a proper secret key for the OPEN request provided by the client, namely the metadata cached locally is determined to be effective, so that the client does not need to send a GetAttr request, and the blockage of IO read-write operation of the network file system is reduced; the user mode network file system has the characteristics of high concurrency, higher safety, easiness in use and the like in practical application, the request time delay of IO read-write operation of the user mode network file system file is reduced, the load capacity of the user mode network file system service is improved, the distributed system resources are effectively utilized, and the cost is saved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.

FIG. 1 is a diagram illustrating an embodiment of a file request method based on a network file system according to the present invention;

fig. 2 is a schematic diagram of an embodiment of a file request system based on a network file system according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.

It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two non-identical entities with the same name or different parameters, and it is understood that "first" and "second" are only used for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements does not include all of the other steps or elements inherent in the list.

The following are explanations of the words relating to English in the embodiments of the present invention:

OPEN request: refers to an open request for a file in a network file system;

RPC: remote Procedure Call (Remote Procedure Call);

IP header: the IP refers to Internet Protocol (Internet Protocol), and the IP header refers to a header structure of an IP Protocol packet transmitted over a network;

a TCP head: TCP refers to Transmission Control Protocol (Transmission Control Protocol), and a TCP header refers to a header structure of a TCP Protocol packet transmitted over a network.

In view of the above, a first aspect of the embodiments of the present invention provides an embodiment of a file request method based on a network file system. Fig. 1 is a schematic diagram illustrating an embodiment of a file request method based on a network file system according to the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:

step S10, receiving an OPEN request of a first client;

step S20, distributing the first secret key to the first client;

step S30, if an OPEN request of another client is received, retrieving the first key from the first client;

and step S40, establishing RPC-B link with other clients and distributing the marking key to the other clients, wherein the RPC-B link is used for binding the marking key and the file lock of the file requested by the OPEN.

In the embodiment of the invention, the server in the user mode network file system directly provides a proper secret key for an OPEN request provided by the client and provides a proper secret key through link processing, namely, the metadata of the local cache is determined to be effective, so that the client does not need to send a GETATTR request, and the blockage of IO read-write operation of the network file system is reduced; the user mode network file system has the characteristics of high concurrency, higher safety, easiness in use and the like in practical application, the request time delay of IO read-write operation of the user mode network file system file is reduced, the load capacity of the user mode network file system service is improved, the distributed system resources are effectively utilized, and the cost is saved.

In some embodiments, the marker key comprises a valid key and an invalid key, the file lock comprises a count 1 for binding the valid key and a count 0 for binding the invalid key; count 1 and count 0 are superimposed to record the number of times the file is opened. In this embodiment, the key information corresponding to the same file has a file lock, and the count 1 and the count 0 are set for the file lock, so that the number of times that the same file is opened can be counted, and the file can be managed by the network file system conveniently. The mark key is identified by an identifier, the identifier of the invalid key is RECLAIM, and the identifier of the valid key is REFAG.

In some embodiments, the invalid key comprises a key to be updated. In this embodiment, when the key needs to be updated, the key is considered to be in an invalid state.

In some embodiments, if an OPEN request from another client is received, reclaiming the first secret key from the first client includes: and after the first key is recovered to the first client, a file lock is created for the file requested by the OPEN. In this embodiment, if a file is opened for the first time, a first client that opens the file for the first time performs file lock creation, and when other clients open the file, the file lock creation is not performed.

In some embodiments, if an OPEN request of another client is received, reclaiming the first key from the first client further includes: if the OPEN requests of other clients are received, a notification request is sent to the first client; and in response to the first client receiving the notification request and updating the attribute information of the metadata cached by the first client, recovering the first key. Generally, data in a file system is divided into data and metadata, wherein the data refers to actual data in a file, and the metadata is information describing data attributes and the like, including access permissions, file owner's file access, modification, creation time, inode file data index information and the like. In a distributed cluster file system, the distribution of data includes the location of a new file on a disk and the location of the disk in a cluster, and when a file is operated, metadata information must be obtained first, and then the file is located to the location of the file and the content or related attributes of the file are obtained. Therefore, the performance of accessing metadata directly determines the performance of the file data IO read-write operation.

In some embodiments, the attribute information includes an expiration time for determining a cache miss. In this embodiment, if the cache is valid, GetAttr processing is not performed, so that the block of the IO read-write operation of the network file system is reduced.

In some embodiments, receiving the OPEN request of the first client comprises: the network file system calls the OPEN request to the first client through the VFS virtual file system layer and sends the OPEN request to the RPC layer, and the server receives the OPEN request of the RPC layer. The NFS belongs to a C/S model architecture, so that an NFS client sends a request through system call.

In a second aspect of the embodiments of the present invention, a file request system based on a network file system is further provided. Fig. 2 is a schematic diagram illustrating an embodiment of a file request system based on a network file system according to the present invention. A file request system based on a network file system comprising: the system comprises a receiving request module 10, a key distribution module 20, a key recovery module 30 and a link processing module 40; the receiving request module 10 is configured to receive an OPEN request of a first client; the key distribution module 20 is configured to distribute the first secret key to the first client; the key recovering module 30 is configured to recover the first key from the first client if an OPEN request of another client is received; the link handling module 40 is configured to establish RPC-B links with other clients and distribute the marking key thereto, wherein the RPC-B links are used to bind the marking key and the file lock of the file requested by the OPEN.

In the file request system based on the network file system, the server in the user mode network file system directly provides a proper secret key for an OPEN request provided by the client, that is, the metadata cached locally is determined to be valid, so that the client does not need to send a GetAttr request, the blockage of the IO read-write operation of the network file system is reduced, and the request delay of the IO read-write operation of the network file system is reduced; meanwhile, the load capacity of the user mode network file system service is improved, the distributed system resources are effectively utilized, and the cost is saved.

In a third aspect of the embodiments of the present invention, a computer storage medium is further provided, where the computer storage medium stores computer program instructions, and the computer program instructions, when executed, implement any one of the above-mentioned embodiment methods.

It is to be understood that all embodiments, features and advantages set forth above with respect to the network file system based file request method according to the present invention are equally applicable to the network file system based file request system and the storage medium according to the present invention without conflicting therewith. That is, all of the embodiments and variations thereof described above as applied to the network file system based file request method may be directly transferred to and applied to the system and storage medium according to the present invention, and are directly incorporated herein. For the sake of brevity of the present disclosure, no repeated explanation is provided herein.

Finally, it should be noted that, as one of ordinary skill in the art can appreciate that all or part of the processes of the methods of the above embodiments can be implemented by a computer program to instruct related hardware, and the program of the file request method based on the network file system can be stored in a computer readable storage medium, and when executed, the program can include the processes of the embodiments of the methods described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.

Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.

Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.

The various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed with the following components designed to perform the functions herein: a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.

The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.

It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items. The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.

Claims

1. A file request method based on a network file system is characterized by comprising the following steps:

receiving an OPEN request of a first client;

distributing a first key to the first client;

if an OPEN request of other clients is received, the first secret key is recovered to the first client;

and establishing an RPC-B link with other clients and distributing a marking key to the other clients, wherein the RPC-B link is used for binding the marking key and a file lock of the file requested by the OPEN.

2. The method of claim 1, wherein the marker key comprises a valid key and an invalid key, and wherein the file lock comprises a count of 1 for binding the valid key and a count of 0 for binding the invalid key; the count 1 and count 0 are superimposed to record the number of times the file is opened.

3. The method of claim 2, wherein the invalid key comprises a key to be updated.

4. The method of claim 1, wherein reclaiming the first secret key from the first client if an OPEN request from another client is received comprises:

and after the first key is recovered to the first client, creating the file lock for the file requested by the OPEN.

5. The method of claim 1, wherein reclaiming the first key from the first client if an OPEN request from another client is received further comprises:

if the OPEN requests of other clients are received, a notification request is sent to the first client;

and in response to the first client receiving the notification request and updating the attribute information of the metadata cached by the first client, recovering the first key.

6. The method of claim 5, wherein the attribute information comprises an expiration time for determining the cache miss.

7. The method of claim 1, wherein receiving the OPEN request of the first client comprises:

the network file system calls the OPEN request to the first client through a VFS virtual file system layer and sends the OPEN request to an RPC layer, and a server receives the OPEN request of the RPC layer.

8. The method of claim 7, wherein receiving the OPEN request of the RPC layer comprises:

receiving the OPEN request of the RPC layer through an RPC channel, wherein the RPC channel comprises a request message, and the request message comprises an Ethernet header, an IP header, a TCP header, an RPC header and network file system data.

9. A file request system based on a network file system, comprising:

a key distribution module configured to distribute a first secret key to the first client;

and the link processing module is configured to establish an RPC-B link with other clients and distribute a marking key to the RPC-B link, wherein the RPC-B link is used for binding the marking key and a file lock of the file requested by the OPEN.

10. A computer storage medium having stored thereon computer program instructions which, when executed, implement the method of any one of claims 1 to 8.