CN111966634A

CN111966634A - File operation method, system, device and medium

Info

Publication number: CN111966634A
Application number: CN202010731345.XA
Authority: CN
Inventors: 臧林劼
Original assignee: Suzhou Inspur Intelligent Technology Co Ltd
Current assignee: Suzhou Inspur Intelligent Technology Co Ltd
Priority date: 2020-07-27
Filing date: 2020-07-27
Publication date: 2020-11-20

Abstract

The invention discloses a file operation method, which comprises the following steps: receiving a remote operation request sent by a client; responding to the first type of operation request of the remote operation request, and setting a corresponding open mark; judging whether a mark corresponding to the opening mode exists in the cache; responding to the fact that no mark corresponding to the opening mode exists in the cache, calling an open interface, opening a file to be operated according to the opening mode corresponding to the open mark, further operating the file to be operated, and caching the mark corresponding to the opening mode; and responding to the mark corresponding to the opening mode in the cache, and directly operating the file to be operated. The invention also discloses a system, a computer device and a readable storage medium. The scheme provided by the invention caches the marks corresponding to the open mode, effectively solves the problem of frequent calling of the file open, reduces the operation of system calling, reduces the time delay of the file operation request of the stored file system, and is beneficial to improving the performance of the system and the resource utilization rate.

Description

File operation method, system, device and medium

Technical Field

The invention relates to the field of file systems, in particular to a file operation method, a file operation system, file operation equipment and a storage medium.

Background

At present, when a file system reads and writes a file, a file to be operated by open is needed, the main purpose of the file system is to call control information of the file from a storage device such as an external memory and the like into an internal memory for processing, the file control information is a basic data structure for controlling the reading and writing of the file and managing the file, when a process needs to use a certain file, open () is called to open the file, a file control block of the file is called from the external memory into the internal memory by the call, the file control block is stored in a user activity file table in a process table, the opening times of the file are recorded in a system activity file table, and if the file is shared, the number of users linked with the file is also added by one. Since the control block for the file is stored in the process table, the user process can find the location of the file and access the contents of the file when calling read (). In a mass storage system, when a file is frequently operated, particularly in some scenes of frequently reading and writing the same file, the request/response times of the same file of the open are increased, the processing time delay of the whole operation request is increased, the whole service operation request is influenced, and the performance of the distributed storage server on the file operation service is slowed down; when the same file is frequently opened, in the process of frequently calling in and calling out the file control information, the key resources of the server systems such as the memory/CPU are wasted, and great influence is generated.

Disclosure of Invention

In view of the above, in order to overcome at least one aspect of the above problems, an embodiment of the present invention provides a file operating method, including:

receiving a remote operation request sent by a client;

responding to the first type of operation request of the remote operation request, and setting a corresponding open mark;

judging whether a mark corresponding to the opening mode exists in the cache;

responding to the cache that no mark corresponding to the open mode exists, calling an open interface, opening a file to be operated according to the open mode corresponding to the open mark, further operating the file to be operated, and caching the mark corresponding to the open mode;

and responding to the mark corresponding to the opening mode in the cache, and directly operating the file to be operated.

In some embodiments, in response to the remote operation request being a first type of operation request, setting a corresponding open flag, further includes:

setting the open flag to a first flag in response to the remote operation request being a read operation request;

setting the open flag to a second flag in response to the remote operation request being a write operation request;

and in response to the remote operation request being a synchronous write operation request, setting the open flag to be a third flag.

In some embodiments, opening the file to be operated according to the open mode corresponding to the open flag further comprises:

responding to the open mark as a first mark or a second mark, and opening the file to be operated in a read-write mode;

and responding to the open mark as a third mark, opening a synchronous switch, and further opening the file to be operated in a synchronous writing mode.

In some embodiments, in response to a flag corresponding to an open mode in the cache, directly operating the file to be operated, further includes:

judging whether the mark corresponding to the opening mode acquired from the cache corresponds to the open mark of the first type of operation request or not;

and directly operating the file to be operated in response to the fact that the mark corresponding to the opening mode acquired from the cache corresponds to the open mark of the first type of operation request.

In some embodiments, further comprising:

and in response to that the mark corresponding to the opening mode acquired in the cache does not correspond to the open mark of the first type of operation request, calling the open interface, opening the file to be operated according to the opening mode corresponding to the open mark, further operating the file to be operated, and caching the mark corresponding to the opening mode.

In some embodiments, further comprising:

and calling a create interface to create a file in response to the remote operation request being the second type operation request.

In some embodiments, further comprising:

and setting the opening mode of the file as a read-write mode, and caching a mark corresponding to the read-write mode.

Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a file operating system, including:

the receiving module is configured to receive a remote operation request sent by a client;

the setting module is configured to respond to the first type of operation request of the remote operation request and set a corresponding open mark;

the judging module is configured to judge whether a mark corresponding to the opening mode exists in the cache;

a first response module, configured to respond to that no mark corresponding to the open mode exists in the cache, invoke an open interface and open a file to be operated according to the open mode corresponding to the open mark, further operate the file to be operated, and cache the mark corresponding to the open mode;

a second response module configured to respond to the mark corresponding to the open mode in the cache and directly operate the file to be operated.

Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer apparatus, including:

at least one processor; and

a memory storing a computer program operable on the processor, wherein the processor executes the program to perform any of the steps of the file manipulation method as described above.

Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program which, when executed by a processor, performs the steps of any of the file manipulation methods described above.

The invention has one of the following beneficial technical effects: the scheme provided by the invention caches the opened file handle, caches the open file handle, effectively solves the problem of frequent calling of the open file, reduces the operation of system calling, reduces the time delay of the file operation request of the file storage system, and is beneficial to improving the performance and the resource utilization rate of the distributed storage system.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.

FIG. 1 is a flowchart illustrating a file operation method according to an embodiment of the present invention;

FIG. 2 is a block flow diagram of a NAS-based fusion architecture according to an embodiment of the present invention;

FIG. 3 is a block flow diagram of a file manipulation method according to an embodiment of the present invention;

FIG. 4 is another flow chart diagram of a file operation method according to an embodiment of the present invention;

FIG. 5 is a block diagram of a file operating system according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a computer device provided in an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.

It should be noted that, in the embodiment of the present invention, the OPEN file handle is a flag corresponding to the OPEN mode of the FSAL layer.

It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.

It should be noted that, in the embodiment of the present invention, nas (network Attached storage) is defined as a special dedicated data storage server, which includes a storage device (e.g., a disk array, a CD/DVD drive, a tape drive, or a removable storage medium) and embedded system software, and can provide a cross-platform file sharing function. The NAS usually possesses its own node on a LAN, and allows users to access data on the network without the intervention of an application server.

The NAS can support a variety of protocols (e.g., NFS, CIFS, FTP, HTTP, etc.), and can support various operating systems. By any workstation, the NAS device can be intuitively and conveniently managed by adopting an IE or Netscape browser. NAS connects storage devices to existing networks to provide data and file services. The NAS server generally consists of several parts, including storage hardware, an operating system, and a file system thereon. The NAS is directly connected with a disk storage array through a network, and the magnetic array has the characteristics of high capacity, high efficiency, high reliability and the like. The NAS connects the storage equipment to the Internet directly without a server through a standard network topology structure, does not depend on a general operating system, adopts a simplified operating system which is designed for users and is specially used for data storage, and embeds a protocol required by network connection, thereby leading the management and the setting of the whole system to be simpler.

In the field of distributed storage technology, the NFS protocol is widely applied at present, and belongs to a presentation layer (presentation layer protocol) of the UNIX protocol, and the NFS protocol is implemented mainly by using a Remote Procedure Call (RPC) mechanism (remote Procedure Call), where RPC provides a set of operations for accessing remote files that are unrelated to a machine, an operating system, and a low-level transport protocol. RPC employs support of XDR. The XDR is a data description coding protocol independent of a machine, encodes and decodes data transmitted on the network in a format independent of an arbitrary machine architecture, and supports the transmission of the data between heterogeneous systems.

The NAS fusion architecture is that after receiving a request, an NFS server processes the request through an NFS server protocol, and sends a corresponding file processing request to a File System Abstraction Layer (FSAL) (file system abstraction layer) in a distributed cluster through convention configuration, wherein the layer is also called a client layer of a distributed storage file system, and after the client layer processes the request, a processing result is returned to the NFS client to complete NFS request processing.

According to an aspect of the present invention, an embodiment of the present invention provides a file operating method, as shown in fig. 1, which may include the steps of:

s1, receiving a remote operation request sent by the client;

s2, responding to the remote operation request as a first type operation request, and setting a corresponding open mark;

s3, judging whether a mark corresponding to the opening mode exists in the cache;

s4, responding to the fact that the cache does not have the mark corresponding to the opening mode, calling an open interface, opening a file to be operated according to the opening mode corresponding to the open mark, further operating the file to be operated, and caching the mark corresponding to the opening mode;

and S5, responding to the mark corresponding to the opening mode in the cache, and directly operating the file to be operated.

The scheme provided by the invention caches the opened file handle, caches the open file handle, effectively solves the problem of frequent calling of the open file, reduces the operation of system calling, reduces the time delay of the file operation request of the file storage system, and is beneficial to improving the performance and the resource utilization rate of the distributed storage system.

In some embodiments, in step S1, in receiving the remote operation request sent by the client, specifically, as shown in fig. 2, the NFS client performs a read-write request call to perform a file operation, and enters a VFS (virtual file system layer) through a system call, in which generally, the client request interacts with a VFS layer of a Linux kernel through libc, the VFS layer is an abstraction of a bottom file system, and the bottom file system of the client in the present invention refers to an NFS client program. The VFS provides a common interface to the various file systems, and each specific file system converts its physical organization into a common model of a virtual file system. The NFS Client request needs to send a file operation request to an RPC layer through RPC remote procedure call; when receiving a request of an NFS Client, an RPC layer sends the request to an NFS server through a network, and the RPC layer needs to assemble a request message structure, particularly an Ethernet header, an IP header, a TCP header, an RPC header and NFS data to form a standard NFS request; the NFS server monitors and receives an IO request of the NFS client, and in the embodiment of the invention, the IO request is a remote operation request.

In some embodiments, in step S2, in response to the remote operation request being a first type operation request, setting a corresponding open flag, further includes:

Specifically, as shown in fig. 3, when the NFS client initiates a read-write request, the read-write type is first determined, different open flags are respectively set, and the read-write is distinguished. If the NFS client initiates a READ request, an open flag is set to O _ READ (a first flag) in the NFS service program, if the NFS client initiates a write request, the open flag is set to O _ write (a second flag) in the NFS service program, and if the NFS client initiates a write request, a sync and write flag (a third flag) is set in the NFS service program, which represents a synchronous write request.

In some embodiments, in step S3, it is determined whether there is a flag corresponding to the OPEN mode in the cache, specifically, before the OPEN interface of the FSAL distributed file system is called, the flag corresponding to the OPEN mode of the FSAL layer (that is, the OPEN file handle) is to be obtained, so as to determine whether the OPEN interface needs to be called again by obtaining the OPEN mode flag.

In some embodiments, step S4, opening the file to be operated according to the open mode corresponding to the open flag, further includes:

Specifically, in the embodiment of the present invention, as shown in fig. 3, the opening modes corresponding to the read or write requests are all set to RDWR, that is, are all opened by using the read-write mode, and after the FSAL layer open interface is called, the flag corresponding to the opening mode is set and cached as RDWR in the FSAL layer. As shown in fig. 4, if the open flag is the third flag, that is, the synchronous write request, the synchronous switch is turned on, so that the file to be operated is opened in the synchronous write mode, and the flag corresponding to the open mode is set and cached as sync and write.

Note that, when the synchronization switch is turned on, the synchronous write operation can be performed.

In some embodiments, step S5, in response to the flag corresponding to the open mode being in the cache, directly operating the file to be operated, further includes:

In some embodiments, step S5 further includes:

Specifically, as shown in fig. 3 and 4, when the Open interface of the FSAL distributed file system is called, the flag corresponding to the Open mode of the FSAL layer is to be obtained, so as to determine whether the Open interface needs to be called again by obtaining the flag corresponding to the Open mode, if the Open flag is the first flag or the second flag, and the flag corresponding to the Open mode obtained in the FSAL layer is RDWR, the Open operation is not performed, otherwise, the Open interface needs to be flushed and called. Or, if the Open is marked as the third mark and the marks corresponding to the Open modes obtained in the FSAL layer are sync and write, then Open operation is no longer performed, otherwise, the Open interface needs to be called again.

In some embodiments, the method further comprises:

and S6, responding to the remote operation request as a second type operation request, and calling a create interface to create a file.

In some embodiments, step S6 further includes:

Specifically, as shown in fig. 2, when the NFS server monitors and receives an IO request of the NFS client and the IO request is a create operation request, processing is performed at an NFS server program layer, the NFS protocol performs an operation on a file through filehandle, when the file is created, a cache entry of a parent directory is first constructed, attribute information of the parent directory is obtained, it is ensured that the file is consistent when the attribute is updated for the file operation, whether the parent directory has a write right or not is verified, and the file creation can be performed only if the parent directory has the write right. And then calling a Client-FSAL layer of the distributed file system to perform creation operation on the file. In some embodiments, a create _ open switch may also be set, and when the switch is opened, the distributed client layer create interface processing is called, an open file descriptor is obtained, cache reservation is performed in the client layer handle structure, and open flags in the handle structure are set as a read-write mode.

It should be noted that after the file is created, the file handle may be cached, or the file handle may not be cached, that is, the corresponding tag corresponding to the open mode may be cached, or the tag corresponding to the open mode may not be cached. After the mark corresponding to the cache open mode is received, the open interface does not need to be called after the corresponding operation request is received.

Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a file operation 400, as shown in fig. 5, including:

a receiving module 401, where the receiving module 401 is configured to receive a remote operation request sent by a client;

a setting module 402, wherein the setting module 402 is configured to set a corresponding open flag in response to the remote operation request being a first type of operation request;

a judging module 403, where the judging module 403 is configured to judge whether a flag corresponding to an open mode exists in a cache;

a first response module 404, where the first response module 404 is configured to, in response to that there is no tag corresponding to the open mode in the cache, invoke an open interface and open a file to be operated according to the open mode corresponding to the open tag, further operate the file to be operated, and cache the tag corresponding to the open mode;

a second response module 405, where the second response module 405 is configured to respond to the mark corresponding to the open mode in the cache, and directly operate the file to be operated.

In some embodiments, the setup module 402 is further configured to:

In some embodiments, the first response module 404 is further configured to:

In some embodiments, the second response module 405 is further configured to:

In some embodiments, further comprising a third response module, the third response module further configured to:

In some embodiments, the third response module is further configured to:

Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 6, an embodiment of the present invention further provides a computer apparatus 501, including:

at least one processor 520; and

the memory 510, the memory 510 storing a computer program 511 executable on the processor, the processor 520 executing the program to perform the steps of any of the above file manipulation methods.

Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 7, an embodiment of the present invention further provides a computer-readable storage medium 601, where the computer-readable storage medium 601 stores computer program instructions 610, and the computer program instructions 610, when executed by a processor, perform the steps of any of the above file operating methods.

Finally, it should be noted that, as will be understood by those skilled in the art, all or part of the processes of the methods of the above embodiments may be implemented by a computer program to instruct related hardware to implement the methods.

Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.

The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.

It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.

The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.

It will be understood by those skilled in the art that all or part of the steps of implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.

Claims

1. A file operation method is characterized by comprising the following steps:

receiving a remote operation request sent by a client;

judging whether a mark corresponding to the opening mode exists in the cache;

2. The method of claim 1, wherein in response to the remote operation request being a first type of operation request, setting a corresponding open flag, further comprising:

3. The method of claim 2, wherein opening a file to be operated on according to an open mode corresponding to the open flag, further comprises:

4. The method of claim 3, wherein the file to be operated is directly operated in response to a flag corresponding to an open mode in the cache, further comprising:

5. The method of claim 4, further comprising:

6. The method of claim 1, further comprising:

7. The method of claim 6, further comprising:

8. A file operating system, comprising:

9. A computer device, comprising:

at least one processor; and

memory storing a computer program operable on the processor, wherein the processor executes the program to perform the steps of the method according to any of claims 1-7.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1 to 7.