CN117852002A

CN117852002A - File system locking method, device, equipment and storage medium

Info

Publication number: CN117852002A
Application number: CN202410130855.XA
Authority: CN
Inventors: 唐蜜; 过晓春
Original assignee: China United Network Communications Group Co Ltd; Unicom Digital Technology Co Ltd; Unicom Cloud Data Co Ltd
Current assignee: China United Network Communications Group Co Ltd; Unicom Digital Technology Co Ltd; Unicom Cloud Data Co Ltd
Priority date: 2024-01-30
Filing date: 2024-01-30
Publication date: 2024-04-09

Abstract

The application provides a file system locking method, a device, equipment and a storage medium. The method comprises the following steps: acquiring multithreaded access requests sent by a client, wherein each access request carries an identification of a file to be accessed; acquiring at least two target threads accessing the same target file according to the identification of the file to be accessed; and locking the target file at different times according to the access types of the at least two target threads, so that the request phases of the at least two target threads are executed concurrently, and the data access phases read and write mutually exclusive for the same data domain of the target file. The method improves the concurrency processing capacity of the file system.

Description

File system locking method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of distributed file systems, and in particular, to a method, an apparatus, a device, and a storage medium for locking a file system.

Background

With the rapid growth of the internet and big data, the data volume grows exponentially. Conventional stand-alone file systems cannot meet the storage requirements of huge amounts of data, and thus a distributed file system is required to store and manage data. The distributed file system can provide distributed object, file and block storage services for users, and is used for big data and cloud computing fields. The distributed file system can provide concurrent access functions for users, process the requests of the users through a multi-thread mode, and internally control multi-thread access through a global lock mode, thereby allowing the users to use the distributed file system like operating a local file system.

The control of the multi-thread access through the global lock ensures the consistency of data, but the existence of the global lock also leads to longer holding time of the lock by a single thread, so that the concurrent processing capacity of the multi-thread cannot be fully utilized, and particularly, under the condition of multi-user large-pressure access, the distributed file system cannot fully and concurrently process the request.

Thus, there is a need for a method that increases the concurrent processing capacity of a file system.

Disclosure of Invention

The application provides a file system locking method, a device, equipment and a storage medium, which are used for solving the technical problem of low concurrent processing capacity of a file system.

In a first aspect, the present application provides a method for locking a file system, including:

acquiring multithreaded access requests sent by a client, wherein each access request carries an identification of a file to be accessed;

acquiring at least two target threads accessing the same target file according to the identification of the file to be accessed;

and locking the target file at different times according to the access types of the at least two target threads, so that the request phases of the at least two target threads are executed concurrently, and the data access phases read and write mutually exclusive for the same data domain of the target file.

In one possible design, the locking the target file at different times according to the access types of the at least two target threads includes:

if the access types of the at least two target threads comprise a write request type and a getattr access type, adding a global lock to the target file when a first target thread corresponding to the write request type requests the target file, and controlling a second target thread corresponding to the getattr access type to execute a request stage, wherein the request stage of the second target thread is executed in parallel with the request processing, and the request processing comprises a request stage and a data access stage.

In one possible design, the controlling the second target thread corresponding to the getattr access type to execute the request phase includes:

performing authentication on the client to execute the request phase;

after the second target thread corresponding to the getattr access type is controlled to execute the request stage, the method further comprises:

after the authentication is passed, after the target file is unlocked, reading and locking the cached metadata, and acquiring a historical timestamp and a current timestamp for caching the metadata;

Acquiring the life duration of the metadata according to the current timestamp and the historical timestamp;

if the metadata are not expired according to the life duration, reading target data corresponding to the access request from the metadata and returning the target data;

and if the metadata is determined to be out of date according to the life time, sending a request to an MDS metadata server, adding a write lock to the metadata, and updating the metadata obtained by the request into a cache through the write lock.

if the access types of the at least two target threads are write request types, the request phases of the at least two target threads are executed concurrently;

and adding a domain lock to the data domains of the write requests of the at least two target threads according to the data domains of the write request target files of the at least two target threads, so as to perform write operation, and leading the write requests of different target threads to be mutually exclusive to the same data domain.

In one possible design, the adding a scope lock to the data fields of the write requests of the at least two target threads according to the data fields of the write requests of the at least two target threads, so that the write requests of different target threads to the same data field are mutually exclusive, includes:

According to the data fields of the target files of the write requests corresponding to the at least two target threads, respectively and concurrently adding a scope lock to different data fields of the at least two target threads in the same time period to perform write operation, and unlocking each data field after the write operation of the different data fields is completed;

and in the next time period, simultaneously locking other different data fields of at least two target threads to perform writing operation, unlocking each data field after the writing operation of the other different data fields is completed, and repeatedly executing the locking and unlocking operation until each target thread completes the writing operation.

if the access types of the at least two target threads are both getattr access types, the request phases corresponding to the at least two target threads are executed concurrently;

and when the metadata cached by at least two target threads are out of date, concurrently locking the metadata, sending a request to an MDS metadata server, locking the metadata, updating the metadata obtained by the request into the cache through the write lock, and mutually exclusive write requests of the at least two target threads.

In one possible design, the write request type of access request is a request to create a directory mkdir.

In a second aspect, the present application provides a file system locking apparatus, including:

the receiving module is used for acquiring multithreaded access requests sent by the client, and each access request carries an identification of a file to be accessed;

the acquisition module is used for acquiring at least two target threads accessing the same target file according to the identification of the file to be accessed;

and the locking module is used for locking the target file at different times according to the access types of the at least two target threads so as to enable the request phases of the at least two target threads to be executed concurrently, and the data access phases read and write mutual exclusion aiming at the same data domain of the target file.

In a third aspect, the present application provides a file system locking apparatus, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to cause the at least one processor to perform the file system locking method as described above in the first aspect and the various possible designs of the first aspect.

In a fourth aspect, the present application provides a computer readable storage medium having stored therein computer executable instructions which, when executed by a processor, implement the file system locking method according to the first aspect and the various possible designs of the first aspect.

According to the file system locking method, device, equipment and storage medium, at least two target threads accessing the same target file are obtained according to the identification of the file to be accessed carried in each access request by obtaining the multithreaded access request sent by the client. By dividing each target thread into a request phase and a data access phase, the request phases of at least two target threads can be executed concurrently, and further, when at least two target threads lock target files in different time periods, the request phases of at least two target threads can be processed in parallel, then the same data domain of the target files are read and written mutually exclusively, the concurrent processing capacity of a file system is improved, and the problem of low concurrent processing capacity of the file system caused by overlong locking time of each request on the target files when a plurality of target threads process the same target files is avoided.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

FIG. 1 is a system architecture diagram of an embodiment of the present application;

FIG. 2 is a flowchart of a method for locking a file system according to an embodiment of the present application;

FIG. 3 is a flowchart II of a method for locking a file system according to an embodiment of the present application;

FIG. 4A is a schematic diagram of a locking manner of a file system according to an embodiment of the present disclosure;

FIG. 4B is a second schematic diagram of a locking manner of a file system according to an embodiment of the present disclosure;

FIG. 5 is a flowchart III of a method for locking a file system according to an embodiment of the present application;

FIG. 6A is a third schematic diagram of a locking manner of a file system according to an embodiment of the present disclosure;

FIG. 6B is a fourth schematic diagram of a locking manner of a file system according to an embodiment of the present disclosure;

FIG. 7 is a flowchart of a method for locking a file system according to an embodiment of the present disclosure;

FIG. 8 is a fifth exemplary diagram of a locking manner of a file system according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a locking device of a file system according to an embodiment of the present application;

fig. 10 is a hardware schematic diagram of a file system locking device according to an embodiment of the present application.

Specific embodiments thereof have been shown by way of example in the drawings and will herein be described in more detail. These drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.

The distributed file system can provide concurrent access functions for users, process the requests of the users through a multi-thread mode, and internally control multi-thread access through a global lock mode, thereby allowing the users to use the distributed file system like operating a local file system. The control of the multi-thread access through the global lock ensures the consistency of data, but the existence of the global lock also leads to longer holding time of the lock by a single thread, so that the concurrent processing capacity of the multi-thread cannot be fully utilized, and particularly, under the condition of multi-user large-pressure access, the distributed file system cannot fully and concurrently process the request.

In order to effectively improve concurrent processing capacity of a file system, the invention designs a file system locking method, and a file system client side simultaneously processes a plurality of target threads accessing the same target file according to a multithreaded access request sent by a client machine. By dividing the target threads into a request phase and a data access phase, when a file system client aims at a plurality of target threads of the same target file, the request phases of the target threads can be simultaneously processed concurrently, and then the target file is locked in different time periods according to the respective access types of different target threads, so that the data access phases of different target threads are mutually exclusive.

The method for locking the file system aims at solving the technical problems in the prior art.

Fig. 1 is a system architecture diagram of an embodiment of the present application, and the system architecture of the present application is described in conjunction with fig. 1. As shown in fig. 1, the system of the present application includes a client machine 101 and a file system client 102. The client machine 101 is able to run a corresponding process, which in turn causes the file system client 102 to provide access to the file system for the process. The file system client 102 can be a Ceph-fuse client. In the system of the present application, a user program on client machine 101 can access the mount point, thereby enabling requests on the mount point to enter a kernel-mode fuse module through system calls. The fuse module sends a request to the user-mode libfuse module via the character device. The libfuse module processes the request through multithreading and sends the request to the file system client 102. During the mount process, the file system client 102 can establish communication with a MetaData Server (MDS) and obtain MetaData information of the root directory. After mounting is complete, file system client 102 can provide a standard file system interface, such as read, write, create, delete files, etc., for client machine 101. When a thread in the file system client 102 processes a request, the lock module can lock the requested target file, then process metadata or data, and the lock is not released until the request needs to be sent to the MDS or Rados (useable, autonomic Distributed Object Store) through the message module. After the data operation corresponding to the request is completed, the file system client 102 can return the operation result to the client machine 101.

The following describes the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 2 is a flowchart of a file system locking method according to an embodiment of the present application. As shown in fig. 2, the execution body of the present embodiment may be the file system client 102 in the embodiment of fig. 1, where the method includes:

s201, acquiring multithreaded access requests sent by a client, wherein each access request carries an identification of a file to be accessed.

Specifically, the access request sent by the client can be used to operate files and directories corresponding to the files in the file system client, as well as access metadata information. For example, the access request may be a getattr request, a write request, or the like. The getattr request can be used to obtain attribute information of a file or directory. The write request can be used to write data to a file.

Alternatively, the identifier of the file to be accessed may be a request path of the target file, and the file system client enables the client to access the specified target file or directory according to the request path carried by each access request.

In a specific implementation process, a file system client acquires multithreaded access requests sent by the client, and each access request carries an identification of a file to be accessed.

S202, according to the identification of the file to be accessed, at least two target threads accessing the same target file are obtained.

Specifically, the file system client can determine a target file to be accessed by the client according to the identification in at least two access requests sent by the client, select at least two target threads from a thread pool, and use the at least two target threads as target threads for processing the at least two access requests.

S203, locking the target file at different times according to the respective access types of the at least two target threads, so that the request phases of the at least two target threads are executed concurrently, and the data access phases read and write mutually exclusive for the same data domain of the target file.

Specifically, the target thread may be divided into a request phase and a data access phase. The request phase may include client authentication and rights verification. For example, authentication may be achieved by a user name and password or token provided by the client, etc. Further, the authentication phase does not require locking of the accessed resource. Since the authentication phase described above does not require access to or modification of shared resources in the file system client, the authentication phase does not require acquisition of a lock. For another example, the rights verification may check the access rights of the client to the requested target file or directory, thereby ensuring that the client is entitled to perform the requested operation. The data access stage can execute corresponding read-write operations according to respective access types of the target threads.

In a specific implementation process, the file system client can process the request phases of at least two target threads simultaneously, so that the request phases of at least two target threads can be executed concurrently, and lock the target file at different times according to respective access types of different target threads, so that the same data domain of the target file can read and write mutually exclusive.

According to the file system locking method, the plurality of target threads accessing the same target file are divided into the request phase and the data access phase, so that when a file system client aims at the plurality of target threads of the same target file, the request phases of the plurality of target threads can be processed simultaneously, then the target file is locked in different time periods according to the respective access types of different target threads, and the data access phases of different target threads are mutually exclusive.

Fig. 3 is a flowchart of a second method for locking a file system according to an embodiment of the present application. In this embodiment, on the basis of the embodiment of fig. 2, in the case that the access type of the target thread includes a write request type and a getattr access type, a process of locking the target file by the write request type and the getattr access type is described in detail. As shown in fig. 3, the method includes:

S301, acquiring multithreaded access requests sent by a client, wherein each access request carries an identification of a file to be accessed.

S302, according to the identification of the file to be accessed, at least two target threads accessing the same target file are obtained.

Steps S301 to S302 are similar to steps S201 to S202 in the embodiment of fig. 2, and the description of this embodiment is omitted here.

S303, if the access types of the at least two target threads comprise a write request type and a getattr access type, adding a global lock to the target file when the first target thread corresponding to the write request type requests the target file.

In one possible design, if the access types of the at least two target threads include a write request type and a getattr access type, the write request type is preferentially processed. And when the first target thread corresponding to the write request type writes data into the target file, the file system client can acquire the global lock for the first target thread. A global lock can only allow one thread to acquire the lock at any one time and block access by other threads. The global lock can avoid simultaneous access of multiple threads to the critical section and avoid conflict of simultaneous operation of the multiple threads. Critical sections can avoid race conditions and data inconsistency problems caused by concurrent access. For example, a critical section may be a shared variable or a shared resource in a multi-threaded environment.

In one possible design, the access request of the write request type is a request to create a directory mkdir.

In one possible design, when the access types of the at least two target threads include a write request type and a getattr access type, at least two request phases of the at least two target threads can be processed concurrently, after the at least two request phases of the at least two target threads are processed concurrently, a write lock is configured for a data access phase of any one of the at least two target threads, and the data access phases of different target threads are mutually exclusive.

S304, when the first target thread corresponding to the write request type requests the target file, the second target thread corresponding to the getattr access type is authenticated to execute the request stage, wherein the request stage of the second target thread is executed in parallel with the request processing.

Specifically, the request processing of the target file by the first target thread includes a request phase of the first target thread and a data access phase of the first target thread. The request phase of the second target thread is executed in parallel with the request processing of the first target thread. Further, the request phase of the second target thread can be executed in parallel with any phase in the request processing of the first target thread. For example, the request phase of the second target thread can be executed in parallel with the request phase in the first target thread request process, and the request phase of the second target thread can also be executed in parallel with the data access phase in the first target thread request process.

In particular, the embodiment of FIG. 3 is further described in connection with FIGS. 4A-4B. Fig. 4A is a schematic diagram of a locking manner of a file system provided in the embodiment of the present application in a case where an access type of a target thread includes a write request type and a getattr access type. Fig. 4B is a schematic diagram two of a locking manner of a file system provided in the embodiment of the present application in a case where an access type of a target thread includes a write request type and a getattr access type.

FIG. 4A can represent an existing file system locking scheme. As shown in fig. 4A, if the access types of at least two target threads include a write request type and a getattr access type, the conventional file system locking manner is: the file system client configures a global lock for the write request of the first thread, after the identity verification and the authority verification of the first thread in the request stage are completed, the client writes data into the target file or modifies the data, and after the target file is modified, the file system client releases the global lock. After the file system client releases the global lock, the global lock is configured for the getattr access request. The request phase and the data access phase of the write request are mutually exclusive with the request phase and the data access phase of the getattr access request.

FIG. 4B can represent a file system locking scheme in accordance with an embodiment of the present application. As shown in fig. 4B, if the access types of at least two target threads include a write request type and a getattr access type, the file system locking manner in the embodiment of the present application is: the file system client configures a global lock for the write request of thread one, and performs authentication on the getattr access request in line Cheng San while the request is processed by thread one for the target file. After the file system client releases the lock for the write request of the thread one, after the identity verification of the getattr access request in the thread three passes, the lock is configured for the data access stage of the getattr access request in the thread three. In this way, the global lock is configured for the write request of the first thread, and the target file of the first thread is processed, and the request stage of the getattr access in the third thread is concurrently executed, so that the concurrency processing capability of the file system can be improved.

And S305, after the identity verification is passed and the target file is unlocked, reading and locking the cached metadata to obtain a historical timestamp and a current timestamp for caching the metadata.

Specifically, metadata information corresponding to metadata is cached in a memory of a file system client. For example, storage location information of metadata information data, history data information, resource search information, file record information, and the like. Caching metadata information in the memory of a file system client can improve the performance of the file system client and can reduce frequent accesses of the file system client by the client. The historical timestamp may be the time at which the metadata information was last cached by the file system client. The current timestamp may be the time at which the file system client read-locks the cached metadata.

In one possible design, a file system client can utilize its cached metadata information to meet the client's access needs to a file, without having to communicate with the MDS in the file system client each time, and the file system client can directly return the cached metadata information to the client in the event that the cached metadata information is not expired.

S306, acquiring the life duration of the metadata according to the current timestamp and the historical timestamp.

Specifically, the lifetime of metadata is the difference between the current timestamp and the historical timestamp. The preset range of the life duration can be adaptively adjusted according to the modification frequency and importance of the target file. For example, the preset range of the life duration can be 5s or 5min.

S307, judging whether the life duration exceeds a preset range, if so, executing S308; if not, S309 is performed.

S308, determining that the metadata is not expired, reading the target data corresponding to the access request from the metadata, and returning.

Specifically, the target data may be content data of the target file. Such as text files, image files, video files, etc.

In the specific implementation process, when determining that the cached metadata of the file system client is not expired, the file system client can read target data in the metadata of the target file corresponding to the getattr access type in the second target thread from the cached metadata, and return the target data to the client.

S309, determining that the metadata has expired, sending a request to the MDS metadata server, adding a write lock to the metadata, and updating the metadata obtained by the request into a cache through the write lock.

In a specific implementation process, a file system client sends a request to an MDS metadata server to acquire latest metadata. Writing a lock to cached metadata in a file system client ensures that other threads cannot read or modify the cached metadata during an update. And updating the latest metadata into the cache through the acquired write lock, and returning the updated metadata to the client.

According to the file system locking method, when the access types of at least two target threads comprise the write request type and the getattr access type, the request stage of the second target thread corresponding to the getattr access type and the request processing corresponding to the write request type can be executed in parallel, after the identity of the second target thread passes verification, after the target file corresponding to the first target thread is unlocked, the data access stage of the second target thread can be locked, and therefore the concurrent processing capacity of the file system is improved.

Fig. 5 is a flowchart III of a file system locking method according to an embodiment of the present application. In this embodiment, on the basis of the embodiment of fig. 2, in the case where the access types of the target threads are all write request types, a process of locking the write request types for the target files is described in detail. As shown in fig. 5, the method includes:

S501, acquiring multithreaded access requests sent by a client, wherein each access request carries an identification of a file to be accessed.

S502, according to the identification of the file to be accessed, at least two target threads accessing the same target file are obtained.

Steps S501-S502 are similar to steps S201-S202 in the embodiment of fig. 2, and the description of this embodiment is omitted here.

S503, if the access types of at least two target threads are write request types, the request phases of the at least two target threads are executed concurrently.

S504, according to the data fields of the target files of the write requests corresponding to the at least two target threads, respectively and concurrently adding a scope lock to different data fields of the at least two target threads in the same time period to perform write operation, and unlocking each data field after the write operation of the different data fields is completed.

Specifically, the data fields of the target file are different data portions in the target file, and can be divided by functions. Such as quota data fields, inline information data fields, cap rights data fields, and the like.

In one possible design, a target thread can be divided into multiple scopes, and when writing to at least two target threads, a file system client can write to different data domains in different target threads concurrently in the same time period.

In particular, the embodiment of FIG. 5 is further described in connection with FIGS. 6A-6B. Fig. 6A is a schematic diagram three of a locking manner of a file system according to an embodiment of the present application. Fig. 6B is a schematic diagram of a locking manner of a file system according to an embodiment of the present application. Fig. 6A can represent a locking manner of an existing file system when all access types of a target thread are write request types, and fig. 6B can represent a locking manner of a file system according to an embodiment of the present application when all access types of a target thread are write request types.

As shown in fig. 6A, in the case where the access types of at least two target threads are both write request types, the conventional file system locking manner is: the file system client configures a global lock for the write request of the first thread, after the identity verification and the authority verification of the first thread in the request stage are completed, the client writes data into the target file or modifies the data, and after the target file is modified, the file system client releases the global lock. After the file system client releases the global lock, the global lock is configured for the write request of thread three. The request phase and the data access phase of the write request of the thread one are mutually exclusive with the request phase and the data access phase of the write request of the thread three.

As shown in fig. 6B, in the case where the access types of at least two target threads are both write request types, the file system locking manner in the embodiment of the present application is: the file system client divides write requests for thread one and thread three into a request phase and a data access phase. The request begins with a request phase for verifying identity and rights information. The data access stage is divided into a scope 1 and a scope 2 according to functions. The file system client performs concurrent processing on the request phases of the first thread and the third thread, locks the scope 1 of the first thread and the scope 2 of the third thread after the concurrent processing on the request phases of the first thread and the third thread, and performs write operation on the scope 1 of the first thread and the scope 2 of the third thread. After concurrent execution of a write operation to scope 1 of thread one and scope 2 of thread three, the lock is released. And lock up the scope 2 of the thread one and the scope 1 of the thread three, and further execute the write operation on the scope 2 of the thread one and the scope 1 of the thread three concurrently. In this way, according to the embodiment of the application, the target threads with the access types of the at least two target threads being the write request types are divided according to the functions, and the granularity of the requests is refined, so that when the file system client processes different data fields of different target threads, the two target threads can be executed concurrently, the waiting time of locking the data fields is reduced, and the concurrent processing capacity of the file system is improved.

S505, in the next time period, locking other different data fields of at least two target threads are respectively and concurrently locked for writing operation, after the writing operation of the other different data fields is finished, unlocking the data fields, and repeatedly executing the locking and unlocking operation until each target thread completes the writing operation.

In one possible design, if the number of data fields of at least two target threads is different, then there is a case where a part of the target threads complete request processing first and a part of the target threads complete request processing later. The target thread in the file system client which completes the request processing first can wait to complete the target thread of the request processing until the last target thread completes the request processing.

According to the file system locking method, when the access types of at least two target threads are the write request types, the target threads are divided according to the functions, and the granularity of the write requests is refined, so that when the file system client processes different data fields of different target threads, the file system client can execute the target threads and the write request according to the functions, the waiting time of locking the data fields is reduced, and the concurrent processing capacity of the file system is improved.

Fig. 7 is a flowchart of a file system locking method according to an embodiment of the present application. In this embodiment, on the basis of the embodiment of fig. 2, in the case that the access types of the target threads are both getattr access types, a process of locking the target files with the getattr access types is described in detail. As shown in fig. 7, the method includes:

s701, acquiring multithreaded access requests sent by a client, wherein each access request carries an identification of a file to be accessed.

S702, according to the identification of the file to be accessed, at least two target threads accessing the same target file are obtained.

Steps S701 to S702 are similar to steps S201 to S202 in the embodiment of fig. 2, and the description of this embodiment is omitted here.

S703, if the access types of the at least two target threads are both getattr access types, the request phases corresponding to the at least two target threads are executed concurrently.

In one possible design, after the access types of the at least two target threads are both getattr access types and the request phases corresponding to the at least two target threads are concurrently executed, the method further includes:

and judging whether the metadata cached by at least two target threads are out of date. If the metadata cached by at least two target threads are all out of date, the file system client returns the cached metadata to the client.

And S704, when the metadata cached by at least two target threads are expired, concurrently locking the metadata, sending a request to the MDS metadata server, locking the metadata, updating the metadata obtained by the request into the cache through the write lock, and mutually exclusive the write locks of the at least two target threads.

Specifically, the embodiment of fig. 7 is further described in connection with fig. 8. Fig. 8 is a schematic diagram of a locking manner of a file system according to an embodiment of the present application. As shown in fig. 8, if the access types of at least two target threads are both getattr access types, the file system locking manner in the embodiment of the present application is: the file system client concurrently processes the request phases of thread one and thread three. After the identity verification of the getattr access request of the thread I and the thread III is passed, configuring a read lock for the metadata cached in the file system client of the target files corresponding to the thread I and the thread III, and further judging whether the cached metadata is out of date. If the cache metadata of the target files corresponding to the first thread and the third thread are expired, after the read locks are released by the first thread and the third thread, write locks are respectively configured for the cache metadata corresponding to the first thread and the third thread. And, the write requests of thread one and thread three are mutually exclusive to execute.

According to the file system locking method, when the access types of at least two target threads are the getattr request types, the request phases of the at least two target threads and the configuration read lock phases in the data request phases are processed in parallel, so that the concurrent processing capacity of the file system can be improved. When multiple target threads perform getattr requests on the same target file, a global lock is configured for each request, so that a request stage of different getattr requests of different target threads and an acquisition read lock stage in a data request stage cannot be processed in parallel, and the concurrent processing capacity of a file system is reduced.

Fig. 9 is a schematic structural diagram of a locking device for a file system according to an embodiment of the present application. As shown in fig. 9, the file system locking apparatus 90 includes: a receiving module 901, an acquiring module 902, a locking module 903.

The receiving module 901 is configured to obtain multithreaded access requests sent by a client, where each access request carries an identifier of a file to be accessed.

And the acquiring module 902 is configured to acquire at least two target threads accessing the same target file according to the identifier of the file to be accessed.

The locking module 903 is configured to lock the target file at different times according to respective access types of at least two target threads, so that request phases of the at least two target threads are executed concurrently, and a data access phase reads and writes mutual exclusion with respect to a same data domain of the target file.

In one possible design, the lock module 903 is also used to:

if the access types of the at least two target threads comprise a write request type and a getattr access type, adding a global lock to the target file when a first target thread corresponding to the write request type requests the target file, and controlling a second target thread corresponding to the getattr access type to execute a request stage, wherein the request stage of the second target thread is executed in parallel with the request process, and the request process comprises a request stage and a data access stage.

In one possible design, the lock module 903 is also used to:

performing identity verification on the client to execute a request phase;

after the second target thread corresponding to the get ttr access type is controlled to execute the request stage, the method further comprises:

acquiring the life time of the metadata according to the current time stamp and the historical time stamp;

if the metadata are not expired according to the life time, reading the target data corresponding to the access request from the metadata and returning the target data;

if the metadata is determined to be out of date according to the life time, sending a request to the MDS metadata server, adding a write lock to the metadata, and updating the metadata obtained by the request into a cache through the write lock.

In one possible design, the lock module 903 is also used to:

if the access types of the at least two target threads are the write request types, the request phases of the at least two target threads are executed concurrently;

and adding a domain lock to the data domains of the write requests of the at least two target threads according to the data domains of the write requests of the at least two target threads, so as to perform write operation, and leading the write requests of different target threads to be mutually exclusive to the same data domain.

In one possible design, the lock module 903 is also used to:

If the access types of the at least two target threads are the getattr access types, the request phases corresponding to the at least two target threads are executed concurrently;

and when the metadata cached by at least two target threads are out of date, concurrently locking the metadata, sending a request to the MDS metadata server, locking the metadata, updating the metadata obtained by the request into the cache through the write lock, and mutually exclusive the write locks of the at least two target threads.

Fig. 10 is a hardware schematic diagram of a file system locking device according to an embodiment of the present application. As shown in fig. 10, the file system locking apparatus 100 provided in this embodiment includes: at least one processor 1001 and memory 1002. The device 100 further comprises a communication component 1003. The processor 1001, the memory 1002, and the communication unit 1003 are connected by a bus 1004.

In a particular implementation, at least one processor 1001 executes computer-executable instructions stored in memory 1002, causing the at least one processor 1001 to perform a file system locking method as described above.

The specific implementation process of the processor 1001 may refer to the above method embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described herein again.

The application also provides an electronic device comprising: the system comprises a processor and a memory which is in communication connection with the processor, wherein the memory stores computer-executable instructions, and the processor executes the computer-executable instructions stored in the memory to realize the file system locking method.

The application also provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and when a processor executes the computer execution instructions, the locking method of the visual file system is realized.

The computer readable storage medium described above may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. A readable storage medium can be any available medium that can be accessed by a general purpose or special purpose computer.

It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of action combinations, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all alternative embodiments, and that the acts and modules referred to are not necessarily required in the present application.

It should be further noted that, although the steps in the flowchart are sequentially shown as indicated by arrows, the steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least a portion of the steps in the flowcharts may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order in which the sub-steps or stages are performed is not necessarily sequential, and may be performed in turn or alternately with at least a portion of the sub-steps or stages of other steps or other steps.

It should be understood that the above-described device embodiments are merely illustrative, and that the device of the present application may be implemented in other ways. For example, the division of the units/modules in the above embodiments is merely a logic function division, and there may be another division manner in actual implementation. For example, multiple units, modules, or components may be combined, or may be integrated into another system, or some features may be omitted or not performed.

In addition, each functional unit/module in each embodiment of the present application may be integrated into one unit/module, or each unit/module may exist alone physically, or two or more units/modules may be integrated together, unless otherwise specified. The integrated units/modules described above may be implemented either in hardware or in software program modules.

The integrated units/modules, if implemented in hardware, may be digital circuits, analog circuits, etc. Physical implementations of hardware structures include, but are not limited to, transistors, memristors, and the like. The processor may be any suitable hardware processor, such as CPU, GPU, FPGA, DSP and ASIC, etc., unless otherwise specified. Unless otherwise indicated, the storage elements may be any suitable magnetic or magneto-optical storage medium, such as resistive Random Access Memory RRAM (Resistive Random Access Memory), dynamic Random Access Memory DRAM (Dynamic Random Access Memory), static Random Access Memory SRAM (Static Random-Access Memory), enhanced dynamic Random Access Memory EDRAM (Enhanced Dynamic Random Access Memory), high-Bandwidth Memory HBM (High-Bandwidth Memory), hybrid Memory cube HMC (Hybrid Memory Cube), etc.

The integrated units/modules may be stored in a computer readable memory if implemented in the form of software program modules and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments. The technical features of the above embodiments may be combined in any way, and for brevity, all of the possible combinations of the technical features of the above embodiments are not described, but should be considered as the scope of the description

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method of locking a file system, the method comprising:

2. The method of claim 1, wherein said locking the target file at different times according to the access type of each of the at least two target threads comprises:

3. The method of claim 2, wherein controlling the second target thread corresponding to the getattr access type to execute the request phase comprises:

performing authentication on the client to execute the request phase;

4. The method of claim 1, wherein said locking the target file at different times according to the access type of each of the at least two target threads comprises:

5. The method according to claim 4, wherein the adding a scope lock to the data fields of the write requests of the at least two target threads according to the data fields of the target files of the write requests of the at least two target threads, so that the write requests of different target threads to the same data field are mutually exclusive, comprises:

6. The method of claim 1, wherein said locking the target file at different times according to the access type of each of the at least two target threads comprises:

7. The method of claim 2, wherein the access request of the write request type is a request to create a directory mkdir.

8. A file system locking apparatus, comprising:

9. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 7.

10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1 to 7.