CN118312105A

CN118312105A - Control method, medium, electronic device and program product for distributed storage system

Info

Publication number: CN118312105A
Application number: CN202410565007.1A
Authority: CN
Inventors: 田杨锋; 王杰; 李鹏; 杨金峰; 耿英杰
Original assignee: Beijing Volcano Engine Technology Co Ltd
Current assignee: Beijing Volcano Engine Technology Co Ltd
Filing date: 2024-05-08
Publication date: 2024-07-09

Abstract

The disclosure relates to a control method, a medium, an electronic device and a program product of a distributed storage system, and relates to the technical field of computers. The proxy server manages the held segment locks through the first list and the second list, so that the search and the use of the segment locks can be facilitated, the segment locks held by the proxy server are orderly managed, and the use efficiency of the segment locks is improved.

Description

Control method, medium, electronic device and program product for distributed storage system

Technical Field

The present disclosure relates to the field of computer technology, and in particular, to a control method, medium, electronic device, and program product for a distributed storage system.

Background

NAS (Network Attached Storage ) storage architecture is a technical architecture specifically designed to provide file-level data storage access over a network. In NAS storage architecture, proxy services (Proxy) are responsible for resolving protocol requests of users and sending the resolved protocol requests to the backend storage clusters. In this process, the proxy service needs to interact with the metadata management server and the file storage server, where the metadata management server is responsible for metadata service and distributed lock service, manages all metadata attributes of the file, and the file storage server is responsible for file reading and writing operations and data disc dropping.

In the NAS storage architecture, if an event of concurrently reading and writing the same data interval occurs, the sequence of completing the read and write requests cannot be guaranteed, so that the data content of the data interval cannot be expected, and thus, erroneous data is read or erroneous data is written.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, the present disclosure provides a control method of a distributed storage system, the method including:

Receiving a resource access request initiated by a client, wherein the resource access request is used for indicating to execute a resource access operation aiming at data of a first data interval;

Determining a first set of segment locks matched with the first data interval based on a first list in response to the resource access request, wherein the first set of segment locks comprises segment locks in the range of the first data interval, and the first list is used for storing segment locks in unused states held by proxy servers in a distributed storage system;

Executing the resource access request based on the first segment lock set, and storing the segment locks included in the first segment lock set to a second list, wherein the second list is used for storing the segment locks in a use state held by the proxy server, the segment locks included in the first list and the second list are distributed to the proxy server by a metadata management server in the distributed storage system in response to the segment lock request sent by the proxy server, and the segment lock request is used for requesting the metadata management server to distribute the segment locks of a target data interval to the proxy server.

In a second aspect, the present disclosure provides a control method of a distributed storage system, the method including:

Receiving a segment lock request sent by a proxy server in a distributed storage system, wherein the segment lock request is used for requesting a metadata management server in the distributed storage system to distribute a segment lock of a target data interval to the proxy server;

responding to the segment lock request, and distributing the segment lock of the target data interval to the proxy server;

The method comprises the steps that a segmented lock of a target data interval is used for enabling a proxy server to respond to a resource access request initiated by a client and used for indicating to execute a resource access operation on data of a first data interval, a first segmented lock set matched with the first data interval is determined based on a first list, the resource access request is executed based on the first segmented lock set, and segmented locks included in the first segmented lock set are stored in a second list;

The first set of segment locks includes segment locks within the first data interval, the first list is used for storing segment locks in an unused state held by the proxy server, and the second list is used for storing segment locks in a used state held by the proxy server.

In a third aspect, the present disclosure provides a computer readable medium having stored thereon a computer program which, when executed by a processing device, implements the steps of the method of the first aspect, or implements the steps of the method of the second aspect.

In a fourth aspect, the present disclosure provides an electronic device comprising:

a storage device having a computer program stored thereon;

Processing means for executing said computer program in said storage means to carry out the steps of the method according to the first aspect or to carry out the steps of the method according to the second aspect.

In a fifth aspect, the present disclosure provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of the first aspect, or implements the steps of the method of the second aspect.

Based on the technical scheme, the resource access request initiated by the client is received, the first segmented lock set matched with the first data interval is determined based on the first list in response to the resource access request, the first segmented lock set comprises segmented locks in the range of the first data interval, then the resource access request is executed based on the first segmented lock set, the segmented locks included in the first segmented lock set are stored in the second list, and the proxy server can perform the resource access request on the first data interval through the segmented locks, so that the accuracy of read data can be guaranteed, and the consistency of the concurrently read and written data can be guaranteed under the concurrent read and write scene. In addition, the proxy server manages the held segment locks through the first list and the second list, so that the segment locks can be conveniently searched and used, the segment locks held by the proxy server are orderly managed, and the use efficiency of the segment locks is improved.

Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale. In the drawings:

FIG. 1 is a block diagram of a distributed storage system, according to an example embodiment.

Fig. 2 is a flow chart illustrating a method of controlling a distributed storage system according to an exemplary embodiment.

Fig. 3 is a schematic diagram of a first list and a second list shown according to an exemplary embodiment.

Fig. 4 is a schematic diagram illustrating a control method of a distributed storage system according to another exemplary embodiment.

FIG. 5 is a schematic diagram illustrating write-read conflicts in accordance with an example embodiment.

FIG. 6 is a schematic diagram illustrating read-write conflicts in accordance with an example embodiment.

FIG. 7 is a schematic diagram illustrating read-write conflicts in accordance with an example embodiment.

Fig. 8 is a flowchart illustrating a control method of a distributed storage system according to still another exemplary embodiment.

Fig. 9 is a schematic diagram illustrating a control method of a distributed storage system according to another exemplary embodiment.

Fig. 10 is a flowchart illustrating a control method of a distributed storage system according to an exemplary embodiment.

Fig. 11 is a schematic diagram of a third list and a fourth list shown according to an example embodiment.

Fig. 12 is a schematic diagram showing the module connection of a control device of a distributed storage system according to an exemplary embodiment.

Fig. 13 is a schematic block diagram illustrating a control apparatus of a distributed storage system according to still another exemplary embodiment.

Fig. 14 is a schematic structural view of an electronic device shown according to an exemplary embodiment.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. Related definitions of other terms will be given in the description below.

It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.

It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.

The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

FIG. 1 is a block diagram of a distributed storage system, according to an example embodiment. As shown in fig. 1, the control method of the distributed storage system provided in the embodiment of the present disclosure is applicable to the distributed storage system shown in fig. 1, and the system may include a proxy server 101, a metadata management server 102, and a file storage server 103. The proxy server 101, the metadata management server 102 and the file storage server 103 may perform data interaction through wired connection or wireless connection.

In the distributed storage system, the proxy server 101 is configured to parse the protocol request sent by the client 104, and send the parsed protocol request to the backend storage cluster (the metadata management server 102 and the file storage server 103). The metadata management server 102 is used for managing a file directory tree, is responsible for metadata services and distributed lock services, and manages all metadata attributes of files. The file storage server 103 is used for taking charge of reading and writing operations of files and data disc dropping. For example, the file storage server 103 manages Segment units, and one file is scattered on 16 segments in 64KB split. The proxy server 101 calculates a corresponding Segment from the offset and length, and then initiates an IO (Input/Output) request to the file storage server 103. The IO request may be understood as a resource access request, and may be a read/write operation for a file in the file storage server 103.

The client 104 may send a resource access request to the proxy server 101, and the proxy server 101 performs a read/write operation on a file stored in the file storage server 103 indicated by the resource access request based on the segment lock allocated by the metadata management server 102.

The proxy server 101 transmits a segment lock request to the metadata management server 102, and the metadata management server 102 receives the segment lock request transmitted by the proxy server 101 and allocates a segment lock of the target data section to the proxy server 101 in response to the segment lock request. The proxy server 101 receives the segment locks allocated by the metadata management server 102, determines a first segment lock set from the first list according to a first data interval indicated by the resource access request when the proxy server 101 initiates the resource access request, executes the resource access request based on the first segment lock set, and moves the segment locks included in the first segment lock set to the second list. The first set of segment locks includes segment locks within a first data interval, a first list for storing segment locks held by proxy server 101 in an unused state, and a second list for storing segment locks held by proxy server 101 in an used state.

Based on the method, when the proxy server initiates a resource access request, a first segmented lock set is determined from a first list according to a first data interval indicated by the resource access request, the first segmented lock set comprises segmented locks in the range of the first data interval, the resource access request is executed based on the first segmented lock set, the segmented locks included in the first segmented lock set are moved to a second list, the proxy server performs the resource access request on the first data interval through the segmented locks, the accuracy of read data can be guaranteed, and the consistency of the concurrently read-written data can be guaranteed under the concurrent read-write scene. In addition, the proxy server manages the segment locks through the first list and the second list, so that the search and the use of the segment locks can be facilitated, and the use efficiency of the segment locks is improved.

Fig. 2 is a flow chart illustrating a method of controlling a distributed storage system according to an exemplary embodiment. As shown in fig. 2, an embodiment of the present disclosure provides a control method of a distributed storage system, which is performed by a proxy server shown in fig. 1, and may include the following steps.

In step 210, a client-initiated resource access request is received.

Here, the resource access request is used to instruct the execution of the resource access operation with respect to the data of the first data section. The resource access request may be a read-write operation of the pointer to a first data interval of the target file in the file storage server. It should be appreciated that in a distributed file storage system, the resource access request may be client-triggered, accessing the data content of the first data interval of the target file in the file storage server through the proxy server. Correspondingly, the proxy server executes a resource access request triggered by the client, and performs read-write operation on the data content of the first data interval of the target file in the file storage server.

For example, when a client needs to read data of a first data section of a target file in a file storage server, the client may send a resource access request to a proxy server. For another example, the client may send a resource access request to the proxy server when the client needs to write new data to the first data interval of the target file in the file storage server.

In step 220, a first set of segment locks that match the first data interval is determined based on the first list in response to the resource access request.

Here, a segment lock is a fine-grained lock mechanism that is commonly used for concurrent access control to large-scale data sets. It divides a data into a plurality of separate, smaller segments, each with its own lock. Segment locks allow multiple concurrent operations to be performed on different segments of data simultaneously without blocking each other, thereby improving the parallel processing capacity and overall performance of the distributed storage system.

It should be noted that the segment locks may be classified into read segment locks and write segment locks. The data is read and read compatible, and the same data interval is mutually exclusive in reading, writing and writing. For the read segment lock of the [ start, end) data interval, when the proxy server initiates a read operation to the [ start, end) data interval of the target file, the read segment lock of the [ start, end) data interval needs to be applied to the metadata management server. For the write segment lock of the [ start, end) data section, when the proxy server initiates a write operation or a truncate operation to the [ start, end) data section of the target file, the metadata management server needs to apply for the write segment lock of the [ start, end) data section.

In some embodiments, the segmented lock may include a start-stop location, a lock identifier to identify the segmented lock, a read-write type, and a status attribute.

The start and stop positions include a start point (start) and an end point (end) of the data section, for example, [0,64k ], where the start point is 0 and the end point is 64k. The lock identifier (lock_id) may be an identifier for identifying a sequence number of the lock, e.g., the lock identifier of the segment lock may be 0001, 0002, 0003, 0004, etc. The metadata management server assigns a new segment lock to the proxy server, the lock identifier of which may also be newly assigned. That is, the lock identifiers may be sequentially assigned by the metadata management server. The READ-WRITE type is used to indicate the type of the segment lock, such as READ segment lock, WRITE segment lock. The state attribute of the segment lock is used to identify whether the segment lock is in a valid state or an invalid state. For example, the state attribute of the segment lock may be represented by a value of valid, if the value of valid is true, the segment lock is in a valid state, and if the value of valid is false, the segment lock is in a failure state.

And the proxy server searches the segment locks in the range of the first data interval from the first list according to the first data interval to obtain a first segment lock set. Wherein the first list is used for storing segment locks held by the proxy server in an unused state. It should be noted that a segment lock held by a proxy server in an unused state may refer to a segment lock assigned to the proxy server by a metadata management server, and that the proxy server does not use the segment lock to perform a resource access request.

It is noted that the number of segment locks included in the first set of segment locks may be one or more. A segment lock within the range of the first data interval may refer to a segment lock in which all segment locks included in the first set of segment locks are capable of completely covering the first data interval. For example, the first data interval is [0,192 k), then the segment locks included in the first set of segment locks may be segment locks for data interval [0,64 k), segment locks for data interval [64k,128 k), segment locks for data interval [128k,192 k).

In step 230, a resource access request is performed based on the first set of segment locks, and the segment locks included in the first set of segment locks are stored in a second list, where the second list is used to store segment locks in use held by the proxy server.

Here, the proxy server executing the resource access request based on the first segment lock set may mean that the proxy server holds the segment locks included in the first segment lock set, and performs a read-write operation on the data in the first data interval based on the segment locks included in the held first segment lock set.

The second list is used to store the segment locks in use held by the proxy server, which may refer to the segment locks assigned to the proxy server by the metadata management server, and which are used by the proxy server to perform any client-initiated resource access requests.

The first list and the second list include segment locks that are assigned to the proxy server by the metadata management server in response to segment lock requests sent by the proxy server. That is, all segment locks held by the proxy server are assigned to the proxy server by the metadata management server in response to segment lock requests sent by the proxy server. When the proxy server initiates a resource access request aiming at a first data interval, if the proxy server does not hold the segment lock of the first data interval, the proxy server sends a segment lock request to the metadata management server so that the metadata management server responds to the segment lock request and distributes the segment lock of the first data interval to the proxy server.

It should be noted that the segment locks in the second list are all memory locked such that the state attribute of the segment lock and the data interval cannot be changed, and thus, the data interval of any segment lock stored in the second list cannot be changed.

It should be appreciated that since the first list is used to store the segment locks held by the proxy server in an unused state, in the event that the proxy server performs a resource access request based on the segment locks included in the first set of segment locks, the segment locks included in the first set of segment locks are in an in-use state and then the segment locks included in the first set of segment locks need to be moved from the first list to the second list for storage.

Fig. 3 is a schematic diagram of a first list and a second list shown according to an exemplary embodiment. As shown in FIG. 3, the first list may include segment locks for data intervals of [0,64k ], [128k,192 k), and [512k,576 k), and the second list may include segment locks for data intervals of [64k,128k ], [512k,576 k), and [2049k,2113 k).

It should be appreciated that segment locks belong to the range lock. In the embodiment of the present disclosure, the left end point of the data interval corresponding to the segment lock may be aligned according to 64k, the aligned length may be adjusted, and the right end point may not be aligned, for example, the right end point may be the size of the file. It should be noted that in the first list and the second list, the segment locks may be ordered according to the right end point of the data interval, where the search complexity is O (logn), and n is the number of segment locks.

The proxy server manages the segment locks held by the proxy server through the first list and the second list, and can distinguish the use state of each held segment lock, so that the proxy server can orderly search the needed segment lock from the first list or the second list according to different executed operations.

It should be noted that, the proxy server executes the resource access request through the segment locks included in the first segment lock set, and since the proxy server executes the resource access request under the condition of holding the segment locks, the accuracy of the read-write data can be ensured.

For example, assuming that the size (size) of the file a is 100, if the first proxy server reads the data of the [90, 100 ] data section of the file a, if the second proxy server completes the truncate operation (one file system operation for adjusting the size of an existing file) for the file a, the size of the file a is changed to 50, and the first proxy server reads a section of out-of-range data because the first proxy server cannot perceive the change in the size of the file a.

Based on the method provided by the disclosure, when the first proxy server executes the resource access request, the first proxy server holds the segment lock of the [90, 100) data interval, if the second proxy server wants to mount the size of the file a to 50, the segment lock needs to be added in the [50, 100) data interval, and because the first proxy server already holds the segment lock of the [90, 100) data interval, segment lock conflict can occur, so that the second proxy server cannot complete the mount operation, and the data of the [90, 100) data interval accessed by the first proxy server is ensured to be accurate.

For another example, in a read-write concurrency scenario, since a resource access request sent by a client penetrates to a file storage server through a proxy server, the proxy server does not cache data, and when the proxy server reads and writes data to the file storage server, the read-write request (IO request) is split into 16 Striping (stripes), and the IO request is issued to different segments. Assuming that two proxy servers initiate concurrent writing to the same data interval, one proxy server writes a and one proxy server writes B completely, because one writing request is divided into 16 requests, the sequence of the 16 requests cannot be guaranteed, and finally, the results such as AABB or ABAB may appear, so that the data content of the whole data interval is unpredictable, and similarly, when the same data interval is read and written concurrently, the read data is also unpredictable.

Based on the method provided by the disclosure, under the scene of concurrent reading and writing aiming at the same data interval, the proxy server needs to hold the segment lock of the data interval to perform reading and writing operation, and the final data of the data interval is either AABB or BBBBBB due to the mutual exclusivity of the segment lock of the same data interval, which depends on which proxy server firstly acquires the segment lock, so that the consistency of the concurrent reading and writing data is ensured.

Therefore, by receiving a resource access request initiated by a client, responding to the resource access request, determining a first segmented lock set matched with a first data interval based on a first list, wherein the first segmented lock set comprises segmented locks within the range of the first data interval, then executing the resource access request based on the first segmented lock set, storing the segmented locks included in the first segmented lock set into a second list, and enabling a proxy server to perform the resource access request on the first data interval through the segmented locks, so that the accuracy of read data can be ensured, and the consistency of the concurrently read-write data can be ensured under the concurrent read-write scene. In addition, the proxy server manages the held segment locks through the first list and the second list, so that the segment locks can be conveniently searched and used, the segment locks held by the proxy server are orderly managed, and the use efficiency of the segment locks is improved.

In some implementations, in step 220, from the data interval in charge of each segment lock recorded in the first list, searching for a candidate segment lock in charge of the data area including the first data interval, determining whether the data area in charge of the candidate segment lock exceeds the first data interval, taking the candidate segment lock as a first target segment lock when the data interval in charge of the candidate segment lock does not exceed the first data interval, splitting the data interval in charge of the candidate segment lock with the first data interval as a reference when the data interval in charge of the candidate segment lock exceeds the first data interval, obtaining a target sub-segment lock in charge of the first data interval, and determining a first segment lock set based on the first target segment lock and the target sub-segment lock.

Here, the proxy server may find, through the first data interval, candidate segment locks including the first data interval in all responsible data intervals among the data intervals responsible for each segment lock recorded in the first list. The responsible data area includes the first data interval and may be a segment lock that at least partially covers the first data interval in the corresponding data interval in the first list.

The candidate segment locks may be segment locks in which the corresponding data interval in the first list includes a first data interval and the corresponding data interval at least partially exceeds the first data interval. The candidate segment lock may be configured to determine that the data interval of the first target segment lock is at least partially beyond the first data interval, and the data interval of the candidate segment lock may be at least partially beyond the first data interval. Of course, the candidate segment locks may also be segment locks in which the corresponding data section in the first list includes the first data section and the corresponding data section does not exceed the first data section. That is, the candidate segment locks that are found may include two types, one type is a segment lock that includes a first data section and the corresponding data section does not exceed the first data section, and the other type is a segment lock that includes a first data section and the corresponding data section exceeds the first data section.

For example, if the first data interval is [64k,128 k), if there is a segment lock for data interval [0,192 k) in the first list, [0,192 k) contains [64k,128 k), then the corresponding segment lock of [0,192 k) is a candidate segment lock.

And when the data interval in charge of the candidate segment lock does not exceed the first data interval, taking the candidate segment lock as a first target segment lock. For example, if the first data interval is [64k,128 k), if there are segment locks for the data interval [64,120 k) in the first list, [64,120 k) is within [64k,128 k), and [64,120 k) does not exceed the range of [64k,128 k), then the corresponding segment lock is the first target segment lock.

When the data interval in charge of the candidate segment lock exceeds the first data interval, splitting the data interval in charge of the candidate segment lock by taking the first data interval as a reference to obtain the target sub-segment lock in charge of the first data interval.

For example, if the first data interval is [64k,128 k), if there is a segment lock for data interval [0,192 k) in the first list, [0,192 k) contains [64k,128 k), then the corresponding segment lock of [0,192 k) is a candidate segment lock. Since the proxy server actually uses a partial data interval of [0,192 k), the interval endpoint of [0,192 k) is beyond the range of [64k,128 k), the data interval for which the candidate segment lock of [0,192 k) is responsible needs to be split into sub-segment locks of multiple different data intervals.

The plurality of sub-segment locks comprise a third sub-segment lock corresponding to the first data interval and a fourth sub-segment lock of other data intervals except the first data interval.

For example, [0,192 k) may split into a third sub-segment lock with a data interval of [64k,128 k) and a fourth sub-segment lock with a data interval of [0,64 k) and a data interval of [128k,192 k). Then, the third sub-segment lock is determined to be the target sub-segment lock.

Next, the first target segment lock and the target sub-segment lock are determined to be a first set of segment locks. That is, if the first data interval includes other data intervals than the data interval corresponding to the target sub-segment lock, the first set of segment locks may be determined based on the target sub-segment lock and the first target segment lock.

It should be noted that, when the proxy server executes the resource access request based on the target sub-segment lock, the target sub-segment lock may be deleted from the first list, and the target sub-segment lock may be moved to the second list for storage, so as to change the use state of the target sub-segment lock. That is, the fourth sub-segment lock with data interval [0,64k ], data interval [128k,192 k) is stored in the first list, and the third sub-segment lock with data interval [64k,128 k) is moved to the second list for storage.

Therefore, through the embodiment, when the data interval in charge of the candidate segment lock exceeds the first data interval, the proxy server can split the candidate segment lock and execute the resource access request based on the target sub-segment lock obtained by splitting, so that the segment lock occupying the overlong data interval is avoided.

In some implementations, when the first set of segment locks cannot fully cover the first data interval, determining the data interval in the first data interval that the first set of segment locks cannot cover as the first remaining data interval; and searching a second segmented lock set capable of covering the first residual data interval from the second list, wherein the second segmented lock set comprises segmented locks capable of covering the first residual data interval. Accordingly, in step 230, a resource access request may be performed based on the first set of segment locks and the second set of segment locks, and each segment lock in the first set of segment locks is stored to the second list.

Here, the failure of the first set of segment locks to completely cover the first data interval may mean that all segment locks within the range of the first data interval cannot be found in the first list, that is, the data intervals corresponding to all segment locks included in the first set of segment locks cannot completely cover the first data interval. For example, assuming that the first data interval is [0,192 k), if the first list includes data intervals of [0,64 k) and [64k,128 k), the first set of segment locks found from the first list lacks segment locks of [128k,192 k), that is, the data interval corresponding to the segment lock included in the first set of segment locks cannot completely cover the first data interval.

The first remaining data interval is a data interval except the data interval included in the first segment lock set in the first data interval.

For example, the first remaining data interval may be obtained by subtracting the data interval corresponding to all the segment locks included in the first set of segment locks from the first data interval. For example, assuming the first data interval is [0,192 k), if the first set of segment locks includes segment locks for data intervals of [0,64 k) and [64k,128 k), then the first remaining data interval is [128k,192 k).

The proxy server searches for segment locks in the range of the first remaining data interval from the second list through the first remaining data interval, and obtains a second set of segment locks based on the segment locks found from the second list. Wherein the second set of segment locks comprises segment locks capable of covering the first remaining data interval.

The segment locks that can cover the first remaining data interval may be segment locks that just cover the first remaining data interval for the data interval. For example, the first remaining data interval is [128k,192 k), then the segment locks for the data interval of [128k,192 k) are looked up from the second list to obtain a second set of segment locks. Of course, a segment lock that can cover the first remaining data section may also refer to a segment lock that can completely cover the first remaining data section and a portion of the data section is beyond the range of the first remaining data section. For example, the first remaining data interval is [128k,192 k), and the second set of segment locks obtained from the second list may include [64k,192 k).

It should be noted that, because the second list stores the segment locks in the in-use state, if the segment locks that can completely cover the first remaining data interval and have a part of the data interval beyond the range of the first remaining data interval are found in the second list, the found segment locks do not need to be split, but the found segment locks can be directly used to make a resource access request.

Accordingly, the proxy server holds all segment locks included based on the first set of segment locks and the second set of segment locks and performs resource access requests based on the segment locks. It should be noted that the proxy server needs to move each segment lock included in the first segment lock set to the second list for storage, so as to change the use state of each segment lock included in the first segment lock set.

Therefore, through the embodiment, the proxy server can orderly search the needed segment locks of the first data interval from the first list and the second list, so that the service efficiency of the segment locks is improved.

In some implementations, in a case where the segment lock held by the proxy server is not capable of fully covering the first data interval, a segment lock request is sent to the metadata management server, the segment lock request being used to request the metadata management server to assign a segment lock for the first data interval to the proxy server.

Here, the segment lock held by the proxy server may not cover the first data section in its entirety, which may mean that the first list and the second list do not have all the data sections of the first data section. For example, in the case that the first list cannot find a segment lock that entirely covers the first data section, the proxy server may then find a segment lock of the first remaining data section in the second list, and in the case that the second list does not have a segment lock that entirely covers the first remaining data section, it may be determined that the proxy server does not fully hold the segment lock of the first data section.

The absence of a segment lock in the second list that completely covers the first remaining data interval may mean that there is no segment lock in the second list that matches the corresponding data interval with the first remaining data interval. For example, the first remaining data interval is [128k,192 k), if there is a segment lock of [128k,160 k) in the second list and there is no segment lock of [160k,192 k), then it indicates that there is no segment lock of the second list that completely covers the first remaining data interval.

If there is no segment lock in the second list that completely covers the first remaining data interval, it indicates that the proxy server does not hold all segment locks required for the first data interval. Accordingly, the proxy server applies for the segment lock of the first data interval to the metadata management server.

It should be noted that, the proxy server does not hold all the segment locks required by the first data interval, and the proxy server not only needs to apply for the segment locks of the first data interval to the metadata management server, but also needs to return all the segment locks included in the first segment lock set obtained from the first list to the first list, so that when applying for the segment locks of the first data interval to the metadata management server, the segment locks included in the first list and belonging to the first segment lock set can be released.

Thus, according to the embodiment, when all the segment locks required for the first data section cannot be found, the segment locks of the first data section can be reapplied to the metadata management server.

In some implementations, in step 230, a memory lock may be added to each of the set of segmented locks included in the first set of segmented locks, validity of each of the segmented locks included in the first set of segmented locks is determined, and in the event that each of the segmented locks has validity, a resource access request is performed based on the segmented locks included in the first set of segmented locks.

Here, memory Locking (Memory Locking) is a computer programming technique, mainly used to ensure exclusive access to a Memory area by a process. In the embodiment of the disclosure, the use state and the data interval of the memory lock for controlling the segment lock cannot be changed.

Wherein, when the proxy server executes the resource access request based on the first set of segment locks, the proxy server adds a memory lock to each segment lock in the first set of segment locks to prevent the segment lock from being released or the same segment lock from being used by other concurrent resource access requests. It should be noted that when the proxy server performs a resource access request based on the first set of segment locks and the second set of segment locks, the proxy server adds a memory lock to each of the first set of segment locks and the second set of segment locks.

In some embodiments, under the condition that adding a memory lock to any segment lock in the first segment lock set fails, unlocking the segment lock in the first segment lock set, which has successfully added the memory lock, and adding the memory lock to the segment lock included in the first segment lock set again until all the segment locks included in the first segment lock set successfully add the memory lock.

If the memory lock fails to be added to each segment lock included in the first segment lock set, the segment locks which fail to be added to the memory lock are indicated to be used by other resource access requests. In order to prevent the problem of deadlock, once any one of the first set of segment locks fails to lock the memory, all segment locks that have successfully locked the memory need to be unlocked, and then, memory locking is performed on all segment locks included in the first set of segment locks in a re-attempt mode until each segment lock in the first set of segment locks successfully locks the memory.

The proxy server further checks whether a state attribute of each segment lock in the first set of segment locks is valid if each segment lock included in the first set of segment locks has successfully been memory-locked, and performs a resource access request based on the first set of segment locks if the state attribute of all segment locks included in the first set of segment locks is valid.

For example, the proxy server may check whether the values of valid of all the segment locks included in the first segment lock set are true, and if true, indicate that the state attribute of all the segment locks included in the first segment lock set are valid states.

It should be appreciated that in some scenarios, the segment locks taken by the proxy server may already be invalid. For example, the proxy server obtains the segment lock of [64k,128 k), before the memory lock is added to the segment lock of [64k,128 k), the proxy server receives a release request sent by the metadata management server for instructing to release the segment lock of [64k,128 k), the release thread of the proxy server adds the memory lock to the segment lock of [64k,128 k), and sets the state attribute of the segment lock of [64k,128 k) to an invalid state, and after the segment lock of [64k,128 k) is released, the memory lock of the segment lock of [64k,128 k) is released, and the segment lock of [64k,128 k) held by the read-write thread of the proxy server has been released.

Therefore, by checking the state attribute of all the segment locks included in the first segment lock set, and in the case that the state attribute corresponding to the segment lock included in the first segment lock set is a valid state, it can be determined that each segment lock included in the first segment lock set has validity. The proxy server then performs the resource access request based on the segment locks included in the first set of segment locks, which may ensure that the segment locks are actually held by the proxy server.

Therefore, through the embodiment, the segment lock can be prevented from being used by other resource access requests, and the validity of the segment lock can be ensured.

It should be noted that, in the case that the proxy server performs the resource access request based on the first set of segment locks and the second set of segment locks, the proxy server may add the memory lock to the segment locks included in the first set of segment locks and the second set of segment locks to perform the resource access request based on the foregoing embodiment as well.

In some implementations, after the execution of the resource access request is completed, if the segment locks included in the first segment lock set are not referenced by other operations, each segment lock in the first segment lock set is transferred to a first list, a second target segment lock having the same read-write type, the same lock identifier and belonging to an adjacent data interval as the segment lock in the first list is searched in the first list, and segment lock merging processing is performed on the first segment lock set and the second target segment lock in the first list.

Here, after completing execution of the resource access request based on the first set of segment locks, the proxy server returns the segment locks included in the first set of segment locks to the first list if the segment locks used by the proxy server are not referenced by other operations. The other operation may be a new resource access request, for example, a resource access request initiated by another client using a segment lock of any data interval in the first set of segment locks, and of course, the other operation may also be a release operation for releasing the segment lock.

It should be appreciated that where the first set of segment locks includes segment locks that are referenced by other operations, then the referenced segment locks may be stored to the second list to cause other operations to use the segment locks.

It should be noted that, in the case that the proxy server performs the resource access request based on the first set of segment locks and the second set of segment locks, the proxy server may also return the segment locks included in the first set of segment locks and the second set of segment locks to the first list.

The second target segment lock is a segment lock in the first list that has the same read-write type, the same lock identifier, and belongs to a neighboring data interval as the segment lock that is transferred into the first list.

And combining the second target segment lock with the segment lock transferred to the first list into one segment lock under the condition that the second target segment lock exists in the first list. The second target segment lock is a segment lock belonging to the same read-write type as the segment lock transferred to the first list, having the same lock identifier as the segment lock transferred to the first list, and belonging to an adjacent data interval as the segment lock transferred to the first list.

For example, in the above embodiment, after the proxy server performs the completion of the resource access request using the third sub-segment lock with the data interval [64k,128 k), the data interval [0,64k ] in the first list, and the fourth sub-segment lock with the data interval [128k,192 k) may be locked and the segment lock with the data interval [0,192 k).

Thus, by combining the second target segment lock and the segment lock transferred to the first list into one segment lock, excessive memory resources can be avoided. For example, for an oversized file, if the segment locks are instantiated to a size of 64k, the number of segment locks held by the file can be very large, resulting in excessive memory resources being consumed. The number of segment locks may be reduced by merging segment locks belonging to the same read-write type as the segment locks that are dumped to the first list, having the same lock identifier as the segment locks that are dumped to the first list, and segment locks belonging to adjacent data intervals as the segment locks that are dumped to the first list with the segment locks that are dumped to the first list.

It should be appreciated that if the proxy server also needs to use the partial data intervals of the segment locks obtained by merging to make a resource access request, the proxy server may split the segment locks obtained by merging in the manner provided in the above embodiment.

Fig. 4 is a schematic diagram illustrating a control method of a distributed storage system according to another exemplary embodiment. Assuming that the proxy server needs to access the data intervals a, F, the proxy server searches the first list, finds sub-data intervals a, B of a, F), removes a, B) from the first list, and uses a segment lock of a, B), as shown in fig. 4.

Regarding the remaining data section [ B, F), the proxy server searches the second list, if the second list has the segment lock of [ C, D) and cannot cover the data section of [ B, F), the proxy server returns [ a, B) to the first list and applies the segment lock of [ a, F) to the metadata management server, and then retries searching the segment lock from the first list and the second list.

Regarding the remaining data section [ B, F), the proxy server searches the second list, adds [ a, B) to the second list if the second list has the segment locks of [ B, H) capable of covering the data section of [ B, F), changes the usage status, and performs a resource access request based on the searched segment locks of [ a, B) and [ B, F).

In some implementations, the resource access request can include an overwrite operation (content modification without CHANGING FILE size). Accordingly, the proxy server may perform an overwriting operation based on the first segment lock set, and return the content modification time and/or the state modification time of the file corresponding to the first data interval to the metadata management server under the condition that the segment lock included in the first segment lock set is released, so that the metadata management server updates the file according to the content modification time and/or the state modification time.

Here, the proxy server executes the overwriting operation based on the first segment lock set, and after the overwriting operation is executed, if the metadata management server instructs the proxy server to release any segment lock included in the first segment lock set, the proxy server returns the content modification time and/or the state modification time of the file corresponding to the first data interval to the metadata management server. The metadata management server receives the returned content modification time and/or state modification time and updates the file according to the content modification time and/or state modification time.

Wherein, the content modification time (mtime) represents the time when the file content was modified last time. The state modification time (ctime) represents the time when the metadata of the file or directory was last changed.

When the content of the file is modified, the m/ctime will be updated to the current modification time. Since the update of m/ctime is bound with write lock of size, in the related art, the proxy server does not hold write lock of size when performing the overwriting operation, which results in that m/ctime cannot always be synchronized to the metadata management server.

In the embodiment of the disclosure, when the proxy server executes the overwriting operation, the proxy server holds the corresponding segment lock, and then modifies the content of the file and updates the m/ctime. When the segment lock is released, the proxy server synchronizes the new m/ctime to the metadata management server, so that the m/ctime can be updated in time in the overwriting scene.

FIG. 5 is a schematic diagram illustrating write-read conflicts in accordance with an example embodiment. As shown in fig. 5, the second proxy server holds a write segment lock of a certain data interval, and the first proxy server applies for a read segment lock of the interval, which may include the following steps.

S501, a first proxy server requests a metadata management server to read a segment lock;

s502, the metadata management server informs a second proxy server of releasing the write segment lock;

S503, the second proxy server releases the writing segmentation lock;

S504, the second proxy server returns mtime and ctime to the metadata management server;

s505, the metadata management server updates mtime and ctime;

s506, the metadata management server releases the write segment lock held by the second proxy server;

S507, the metadata management server distributes a read segment lock to the first proxy server;

S508, the metadata management server sends a read segment lock to the first proxy server;

s509, the first proxy server records the read segment lock.

FIG. 6 is a schematic diagram illustrating read-write conflicts in accordance with an example embodiment. As shown in fig. 6, the second proxy server holds a read segment lock of a certain data interval, and the first proxy server applies for a write segment lock of the interval, which may include the following steps.

S601, a first proxy server requests a metadata management server to write a segment lock;

S602, the metadata management server informs a second proxy server of releasing the read segment lock;

S603, the second proxy server releases the read segment lock;

S604, the second proxy server informs the metadata management server that the read segment lock is released;

s605, the metadata management server releases the read segment lock held by the second proxy server;

s606, the metadata management server distributes a write segment lock to the first proxy server;

S607, the metadata management server sends a write segment lock to the first proxy server;

S608, the first proxy server records the write segment lock.

FIG. 7 is a schematic diagram illustrating read-write conflicts in accordance with an example embodiment. As shown in fig. 7, the second proxy server holds a write segment lock of a certain data interval, and the first proxy server applies for the write segment lock of the interval, which may include the following steps.

S701, a first proxy server requests a metadata management server to write a segment lock;

s702, the metadata management server informs a second proxy server of releasing the write segment lock;

s703, the second proxy server releases the write segment lock;

S704, the second proxy server returns mtime and ctime to the metadata management server;

S705, the metadata management server updates mtime and ctime;

s706, the metadata management server releases the write segment lock held by the second proxy server;

s707, the metadata management server distributes a write segment lock to the first proxy server;

s708, the metadata management server sends a write segment lock to the first proxy server;

s709, the first proxy server records the write segment lock.

As shown in fig. 5 to 7, in the overwriting scenario, when the proxy server releases the writing segmentation lock, mtime and ctime are returned to the metadata management server, so that m/ctime can be updated in time in the overwriting scenario.

In some implementations, the proxy server may send a segment lock request to the metadata management server for applying for a segment lock of the target data interval, receive the segment lock of the target data interval allocated by the metadata management server, and store the segment lock of the target data interval in the first list or the second list of the proxy server.

Here, the segment lock request is used to cause the metadata management server to determine whether the segment lock of the target data section is allocated to another proxy server according to a third list or a fourth list that matches the segment lock type of the segment lock request, and allocate the segment lock of the target data section to the proxy server if the segment lock of the target data section is not allocated to the other proxy server, the third list is used to store the segment lock of the allocated read type, and the fourth list is used to store the segment lock of the allocated write type.

In the metadata management server, there are a third list for storing the assigned read-type segment locks and a fourth list for storing the assigned write-type segment locks to manage the assigned read-type segment locks and the write-type segment locks, respectively, so as to facilitate management of lock conflicts.

The proxy server sends a segment lock request to the metadata management server, and the metadata management server determines whether the segment lock of the target data interval is distributed to other proxy servers or not from the third list or the fourth list according to the segment lock type of the segment lock request so as to perform lock conflict check. For example, if the segmented lock requests a read type segmented lock, a lock conflict check is performed in the third list. If the segmented lock request applies for a write type segmented lock, a lock conflict check is performed in the fourth list.

In the case that there is no segment lock for the target data interval in the third list or the fourth list, it is determined that the segment lock for the target data interval is not allocated to the other proxy server, that is, there is no lock conflict. If there is a lock conflict, the metadata management server may assign the segmented lock of the target data interval to the proxy server after resolving the lock conflict.

It should be noted that each time the metadata management server assigns a segment lock to a proxy server, a new lock identifier may be generated and the segment lock recorded in the third list or the fourth list. Since each segment lock has a different lock identifier, the newly assigned segment lock may not be consolidated with the segment locks in the third or fourth list when inserted into the third or fourth list.

In the case that the segment lock of the target data section is not allocated to the other proxy server, the metadata management server allocates the segment lock of the target data section to the proxy server. The proxy server receives and records the segment locks of the target data intervals distributed by the metadata management server, and locks the segments of the target data intervals in the first list or the second list of the proxy server.

It should be appreciated that latching the segments of the target data interval in the first list or the second list of the storage proxy server may be based on the usage status of the segment locks, which may be stored in the corresponding first list or second list.

Thus, with the above embodiment, since the metadata management server manages the different types of segment locks through the third list and the fourth list, the proxy server can be made to quickly obtain the segment locks from the metadata management server.

Fig. 8 is a flowchart illustrating a control method of a distributed storage system according to still another exemplary embodiment. As shown in fig. 8, in some implementations that may be implemented, the following steps may also be included.

In step 810, a release request sent by the metadata management server is received, wherein the release request is used to instruct the proxy server to release the segment lock of the second data interval.

Here, the metadata management server may transmit a release request to the proxy server to instruct the proxy server to release the segment lock of the second data section in case the segment lock needs to be released. The proxy server receives the release request and performs an operation of releasing the segment lock of the second data interval based on the release request.

It is noted that the second data interval may be any data interval. For example, the second data section may be the first data section, the target data section, or the like of the above-described embodiment.

In step 820, a third set of segment locks is determined from the first list in response to the release request.

Here, the proxy server determines a third set of segment locks from the first list in response to the release request. Wherein the third set of segment locks comprises segment locks within the range of the second data interval.

It should be understood that a segment lock within the range of the second data interval may refer to a segment lock that just covers the second data interval. One or more segment locks may be included in the third set of segment locks, the data interval sum of the one or more segment locks being equal to the second data interval.

In step 830, the segment locks included in the third set of segment locks are deleted from the first list.

Here, since the segment locks included in the first list are in an unused state and no memory locks are added, the segment locks included in the third segment lock set may be directly deleted from the first list to release the segment locks of the second data interval.

Thus, the proxy server may quickly release the segment lock of the data interval indicated by the metadata management server through steps 810 to 830.

In some implementations, in step 830, it is determined that the third set of segment locks includes a third target segment lock, and in response to the third set of segment locks including the third target segment lock, the segment locks included in the third set of segment locks are deleted from the first list and the segment locks corresponding to the first sub-data interval are stored in the first list.

Here, the third target segment lock is a segment lock in which at least a portion of the data interval is within the range of the second data interval and another portion of the data interval exceeds the second data interval.

For example, the second data interval is [64k,128k ], and if there is a data interval of [0,192k ] in the first list, this indicates that there is a third target segment lock in the first list. Of course, if there is a data interval [64k,192k ] in the first list, this also indicates that there is a third target segment lock in the first list.

Correspondingly, the proxy server deletes the segment lock included in the third segment lock set from the first list, and stores the segment lock corresponding to the first sub-data interval in the first list. The first sub-data interval is a data interval exceeding the second data interval in the data intervals of the third target segment lock.

For example, the second data interval is [64k,128 k), if there is a data interval of [0,192 k) in the first list, it is necessary to split [0,192 k) into [0,64k ], [64k,128 k) and [128k,192 k), then delete the segment locks of [64k,128 k) from the first list, and add the segment locks of [0,64 k) and [128k,192 k) to the first list.

Thus, according to the above embodiment, the segment locks of the second data section held by the proxy server can be accurately released by deleting the segment locks included in the third segment lock set from the first list and storing the segment locks corresponding to the first sub-data section in the first list.

In some possible implementations, when the data interval corresponding to all the segment locks included in the third segment lock set cannot completely cover the second data interval, determining a second remaining data interval according to the data interval corresponding to the segment locks included in the third segment lock set and the second data interval, where the second remaining data interval is a data interval other than the data interval included in the third segment lock set in the second data interval, determining a fourth segment lock set from the second list according to the second remaining data interval, and deleting the segment locks included in the fourth segment lock set from the second list.

Here, the proxy server traverses the first list first, if the segment locks included in the third segment lock set traversed from the first list cannot completely cover the second data interval, the proxy server indicates that the segment locks of the second data interval cannot be completely released through the first list, and correspondingly, determines the second remaining data interval according to the data interval corresponding to the segment locks included in the third segment lock set and the second data interval.

For example, the second remaining data interval may be determined according to a difference between the data interval corresponding to the segment lock included in the third segment lock set and the second data interval.

The proxy server then determines a fourth set of segment locks from the second list based on the second remaining data interval. Wherein the fourth set of segment locks comprises segment locks that are within the range of the second remaining data interval.

It should be noted that, since the second list stores the segment locks in the use state, the data intervals of the segment locks are not detachable, so that the segment locks in the range of the second remaining data intervals may refer to the segment locks in the second list, where the corresponding data intervals can completely cover the second remaining data intervals. For example, the second remaining data interval is [128k,192 k), then the segment locks within the range of the second remaining data interval may be [128k,192 k) or [64k,192 k) or [0,192 k) or [128k,256 k).

Next, the proxy server deletes the segment locks included in the fourth set of segment locks from the second list.

In some embodiments, the proxy server may add a memory lock to each of the segment locks included in the fourth set of segment locks, adjust the state attribute of each of the segment locks included in the fourth set of segment locks to an invalid state, and delete the segment locks included in the fourth set of segment locks from the second list.

Here, the use state and data interval of the memory lock for controlling the segment lock may not be changed. In order to avoid that the segment locks held by the original resource access requests are invalid due to the release of the segment locks, in the process of deleting the fourth segment lock set from the second list, a memory lock may be added to each segment lock included in the fourth segment lock set, the state attribute of the segment lock included in the fourth segment lock set is adjusted to be an invalid state, and then the segment lock included in the fourth segment lock set is deleted from the second list.

Illustratively, the value of the valid of the segment lock included in the fourth set of segment locks may be set to false to adjust the state attribute of the segment lock to an invalid state.

Therefore, through the embodiment, the proxy server can release the segment lock through the first list and the second list, so that the release efficiency of the segment lock is improved.

In some implementations, determining that the fourth set of segment locks includes the fourth target segment lock, responsive to the fourth set of segment locks including the fourth target segment lock, deleting the segment locks included in the fourth set of segment locks from the second list, and storing the segment locks corresponding to the second sub-data interval in the first list.

Here, the fourth target segment lock is a segment lock in which at least a portion of the data interval is within the range of the second remaining data interval and another portion of the data interval exceeds the second remaining data interval. For example, the second remaining data interval is [128k,192 k), if the fourth set of segment locks has a segment lock of [64k,192 k), then it indicates that the fourth set of segment locks has a fourth target segment lock.

It should be appreciated that the fourth set of segment locks exists for the fourth target segment lock, indicating that the data interval of the segment lock released from the second list is greater than the second remaining data interval that needs to be released.

Correspondingly, the proxy server deletes the segment lock included in the fourth segment lock set from the second list, and stores the segment lock corresponding to the second sub-data interval in the first list. The second sub-data section is a data section exceeding the second remaining data section in the data section of the fourth target segment lock.

For example, the second remaining data interval is [128k,192 k) and the data interval of the fourth target segment lock is [64k,192 k), then the fourth target segment lock of [64k,192 k) needs to be deleted from the second list and the segment lock of [64k,128 k) is written to the first list.

It is noted that since the segment locks of [64k,128 k) are in an unused state, the segment locks of [64k,128 k) may be stored in the first list.

Thus, through the embodiment, the proxy server can precisely release the segment locks in the second list.

Fig. 9 is a schematic diagram illustrating a control method of a distributed storage system according to another exemplary embodiment. As shown in fig. 9, the proxy server releases the segment lock of the data section [ B, F), finds that [ a, C) exists in the sub-data section [ B, C) from the first list, deletes [ a, C) from the first list, and returns [ a, B). That is, [ A, C ] in the first list is changed to [ A, B). The proxy server then finds from the second list all segment locks [ C, D) and [ D, G ] for the data interval containing [ C, F). Then, add segment locks to [ C, D) and [ D, G), and set valid for [ C, D) and [ D, G) to false, and delete [ C, D) and [ D, G) from the second list, and return [ F, G) to the first list, releasing the memory lock. That is, the first list includes [ A, B) and [ F, G).

Fig. 10 is a flowchart illustrating a control method of a distributed storage system according to an exemplary embodiment. As shown in fig. 10, an embodiment of the present disclosure provides a control method of a distributed storage system, which is performed by a metadata management server shown in fig. 1, and may include the following steps.

In step 1010, a segment lock request is received from a proxy server in a distributed storage system, the segment lock request being used to request a metadata management server in the distributed storage system to allocate a segment lock for a target data interval to the proxy server.

Here, the proxy server may send a segment lock request to the metadata management server for requesting the metadata management server in the distributed storage system to allocate a segment lock of the target data interval to the proxy server. The metadata management server receives a segment lock request sent by the proxy server and assigns a segment lock to the proxy server based on the segment lock request.

In step 1020, a segment lock for the target data interval is assigned to the proxy server in response to the segment lock request.

Here, the metadata management server allocates a segment lock of the target data section to the proxy server in response to the segment lock request transmitted by the proxy server.

The method comprises the steps that a proxy server responds to a resource access request initiated by a client and used for indicating to execute a resource access operation on data of a first data interval, a first segmented lock set matched with the first data interval is determined based on a first list, the resource access request is executed based on the first segmented lock set, and segmented locks included in the first segmented lock set are stored in a second list.

The first set of segment locks includes segment locks within a first data interval, a first list for storing segment locks held by the proxy server in an unused state, and a second list for storing segment locks held by the proxy server in an used state.

It should be noted that, regarding how the proxy server performs the resource access request based on the segment lock allocated by the metadata management server, reference may be made to the related description on the proxy server side, which is not repeated herein.

Therefore, the segmented lock is distributed to the proxy server, and the proxy server performs resource access request through the segmented lock, so that not only can the accuracy of read data be ensured, but also the consistency of the data read and written concurrently can be ensured under the scenario of read and write concurrently. In addition, the proxy server manages the segment locks through the first list and the second list, so that the segment locks can be conveniently searched and used, the segment locks held by the proxy server are orderly managed, and the use efficiency of the segment locks is improved.

In some implementations, in step 1020, the metadata management server may determine whether the segment lock of the target data interval is assigned to the other proxy server according to the third list or the fourth list matching the segment lock type of the segment lock request in response to the segment lock request, and in case the segment lock of the target data interval is not assigned to the other proxy server, assign the segment lock of the target data interval to the proxy server, and store the segment lock in the corresponding third list or fourth list.

Here, in the metadata management server, there are a third list for storing the assigned read-type segment locks and a fourth list for storing the assigned write-type segment locks to manage the assigned read-type segment locks and the write-type segment locks, respectively, so that the lock conflict can be managed conveniently.

Fig. 11 is a schematic diagram of a third list and a fourth list shown according to an example embodiment. As shown in FIG. 11, the third list is for storing assigned READ (READ) type segment locks, and the fourth list is for storing assigned WRITE (WRITE) type segment locks. Wherein in each segment lock, the proxy server holding the segment lock may also be marked. For example, proxy:1 denotes a proxy server holding a read fragment lock of [0,192 k) as 1.

It should be noted that, in the third list and the fourth list, since the segment locks are compatible for reading and reading the same segment, there may be a case that the data segments overlap in the third list of the metadata management server. Because the writing mutual exclusion and the reading mutual exclusion exist, the fourth list does not have data interval overlapping and does not have intersection of the data interval with the third list.

In some implementations, in step 1020, the metadata management server may determine a data interval to be allocated from the target data interval in response to the segment lock request, and allocate the segment lock of the data interval to be allocated to the proxy server.

Here, the starting point of the data interval to be allocated is consistent with the starting point of the target data interval, and the end point of the data interval to be allocated is greater than the end point of the target data interval.

That is, the data interval of the segment lock actually allocated to the proxy server by the metadata management server is larger than the target data interval applied to the metadata management server by the proxy server. When the metadata management server allocates the segment lock, the metadata management server allocates the data interval as large as possible to the proxy server from the starting point of the target data interval. For example, the target data interval is [64k,128 k), and the metadata management server may assign a segment lock for the data interval of [64k,192 k) to the proxy server.

It should be appreciated that the metadata management server actually assigns segment locks to proxy servers in an optimistic lock policy. Under the optimistic lock strategy, when the first application of the segment lock is performed under the continuous additional writing scene, the metadata management server actively distributes a section of interval backwards, and the subsequent additional writing of the proxy server does not need to apply the segment lock to the metadata management server again, so that the network overhead is reduced.

Therefore, through the embodiment, the metadata management server distributes the segment locks of the data intervals to the proxy server as large as possible, so that the proxy server can be prevented from repeatedly applying the segment locks to the metadata management server, and network overhead is reduced.

In some implementations, a fifth target segment lock is determined in a third or fourth list that matches the segment lock type, the proxy server is notified to release the fifth target segment lock, and after the fifth target segment lock is successfully released, the proxy server is assigned a segment lock for the target data interval and the segment lock is stored in the corresponding third or fourth list.

Here, the fifth target segment lock is a segment lock that has been assigned to the proxy server and that is responsible for at least partially overlapping the target data interval.

The fifth target segment lock may be understood as a segment lock of a data interval that has been allocated to the proxy server in the third list or the fourth list and that partially coincides with the target data interval that the proxy server re-applies for. For example, the first proxy server applies for a read segment lock for the target data interval of [0,128 k), if there is a read segment lock of [0,64 k) in the third list and the read segment lock has been assigned to the first proxy server, it indicates that there is a fifth target segment lock in the third list.

Since the fifth target segment lock is already assigned to the proxy server and there is at least a partial overlap of the target data interval re-applied by the proxy server, the metadata management server may release the fifth target segment lock to reclaim the fifth target segment lock from the proxy server in order to prevent over-fragmentation of the data interval of the segment lock. After the fifth target segment lock is successfully released, the metadata management server allocates the segment lock of the target data interval to the proxy server and stores the segment lock in the corresponding third list or fourth list.

For example, the metadata management server releases the segment locks of [0,64 k) back before reassigning the segment locks of [0,128 k) to the proxy server.

Illustratively, the metadata management server releases the fifth target segment lock, which may be that the metadata management server sends a release request to the proxy server, so that the proxy server releases the corresponding segment lock in response to the release request, and returns response information to the metadata management server. And the metadata management server responds to the response information, deletes the corresponding segment lock and completes release of the segment lock.

Therefore, according to the embodiment, one large data interval can be prevented from being split into a plurality of fragmented data intervals, so that the efficiency of adding, deleting and searching the segmented locks of the third list and the fourth list can be improved, and excessive occupied consumption of the memory can be avoided.

In some implementations, in a case where the metadata management server needs to release the segment locks of the second data interval, a release request is sent to a proxy server corresponding to the segment locks of the second data interval, where the release request is used to enable the proxy server to determine a third set of segment locks from the first list in response to the release request, and delete the segment locks included in the third set of segment locks from the first list, where the third set of segment locks includes segment locks within the range of the second data interval.

Here, the second data interval may be an arbitrary data interval. For example, the second data interval may be the first data interval, the target data interval, the data interval corresponding to the fifth target segment lock, and so on in the above embodiments.

The metadata management server sends a release request to a proxy server holding the segment lock for the second data interval. The proxy server determines a third set of segment locks from the first list in response to the release request, and deletes the segment locks included in the third set of segment locks from the first list to complete release of the segment locks.

It should be appreciated that after completing releasing the segment locks, the proxy server may return a reply message to the metadata management server to inform the metadata management server that the segment locks have been successfully released, and the metadata management server deletes the segment locks in the metadata management server in response to the reply message, completing reclaiming the segment locks.

It should be noted that, for a detailed description of how the proxy server determines the third set of segment locks from the first list and the third set of segment locks, reference may be made to a related description on the proxy server side, which is not repeated herein.

Thus, with the above embodiments, the metadata management server can quickly retrieve the segment locks from the proxy server.

In some possible embodiments, when the metadata management server needs to release the segment lock of the second data interval, determining a sixth target segment lock in a third list or a fourth list corresponding to the segment lock of the second data interval, where the sixth target segment lock is a segment lock at least some of which exceeds the second data interval;

and splitting the data interval of the sixth target segment lock by taking the second data interval as a reference to obtain a plurality of sub-segment locks, deleting the first sub-segment lock from the third list or the fourth list, and sending a release request to a proxy server corresponding to the first sub-segment lock.

Here, the sixth target segment lock is a segment lock in which at least a portion of the data interval exceeds the second data interval. A segment lock with at least a portion of the data interval exceeding the second data interval may mean that the data interval of the segment lock is capable of fully covering the second data interval, but that the left and/or right end points in the data interval of the segment lock exceed the second data interval.

For example, if the second data interval is [64k,128 k), if there is a sixth target segment lock of data interval [0,192 k) in the third list or the fourth list, the data interval of the sixth target segment lock, which indicates that the third list or the fourth list records, that has been allocated to the proxy server is larger than the data interval that needs to be released, and since only a part of the data interval in the sixth target segment lock needs to be released, the sixth target segment lock of [0,192 k) needs to be split into a plurality of sub-segment locks.

The plurality of sub-segment locks comprise a first sub-segment lock corresponding to the second data interval and a second sub-segment lock of other data intervals except the second data interval.

For example, [0,192 k) may be split into [0,64 k), [64k,128 k), and [128k,192 k), based on the end point of the second data interval.

The metadata management server then deletes the first sub-segment lock of the second data interval from the third or fourth list, and the other sub-segment locks remain in the third or fourth list. For example, [0,64 k) and [128k,192 k) remain in the third list or the fourth list, and [64k,128 k) are deleted from the third list or the fourth list.

And the metadata management server sends a release request to the proxy server corresponding to the first sub-segment lock, so that the proxy server releases the first sub-segment lock.

Therefore, through the embodiment, the metadata management server can accurately recycle the segment locks of the second data interval, and the segment locks of other data intervals are prevented from being influenced.

In some implementations, in a case that the proxy server corresponding to the first sub-segment lock does not successfully release the first sub-segment lock, the first sub-segment lock is stored in a corresponding third list or fourth list, and the first sub-segment lock and the corresponding second sub-segment lock are combined into a sixth target segment lock.

Here, if the proxy server corresponding to the first sub-segment lock does not release the first sub-segment lock successfully, it indicates that the first sub-segment lock is also allocated for external use. Accordingly, the metadata management server cannot directly discard the first sub-segment lock.

Thus, the metadata management server returns the first sub-segment lock to the corresponding third or fourth list and merges the first sub-segment lock with the corresponding second sub-segment lock into a sixth target segment lock.

It should be noted that, since the first sub-segment lock and the corresponding second sub-segment lock are a plurality of sub-segment locks split from the same sixth target segment lock, which have the same lock identifier, segment lock type, and are located in adjacent data intervals, the first sub-segment lock and the second sub-segment lock may be combined into one segment lock.

For example, the metadata management server returns [64k,128 k) to the third or fourth list, [64k,128 k) may perform data interval merging with [0,64 k) and [128k,192 k) in the third or fourth list, merging [64k,128 k), [0,64 k), and [128k,192 k) into [0,192 k).

Therefore, through the implementation mode, fragmentation of the data interval can be avoided, and therefore memory consumption is reduced.

Fig. 12 is a schematic diagram showing the module connection of a control device of a distributed storage system according to an exemplary embodiment. As shown in fig. 12, an embodiment of the present disclosure provides a control apparatus 1200 of a distributed storage system, where the control apparatus 1200 of the distributed storage system is disposed on a proxy server, the control apparatus 1200 of the distributed storage system may include:

A request receiving module 1201 configured to receive a resource access request initiated by a client, the resource access request being used to instruct to perform a resource access operation on data of a first data interval;

A first determining module 1202 configured to determine, in response to the resource access request, a first set of segment locks matching the first data interval based on a first list, the first set of segment locks including segment locks within the first data interval, the first list being for storing segment locks held by proxy servers in a distributed storage system in an unused state;

An executing module 1203, configured to execute the resource access request based on the first set of segment locks, and store segment locks included in the first set of segment locks to a second list, where the second list is used to store segment locks in use held by the proxy server, and the segment locks included in the first list and the second list are distributed to the proxy server by a metadata management server in the distributed storage system in response to a segment lock request sent by the proxy server, where the segment lock request is used to request the metadata management server to distribute segment locks of a target data interval to the proxy server.

Optionally, the first determining module 1202 is specifically configured to:

Searching candidate segment locks of which the responsible data areas contain the first data interval from the data intervals responsible for the segment locks recorded in the first list;

Determining whether a data area for which the candidate segment lock is responsible exceeds the first data interval;

when the data interval in charge of the candidate segment lock does not exceed the first data interval, the candidate segment lock is used as a first target segment lock;

When the data interval in charge of the candidate segment lock exceeds the first data interval, splitting the data interval in charge of the candidate segment lock by taking the first data interval as a reference to obtain a target sub-segment lock in charge of the first data interval;

the first set of segment locks is determined based on the first target segment lock and the target sub-segment lock.

Optionally, the control device 1200 of the distributed storage system further includes:

a second determining module configured to determine a data interval of the first data interval that the first set of segment locks fails to cover as a first remaining data interval when the first set of segment locks fails to cover the first data interval;

A third determining module configured to find a second set of segment locks capable of covering the first remaining data interval from the second list, the second set of segment locks comprising segment locks capable of covering the first remaining data interval;

The execution module 1203 is specifically configured to:

And executing the resource access request based on the first segmented lock set and the second segmented lock set, and storing each segmented lock in the first segmented lock set to the second list.

And the sending module is configured to send a segment lock request to the metadata management server, wherein the segment lock request is used for requesting the metadata management server to distribute the segment lock for the first data interval to the proxy server when the segment lock held by the proxy server cannot fully cover the first data interval.

Optionally, the execution module 1203 is specifically configured to:

Adding a memory lock to each segment lock included in the first segment lock set, wherein the memory lock is used for controlling the use state and the data interval of the segment lock to be unchangeable;

And determining the validity of each segment lock included in the first segment lock set, and executing the resource access request based on the segment locks included in the first segment lock set under the condition that each segment lock has validity.

the transfer module is configured to transfer each segment lock in the first segment lock set to the first list after completing execution of the resource access request if the segment locks included in the first segment lock set are not referenced by other operations;

the first merging module is configured to search the first list for a second target segment lock which has the same read-write type, the same lock identifier and belongs to an adjacent data interval as the segment lock transferred into the first list;

And segment locking and combining the first segment lock set and the second target segment lock in the first list.

Optionally, the resource access request includes an overwriting operation, and the execution module 1203 is specifically configured to:

performing the overwriting operation on the data of the first data interval based on the first set of segment locks;

And under the condition of releasing the segment locks included in the first segment lock set, returning the content modification time and/or the state modification time of the file corresponding to the first data interval to the metadata management server so that the metadata management server updates the file according to the content modification time and/or the state modification time.

An application module configured to send a segment lock request for applying for a segment lock of a target data interval to a metadata management server, where the segment lock request is used to enable the metadata management server to determine whether the segment lock of the target data interval is allocated to another proxy server according to a third list or a fourth list that is matched with a segment lock type of the segment lock request, and allocate the segment lock of the target data interval to the proxy server if the segment lock of the target data interval is not allocated to the other proxy server, where the third list is used to store the segment lock of the allocated read type, and the fourth list is used to store the segment lock of the allocated write type;

And the first receiving module is configured to receive the segment lock of the target data interval distributed by the metadata management server and store the segment lock of the target data interval in the first list or the second list of the proxy server.

a second receiving module configured to receive a release request sent by the metadata management server, where the release request is used to instruct the proxy server to release the segment lock of the second data interval;

A fourth determination module configured to determine, in response to the release request, a third set of segment locks from the first list, the third set of segment locks including segment locks that are within the range of the second data interval;

A first deletion module configured to delete segment locks included in the third set of segment locks from the first list.

Optionally, the first deletion module is specifically configured to:

Determining that the third set of segment locks includes a third target segment lock, the third target segment lock being a segment lock having at least a portion of a data interval within the range of the second data interval and another portion of the data interval exceeding the second data interval;

And deleting the segment locks included in the third segment lock set from the first list in response to the third segment lock set including the third target segment lock, and storing the segment locks corresponding to a first sub-data interval in the first list, wherein the first sub-data interval is a data interval exceeding the second data interval in the data intervals of the third target segment lock.

A fifth determining module, configured to determine, according to the data interval corresponding to the segment lock included in the third segment lock set and the second data interval, a second remaining data interval, where the second remaining data interval is a data interval other than the data interval included in the third segment lock set in the second data interval, where the data intervals corresponding to all segment locks included in the third segment lock set cannot completely cover the second data interval;

a sixth determining module configured to determine a fourth set of segment locks from the second list according to the second remaining data interval, the fourth set of segment locks including segment locks that are within the range of the second remaining data interval;

and a second deleting module configured to delete segment locks included in the fourth set of segment locks from the second list.

Optionally, the second deletion module is specifically configured to:

Adding a memory lock to each segment lock included in the fourth segment lock set, wherein the memory lock is used for controlling the use state and the data interval of the segment lock to be unchangeable;

And adjusting the state attribute of each segment lock included in the fourth segment lock set to be an invalid state, and deleting the segment lock included in the fourth segment lock set from the second list.

Optionally, the second deletion module is specifically configured to:

Determining that the fourth set of segment locks includes a fourth target segment lock, the fourth target segment lock being a segment lock having at least a portion of a data interval within the range of the second remaining data interval and another portion of the data interval exceeding the second remaining data interval;

And deleting the segment locks included in the fourth segment lock set from the second list in response to the fourth segment lock set including the fourth target segment lock, and storing the segment locks corresponding to a second sub-data interval in the first list, wherein the second sub-data interval is a data interval exceeding the second remaining data interval in the data intervals of the fourth target segment lock.

The logic of the method executed by each functional module in the control device 1200 of the distributed storage system may refer to the part of the method related to the foregoing embodiment, which is not described herein again.

Fig. 13 is a schematic block diagram illustrating a control apparatus of a distributed storage system according to still another exemplary embodiment. As shown in fig. 13, an embodiment of the present disclosure provides a control apparatus 1300 of a distributed storage system, where the control apparatus 1300 of the distributed storage system is disposed on a metadata management server, the control apparatus 1300 of the distributed storage system may include:

A third receiving module 1301, configured to receive a segment lock request sent by a proxy server in a distributed storage system, where the segment lock request is used to request a metadata management server in the distributed storage system to allocate a segment lock of a target data interval to the proxy server;

an allocation module 1302 configured to allocate a segment lock of the target data interval to the proxy server in response to the segment lock request;

Optionally, the allocation module 1302 is specifically configured to:

Determining whether the segmented locks of the target data interval are allocated to other proxy servers according to a third list or a fourth list matched with the segmented lock types corresponding to the segmented lock requests, wherein the third list is used for storing the segmented locks of the allocated read types, and the fourth list is used for storing the segmented locks of the allocated write types;

And if the segment lock of the target data interval is not allocated to other proxy servers, the segment lock of the target data interval is allocated to the proxy servers, and the segment lock is stored in the corresponding third list or fourth list.

Optionally, the allocation module 1302 is specifically configured to:

Responding to the segmented lock request, and determining a data interval to be allocated according to the target data interval, wherein the starting point of the data interval to be allocated is consistent with the starting point of the target data interval, and the end point of the data interval to be allocated is larger than the end point of the target data interval;

And distributing the segment lock of the data interval to be distributed to the proxy server.

Optionally, the allocation module 1302 is specifically configured to:

determining a fifth target segment lock in a third list or a fourth list matched with the segment lock type, wherein the fifth target segment lock is a segment lock which is allocated to the proxy server and is at least partially overlapped with the target data interval;

notifying the proxy server to release the fifth target segment lock, and after the fifth target segment lock is successfully released, distributing the segment lock of the target data interval to the proxy server, and storing the segment lock in the corresponding third list or fourth list.

Optionally, the control apparatus 1300 of the distributed storage system further includes:

And the release module is configured to send a release request to a proxy server corresponding to the segment lock of the second data interval when the metadata management server needs to release the segment lock of the second data interval, wherein the release request is used for enabling the proxy server to respond to the release request, determine a third segment lock set from the first list, and delete the segment lock included in the third segment lock set from the first list, and the third segment lock set includes the segment locks in the range of the second data interval.

Optionally, the release module is specifically configured to:

Under the condition that the metadata management server needs to release the segment lock of the second data interval, determining a sixth target segment lock in the third list or the fourth list corresponding to the segment lock of the second data interval, wherein the sixth target segment lock is a segment lock of which at least part of the data interval exceeds the second data interval;

Splitting the data interval of the sixth target segment lock by taking the second data interval as a reference to obtain a plurality of sub-segment locks, wherein the plurality of sub-segment locks comprise a first sub-segment lock corresponding to the second data interval and a second sub-segment lock of other data intervals except the second data interval;

deleting the first sub-segment lock from the third list or the fourth list, and sending a release request to a proxy server corresponding to the first sub-segment lock, wherein the release request is used for indicating the proxy server to release the first sub-segment lock.

Optionally, the release module is further configured to:

And under the condition that the proxy server corresponding to the first sub-segment lock does not successfully release the first sub-segment lock, storing the first sub-segment lock into the corresponding third list or fourth list, and combining the first sub-segment lock and the corresponding second sub-segment lock into the sixth target segment lock.

The logic of the method executed by each functional module in the control device 1300 of the distributed storage system may refer to the part of the method related to the foregoing embodiment, which is not described herein again.

Referring now to fig. 14, a schematic diagram of an architecture of an electronic device (e.g., a proxy server or metadata management server in fig. 1) 1400 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 14 is merely an example, and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.

As shown in fig. 14, the electronic device 1400 may include a processing apparatus (e.g., a central processor, a graphics processor, etc.) 1401, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1402 or a program loaded from a storage apparatus 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the electronic device 1400 are also stored. The processing device 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input/output (I/O) interface 1405 is also connected to the bus 1404.

In general, the following devices may be connected to the I/O interface 1405: input devices 1406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 1407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 1408 including, for example, magnetic tape, hard disk, etc.; and communication means 1409. The communication means 1409 may allow the electronic device 1400 to communicate wirelessly or by wire with other devices to exchange data. While fig. 14 shows an electronic device 1400 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 1409, or installed from the storage means 1408, or installed from the ROM 1402. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 1401.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some embodiments, the client, proxy server, metadata management server, file storage server may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a resource access request initiated by a client, wherein the resource access request is used for indicating to execute a resource access operation aiming at data of a first data interval; determining a first set of segment locks matched with the first data interval based on a first list in response to the resource access request, wherein the first set of segment locks comprises segment locks in the range of the first data interval, and the first list is used for storing segment locks in unused states held by proxy servers in a distributed storage system; executing the resource access request based on the first segment lock set, and storing the segment locks included in the first segment lock set to a second list, wherein the second list is used for storing the segment locks in a use state held by the proxy server, the segment locks included in the first list and the second list are distributed to the proxy server by a metadata management server in the distributed storage system in response to the segment lock request sent by the proxy server, and the segment lock request is used for requesting the metadata management server to distribute the segment locks of a target data interval to the proxy server.

Or the computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a segment lock request sent by a proxy server in a distributed storage system, wherein the segment lock request is used for requesting a metadata management server in the distributed storage system to distribute a segment lock of a target data interval to the proxy server; responding to the segment lock request, and distributing the segment lock of the target data interval to the proxy server; the method comprises the steps that a segmented lock of a target data interval is used for enabling a proxy server to respond to a resource access request initiated by a client and used for indicating to execute a resource access operation on data of a first data interval, a first segmented lock set matched with the first data interval is determined based on a first list, the resource access request is executed based on the first segmented lock set, and segmented locks included in the first segmented lock set are stored in a second list; the first set of segment locks includes segment locks within the first data interval, the first list is used for storing segment locks in an unused state held by the proxy server, and the second list is used for storing segment locks in a used state held by the proxy server.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present disclosure may be implemented in software or hardware. The name of a module does not in some cases define the module itself.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).

Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims. The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Claims

1. A method of controlling a distributed storage system, the method comprising:

2. The method of claim 1, wherein the determining a first set of segment locks that match the first data interval based on the first list comprises:

3. The method according to claim 1, wherein the method further comprises:

Determining a data interval in the first data interval which is not covered by the first segment lock set as a first residual data interval when the first segment lock set cannot completely cover the first data interval;

searching a second segmented lock set capable of covering the first residual data interval from the second list, wherein the second segmented lock set comprises segmented locks capable of covering the first residual data interval;

The executing the resource access request based on the first set of segment locks, and storing the segment locks included in the first set of segment locks to a second list, includes:

4. The method according to claim 1, wherein the method further comprises:

And if the segment lock held by the proxy server is not capable of fully covering the first data interval, sending a segment lock request to the metadata management server, wherein the segment lock request is used for requesting the metadata management server to distribute the segment lock for the first data interval to the proxy server.

5. The method of claim 1, wherein the executing the resource access request based on the first set of segment locks comprises:

6. The method according to claim 1, wherein the method further comprises:

After the execution of the resource access request is completed, if the segment locks included in the first segment lock set are not referenced by other operations, each segment lock in the first segment lock set is saved to the first list;

Searching a second target segment lock which has the same read-write type, the same lock identifier and belongs to an adjacent data interval from the first list and the segment lock transferred to the first list;

7. The method of claim 1, wherein the resource access operation comprises an overwrite operation, the executing the resource access request based on the first set of segment locks comprising:

8. The method according to any of claims 1 to 7, wherein the segment lock held by the proxy server is obtained by:

Sending a segment lock request for applying for a segment lock of a target data interval to a metadata management server, wherein the segment lock request is used for enabling the metadata management server to determine whether the segment lock of the target data interval is allocated to other proxy servers according to a third list or a fourth list matched with the segment lock type of the segment lock request, and if the segment lock of the target data interval is not allocated to other proxy servers, the segment lock of the target data interval is allocated to the proxy servers, the third list is used for storing the segment lock of the allocated read type, and the fourth list is used for storing the segment lock of the allocated write type;

and receiving the segment lock of the target data interval distributed by the metadata management server, and storing the segment lock of the target data interval in the first list or the second list of the proxy server.

9. The method according to any one of claims 1 to 7, further comprising:

receiving a release request sent by the metadata management server, wherein the release request is used for indicating the proxy server to release the segment lock of the second data interval;

in response to the release request, determining a third set of segment locks from the first list, the third set of segment locks including segment locks that are within the range of the second data interval;

And deleting the segment locks included in the third segment lock set from the first list.

10. The method of claim 9, wherein the deleting the segment locks included in the third set of segment locks from the first list comprises:

11. The method according to claim 9, wherein the method further comprises:

determining a second remaining data interval according to the data interval corresponding to the segment locks included in the third segment lock set and the second data interval when the data interval corresponding to all the segment locks included in the third segment lock set cannot completely cover the second data interval, wherein the second remaining data interval is a data interval except the data interval included in the third segment lock set in the second data interval;

Determining a fourth set of segment locks from the second list according to the second remaining data interval, the fourth set of segment locks including segment locks within the range of the second remaining data interval;

And deleting the segment locks included in the fourth segment lock set from the second list.

12. The method of claim 11, wherein said deleting the segment locks included in the fourth set of segment locks from the second list comprises:

13. The method of claim 11, wherein said deleting the segment locks included in the fourth set of segment locks from the second list comprises:

14. A method of controlling a distributed storage system, the method comprising:

15. The method of claim 14, wherein said assigning the segment lock of the target data interval to the proxy server in response to the segment lock request comprises:

16. The method of claim 14, wherein said assigning the segment lock of the target data interval to the proxy server in response to the segment lock request comprises:

17. The method of claim 15, wherein the assigning the segment lock of the target data interval to the proxy server and storing the segment lock in the corresponding third list or fourth list comprises:

18. The method of claim 15, wherein the method further comprises:

And under the condition that the metadata management server needs to release the segment locks of the second data interval, sending a release request to a proxy server corresponding to the segment locks of the second data interval, wherein the release request is used for enabling the proxy server to respond to the release request, determining a third segment lock set from the first list, deleting the segment locks included in the third segment lock set from the first list, and the third segment lock set comprises the segment locks in the range of the second data interval.

19. The method according to claim 18, wherein, in the case that the metadata management server needs to release the segment lock of the second data interval, sending a release request to the proxy server corresponding to the segment lock of the second data interval, includes:

20. The method of claim 19, wherein the method further comprises:

21. A computer readable medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processing device, implements the steps of the method of any one of claims 1 to 13 or the steps of the method of any one of claims 14 to 20.

22. An electronic device, comprising:

a storage device having a computer program stored thereon;

Processing means for executing the computer program in the storage means to carry out the steps of the method of any one of claims 1 to 13 or to carry out the steps of the method of any one of claims 14 to 20.

23. A computer program product comprising a computer program, characterized in that the computer program when executed by a processor realizes the steps of the method of any one of claims 1 to 13 or the steps of the method of any one of claims 14 to 20.