CN114153390A - Volume deletion optimization method and device in distributed storage system and storage medium - Google Patents
Volume deletion optimization method and device in distributed storage system and storage medium Download PDFInfo
- Publication number
- CN114153390A CN114153390A CN202111399481.4A CN202111399481A CN114153390A CN 114153390 A CN114153390 A CN 114153390A CN 202111399481 A CN202111399481 A CN 202111399481A CN 114153390 A CN114153390 A CN 114153390A
- Authority
- CN
- China
- Prior art keywords
- volume
- deletion
- node
- target
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012217 deletion Methods 0.000 title claims abstract description 140
- 230000037430 deletion Effects 0.000 title claims abstract description 140
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000005457 optimization Methods 0.000 title description 7
- 238000012163 sequencing technique Methods 0.000 claims abstract description 21
- 230000008569 process Effects 0.000 claims description 20
- 238000010276 construction Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 description 6
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method and a device for optimizing volume deletion in a distributed storage system and a storage medium. In the invention, the cluster main node responds to the deletion request of the target volume, deletes the metadata of the target volume and the volume record in the storage pool, records the volume information of the target volume and feeds back the success of the deletion of the target volume; the volume information includes volume name, volume ID, deletion time, priority and data object information; constructing a deletion task by using the volume information, putting the deletion task into a volume deletion queue to be processed, and sequencing according to a preset sequencing rule; when the cluster service pressure is lower than a set pressure threshold value, deleting task load conditions according to each node of a cluster, and acquiring a target node for executing a task; and selecting the deletion tasks from the volume deletion pending queue in sequence, and distributing the tasks to the target node for execution. According to the method and the device, the metadata deletion and the data object deletion of the target volume are separated, the deletion speed is increased in sense, the data object is deleted according to the cluster service pressure, and the influence on the cluster service execution is reduced.
Description
Technical Field
The present invention relates to the field of volume deletion, and in particular, to a method and an apparatus for optimizing volume deletion in a distributed storage system, and a storage medium.
Background
The volume deletion function is a basic function of block storage, in a traditional distributed block storage system, volume deletion operations include metadata deletion and data object deletion of a volume, and the volume deletion needs to be completed after the two parts of data are completely deleted.
In the process of deleting the data object of the volume, the data volume of the data object is huge, the deleting operation of the data object can seriously affect the IO of the system, the integral service execution of the cluster is greatly affected, and the user service can be affected. However, if the influence on the cluster service is controlled by slowing down the speed of deleting the data object, the time for deleting the volume is increased, so that not only the user feels that the time for deleting the volume is too long, but also the user operation may be blocked or enter other abnormal scenes due to overtime.
Disclosure of Invention
In order to solve the above technical problem or at least partially solve the above technical problem, the present invention provides a method, an apparatus, and a storage medium for optimizing volume deletion in a distributed storage system.
In a first aspect, the present invention provides a method for optimizing volume deletion in a distributed storage system, including:
the cluster main node responds to the deletion request of the target volume, deletes the metadata of the target volume and the volume record of the target volume in the storage pool, records the volume information of the target volume and feeds back the success of the deletion of the target volume; the volume information comprises volume name, volume ID, deletion time, priority and data object information;
constructing a deletion task by using the volume information, putting the deletion task into a volume deletion queue to be processed, and sequencing according to a preset sequencing rule;
when the cluster service pressure is lower than a set pressure threshold value, deleting task load conditions according to each node of a cluster, and acquiring a target node for executing a task;
and selecting deletion tasks from the volume deletion pending queue in sequence, and distributing the tasks to the target node for execution so as to delete the data object of the target volume.
Further, the cluster master node responds to the deletion request of the target volume, firstly judges whether the target volume is a clone volume, if so, firstly disconnects the association between the target volume and the parent snapshot of the target volume, and then deletes the metadata of the target volume and the volume record of the target volume in the storage pool, otherwise, directly deletes the metadata of the volume and the volume record of the target volume in the storage pool.
Further, the priority in the volume information is any one of a boolean type and a numeric type;
wherein: for Boolean type priority, Boolean true indicates that priority is enabled, Boolean false indicates that priority is not enabled, default settings are false, and a user can actively configure Boolean type priority of a target volume to true;
for the numerical value type priority, the priority is determined by the value size and is defaulted to be a set value, a user can actively configure a designated value for the numerical value type priority of the target volume, and the designated value is in the value range of the preset numerical value type priority.
Further, the deleting task is placed into a queue to be processed for volume deletion and is sorted according to a preset sorting rule, wherein the preset sorting rule comprises:
sorting the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion according to the priority of the volume information;
if the priorities are consistent, sorting the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion according to the deletion time of the volume information, wherein the earlier the deletion time is, the earlier the position sorting is;
and if the deleting time is consistent, sequencing the positions of the deleting tasks created by the volume information in the queue to be processed for volume deletion according to the letter sequencing of the volume names of the volume information, wherein the position sequencing is more advanced the letter sequencing of the volume names is.
Further, when the cluster service pressure is lower than the set pressure threshold, acquiring a target node for executing a task according to a task load condition deleted by each node of the cluster includes:
the cluster main node monitors and collects the condition of cluster service pressure through a background;
comparing whether the cluster service pressure is smaller than a set pressure threshold value, if not, not distributing a new task to be deleted, and if so, acquiring the load of the current task to be deleted of each node of the cluster;
sorting the nodes according to the deleted task load from low to high, and sorting the nodes according to the size of the node serial number if the deleted task loads are the same;
and selecting the node with the top ranking as a target node.
Furthermore, the cluster master node monitors whether a node executing the delete task in the cluster fails, and when the node executing the delete task fails, the cluster master node reacquires the target node and distributes the delete task executed by the failed node to the reacquired target node.
Furthermore, the nodes in the cluster periodically and asynchronously send applications to become master node applications; if the master node is normal, the application for becoming the master node of any node is fed back and rejected; if the main nodes are in failure, the first node becomes the main node after the main nodes are in failure and applies for feedback approval, the corresponding node becomes the main node, and the feedback refusal is applied for the rest nodes.
Furthermore, the execution process of the deleting task of the target volume is monitored, a corresponding volume state is configured for the target volume according to the execution process of the deleting task of the target volume, and a volume state query interface is provided for a user, wherein the volume state comprises to-be-processed state, allocated state, deleting state and completed state.
In a second aspect, the present invention provides a device for optimizing volume deletion in a distributed storage system, including:
a first deletion module that deletes metadata of the target volume and volume records in the storage pool in response to a deletion request of the target volume;
a volume information recording module that records volume information of a target volume;
the feedback module feeds back the successful deletion of the target volume to the user after deleting the metadata of the target volume and the volume record in the storage pool;
the task construction module is used for creating a corresponding deletion task according to the volume information;
the queue module stores the deleted tasks and sorts the deleted tasks according to a preset sorting rule;
the task allocation module selects a target node for executing the deleted tasks when the cluster service pressure is lower than a pressure threshold value, and sequentially selects the deleted tasks from the queue module to be allocated to the target node and records the deleted tasks;
and the target node executes the deleting task through the second deleting module.
In a third aspect, the present invention provides a storage medium for implementing a method for optimizing deletion of a volume in a distributed storage system. The storage medium for realizing the optimization method for deleting the volume in the distributed storage system stores at least one instruction, reads and executes the instruction to realize the optimization method for deleting the volume in the distributed storage system.
Compared with the prior art, the technical scheme provided by the embodiment of the invention has the following advantages:
the method and the device have the advantages that the metadata of the target volume to be deleted is deleted and the data object is deleted in a separated mode, the successful deletion is fed back to a user after the metadata of the target volume and the volume records in the storage pool are deleted, and the deletion speed is increased in a sense.
The method includes the steps of recording volume information, constructing a deleting task of a volume data object by utilizing the volume information, sequencing the deleting task according to a preset sequencing rule through a to-be-deleted volume queue, giving consideration to the priority of the deleting task according to the sequencing rule, and deleting metadata of a target volume corresponding to the deleting task and deleting time of volume records in a storage pool. The distribution of the deleting tasks has flexibility and timeliness.
When the cluster service pressure is lower than the set pressure threshold, the node with the small task deleting load is selected as the target node for executing the task, so that the influence of the task deleting on the cluster service execution is reduced, and the balance of the task deleting executed by each node of the cluster is realized.
When the node executing the deleting task in the cluster fails, the target node can be obtained again, the deleting task executed by the failed node is distributed to the obtained target node again, and smooth execution of the deleting task is guaranteed. When the cluster main node fails, the node which is applied for becoming the main node first is selected as the main node, and normal operation of the cluster is guaranteed. And a set of complete volume deletion fault protection mechanism is formed for the fault conditions of the nodes and the main nodes for deleting the tasks, so that the stability of volume deletion is ensured, and the reliability of the system is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic diagram of a method for optimizing volume deletion in a distributed storage system according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating that when a cluster service pressure is lower than a set pressure threshold, a target node for executing a task is obtained according to a task load deletion condition of each node of a cluster according to an embodiment of the present invention;
fig. 3 is a flowchart for monitoring and recording test actions performed outside the test server terminal according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Example 1
Referring to fig. 1, an embodiment of the present invention provides a method for optimizing volume deletion in a distributed storage system, including:
s100, the cluster main node receives and analyzes the deletion request of the target volume sent by the user.
S200, the cluster main node responds to the deletion request of the target volume, firstly judges whether the target volume is a clone volume, if so, the association between the target volume and the parent snapshot of the target volume is disconnected, and then S300 is executed, otherwise, S300 is directly executed.
S300, the cluster main node responds to the deletion request of the target volume, deletes the metadata of the target volume and the volume record of the target volume in the storage pool, records the volume information of the target volume and feeds back the successful deletion of the target volume; the volume information includes a volume name, a volume ID, a deletion time, a priority, and data object information.
Specifically, the priority in the volume information is configured by the user and passed through the delete request. In the specific implementation process, a priority configuration interface is provided for a user, the user provides the priority configuration interface for the user when executing a volume deletion operation, and the user configures the priority of the target volume and sends the target volume to the cluster master node through the configuration in the deletion request.
The priority is any one of a boolean type and a numeric type;
wherein: for Boolean type priority, Boolean true indicates that priority is enabled, Boolean false indicates that priority is not enabled, default settings are false, and a user can actively configure Boolean type priority of a target volume to true;
for the numerical value type priority, the priority is determined by the value size and is defaulted to be a set value, a user can actively configure a designated value for the numerical value type priority of the target volume, and the designated value is in the value range of the preset numerical value type priority.
Specifically, the deletion time is the deletion time of the target volume metadata and the volume record in the storage pool, and reflects the timing of the user deleting the target volume.
S400, constructing a deletion task by using the volume information, putting the deletion task into a volume deletion queue to be processed, and sequencing according to a preset sequencing rule; specifically, the preset ordering rule includes:
and sequencing the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion according to the priority of the volume information. When the priority in the volume information is Boolean, the deleting tasks are sorted according to the true and false of the corresponding volume information priority, namely, all deleting tasks with true priority in the volume information are arranged before the deleting tasks with false priority. The priority in the volume information is a numerical value, namely, the deleting tasks are sorted according to the size of the priority numerical value, namely, the deleting tasks with the large priority numerical value in the volume information are arranged before the deleting tasks with the small priority numerical value.
After the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion are sorted according to the priority of the volume information, if the priorities are consistent, the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion are sorted according to the deletion time of the volume information, and the earlier the deletion time is, the earlier the position sorting is.
After the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion are sorted according to the deletion time of the volume information, if the deletion times are consistent, the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion are sorted according to the letter sorting of the volume names of the volume information, and the position sorting is more advanced the letter sorting of the volume names is.
S500, when the cluster service pressure is lower than a set pressure threshold value, deleting task load conditions according to each node of the cluster, and acquiring a target node for executing a task;
in a specific implementation process, referring to fig. 2, when the cluster service pressure is lower than the set pressure threshold, the obtaining a target node for executing a task according to a task load condition of each node of the cluster includes:
s501, the cluster master node monitors and collects the condition of cluster service pressure through a background;
s502, comparing whether the cluster service pressure is smaller than a set pressure threshold value, if not, not performing new deletion task allocation, and if so, executing S503.
S503, collecting the load of the current task deletion of each node of the cluster;
s504, sorting the nodes according to the deleted task load from low to high, and sorting the nodes with the same deleted task load according to the size of the node serial number; in the specific implementation process, the nodes with small node sequence numbers are arranged in front.
And S505, selecting the node with the top ranking as a target node.
S600, selecting deletion tasks from the volume deletion pending queue in sequence, and distributing the tasks to the target node for execution. In the specific implementation process, the distributed deleting task and the target node executing the deleting task are recorded. And after the deletion task is executed, clearing the corresponding deletion task record.
Monitoring the execution process of the task of deleting the target volume, configuring a corresponding volume state for the target volume according to the execution process of the task of deleting the target volume, and providing a volume state query interface for a user, wherein the volume state comprises to-be-processed, allocated, deleted and completed volume states. Specifically, after the metadata of the target volume and the volume record in the storage pool are deleted, the volume state corresponding to the process of waiting for task allocation in the volume deletion pending queue to the deletion task corresponding to the target volume is pending. And allocating the deleting task to the target node, wherein the volume state corresponding to the process before the target node executes the deleting task is allocated. And the target node starts to execute the deletion task, and the volume state corresponding to the process before the execution of the deletion task is in deletion. And the volume state corresponding to the target node completing the deleting task is the completed volume state. And when the volume state is completed, removing the completed deleting task in the volume deleting queue to be processed, and clearing the related task information.
In a specific implementation process, the cluster master node monitors whether a node executing a delete task in a cluster fails, and when the node executing the delete task fails, the cluster master node reacquires a target node and distributes the delete task executed by the failed node to the reacquired target node. Specifically, one possible way is: and when the node executing the deleted task fails, setting the volume state to be processed, putting the deleted task into the volume deletion queue to be processed again for sequencing, and selecting a new target node processing volume by the cluster main node according to the selection strategy of the target node to delete the deleted task in the queue to be processed.
In the specific implementation process, the nodes in the cluster periodically and asynchronously send applications to become master node applications; if the master node is normal, the application for becoming the master node of any node is fed back and rejected; if the main nodes are in failure, the first node becomes the main node after the main nodes are in failure and applies for feedback approval, the corresponding node becomes the main node, and the feedback refusal is applied for the rest nodes. After the master node is recovered, whether the master node exists in the cluster is detected, if yes, the authority of the master node as the master node is released actively to become the cluster node.
Example 2
Referring to fig. 3, an embodiment of the present invention provides a device for optimizing volume deletion in a distributed storage system, including:
and the analysis module analyzes the deletion request of the target volume and acquires the target volume from the deletion request.
And the target volume preprocessing module judges whether the target volume is a clone volume or not, and if so, disconnects the association between the target volume and the parent snapshot of the target volume.
A first deletion module to delete the metadata of the target volume and the volume records of the target volume in the storage pool in response to a deletion request of the target volume.
A volume information recording module that records volume information of a target volume; the volume information includes a volume name, a volume ID, a deletion time, a priority, and data object information.
And the feedback module feeds back the successful deletion of the target volume to the user after deleting the metadata of the target volume and the volume record in the storage pool.
And the task construction module is used for creating a corresponding deleting task according to the volume information.
And the queue module stores the deleted tasks and sorts the deleted tasks according to a preset sorting rule.
The task allocation module selects a target node for executing the deleted tasks when the cluster service pressure is lower than a pressure threshold value, and sequentially selects the deleted tasks from the queue module to be allocated to the target node and records the deleted tasks; specifically, the task allocation module comprises a cluster service pressure monitoring unit, a node deletion task monitoring unit and a target node selection unit, wherein the cluster service pressure monitoring unit monitors the condition of collecting cluster service pressure through a background, the node deletion task monitoring unit monitors the load of a current deleted task of each node of a cluster, the target node selection unit sorts the nodes from low to high according to the deleted task load, and for the nodes with the same deleted task load, the nodes are sorted from small to large according to the node serial number, and the node arranged in front is selected as a target node.
And the target node executes a deleting task through the second deleting module and deletes the data object of the target volume.
Example 3
The embodiment of the invention provides a storage medium for realizing a volume deletion optimization method in a distributed storage system. The storage medium for realizing the optimization method for deleting the volume in the distributed storage system stores at least one instruction, reads and executes the instruction to realize the optimization method for deleting the volume in the distributed storage system.
According to the method and the device, the processes of deleting the metadata of the target volume to be deleted and deleting the data object are separated, the success of deletion is fed back to the user after the metadata of the target volume is deleted, and the deleting speed is increased in sense.
The method and the device record volume information, construct a deletion task of a volume data object by using the volume information, sort the deletion task according to a preset sorting rule through a volume deletion queue to be processed, and the sorting rule gives consideration to the priority of the deletion task and deletes the deletion time of metadata of a target volume corresponding to the deletion task. The distribution of the deleting tasks has flexibility and timeliness.
When the cluster service pressure is lower than the set pressure threshold, the node with the small task deleting load is selected as the target node for executing the task, so that the influence of the task deleting on the cluster service execution is reduced, and the balance of the task deleting executed by each node of the cluster is realized.
When the node executing the deleting task in the cluster fails, the target node can be obtained again, the deleting task executed by the failed node is distributed to the obtained target node again, and smooth execution of the deleting task is guaranteed.
When the cluster main node fails, the node which is applied for becoming the main node first is selected as the main node, and normal operation of the cluster is guaranteed.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for optimizing volume deletion in a distributed storage system is characterized by comprising the following steps:
the cluster main node responds to the deletion request of the target volume, deletes the metadata of the target volume and the volume record of the target volume in the storage pool, records the volume information of the target volume and feeds back the success of the deletion of the target volume; the volume information comprises volume name, volume ID, deletion time, priority and data object information;
constructing a deletion task by using the volume information, putting the deletion task into a volume deletion queue to be processed, and sequencing according to a preset sequencing rule;
when the cluster service pressure is lower than a set pressure threshold value, deleting task load conditions according to each node of a cluster, and acquiring a target node for executing a task;
and selecting deletion tasks from the volume deletion pending queue in sequence, and distributing the tasks to the target node for execution so as to delete the data object of the target volume.
2. The method according to claim 1, wherein the cluster master node, in response to the deletion request of the target volume, first determines whether the target volume is a clone volume, if yes, first disconnects the association between the target volume and the parent snapshot of the target volume, and then deletes the metadata of the target volume and the volume record of the target volume in the storage pool, otherwise, directly deletes the metadata of the volume and the volume record of the target volume in the storage pool.
3. The method for optimizing deletion of a volume in a distributed storage system according to claim 1, wherein the priority in the volume information is any one of boolean type and numeric type;
wherein: for Boolean type priority, Boolean true indicates that priority is enabled, Boolean false indicates that priority is not enabled, default settings are false, and a user can actively configure Boolean type priority of a target volume to true;
for the numerical value type priority, the priority is determined by the value size and is defaulted to be a set value, a user can actively configure a designated value for the numerical value type priority of the target volume, and the designated value is in the value range of the preset numerical value type priority.
4. The method for optimizing volume deletion in a distributed storage system according to claim 1, wherein the deletion task is placed in a queue to be processed for volume deletion and sorted according to a preset sorting rule, wherein the preset sorting rule includes:
sorting the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion according to the priority of the volume information;
if the priorities are consistent, sorting the positions of the deletion tasks created by the volume information in the queue to be processed for volume deletion according to the deletion time of the volume information, wherein the earlier the deletion time is, the earlier the position sorting is;
and if the deleting time is consistent, sequencing the positions of the deleting tasks created by the volume information in the queue to be processed for volume deletion according to the letter sequencing of the volume names of the volume information, wherein the position sequencing is more advanced the letter sequencing of the volume names is.
5. The method for optimizing volume deletion in a distributed storage system according to claim 1, wherein when the cluster service pressure is lower than the set pressure threshold, acquiring a target node for executing a task according to a task deletion load condition of each node of the cluster comprises:
the cluster main node monitors and collects the condition of cluster service pressure through a background;
comparing whether the cluster service pressure is smaller than a set pressure threshold value, if not, not distributing a new task to be deleted, and if so, acquiring the load of the current task to be deleted of each node of the cluster;
sorting the nodes according to the deleted task load from low to high, and sorting the nodes according to the size of the node serial number if the deleted task loads are the same;
and selecting the node with the top ranking as a target node.
6. The method according to claim 1, wherein the cluster master node monitors whether a node executing the delete task in the cluster fails, and when the node executing the delete task fails, the cluster master node reacquires the target node and allocates the delete task executed by the failed node to the reacquired target node.
7. The method for optimizing deletion of a volume in a distributed storage system according to claim 1, wherein a node in the cluster periodically and asynchronously sends an application to become a master node application; if the master node is normal, the application for becoming the master node of any node is fed back and rejected; if the main nodes are in failure, the first node becomes the main node after the main nodes are in failure and applies for feedback approval, the corresponding node becomes the main node, and the feedback refusal is applied for the rest nodes.
8. The method according to claim 1, wherein the execution process of the deletion task of the target volume is monitored, a corresponding volume state is configured for the target volume according to the execution process of the deletion task of the target volume, and a volume state query interface is provided for a user, wherein the volume state includes pending, allocated, deleted and completed.
9. An apparatus for optimizing deletion of a volume in a distributed storage system, comprising:
the first deleting module is used for responding to a deleting request of the target volume to delete the metadata of the target volume;
a volume information recording module that records volume information of a target volume;
the feedback module feeds back the successful deletion of the target volume to the user after deleting the metadata of the target volume and the volume record in the storage pool;
the task construction module is used for creating a corresponding deletion task according to the volume information;
the queue module stores the deleted tasks and sorts the deleted tasks according to a preset sorting rule;
the task allocation module selects a target node for executing the deleted tasks when the cluster service pressure is lower than a pressure threshold value, and sequentially selects the deleted tasks from the queue module to be allocated to the target node and records the deleted tasks;
and the target node executes the deleting task through the second deleting module.
10. A storage medium for implementing a method for optimizing volume deletion in a distributed storage system, wherein the storage medium for implementing the method for optimizing volume deletion in a distributed storage system stores at least one instruction, and reads and executes the instruction to implement the method for optimizing volume deletion in a distributed storage system according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111399481.4A CN114153390B (en) | 2021-11-19 | 2021-11-19 | Method and device for optimizing volume deletion in distributed storage system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111399481.4A CN114153390B (en) | 2021-11-19 | 2021-11-19 | Method and device for optimizing volume deletion in distributed storage system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114153390A true CN114153390A (en) | 2022-03-08 |
| CN114153390B CN114153390B (en) | 2023-08-29 |
Family
ID=80457891
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111399481.4A Active CN114153390B (en) | 2021-11-19 | 2021-11-19 | Method and device for optimizing volume deletion in distributed storage system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114153390B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028478A (en) * | 2023-02-13 | 2023-04-28 | 山东英信计算机技术有限公司 | Data set management method and system of artificial intelligent platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140181033A1 (en) * | 2012-12-21 | 2014-06-26 | Commvault Systems, Inc. | Systems and methods to track deleted files |
| CN110888844A (en) * | 2019-11-22 | 2020-03-17 | 浪潮电子信息产业股份有限公司 | Data deleting method, system, equipment and computer readable storage medium |
| CN111400112A (en) * | 2020-03-18 | 2020-07-10 | 深圳市腾讯计算机系统有限公司 | Writing method and device of storage system of distributed cluster and readable storage medium |
| CN111625506A (en) * | 2020-05-29 | 2020-09-04 | 浪潮电子信息产业股份有限公司 | Distributed data deleting method, device and equipment based on deleting queue |
| CN113377535A (en) * | 2021-06-08 | 2021-09-10 | 深圳前海微众银行股份有限公司 | Distributed timing task allocation method, device, equipment and readable storage medium |
-
2021
- 2021-11-19 CN CN202111399481.4A patent/CN114153390B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140181033A1 (en) * | 2012-12-21 | 2014-06-26 | Commvault Systems, Inc. | Systems and methods to track deleted files |
| CN110888844A (en) * | 2019-11-22 | 2020-03-17 | 浪潮电子信息产业股份有限公司 | Data deleting method, system, equipment and computer readable storage medium |
| CN111400112A (en) * | 2020-03-18 | 2020-07-10 | 深圳市腾讯计算机系统有限公司 | Writing method and device of storage system of distributed cluster and readable storage medium |
| CN111625506A (en) * | 2020-05-29 | 2020-09-04 | 浪潮电子信息产业股份有限公司 | Distributed data deleting method, device and equipment based on deleting queue |
| CN113377535A (en) * | 2021-06-08 | 2021-09-10 | 深圳前海微众银行股份有限公司 | Distributed timing task allocation method, device, equipment and readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116028478A (en) * | 2023-02-13 | 2023-04-28 | 山东英信计算机技术有限公司 | Data set management method and system of artificial intelligent platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114153390B (en) | 2023-08-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7313659B2 (en) | System and method for managing storage and program for the same for executing an operation procedure for the storage according to an operation rule | |
| CN110427299B (en) | Log processing method, related device and system for micro-service system application | |
| JPH05173873A (en) | Data processor and storage management method | |
| CN113704052B (en) | Operation and maintenance system, method, equipment and medium of micro-service architecture | |
| CN112559211A (en) | Message data updating method and device, computer equipment and storage medium | |
| CN112506870A (en) | Data warehouse increment updating method and device and computer equipment | |
| CN101196901B (en) | Computer system and method for database query | |
| CN113553179A (en) | Distributed key value storage load balancing method and system | |
| CN109218401B (en) | Log collection method, system, computer device and storage medium | |
| CN114153390B (en) | Method and device for optimizing volume deletion in distributed storage system and storage medium | |
| CN114020368A (en) | Information processing method and device based on state machine and storage medium | |
| JP2019082857A (en) | Computer system and data processing control method | |
| CN115061811A (en) | Resource scheduling method, device, equipment and storage medium | |
| US12040992B2 (en) | Control device, control method, and control program | |
| CN114218044A (en) | Single board management method, device, equipment and medium | |
| JP4034201B2 (en) | Computer resource utilization method and computer resource utilization method | |
| JP2008135054A (en) | Workflow managing method, and its execution system | |
| CN111680018A (en) | Method, device and equipment for responding to request and readable storage medium | |
| CN111767299A (en) | Database operation method, device and system, storage medium and electronic equipment | |
| WO2024212601A1 (en) | Multi-language data access proxy device implemented based on grpc | |
| US20230099545A1 (en) | Iot system and data collection control method | |
| CN106789155A (en) | Metadata arbitrating server, control method and communication system | |
| US10990305B2 (en) | Storage apparatus and configuration management request processing method for storage apparatus | |
| JP2008234264A (en) | Load balancing device for file server, program for load balancing, and load balancing method | |
| US20230054058A1 (en) | Determining data copy resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |