CN115878049A - Thick backup volume creating method, device, equipment and storage medium - Google Patents
Thick backup volume creating method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115878049A CN115878049A CN202310173719.4A CN202310173719A CN115878049A CN 115878049 A CN115878049 A CN 115878049A CN 202310173719 A CN202310173719 A CN 202310173719A CN 115878049 A CN115878049 A CN 115878049A
- Authority
- CN
- China
- Prior art keywords
- volume
- formatting
- task
- instruction
- creating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 87
- 238000004590 computer program Methods 0.000 claims description 15
- 230000004048 modification Effects 0.000 claims description 5
- 238000012986 modification Methods 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 20
- 238000004891 communication Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for creating a thick backup volume, and relates to the technical field of storage. The method comprises the following steps: acquiring a thick backup volume creating instruction through a storage service intermediate layer, creating a corresponding thin volume according to the thick backup volume creating instruction, and configuring the thin volume to be in a usable state so as to provide service by using the thin volume; creating a formatting task in the background through the storage service middle layer, and issuing a formatting instruction for the thin volume to a storage layer by executing the formatting task; and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain a thick backup volume corresponding to the thick backup volume creating instruction. The method can realize the second-level creation rate of the thick backup volume, has the space occupation characteristic of the thick backup volume to prevent capacity insufficiency, and can improve the core service performance and the stability of a storage system in the creation process of the thick backup volume.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a method, an apparatus, a device, and a storage medium for creating a thick backup volume.
Background
Under the background of digital transformation, mass data is continuously growing, a distributed storage system is generally applied, and the distributed storage system generally comprises a main control server, a storage server and a plurality of clients, essentially, a large number of files are uniformly distributed on the plurality of storage servers, and the distributed storage system has the characteristics of high expansibility and high reliability. Meanwhile, the distributed storage system can be applied to various scenes, and the performance requirement on the distributed storage system is higher and higher under the promotion of a commercialization mode.
For a client, an arbitrary file system can be formatted and established like a physical hard disk to use block storage, so that the block storage is suitable for data storage in most general service scenarios and is more and more widely applied. In a block storage scenario, the volume assumes the main functional implementation and business output. The volume is mainly divided into a Thin-provisioning volume (Thin-provisioning volume) and a Thick-provisioning volume (Thick-provisioning volume), wherein the Thin-provisioning volume is pre-allocated with a virtual logical volume capacity size, does not occupy an actual physical space, and only allocates the physical space to the volume when data is actually written; the thin volumes are easily created with a total logical capacity greater than the physical capacity of the storage system, and when the sum of the used capacity of each volume exceeds the physical capacity of the array, no more data can be written, with serious consequences for application errors. However, the operation process of filling the quota space when the thick backup volume is created occupies a large amount of I/O and bandwidth storage resources, occupies an actual physical space, affects core services, and requires a long time for creating the thick backup volume, so that most of the current scenarios abandon the application of the thick backup volume.
In the prior art, the thick backup volume is usually fully written by writing data in different ways during the process of creating the thick backup volume, the creation of the thick backup volume is completed in such a way, the time consumed by the process is proportional to the size of the thick backup volume, and when a large thick backup volume is created, the whole creation process consumes a long time.
Disclosure of Invention
In view of this, an object of the present invention is to provide a method, an apparatus, a device, and a medium for creating a thick backup volume, which can achieve a second-level creation rate of the thick backup volume, and improve core service performance and stability of a storage system in a thick backup volume creation process. The specific scheme is as follows:
in a first aspect, the present application discloses a thick backup volume creation method, including:
acquiring a thick backup volume creating instruction through a storage service intermediate layer, creating a corresponding thin volume according to the thick backup volume creating instruction, and configuring the thin volume to be in a usable state so as to provide service by using the thin volume;
creating a formatting task in the background through the storage service middle layer, and issuing a formatting instruction for the thin volume to a storage layer by executing the formatting task;
and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain a thick backup volume corresponding to the thick backup volume creating instruction.
Optionally, before the storage layer modifies the metadata of the corresponding target object according to the formatting instruction, the method further includes:
judging whether the storage layer has a target object corresponding to the thin volume;
and if the target object corresponding to the thin volume exists, executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction.
Optionally, after determining whether the storage tier has a target object corresponding to the thin volume, the method further includes:
and if the target object corresponding to the thin volume does not exist, creating the target object corresponding to the thin volume, and then executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction.
Optionally, the creating, by the storage service intermediate layer, a formatting task in the background includes:
registering, by the storage service intermediary layer, formatting tasks for the thin volume in the background;
and allocating an executed target node for the formatting task, so that the target node is utilized to execute the operation of issuing the formatting instruction aiming at the thin volume to the storage layer.
Optionally, the registering, by the storage service intermediate layer, a formatting task for the thin volume in the background includes:
allocating a unique identification code for the formatting task;
and generating task information corresponding to the formatted task according to a preset information format, and adding the task information to a task list.
Optionally, the allocating the target node for the formatting task includes:
and checking the task list at regular time through the main node, screening out the newly generated tasks in the task list, and distributing corresponding target nodes for the newly generated tasks.
Optionally, the allocating the target node for the formatting task includes:
and calculating a hash value corresponding to the unique identification code through a hash algorithm, and distributing a corresponding target node for the formatted task according to the hash value and a preset distribution rule.
Optionally, after the target node for executing is allocated to the formatting task, the method further includes:
adding the node information of the target node into the task information corresponding to the formatting task;
and informing all the target nodes to execute the formatting instruction issuing task through broadcasting.
Optionally, the executing, by the target node, the operation of issuing the format instruction for the thin volume to the storage layer includes:
determining a target object corresponding to the thin volume according to the metadata corresponding to the thin volume;
creating a working thread corresponding to the formatting task through the target node;
starting a throttle valve in the work thread; the throttle valve is used for limiting the formatting instructions of the N target objects which are issued at maximum at a time;
and issuing a formatting instruction aiming at the target object to a storage layer through the working thread according to the throttle valve.
Optionally, before the target node performs the operation of issuing the formatting instruction for the thin volume to the storage tier, the method further includes:
the target node judges whether the current formatting task is the node task according to the received task information corresponding to the formatting task and the node information of the target node;
and if the node is the task of the node, executing the operation of issuing the formatting instruction aiming at the compact volume to the storage layer.
Optionally, the starting a throttle valve in the work thread includes:
and adjusting the parameter value corresponding to the throttle valve in real time according to the current load state of the storage cluster.
Optionally, the adjusting the parameter value corresponding to the throttle valve in real time according to the current load state of the storage cluster includes:
determining a target value aiming at the throttle valve according to the current load state of the storage cluster and a preset throttle valve condition rule;
and adjusting the parameter value of the throttle valve in real time according to the target value.
Optionally, the thick backup volume creating method further includes:
and if the formatting task fails to be executed, a new target node is allocated to the formatting task again, so that the new target node is used for executing the operation of issuing the formatting instruction for the thin volume to the storage layer.
Optionally, before the formatting task fails to be executed, the method further includes:
judging whether the formatting task fails according to preset conditions; the preset conditions comprise that the target node has a fault and/or the formatting time exceeds a preset time length.
Optionally, the creating a corresponding thin volume according to the thick backup volume creating instruction includes:
and analyzing the thick backup volume creating instruction to obtain volume parameters, and creating the thin volume according to the volume parameters.
Optionally, the modifying the metadata of the corresponding target object according to the formatting instruction includes:
and modifying the occupied space parameters in the metadata of the target object according to the formatting instruction.
Optionally, the modifying the parameters of the occupied space in the metadata of the target object includes:
determining a target space value corresponding to the target object according to the space occupation size of the target object in the full writing state;
and modifying the occupied space parameters in the metadata according to the target space value.
In a second aspect, the present application discloses a thick backup volume creation apparatus, comprising:
the instruction acquisition module is used for acquiring a thick backup volume creation instruction through a storage service intermediate layer, creating a corresponding thin volume according to the thick backup volume creation instruction, and configuring the thin volume to be in a usable state so as to provide service by using the thin volume;
the formatting instruction issuing module is used for creating a formatting task in the background through the storage service middle layer and issuing a formatting instruction aiming at the simplified volume to the storage layer by executing the formatting task;
and the metadata modification module is used for modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction so as to create and obtain the thick backup volume corresponding to the thick backup volume creating instruction.
In a third aspect, the present application discloses an electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the thick provisioning volume creation method described above.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by the processor implements the thick standby volume creation method described above.
According to the method, a thick backup volume creating instruction is obtained through a storage service intermediate layer, a corresponding thin volume is created according to the thick backup volume creating instruction, and the thin volume is configured to be in a usable state so as to provide service by using the thin volume; creating a formatting task in the background through the storage service middle layer, and issuing a formatting instruction aiming at the thin volume to a storage layer by executing the formatting task; and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain a thick backup volume corresponding to the thick backup volume creating instruction. Therefore, the corresponding thin volume is constructed to provide service for a user, a formatting task is created in the background, thick provisioning of the volume object is executed, only metadata of the target object is modified, data is not really written in a traditional thick provisioning mode, the thick provisioning volume can achieve second-level creation rate like the thin volume, meanwhile, the space occupation characteristic of the thick provisioning volume is provided to prevent capacity insufficiency, and core service performance and stability of a storage system in the thick provisioning volume creation process can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a thick backup roll creation method provided in the present application;
fig. 2 is a flowchart of a specific method for creating a thick backup volume according to the present application;
fig. 3 is a flowchart of a specific formatting command issuing method provided in the present application;
fig. 4 is a schematic structural diagram of a thick backup roll creating apparatus provided in the present application;
fig. 5 is a block diagram of an electronic device provided in the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
In the prior art, the thick backup volume is usually fully written by writing data in different ways during the process of creating the thick backup volume, the creation of the thick backup volume is completed in such a way, the time consumed by the process is proportional to the size of the thick backup volume, and when a large thick backup volume is created, the whole creation process consumes a long time. In order to overcome the technical problem, the application provides a method for creating a thick backup volume, which can realize the second-level creation rate of the thick backup volume and improve the core service performance and the stability of a storage system in the process of creating the thick backup volume.
The embodiment of the application discloses a thick backup roll creating method, and as shown in fig. 1, the method may include the following steps:
step S11: and acquiring a thick backup volume creating instruction through a storage service intermediate layer, creating a corresponding thin volume according to the thick backup volume creating instruction, and configuring the thin volume to be in a usable state so as to provide service by using the thin volume.
In this embodiment, after the thick backup volume creation instruction is obtained through the storage service intermediate layer, the corresponding thin volume is created according to the thick backup volume creation instruction, and the thin volume is configured to be in a usable state, so that the thin volume is used to provide service, that is, since the thin volume does not occupy an actual physical storage space and is created at a high speed, the corresponding thin volume is created first to provide service. Specifically, the thin volume is configured to be in a usable state, then a response of completing the creation of the thick backup volume is returned to the upper layer, at this time, the volume can already provide IO services to the outside, the volume is in a usable state, that is, the thin volume can provide services after being quickly created, and then the corresponding thick backup volume is created through the background. The storage service intermediate layer may specifically be a RBD (radial automated Distributed object store) Block Device), that is, after receiving a request for creating a thick backup volume by using an RBD layer of a Distributed storage system, a similar thin volume is created first, and at this time, it is considered that creation of the thick backup volume may be completed and creation success may be returned to an upper layer, and the volume is provided for a client service to use.
In this embodiment, the creating a corresponding thin volume according to the thick backup volume creating instruction may include: analyzing the thick backup volume creating instruction to obtain volume parameters; and creating a thin volume according to the volume parameters, and configuring the thin volume to be in a usable state. I.e., parse the create instruction, such as volume size, to create a thin volume-like object.
Step S12: and creating a formatting task in the background through the storage service middle layer, and issuing a formatting instruction aiming at the thin volume to the storage layer by executing the formatting task.
In this embodiment, after the thin volume is created, a formatting task is created in the background, where the formatting task is a task of performing thick provisioning on a volume object, and a formatting instruction for the thin volume is issued to a storage layer (distributed object storage) by executing the formatting task, that is, the thick provisioning volume occupies actual storage, so that the creation of the thick provisioning volume is performed in the storage layer.
Step S13: and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain the thick backup volume corresponding to the thick backup volume creating instruction.
In this embodiment, the storage layer has more received formatting instructions to modify metadata of a target object of the thin volume corresponding to the storage layer, so as to create and obtain a corresponding thick backup volume. Therefore, the method segments the flow of volume creation and thick backup, creates the thin volume first, and then performs the background thick backup, that is, the thin volume can be used after the thin volume creation is completed, thereby greatly accelerating the speed of creating the thick backup volume, balancing the resources used by the service and the thick backup volume creation, and better ensuring the stability of the core service in the process of creating the thick backup volume. In the background thick-standby process, only the object metadata is operated, so that the cluster bandwidth is greatly saved, the stability of the core service is better ensured, and the current concepts of quick storage and high-efficiency storage are met.
In this embodiment, before the modifying, by the storage layer, the metadata of the corresponding target object according to the formatting instruction, the method may further include: judging whether the storage layer has a target object corresponding to the thin volume; and if the target object corresponding to the thin volume exists, executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction. After the determining whether the storage tier has the target object corresponding to the thin volume, the method may further include: and if the target object corresponding to the thin volume does not exist, creating the target object corresponding to the thin volume, and then executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction. That is, after receiving a formatting instruction of an object, the bottom layer storage first determines whether a target object corresponding to the thin volume exists in the storage layer, one thin volume corresponds to multiple target objects, and if the corresponding target object exists, the metadata of the object is modified; if the object does not exist, the object is created first, namely, a target object corresponding to the thin volume is created in the storage layer, and then the metadata of the target object is modified. Therefore, the occupied bandwidth in the thick preparation process is greatly saved by the space occupation mode of only modifying the metadata of the object without writing data.
In this embodiment, the modifying the metadata of the corresponding target object according to the formatting instruction may include: and modifying the occupied space parameters in the metadata of the target object according to the formatting instruction. That is, the occupied space parameter in the metadata of the modified target object is specifically modified in this embodiment.
In this embodiment, the modifying the parameters of the occupied space in the metadata of the target object may include: determining a target space value corresponding to the target object according to the space occupation size of the target object in a full-written state; and modifying the occupied space parameters in the metadata according to the target space value. I.e. the occupied space is modified to a corresponding size at which the target object is completely full.
As can be seen from the above, in this embodiment, a thick backup volume creation instruction is obtained through a storage service intermediate layer, a corresponding thin volume is created according to the thick backup volume creation instruction, and the thin volume is configured to be in a usable state, so that the thin volume is used to provide a service; creating a formatting task in the background through the storage service middle layer, and issuing a formatting instruction for the thin volume to a storage layer by executing the formatting task; and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain a thick backup volume corresponding to the thick backup volume creating instruction. Therefore, the corresponding thin volume is constructed to provide service for a user, a formatting task is created in the background, thick provisioning of the volume object is executed, only the metadata of the target object is modified, and data is not really written in the traditional thick provisioning mode, so that the thick provisioning volume can realize the second-level creation rate as the thin volume, meanwhile, the space occupation characteristic of the thick provisioning volume is provided to prevent capacity insufficiency, and the core service performance and the stability of a storage system in the thick provisioning volume creation process can be improved.
The embodiment of the present application discloses a specific thick backup roll creating method, which may include the following steps, as shown in fig. 2:
step S21: the method comprises the steps of obtaining a thick backup volume creating instruction through a storage service intermediate layer, creating a corresponding thin volume according to the thick backup volume creating instruction, and configuring the thin volume to be in a usable state so as to provide service by using the thin volume.
Step S22: registering, by the storage service intermediary layer, formatting tasks for the thin volume in the background.
In this embodiment, the registering, by the storage service intermediate layer, the formatting task for the thin volume in the background may include: allocating a unique identification code for the formatting task; and generating task information corresponding to the formatted task according to a preset information format, and adding the task information to a task list.
That is, a random Unique Identifier, such as a uuid (universal Unique Identifier), is assigned to the formatted task, and task information corresponding to the formatted task is generated, where the task information includes, but is not limited to, a task ID, a task type, an operation target (source volume, destination volume), and an executor ID. The executor ID is the specific node for task execution, and the parameter is null before the node is allocated. Then, the Task information is written in the form of a structure into the Task list, i.e., the Task object (Task _ obj).
Step S23: and allocating an executed target node for the formatting task, so that the target node is utilized to execute the operation of issuing the formatting instruction aiming at the thin volume to the storage layer.
In this embodiment, the target node for executing is allocated to the formatting task, so as to execute the operation of issuing the formatting instruction for the thin volume to the storage layer by using the target node, that is, the execution of the task needs to be performed by a certain node, so that the node selects the target node for executing the formatting task from the storage cluster, and executes the operation of issuing the formatting instruction for the thin volume to the storage layer.
In this embodiment, the allocating the target node for executing the formatting task may include: and checking the task list at regular time through the main node, screening out the newly generated tasks in the task list, and distributing corresponding target nodes for the newly generated tasks. The task list is checked at regular time through the main node, then the task list is compared with the tasks in the current execution task list, newly generated tasks are screened out, and then corresponding target nodes are distributed for the newly generated tasks. And when the formatting of all the existing objects is finished, the background formatting task is finished and quitting is finished.
In this embodiment, the allocating the target node for the formatting task may include: and calculating a hash value corresponding to the unique identification code through a hash algorithm, and distributing a corresponding target node for the formatted task according to the hash value and a preset distribution rule. In order to distribute the tasks to the nodes in the cluster as uniformly as possible, in this embodiment, the hash value corresponding to the unique identifier is calculated through a hash algorithm, and then the corresponding target node is distributed to the formatted task according to the hash value and a preset distribution rule. And the load balance of the tasks is ensured.
In this embodiment, after allocating the executed target node for the formatting task, the method may further include: adding the node information of the target node into the task information corresponding to the formatting task; and informing all the target nodes to execute the formatting instruction issuing task through broadcasting. The ID of the executing node is written into the executor field of the task, so that the specific node executing the task can be inquired through the task information, and all task executors are informed of refreshing the task list in a broadcasting mode, namely, the target node acquires the formatted task and the corresponding task information through broadcasting.
In this embodiment, the method for creating a thick backup volume may further include: and if the formatting task fails to be executed, a new target node is allocated to the formatting task again, so that the new target node is used for executing the operation of issuing the formatting instruction for the thin volume to the storage layer. In other words, in an abnormal situation, for example, when the execution of the formatting task is unsuccessful and the formatting task is not finished normally, the task assignment process is retriggered, and the formatting task is started by reselecting the node. In addition, since the underlying storage operates according to the object existence state, the re-triggering of the formatting task does not affect data security.
In this embodiment, before the formatting task fails to be executed, the method may further include: judging whether the formatting task fails according to preset conditions; the preset conditions comprise that the target node has a fault and/or the formatting time exceeds a preset time length. The nodes which are about to execute the tasks are in problem, and the formatted tasks are considered to be failed to execute when the preset timeout time of the tasks is reached.
Step S24: and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain a thick backup volume corresponding to the thick backup volume creating instruction.
For the specific processes of the steps S21 and S24, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated herein.
As can be seen from the above, in this embodiment, a thick backup volume creation instruction is obtained through a storage service intermediate layer, a corresponding thin volume is created according to the thick backup volume creation instruction, and the thin volume is configured to be in a usable state, so that the thin volume is used to provide a service; registering, by the storage service intermediary layer, formatting tasks for the thin volume in the background; allocating an executed target node for the formatting task, so that the target node is utilized to execute the operation of issuing the formatting instruction aiming at the thin volume to a storage layer; and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain a thick backup volume corresponding to the thick backup volume creating instruction. And the creation of the thick backup volume is realized by allocating an executed target node for a formatting task and then executing the operation of issuing a formatting instruction aiming at the thin volume to a storage layer by using the target node.
On the basis of the foregoing embodiment, an embodiment of the present application further discloses a specific formatting instruction issuing method, as shown in fig. 3, that is, the step of executing, by using the target node, the operation of issuing the formatting instruction for the thin volume to the storage layer may include the following sub-steps:
s31: and determining a target object corresponding to the thin volume according to the metadata corresponding to the thin volume.
S32: and creating a working thread corresponding to the formatting task through the target node.
In this embodiment, a corresponding volume object in the storage layer is determined according to the metadata of the thin volume, and a formatting operation thread is created.
S33: starting a throttle valve in the work thread; the throttle valve is used to limit the single maximum issuing of formatting instructions for the N target objects.
In this embodiment, the throttle valve is started in the work thread; the throttle valve is used for limiting the format instructions of the N target objects which are issued at the maximum at a time, namely the throttle valve is used for adjusting the numerical value of N. Thereby controlling the resource occupancy of the task.
In this embodiment, the starting the throttle valve in the work thread may include: and adjusting the parameter value corresponding to the throttle valve in real time according to the current load state of the storage cluster. Namely, the thick backup volume creation rate is automatically adjusted by adjusting the throttle valve according to the cluster working load, and the stability of the core service is ensured. If the throttle valve is started in the thread, the default configuration of the throttle valve is N, the formatting requests of N objects can be issued at the same time, and the formatting request of the next object is allowed to be issued after the formatting of one object is completed.
In this embodiment, the adjusting the parameter value corresponding to the throttle valve in real time according to the current load state of the storage cluster may include: determining a target value aiming at the throttle valve according to the current load state of the storage cluster and a preset throttle valve condition rule; and adjusting the parameter value of the throttle valve in real time according to the target value.
For example, the current overall IO load condition of the storage cluster is acquired once every T seconds, the value of a throttle valve N is adjusted according to the current IO load, and the gears of three N values of high speed, medium speed and low speed are respectively corresponding to 100, 50 and 25; meanwhile, calculating the overall load S of the cluster according to the number of cluster nodes, wherein the load S = IOPS for storing the number of cluster nodes and the single node can provide, and acquiring the current cluster load, wherein N =25 is set when the current load is more than 70% of the total weight S; setting N =50 when the current load is greater than 40% with s less than 70%; set N =100 when the current load is less than 30%.
S34: and issuing a formatting instruction aiming at the target object to a storage layer through the working thread according to the throttle valve.
In the embodiment, a background formatting task is established through an RBD layer, and formatting requests of all objects in the scroll are automatically sent according to the setting condition of a throttle valve; and after receiving the object formatting request, the storage bottom layer formats the volume object to complete the background creation of the thick backup volume.
In this embodiment, before the target node performs the operation of issuing the formatting instruction for the thin volume to the storage tier, the method may further include: the target node judges whether the current formatting task is the node task according to the received task information corresponding to the formatting task and the node information of the target node; and if the node is the task of the node, executing the operation of issuing the formatting instruction aiming at the compact volume to the storage layer. That is, after receiving the broadcast notification, the task management service of each target node reads the executor ID of the task and compares the executor ID with the task itself, and if the executor ID matches the executor ID, it indicates that the task is executed by the local node.
As can be seen from the above, in this embodiment, a target object corresponding to the thin volume is determined according to metadata corresponding to the thin volume; creating a working thread corresponding to the formatting task through the target node; starting a throttle valve in the work thread; the throttle valve is used for limiting the formatting instructions of the N target objects which are issued at maximum at a time; and issuing a formatting instruction aiming at the target object to a storage layer through the working thread according to the throttle valve. The creation rate of the thick backup volume is controlled through a throttle valve, and the stability of core services is ensured.
Correspondingly, an embodiment of the present application further discloses a thick backup roll creating device, which is shown in fig. 4 and includes:
the instruction obtaining module 11 is configured to obtain a thick provisioning volume creation instruction through a storage service intermediate layer, create a corresponding thin volume according to the thick provisioning volume creation instruction, and configure the thin volume to be in a usable state, so as to provide a service by using the thin volume;
a formatting instruction issuing module 12, configured to create a formatting task in the background through the storage service intermediate layer, and issue a formatting instruction for the thin volume to the storage layer by executing the formatting task;
and a metadata modification module 13, configured to modify, at the storage layer, the metadata of the corresponding target object according to the formatting instruction, so as to create a thick backup volume corresponding to the thick backup volume creation instruction.
As can be seen from the above, in this embodiment, a thick backup volume creation instruction is obtained through a storage service intermediate layer, a corresponding thin volume is created according to the thick backup volume creation instruction, and the thin volume is configured to be in a usable state, so that the thin volume is used to provide a service; creating a formatting task in the background through the storage service middle layer, and issuing a formatting instruction for the thin volume to a storage layer by executing the formatting task; and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain a thick backup volume corresponding to the thick backup volume creating instruction. Therefore, the corresponding thin volume is constructed to provide service for a user, a formatting task is created in the background, thick provisioning of the volume object is executed, only metadata of the target object is modified, data is not really written in a traditional thick provisioning mode, the thick provisioning volume can achieve second-level creation rate like the thin volume, meanwhile, the space occupation characteristic of the thick provisioning volume is provided to prevent capacity insufficiency, and core service performance and stability of a storage system in the thick provisioning volume creation process can be improved.
In some specific embodiments, the thick backup roll creating apparatus may further include:
the object judgment unit is used for judging whether the storage layer has the target object corresponding to the thin volume before the storage layer modifies the metadata of the corresponding target object according to the formatting instruction;
and the execution unit is used for executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction if the target object corresponding to the thin volume exists.
In some specific embodiments, the thick backup roll creating apparatus may further include:
and the object creating unit is used for creating the target object corresponding to the thin volume and then executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction if the target object corresponding to the thin volume does not exist after judging whether the storage layer has the target object corresponding to the thin volume.
In some specific embodiments, the instruction obtaining module 11 may specifically include:
a registration unit, configured to register, in the background, a formatting task for the thin volume through the storage service intermediate layer;
and the node allocation unit is used for allocating an executed target node for the formatting task so as to execute the operation of issuing the formatting instruction aiming at the thin volume to the storage layer by using the target node.
In some specific embodiments, the registration unit may specifically include:
the identification code allocation unit is used for allocating a unique identification code for the formatting task;
and the task information generating unit is used for generating task information corresponding to the formatted task according to a preset information format and adding the task information to a task list.
In some specific embodiments, the node allocating unit may specifically include:
and the list viewing unit is used for regularly viewing the task list through the main node, screening the newly generated task in the task list and distributing a corresponding target node for the newly generated task.
In some specific embodiments, the node allocating unit may specifically include:
and the node determining unit is used for calculating a hash value corresponding to the unique identification code through a hash algorithm and distributing a corresponding target node for the formatting task according to the hash value and a preset distribution rule.
In some specific embodiments, the thick backup roll creating apparatus may specifically include:
an information adding unit, configured to add node information of a target node to the task information corresponding to the formatted task after allocating the executed target node to the formatted task;
and the broadcasting unit is used for informing all the target nodes of executing the formatting instruction issuing task through broadcasting.
In some specific embodiments, the node allocating unit may specifically include:
a target object determining unit, configured to determine a target object corresponding to the thin volume according to metadata corresponding to the thin volume;
the thread creating unit is used for creating a working thread corresponding to the formatting task through the target node;
a throttle valve starting unit for starting a throttle valve in the working thread; the throttle valve is used for limiting the formatting instructions of the N target objects which are issued at maximum at a time;
and the instruction issuing unit is used for issuing a formatting instruction aiming at the target object to the storage layer through the working thread according to the throttle valve.
In some specific embodiments, the thick backup roll creating apparatus may specifically include:
the node judgment unit is used for judging whether the current formatting task is the node task or not according to the received task information corresponding to the formatting task and the node information of the target node before the target node executes the operation of issuing the formatting instruction aiming at the simplified volume to the storage layer;
and the instruction issuing execution unit is used for executing the operation of issuing the formatting instruction aiming at the compact volume to the storage layer if the node is the task.
In some embodiments, the throttle valve actuating unit may specifically include:
and the adjusting unit is used for adjusting the parameter value corresponding to the throttle valve in real time according to the current load state of the storage cluster.
In some specific embodiments, the adjusting unit may specifically include:
the numerical value determining unit is used for determining a target numerical value aiming at the throttle valve according to the current load state of the storage cluster and a preset throttle valve condition rule;
and the real-time adjusting unit is used for adjusting the parameter value of the throttle valve in real time according to the target value.
In some specific embodiments, the thick backup roll creating apparatus may further include:
and the reallocation unit is used for reallocating a new target node for the formatting task if the formatting task fails to be executed, so that the new target node is used for executing the operation of issuing the formatting instruction for the thin volume to the storage layer.
In some specific embodiments, the thick backup roll creating apparatus may specifically include:
the failure judgment unit is used for judging whether the formatting task fails according to preset conditions; the preset conditions comprise that the target node has a fault and/or the formatting time exceeds a preset time length.
In some specific embodiments, the instruction obtaining module 11 may specifically include:
and the analysis unit is used for analyzing the thick backup volume creation instruction to obtain volume parameters and creating the thin volume according to the volume parameters.
In some specific embodiments, the metadata modification module 13 may specifically include:
and the space parameter modifying unit is used for modifying the occupied space parameters in the metadata of the target object according to the formatting instruction.
In some specific embodiments, the spatial parameter modification unit may specifically include:
the target space value determining unit is used for determining a target space value corresponding to the target object according to the space occupation size of the target object in a full-written state;
and the occupied space parameter modifying unit is used for modifying the occupied space parameters in the metadata according to the target space value.
Further, the embodiment of the present application also discloses an electronic device, which is shown in fig. 5, and the content in the drawing cannot be considered as any limitation to the application scope.
Fig. 5 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the thick standby volume creation method disclosed in any one of the foregoing embodiments.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the storage 22 is used as a carrier for storing resources, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., the resources stored thereon include an operating system 221, a computer program 222, and data 223 including metadata, etc., and the storage may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device and the computer program 222 on the electronic device 20, so as to realize the operation and processing of the mass data 223 in the memory 22 by the processor 21, and may be Windows Server, netware, unix, linux, and the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the thick provisioning volume creation method performed by the electronic device 20 disclosed in any of the foregoing embodiments.
Further, an embodiment of the present application further discloses a computer storage medium, where computer-executable instructions are stored in the computer storage medium, and when the computer-executable instructions are loaded and executed by a processor, the steps of the thick standby volume creation method disclosed in any of the foregoing embodiments are implemented.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
The method, the apparatus, the device and the medium for creating a thick backup volume provided by the present invention are described in detail above, and a specific example is applied in the present disclosure to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (20)
1. A thick standby volume creation method, comprising:
acquiring a thick backup volume creating instruction through a storage service intermediate layer, creating a corresponding thin volume according to the thick backup volume creating instruction, and configuring the thin volume to be in a usable state so as to provide service by using the thin volume;
creating a formatting task in the background through the storage service middle layer, and issuing a formatting instruction for the thin volume to a storage layer by executing the formatting task;
and modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction to create and obtain the thick backup volume corresponding to the thick backup volume creating instruction.
2. The method according to claim 1, wherein before the storage layer modifies the metadata of the corresponding target object according to the formatting instructions, the method further comprises:
judging whether the storage layer has a target object corresponding to the thin volume;
and if the target object corresponding to the thin volume exists, executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction.
3. The method according to claim 2, wherein after determining whether the storage tier has a target object corresponding to the thin volume, the method further comprises:
and if the target object corresponding to the compact volume does not exist, creating the target object corresponding to the compact volume, and then executing the operation of modifying the metadata of the corresponding target object according to the formatting instruction.
4. The method of claim 1, wherein the creating a formatting task in the background through the storage service middle layer comprises:
registering, by the storage service intermediary layer, formatting tasks for the thin volume in the background;
and allocating an executed target node for the formatting task, so that the target node is utilized to execute the operation of issuing the formatting instruction aiming at the thin volume to the storage layer.
5. The method of claim 4, wherein the registering, by the storage service intermediary layer, the formatting task for the thin volume in the background comprises:
allocating a unique identification code for the formatting task;
and generating task information corresponding to the formatted task according to a preset information format, and adding the task information to a task list.
6. The method according to claim 5, wherein the allocating the target node for the formatting task to execute comprises:
and checking the task list at regular time through the main node, screening out the newly generated tasks in the task list, and distributing corresponding target nodes for the newly generated tasks.
7. The method according to claim 5, wherein the allocating a target node for the formatting task to execute comprises:
and calculating a hash value corresponding to the unique identification code through a hash algorithm, and distributing a corresponding target node to the formatted task according to the hash value and a preset distribution rule.
8. The method of claim 5, wherein after allocating the target node for the formatting task, the method further comprises:
adding the node information of the target node into the task information corresponding to the formatting task;
and informing all the target nodes to execute the formatting instruction issuing task through broadcasting.
9. The method according to claim 4, wherein the performing, by the target node, the operation of issuing the formatting instruction for the thin volume to the storage layer comprises:
determining a target object corresponding to the thin volume according to the metadata corresponding to the thin volume;
creating a working thread corresponding to the formatting task through the target node;
starting a throttle valve in the work thread; the throttle valve is used for limiting the formatting instructions of the N target objects which are issued at maximum at a time;
and issuing a formatting instruction aiming at the target object to a storage layer through the working thread according to the throttle valve.
10. The method according to claim 4, wherein before the target node performs the operation of issuing the formatting instruction for the thin volume to the storage layer, the method further comprises:
the target node judges whether the current formatting task is the node task according to the received task information corresponding to the formatting task and the node information of the target node;
and if the node is the task of the node, executing the operation of issuing the formatting instruction aiming at the compact volume to the storage layer.
11. The thick standby volume creation method of claim 9, wherein said starting a throttle valve in the work thread comprises:
and adjusting the parameter value corresponding to the throttle valve in real time according to the current load state of the storage cluster.
12. The method for creating the thick backup roll according to claim 9, wherein the adjusting the parameter value corresponding to the throttle valve in real time according to the current load status of the storage cluster comprises:
determining a target value aiming at the throttle valve according to the current load state of the storage cluster and a preset throttle valve condition rule;
and adjusting the parameter value of the throttle valve in real time according to the target value.
13. The thick standby volume creation method according to claim 4, further comprising:
and if the formatting task fails to be executed, a new target node is allocated to the formatting task again, so that the new target node is used for executing the operation of issuing the formatting instruction for the thin volume to the storage layer.
14. The method of claim 13, wherein if the formatting task fails to execute, further comprising:
judging whether the formatting task fails according to preset conditions; the preset conditions comprise that the target node has a fault and/or the formatting time exceeds a preset time length.
15. The method according to claim 1, wherein the creating a corresponding thin volume according to the thick-provisioning volume creation instruction comprises:
and analyzing the thick backup volume creating instruction to obtain volume parameters, and creating the thin volume according to the volume parameters.
16. The method according to any one of claims 1 to 15, wherein the modifying metadata of the corresponding target object according to the formatting instruction comprises:
and modifying the occupied space parameters in the metadata of the target object according to the formatting instruction.
17. The method of claim 16, wherein the modifying the footprint parameter in the metadata of the target object comprises:
determining a target space value corresponding to the target object according to the space occupation size of the target object in the full writing state;
and modifying the occupied space parameters in the metadata according to the target space value.
18. A thick standby volume creation apparatus, comprising:
the instruction acquisition module is used for acquiring a thick backup volume creation instruction through a storage service intermediate layer, creating a corresponding thin volume according to the thick backup volume creation instruction, and configuring the thin volume to be in a usable state so as to provide service by using the thin volume;
the formatting instruction issuing module is used for creating a formatting task in the background through the storage service middle layer and issuing a formatting instruction aiming at the simplified volume to the storage layer by executing the formatting task;
and the metadata modification module is used for modifying the metadata of the corresponding target object in the storage layer according to the formatting instruction so as to create and obtain the thick backup volume corresponding to the thick backup volume creating instruction.
19. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the thick provisioning volume creation method of any of claims 1 to 17.
20. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the thick provisioning volume creation method of any of claims 1 to 17.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310173719.4A CN115878049A (en) | 2023-02-28 | 2023-02-28 | Thick backup volume creating method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310173719.4A CN115878049A (en) | 2023-02-28 | 2023-02-28 | Thick backup volume creating method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115878049A true CN115878049A (en) | 2023-03-31 |
Family
ID=85761658
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310173719.4A Pending CN115878049A (en) | 2023-02-28 | 2023-02-28 | Thick backup volume creating method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115878049A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111061435A (en) * | 2019-12-29 | 2020-04-24 | 浪潮电子信息产业股份有限公司 | Thick backup roll creating method and device, electronic equipment and storage medium |
| CN111176570A (en) * | 2019-12-27 | 2020-05-19 | 浪潮电子信息产业股份有限公司 | Thick backup roll creating method, device, equipment and medium |
| CN114610243A (en) * | 2022-03-21 | 2022-06-10 | 苏州浪潮智能科技有限公司 | Thin volume conversion method, system, storage medium and equipment |
| CN114995762A (en) * | 2022-05-31 | 2022-09-02 | 济南浪潮数据技术有限公司 | Thick backup roll capacity expansion method, device, equipment and storage medium |
-
2023
- 2023-02-28 CN CN202310173719.4A patent/CN115878049A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111176570A (en) * | 2019-12-27 | 2020-05-19 | 浪潮电子信息产业股份有限公司 | Thick backup roll creating method, device, equipment and medium |
| CN111061435A (en) * | 2019-12-29 | 2020-04-24 | 浪潮电子信息产业股份有限公司 | Thick backup roll creating method and device, electronic equipment and storage medium |
| CN114610243A (en) * | 2022-03-21 | 2022-06-10 | 苏州浪潮智能科技有限公司 | Thin volume conversion method, system, storage medium and equipment |
| CN114995762A (en) * | 2022-05-31 | 2022-09-02 | 济南浪潮数据技术有限公司 | Thick backup roll capacity expansion method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112532675B (en) | Method, device and medium for establishing network edge computing system | |
| US8104038B1 (en) | Matching descriptions of resources with workload requirements | |
| CN108881512B (en) | CTDB virtual IP balance distribution method, device, equipment and medium | |
| EP3402131A1 (en) | Resource configuration method, virtualized network function manager and network element management system | |
| US8886690B2 (en) | Distributed data storage and access systems | |
| EP3442201B1 (en) | Cloud platform construction method and cloud platform | |
| US10761869B2 (en) | Cloud platform construction method and cloud platform storing image files in storage backend cluster according to image file type | |
| CN109901798B (en) | Data storage method and device | |
| CN103077197A (en) | Data storing method and device | |
| CN108874502B (en) | Resource management method, device and equipment of cloud computing cluster | |
| CN110244901B (en) | Task allocation method and device and distributed storage system | |
| US20220318071A1 (en) | Load balancing method and related device | |
| CN111831232A (en) | Data storage method and device, storage medium and electronic device | |
| US7707199B2 (en) | Method and system for integrated management computer setting access rights, calculates requested storage capacity of multiple logical storage apparatus for migration | |
| CN114661419A (en) | Service quality control system and method | |
| CN110008029B (en) | ceph metadata cluster directory distribution method, system, device and readable storage medium | |
| CN114745358A (en) | IP address management method, system and controller in load balancing service | |
| CN112491592A (en) | Storage resource grouping method, system, terminal and storage medium | |
| CN115878049A (en) | Thick backup volume creating method, device, equipment and storage medium | |
| CN112468458B (en) | Scheduling method based on neutron layering mechanism | |
| CN115604294A (en) | Method and device for managing storage resources | |
| CN111722783B (en) | Data storage method and device | |
| CN106888244B (en) | Service processing method and device | |
| CN114879907A (en) | Data distribution determination method, device, equipment and storage medium | |
| CN105245569A (en) | Breakpoint uploading method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230331 |