CN118069432B - Method, device, equipment and storage medium for snapshot rollback of distributed block storage volume - Google Patents
Method, device, equipment and storage medium for snapshot rollback of distributed block storage volume Download PDFInfo
- Publication number
- CN118069432B CN118069432B CN202410473071.7A CN202410473071A CN118069432B CN 118069432 B CN118069432 B CN 118069432B CN 202410473071 A CN202410473071 A CN 202410473071A CN 118069432 B CN118069432 B CN 118069432B
- Authority
- CN
- China
- Prior art keywords
- storage volume
- data
- block storage
- distributed block
- snapshot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 239000012634 fragment Substances 0.000 claims abstract description 244
- 238000005096 rolling process Methods 0.000 claims abstract description 27
- 238000013507 mapping Methods 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 12
- 230000004044 response Effects 0.000 claims description 12
- 230000008859 change Effects 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 9
- 230000008520 organization Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1415—Saving, restoring, recovering or retrying at system level
- G06F11/1438—Restarting or rejuvenating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1415—Saving, restoring, recovering or retrying at system level
- G06F11/1435—Saving, restoring, recovering or retrying at system level using file system or storage system metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1471—Saving, restoring, recovering or retrying involving logging of persistent data for recovery
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Retry When Errors Occur (AREA)
Abstract
The application relates to a snapshot rollback method, a device, equipment and a storage medium of a distributed block storage volume. The method comprises the following steps: dividing the distributed block storage volume into a plurality of data fragments with the same size; constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment; performing full-scale caching on the bitmap unit in the distributed block storage volume; after the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot; when the snapshot rolls back, comparing the distributed block storage volume with bitmap unit data of the snapshot, and calculating a difference amount; and determining the storage unit position corresponding to the difference in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot. The application can improve rollback efficiency.
Description
Technical Field
The present application relates to the field of distributed block storage technologies, and in particular, to a method, an apparatus, a computer device, and a storage medium for rolling back a snapshot of a distributed block storage volume.
Background
In distributed block storage, snapshot is a common function, and the definition of Snapshot (snap shot) by storage network industry association SNIA (Storage Networking Industry Association) is: with respect to a fully available copy of a given data set, the copy includes an image of the corresponding data at some point in time (the point in time at which the copy began).
The snapshot may be a copy of the data it represents, or may be a copy of the data. The rollback of the snapshot is an operation of recovering a certain volume by using the data of the snapshot, the purpose of the operation is to recover the data of the volume to the data of a certain snapshot time point before, and the traditional method for rollback of the distributed block storage snapshot is to roll back each storage unit in the data set into the volume, so that the rollback of the whole snapshot is completed, and the rollback speed is slower.
Disclosure of Invention
Based on this, it is necessary to provide a distributed block storage volume snapshot rollback method, apparatus, computer device and storage medium capable of improving the rollback speed, aiming at the technical problem that the rollback speed of the distributed block storage snapshot is slow.
In one aspect, a method for snapshot rollback of a distributed block storage volume is provided, the method comprising:
dividing the distributed block storage volume into a plurality of data fragments with the same size;
Constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment;
Performing full-scale caching on the bitmap unit in the distributed block storage volume;
After the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot;
When the snapshot rolls back, comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating a difference amount;
And determining the storage unit position corresponding to the difference in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
In one embodiment, the step of dividing the distributed block storage volume into a plurality of equally sized data slices comprises:
Dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size, wherein each logic interval is used as a data fragment;
Setting each data fragment to contain a head label, a payload label and a foot label which are continuously stored;
setting the head label to store metadata information;
Setting the payload label for recording the state of the data fragment;
And setting the redundancy check codes of the head label and the redundancy check codes of the corresponding data fragments in the foot label according to the sequence.
In one embodiment, the distributed block storage volume snapshot rollback method further includes:
Setting the value of the payload label to comprise a first value, a second value, a third value and a fourth value;
when a target data fragment object does not exist, assigning the value of the payload label in the target data fragment to be a first value;
When a target data fragment object is created, assigning the value of the payload label in the target data fragment to be a second value;
When the target data fragment object is modified, the value of the payload label in the target data fragment is assigned to be a third value;
And when deleting the target data fragment object, assigning the value of the payload label in the target data fragment to be a fourth value.
In one embodiment, the distributed block storage volume snapshot rollback method further includes:
acquiring modified target data in response to a data change of the distributed block storage volume;
and modifying the value of the payload label corresponding to the modified target data.
In one embodiment, the step of modifying the value of the payload label corresponding to the modified target data includes:
Calculating redundancy check codes of the corresponding head labels according to the modified target data;
Comparing the calculated redundancy check code of the head tag with the redundancy check code recorded in the foot tag of the current data fragment to obtain the data fragment with the same redundancy check code;
Calculating redundancy check codes of the corresponding data fragments according to the modified target data;
Comparing the calculated redundancy check code of the data fragment with the redundancy check code recorded in the foot label of the current data fragment to obtain the data fragment with the same redundancy check code as the target fragment;
And acquiring the real-time state of the target fragment, and modifying the value of the corresponding payload label according to the real-time state of the target fragment.
In one embodiment, the dividing the distributed block storage volume into a plurality of logical intervals with the same size according to the preset block size includes:
the distributed block storage volume is partitioned into logical intervals of 4KB size, wherein the payload tag is less than 4KB.
In one embodiment, the step of comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot when the snapshot rolls back, and calculating the difference amount includes:
when the snapshot rolls back, bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot are obtained;
and comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference quantity of the numerical value of the payload label in the obtained distributed block storage volume.
In one embodiment, the determining a location of the storage unit in the distributed block storage volume corresponding to the difference amount, and the rolling back the storage unit with reference to the snapshot includes:
Determining redundancy check codes of data fragments in the distributed block storage volume according to the difference positions of the numerical values of the payload labels in the distributed block storage volume;
Positioning at least one storage unit with data difference in the distributed block storage volume according to the redundancy check code;
And rolling back each storage unit of the distributed block storage volume one by one according to the snapshot, and rolling back the data of the storage units into the data of the snapshot.
In one embodiment, after determining the storage unit location in the distributed block storage volume corresponding to the difference amount, performing a rollback step on the storage unit with reference to the snapshot further includes:
Acquiring data fragments corresponding to the bitmap units of the distributed block storage volumes and the snapshots thereof for data rollback;
Updating the data fragments, and recalculating the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments stored in the updated data fragments.
In one embodiment, the step of updating the data slice and recalculating the redundancy check code of the header tag and the redundancy check code of the corresponding data slice stored in the updated data slice includes:
updating the data fragments;
recalculating redundancy check codes of the head labels in the updated data fragments;
And updating the redundancy check code of the data fragment stored in the foot tag.
In one embodiment, the step of caching the bitmap unit in the distributed block storage volume includes:
monitoring whether the distributed block storage volume is read for the first time in response to reading the distributed block storage volume;
Responding to the distributed block storage volume for first reading, traversing the foot label in each data fragment of the distributed block storage volume, and acquiring the redundancy check code of the head label and the redundancy check code of the corresponding data fragment stored in the foot label;
The acquired redundancy check codes of the head labels and the redundancy check codes of all the data fragments are orderly arranged to form a redundancy check code list of the distributed block storage volume;
the redundancy check list is stored in the distributed block storage volume.
In one embodiment, the step of caching the bitmap unit in the distributed block storage volume in full further comprises:
and in response to the distributed block storage volume not being read for the first time, acquiring the redundancy check code list.
In another aspect, there is provided a distributed block storage volume snapshot rollback apparatus, the apparatus comprising:
the system comprises a slicing processing module, a data processing module and a data processing module, wherein the slicing processing module is used for dividing a distributed block storage volume into a plurality of data slices with the same size;
The bitmap unit management module is used for constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the state of each data fragment is recorded in the bitmap unit;
The loading bitmap unit control module is used for carrying out full-volume caching on the bitmap unit in the distributed block storage volume;
The snapshot management module is used for associating the distributed block storage volume with the snapshot after the distributed block storage volume builds the snapshot, and copying a current bitmap unit to be associated with the snapshot;
the difference amount calculating module is used for comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot when the snapshot rolls back, and calculating the difference amount;
and the rollback control module is used for determining the storage unit position corresponding to the difference quantity in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
In yet another aspect, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of:
dividing the distributed block storage volume into a plurality of data fragments with the same size;
Constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment;
Performing full-scale caching on the bitmap unit in the distributed block storage volume;
After the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot;
When the snapshot rolls back, comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating a difference amount;
And determining the storage unit position corresponding to the difference in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
In yet another aspect, a computer readable storage medium is provided, having stored thereon a computer program which when executed by a processor performs the steps of:
dividing the distributed block storage volume into a plurality of data fragments with the same size;
Constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment;
Performing full-scale caching on the bitmap unit in the distributed block storage volume;
After the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot;
When the snapshot rolls back, comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating a difference amount;
And determining the storage unit position corresponding to the difference in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
According to the snapshot rollback method, the device, the computer equipment and the storage medium for the distributed block storage volume, the distributed block storage volume is divided into the plurality of data fragments with the same size, so that object lock conflict is reduced, data rollback is realized through refined management, rollback speed is improved, in addition, bitmap units of the distributed block storage volume are subjected to full-quantity caching in the distributed block storage volume, except for first loading, disk reading can be avoided subsequently, read-before-write expense can be reduced, and rollback speed is improved. Compared with the current data organization method, the method can effectively improve snapshot rollback efficiency and improve usability of the distributed storage system.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a prior art distributed block storage volume divided into a plurality of data slices;
FIG. 2 is a diagram of an application environment for a distributed block storage volume snapshot rollback method in accordance with one embodiment of the present application;
FIG. 3 is a flow chart of a method for snapshot rollback of a distributed block storage volume according to one embodiment of the present application;
FIG. 4 is a flowchart illustrating a step of dividing a distributed block storage volume into a plurality of data slices of the same size according to an embodiment of the present application;
FIG. 5 is a flowchart illustrating a full caching process of the bitmap unit in the distributed block storage volume according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a distributed block storage volume divided into a plurality of data slices in one embodiment of the application;
FIG. 7 is a flow chart of the steps of comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot and calculating a difference amount when the snapshot is rolled back in one embodiment of the present application;
FIG. 8 is a flowchart of a rollback procedure for determining storage locations in the distributed block storage volume corresponding to the difference amount with reference to the snapshot in accordance with one embodiment of the present application;
FIG. 9 is a block diagram of a distributed block storage volume snapshot rollback apparatus in accordance with one embodiment of the present application;
Fig. 10 is an internal structural view of a computer device according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The existing ObjectMap is used for recording the state of each fragment of the volume, if the part of data of the volume to be queried changes, only the changed data fragment objects marked as the changed data fragment objects need to be traversed and processed according to the ObjectMap, so that the processing efficiency is improved, and especially when the number of the changed data fragment objects is small; and the volume statistics operation can be completed even by traversing the content of the ObjectMap.
In the current snapshot rollback method of the distributed block storage volume, the ObjectMap currently adopts a bitmap format, and the content of the whole ObjectMap object is divided into three parts: head, body (i.e., payload), and boot, wherein the body portion is divided into 1 to multiple slices in a 4KB size alignment. Wherein: the header stores metadata information such as the size of the bitmap object, and the sender sequentially stores a CRC32 check code of the header and a CRC32 check code of each fragment, as shown in FIG. 1.
Wherein each data slice object corresponds to two bits, namely obj_bits, in its order of order index (obj_idx) in the volume over the entire bitmap space.
One persistent update to ObjectMap typically involves only one data shard object, the main processing of which is described as follows:
1) Reading three parts
Because of the discontinuity, the three parts of the ascription bitmap fragment of header, footer and the obj_bits of the data fragment object need to be read sequentially and respectively.
2) CRC32 check
A) Checking a header: the CRC32 check code of the header is calculated and compared with the record in the boot.
B) Checking bitmap fragmentation: the CRC32 check code of the fragment is calculated and compared with the record in the footer.
3) Preparing the content of the next disc
A) And updating the obj_bits of the data fragment object in the bitmap fragment.
B) And recalculating CRC32 check codes of the updated bitmap slices.
C) And updating CRC32 check codes corresponding to the bitmap fragments in the footer.
4) Writing three parts
Also because of the discontinuity, three parts need to be written one after the other: header, footer and home bitmap slices.
As described above, even if obj_bits of only one data slice object are changed, three reads and three writes are involved, and the performance is low and the consumption of the disk performance is high.
The snapshot rollback method of the distributed block storage volume provided by the application can be applied to an application environment as shown in fig. 2. Wherein the distributed block storage volume 101 implements a storage system that provides storage space in the form of volumes in distributed storage as storage nodes. In a distributed storage system, one way of partitioning resources, multiple volumes may be created in the distributed system. In a distributed storage system, the occupation of storage is divided according to storage units, and each storage unit has the same size. Snapshot 102 of distributed block storage volume 101 is associated with distributed block storage volume 101.
Example 1
In embodiment 1, as shown in fig. 3, there is provided a distributed block storage volume snapshot rollback method, including the steps of:
step S1, dividing a distributed block storage volume into a plurality of data fragments with the same size;
Step S2, constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment;
Step S3, carrying out full-quantity caching on the bitmap unit in the distributed block storage volume;
step S4, after the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot;
Step S5, when the snapshot rolls back, comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating a difference amount;
and S6, determining the storage unit position corresponding to the difference in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
The faster the volume snapshot rollback speed, the smaller the duration of the impact on the reels, the shorter the duration of the process, and the less impact on the business. According to the application, the distributed block storage volume is divided into a plurality of data fragments with the same size, so that the conflict of object locks is reduced, the data rollback by refinement management is realized, the rollback speed is improved, in addition, the bitmap units of the distributed block storage volume are subjected to full-scale caching in the distributed block storage volume, besides the first loading, the subsequent disk reading can be avoided, the read-before-write expense can be reduced, and the rollback speed is improved. Compared with the current data organization method, the method can effectively improve snapshot rollback efficiency and improve usability of the distributed storage system.
When the distributed block storage volume is loaded for the first time, the bitmap unit is cached in the distributed block storage volume in full, so that when the redundancy check codes in the data fragments are acquired later, the distributed block storage volume is not required to be read, and the bitmap unit can be obtained only by loading the cache, thereby reducing the read-before-write expense and improving the rollback speed.
As shown in fig. 4, in this embodiment, the step of dividing the distributed block storage volume into a plurality of data slices with the same size includes:
step S11, dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size, wherein each logic interval is used as a data fragment;
step S12, setting a head label (header), a payload label (body) and a foot label (footer) which are continuously stored in each data fragment;
Step S13, setting the head label to store metadata information;
step S14, setting the payload label for recording the state of the data fragment;
And S15, setting the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments to be stored in the foot labels in sequence.
In this embodiment, the distributed block storage volume snapshot rollback method further includes:
Setting the value of the payload label to comprise a first value, a second value, a third value and a fourth value;
when a target data fragment object does not exist, assigning the value of the payload label in the target data fragment to be a first value;
When a target data fragment object is created, assigning the value of the payload label in the target data fragment to be a second value;
When the target data fragment object is modified, the value of the payload label in the target data fragment is assigned to be a third value;
And when deleting the target data fragment object, assigning the value of the payload label in the target data fragment to be a fourth value.
In this embodiment, the distributed block storage volume snapshot rollback method further includes:
acquiring modified target data in response to a data change of the distributed block storage volume;
and modifying the value of the payload label corresponding to the modified target data.
In this embodiment, the step of modifying the value of the payload label corresponding to the modified target data includes:
Calculating redundancy check codes of the corresponding head labels according to the modified target data;
Comparing the calculated redundancy check code of the head tag with the redundancy check code recorded in the foot tag of the current data fragment to obtain the data fragment with the same redundancy check code;
Calculating redundancy check codes of the corresponding data fragments according to the modified target data;
Comparing the calculated redundancy check code of the data fragment with the redundancy check code recorded in the foot label of the current data fragment to obtain the data fragment with the same redundancy check code as the target fragment;
And acquiring the real-time state of the target fragment, and modifying the value of the corresponding payload label according to the real-time state of the target fragment.
In this embodiment, the dividing the distributed block storage volume into a plurality of logical intervals with the same size according to the preset block size includes:
the distributed block storage volume is partitioned into logical intervals of 4KB size, wherein the payload tag is less than 4KB.
The bitmap object is split according to a certain size (128 KB), the split bitmap object can be hashed to different PGs, and the conflict degree of PG locks can be reduced. Examples: 1 volume of 4TB size contains 1M data slice objects, each data slice object is calculated according to 4bits (3 bits are actually needed at present), objectmap total size is 4Mbits, namely 512KB size, and 4 objects can be split according to 128 KB.
Dividing fragments in a bitmap object based on the size of a minimum allocation unit 4KB of a store layer: each slice contains header, body, footer parts stored consecutively, with a total size of 4KB. It should be noted that the body (payload) portion in each slice is slightly smaller than 4KB in size, i.e., the logically entire bitmap payload is not sliced in an entire 4KB alignment.
As shown in fig. 5, the step of performing full caching on the bitmap unit in the distributed block storage volume includes:
step S31, monitoring whether the distributed block storage volume is read for the first time or not in response to the reading of the distributed block storage volume;
Step S32, responding to the first reading of the distributed block storage volume, traversing the foot label in each data fragment of the distributed block storage volume, and acquiring the redundancy check code of the head label and the redundancy check code of the corresponding data fragment stored in the foot label;
step S33, the obtained redundancy check codes of the head labels and the redundancy check codes of all data fragments are orderly arranged to form a redundancy check code list of the distributed block storage volume;
step S34, storing the redundancy check code list in the distributed block storage volume;
and step S35, acquiring the redundancy check code list in response to the distributed block storage volume not being read for the first time.
As shown in FIG. 6, FIG. 6 is a schematic diagram of a set-up data shard of the distributed block storage volume snapshot rollback method of the present application. The schematic diagram of the division of the distributed block storage volume before optimization into multiple data slices is shown in fig. 1, and in the snapshot rollback process, the difference amount is compared with the volume with the size of 4G, which takes 1 second. The schematic diagram of the division of the optimized distributed block storage volume into a plurality of data fragments is shown in fig. 6, and in the snapshot rollback process, the difference quantity is compared with the volume with the size of 4G, which takes 0.5 seconds, and the improvement is 100%.
As shown in fig. 7, in the present embodiment, when the snapshot rolls back, the step of comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating the difference amount includes:
step S51, when the snapshot rolls back, bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot are obtained;
Step S52, comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference amount of the values of the payload labels in the obtained distributed block storage volume.
As shown in fig. 8, in this embodiment, the determining a location of a storage unit in the distributed block storage volume corresponding to the difference amount, and performing a rollback on the storage unit with reference to the snapshot includes:
Step S61, determining redundancy check codes of data fragments in the distributed block storage volume according to the difference amount positions of the numerical values of the payload labels in the distributed block storage volume;
Step S62, positioning at least one storage unit with data difference in the distributed block storage volume according to the redundancy check code;
And step S63, rolling back each storage unit of the distributed block storage volume one by one according to the snapshot, and rolling back the data of the storage unit into the data of the snapshot.
The redundancy check code is preferably a CRC32 check code. CRC32 CRC itself means "redundancy check", CRC32 means that a 32bit (8 bit hexadecimal number) check value is produced. Since each bit of the source data block participates in the calculation when the CRC32 generates the check value, a different CRC32 value is obtained even if only one bit of the data block is changed.
And if the number of the data fragments is N, and the number of the storage units corresponding to the difference in the distributed block storage volumes is M, M is less than or equal to N.
In this embodiment, after determining the storage unit location corresponding to the difference amount in the distributed block storage volume, performing a rollback step on the storage unit with reference to the snapshot further includes:
Acquiring data fragments corresponding to the bitmap units of the distributed block storage volumes and the snapshots thereof for data rollback;
Updating the data fragments, and recalculating the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments stored in the updated data fragments.
The step of updating the data fragments and recalculating the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments stored in the updated data fragments comprises the following steps:
updating the data fragments;
recalculating redundancy check codes of the head labels in the updated data fragments;
And updating the redundancy check code of the data fragment stored in the foot tag.
In the snapshot rollback method of the distributed block storage volume, the distributed block storage volume is divided into a plurality of data fragments with the same size, so that object lock conflict is reduced, data rollback is realized by fine management, rollback speed is improved, in addition, bitmap units of the distributed block storage volume are subjected to full-quantity caching in the distributed block storage volume, and besides the first loading, disk reading can be avoided subsequently, read-before-write expense can be reduced, and rollback speed is improved. Compared with the current data organization method, the method can effectively improve snapshot rollback efficiency and improve usability of the distributed storage system.
It should be understood that, although the steps in the flowcharts of fig. 3 to 8 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps of fig. 3-8 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor does the order in which the sub-steps or stages are performed necessarily occur in sequence, but may be performed alternately or alternately with at least a portion of other steps or sub-steps or stages of other steps.
Example 2
In embodiment 2, as shown in fig. 9, there is provided a distributed block storage volume snapshot rollback apparatus 10 including: the system comprises a fragment processing module 1, a bitmap unit management module 2, a loading bitmap unit control module 3, a snapshot management module 4, a difference amount calculation module 5 and a rollback control module 6.
The slicing processing module 1 is configured to divide a distributed block storage volume into a plurality of data slices with the same size.
The bitmap unit management module 2 is configured to construct a bitmap unit of the distributed block storage volume, where the bitmap unit and the data fragments form a mapping relationship, and a state of each data fragment is recorded in the bitmap unit.
The loading bitmap unit control module 3 is configured to perform full-volume caching on the bitmap unit in the distributed block storage volume.
The snapshot management module 4 is configured to associate the distributed block storage volume with the snapshot after the snapshot is constructed by the distributed block storage volume, and copy a current bitmap unit to associate the current bitmap unit with the snapshot.
The difference amount calculation module 5 is configured to compare bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot when the snapshot rolls back, and calculate a difference amount.
The rollback control module 6 is configured to determine a location of a storage unit in the distributed block storage volume corresponding to the difference amount, and rollback the storage unit with reference to the snapshot.
In this embodiment, the step of dividing the distributed block storage volume into a plurality of data slices having the same size includes:
Dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size, wherein each logic interval is used as a data fragment;
Setting each data fragment to contain a head label, a payload label and a foot label which are continuously stored;
setting the head label to store metadata information;
Setting the payload label for recording the state of the data fragment;
And setting the redundancy check codes of the head label and the redundancy check codes of the corresponding data fragments in the foot label according to the sequence.
In this embodiment, the distributed block storage volume snapshot rollback method further includes:
Setting the value of the payload label to comprise a first value, a second value, a third value and a fourth value;
when a target data fragment object does not exist, assigning the value of the payload label in the target data fragment to be a first value;
When a target data fragment object is created, assigning the value of the payload label in the target data fragment to be a second value;
When the target data fragment object is modified, the value of the payload label in the target data fragment is assigned to be a third value;
And when deleting the target data fragment object, assigning the value of the payload label in the target data fragment to be a fourth value.
The distributed block storage volume snapshot rollback method further comprises the following steps:
acquiring modified target data in response to a data change of the distributed block storage volume;
and modifying the value of the payload label corresponding to the modified target data.
The step of modifying the value of the payload label corresponding to the modified target data comprises the following steps:
Calculating redundancy check codes of the corresponding head labels according to the modified target data;
Comparing the calculated redundancy check code of the head tag with the redundancy check code recorded in the foot tag of the current data fragment to obtain the data fragment with the same redundancy check code;
Calculating redundancy check codes of the corresponding data fragments according to the modified target data;
Comparing the calculated redundancy check code of the data fragment with the redundancy check code recorded in the foot label of the current data fragment to obtain the data fragment with the same redundancy check code as the target fragment;
And acquiring the real-time state of the target fragment, and modifying the value of the corresponding payload label according to the real-time state of the target fragment.
In this embodiment, the dividing the distributed block storage volume into a plurality of logical intervals with the same size according to the preset block size includes:
the distributed block storage volume is partitioned into logical intervals of 4KB size, wherein the payload tag is less than 4KB.
In this embodiment, when the snapshot rolls back, the step of comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating the difference amount includes:
when the snapshot rolls back, bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot are obtained;
and comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference quantity of the numerical value of the payload label in the obtained distributed block storage volume.
In this embodiment, the determining a location of the storage unit in the distributed block storage volume corresponding to the difference amount, and rolling back the storage unit with reference to the snapshot includes:
Determining redundancy check codes of data fragments in the distributed block storage volume according to the difference positions of the numerical values of the payload labels in the distributed block storage volume;
Positioning at least one storage unit with data difference in the distributed block storage volume according to the redundancy check code;
And rolling back each storage unit of the distributed block storage volume one by one according to the snapshot, and rolling back the data of the storage units into the data of the snapshot.
In this embodiment, after determining the storage unit location corresponding to the difference amount in the distributed block storage volume, performing a rollback step on the storage unit with reference to the snapshot further includes:
Acquiring data fragments corresponding to the bitmap units of the distributed block storage volumes and the snapshots thereof for data rollback;
Updating the data fragments, and recalculating the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments stored in the updated data fragments.
In one embodiment, the step of updating the data slice and recalculating the redundancy check code of the header tag and the redundancy check code of the corresponding data slice stored in the updated data slice includes:
updating the data fragments;
recalculating redundancy check codes of the head labels in the updated data fragments;
And updating the redundancy check code of the data fragment stored in the foot tag.
In the snapshot rollback device for the distributed block storage volume, the distributed block storage volume is divided into a plurality of data fragments with the same size, so that object lock conflict is reduced, data rollback is realized by fine management, rollback speed is improved, in addition, bitmap units of the distributed block storage volume are subjected to full-quantity caching in the distributed block storage volume, and besides the first loading, disk reading can be avoided subsequently, read-before-write expense can be reduced, and rollback speed is improved. Compared with the current data organization method, the method can effectively improve snapshot rollback efficiency and improve usability of the distributed storage system.
For specific limitations on the distributed block storage volume snapshot rollback apparatus, reference may be made to the above limitations on the distributed block storage volume snapshot rollback method, and no further description is given here. The various modules in the distributed block storage volume snapshot rollback apparatus described above may be implemented in whole or in part in software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
Example 3
In embodiment 3, there is provided a computer device which may be a server, and an internal structural diagram thereof may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing distributed block storage volume snapshot rollback data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a distributed block storage volume snapshot rollback method.
It will be appreciated by those skilled in the art that the structure shown in FIG. 10 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of when executing the computer program:
dividing the distributed block storage volume into a plurality of data fragments with the same size;
Constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment;
Performing full-scale caching on the bitmap unit in the distributed block storage volume;
After the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot;
When the snapshot rolls back, comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating a difference amount;
And determining the storage unit position corresponding to the difference in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
In one embodiment, the processor when executing the computer program further performs the steps of:
the step of dividing the distributed block storage volume into a plurality of data slices of the same size includes:
Dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size, wherein each logic interval is used as a data fragment;
Setting each data fragment to contain a head label, a payload label and a foot label which are continuously stored;
setting the head label to store metadata information;
Setting the payload label for recording the state of the data fragment;
And setting the redundancy check codes of the head label and the redundancy check codes of the corresponding data fragments in the foot label according to the sequence.
In one embodiment, the processor when executing the computer program further performs the steps of:
The distributed block storage volume snapshot rollback method further comprises the following steps:
Setting the value of the payload label to comprise a first value, a second value, a third value and a fourth value;
when a target data fragment object does not exist, assigning the value of the payload label in the target data fragment to be a first value;
When a target data fragment object is created, assigning the value of the payload label in the target data fragment to be a second value;
When the target data fragment object is modified, the value of the payload label in the target data fragment is assigned to be a third value;
And when deleting the target data fragment object, assigning the value of the payload label in the target data fragment to be a fourth value.
In one embodiment, the distributed block storage volume snapshot rollback method further includes:
acquiring modified target data in response to a data change of the distributed block storage volume;
and modifying the value of the payload label corresponding to the modified target data.
In one embodiment, the step of modifying the value of the payload label corresponding to the modified target data includes:
Calculating redundancy check codes of the corresponding head labels according to the modified target data;
Comparing the calculated redundancy check code of the head tag with the redundancy check code recorded in the foot tag of the current data fragment to obtain the data fragment with the same redundancy check code;
Calculating redundancy check codes of the corresponding data fragments according to the modified target data;
Comparing the calculated redundancy check code of the data fragment with the redundancy check code recorded in the foot label of the current data fragment to obtain the data fragment with the same redundancy check code as the target fragment;
And acquiring the real-time state of the target fragment, and modifying the value of the corresponding payload label according to the real-time state of the target fragment.
In one embodiment, the processor when executing the computer program further performs the steps of:
the dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size comprises the following steps:
the distributed block storage volume is partitioned into logical intervals of 4KB size, wherein the payload tag is less than 4KB.
In one embodiment, the processor when executing the computer program further performs the steps of:
The step of comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot and calculating a difference amount when the snapshot rolls back includes:
when the snapshot rolls back, bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot are obtained;
and comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference quantity of the numerical value of the payload label in the obtained distributed block storage volume.
In one embodiment, the processor when executing the computer program further performs the steps of:
the determining a storage unit position corresponding to the difference in the distributed block storage volume, and the rolling back the storage unit with reference to the snapshot includes:
Determining redundancy check codes of data fragments in the distributed block storage volume according to the difference positions of the numerical values of the payload labels in the distributed block storage volume;
Positioning at least one storage unit with data difference in the distributed block storage volume according to the redundancy check code;
And rolling back each storage unit of the distributed block storage volume one by one according to the snapshot, and rolling back the data of the storage units into the data of the snapshot.
In one embodiment, the processor when executing the computer program further performs the steps of:
after the determining the storage unit position corresponding to the difference in the distributed block storage volume, performing a rollback step on the storage unit with reference to the snapshot further includes:
Acquiring data fragments corresponding to the bitmap units of the distributed block storage volumes and the snapshots thereof for data rollback;
Updating the data fragments, and recalculating the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments stored in the updated data fragments.
In one embodiment, the step of updating the data slice and recalculating the redundancy check code of the header tag and the redundancy check code of the corresponding data slice stored in the updated data slice includes:
updating the data fragments;
recalculating redundancy check codes of the head labels in the updated data fragments;
And updating the redundancy check code of the data fragment stored in the foot tag.
Specific limitations regarding implementation steps of the processor when executing the computer program may be found in the above limitations of the method for snapshot rollback of a distributed block storage volume, and are not described in detail herein.
Example 4
In embodiment 4, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
dividing the distributed block storage volume into a plurality of data fragments with the same size;
Constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment;
Performing full-scale caching on the bitmap unit in the distributed block storage volume;
After the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot;
When the snapshot rolls back, comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating a difference amount;
And determining the storage unit position corresponding to the difference in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
In one embodiment, the computer program when executed by the processor further performs the steps of:
the step of dividing the distributed block storage volume into a plurality of data slices of the same size includes:
Dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size, wherein each logic interval is used as a data fragment;
Setting each data fragment to contain a head label, a payload label and a foot label which are continuously stored;
setting the head label to store metadata information;
Setting the payload label for recording the state of the data fragment;
And setting the redundancy check codes of the head label and the redundancy check codes of the corresponding data fragments in the foot label according to the sequence.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The distributed block storage volume snapshot rollback method further comprises the following steps:
Setting the value of the payload label to comprise a first value, a second value, a third value and a fourth value;
when a target data fragment object does not exist, assigning the value of the payload label in the target data fragment to be a first value;
When a target data fragment object is created, assigning the value of the payload label in the target data fragment to be a second value;
When the target data fragment object is modified, the value of the payload label in the target data fragment is assigned to be a third value;
And when deleting the target data fragment object, assigning the value of the payload label in the target data fragment to be a fourth value.
In one embodiment, the distributed block storage volume snapshot rollback method further includes:
acquiring modified target data in response to a data change of the distributed block storage volume;
and modifying the value of the payload label corresponding to the modified target data.
In one embodiment, the step of modifying the value of the payload label corresponding to the modified target data includes:
Calculating redundancy check codes of the corresponding head labels according to the modified target data;
Comparing the calculated redundancy check code of the head tag with the redundancy check code recorded in the foot tag of the current data fragment to obtain the data fragment with the same redundancy check code;
Calculating redundancy check codes of the corresponding data fragments according to the modified target data;
Comparing the calculated redundancy check code of the data fragment with the redundancy check code recorded in the foot label of the current data fragment to obtain the data fragment with the same redundancy check code as the target fragment;
And acquiring the real-time state of the target fragment, and modifying the value of the corresponding payload label according to the real-time state of the target fragment.
In one embodiment, the computer program when executed by the processor further performs the steps of:
the dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size comprises the following steps:
the distributed block storage volume is partitioned into logical intervals of 4KB size, wherein the payload tag is less than 4KB.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The step of comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot and calculating a difference amount when the snapshot rolls back includes:
when the snapshot rolls back, bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot are obtained;
and comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference quantity of the numerical value of the payload label in the obtained distributed block storage volume.
In one embodiment, the computer program when executed by the processor further performs the steps of:
the determining a storage unit position corresponding to the difference in the distributed block storage volume, and the rolling back the storage unit with reference to the snapshot includes:
Determining redundancy check codes of data fragments in the distributed block storage volume according to the difference positions of the numerical values of the payload labels in the distributed block storage volume;
Positioning at least one storage unit with data difference in the distributed block storage volume according to the redundancy check code;
And rolling back each storage unit of the distributed block storage volume one by one according to the snapshot, and rolling back the data of the storage units into the data of the snapshot.
In one embodiment, the computer program when executed by the processor further performs the steps of:
after the determining the storage unit position corresponding to the difference in the distributed block storage volume, performing a rollback step on the storage unit with reference to the snapshot further includes:
Acquiring data fragments corresponding to the bitmap units of the distributed block storage volumes and the snapshots thereof for data rollback;
Updating the data fragments, and recalculating the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments stored in the updated data fragments.
In one embodiment, the step of updating the data slice and recalculating the redundancy check code of the header tag and the redundancy check code of the corresponding data slice stored in the updated data slice includes:
updating the data fragments;
recalculating redundancy check codes of the head labels in the updated data fragments;
And updating the redundancy check code of the data fragment stored in the foot tag.
For specific limitations regarding implementation steps when the computer program is executed by a processor, reference may be made to the above limitation of the method for snapshot rollback of a distributed block storage volume, which is not repeated here.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (14)
1. A method for snapshot rollback of a distributed block storage volume, comprising:
dividing the distributed block storage volume into a plurality of data fragments with the same size;
Constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the bitmap unit records the state of each data fragment;
Performing full-scale caching on the bitmap unit in the distributed block storage volume;
After the distributed block storage volume builds a snapshot, associating the distributed block storage volume with the snapshot, and copying a current bitmap unit to associate with the snapshot;
When the snapshot rolls back, comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot, and calculating a difference amount;
determining storage unit positions corresponding to the difference amounts in the distributed block storage volumes, and rolling back the storage units by referring to the snapshots;
the step of dividing the distributed block storage volume into a plurality of data fragments with the same size comprises the following steps:
Dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size, wherein each logic interval is used as a data fragment;
Setting each data fragment to contain a head label, a payload label and a foot label which are continuously stored;
Setting the payload label for recording the state of the data fragment;
Wherein when the snapshot rolls back, comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference amount comprises:
when the snapshot rolls back, bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot are obtained;
and comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference quantity of the numerical value of the payload label in the obtained distributed block storage volume.
2. The method of claim 1, wherein the step of dividing the distributed block storage volume into a plurality of equally sized data slices comprises:
setting the head label to store metadata information;
And setting the redundancy check codes of the head label and the redundancy check codes of the corresponding data fragments in the foot label according to the sequence.
3. The distributed block storage volume snapshot rollback method of claim 2, further comprising:
Setting the value of the payload label to comprise a first value, a second value, a third value and a fourth value;
when a target data fragment object does not exist, assigning the value of the payload label in the target data fragment to be a first value;
When a target data fragment object is created, assigning the value of the payload label in the target data fragment to be a second value;
When the target data fragment object is modified, the value of the payload label in the target data fragment is assigned to be a third value;
And when deleting the target data fragment object, assigning the value of the payload label in the target data fragment to be a fourth value.
4. The distributed block storage volume snapshot rollback method of claim 3, further comprising:
acquiring modified target data in response to a data change of the distributed block storage volume;
and modifying the value of the payload label corresponding to the modified target data.
5. The method of claim 4, wherein the modifying the value of the payload label corresponding to the modified target data comprises:
Calculating redundancy check codes of the corresponding head labels according to the modified target data;
Comparing the calculated redundancy check code of the head tag with the redundancy check code recorded in the foot tag of the current data fragment to obtain the data fragment with the same redundancy check code;
Calculating redundancy check codes of the corresponding data fragments according to the modified target data;
Comparing the calculated redundancy check code of the data fragment with the redundancy check code recorded in the foot label of the current data fragment to obtain the data fragment with the same redundancy check code as the target fragment;
And acquiring the real-time state of the target fragment, and modifying the value of the corresponding payload label according to the real-time state of the target fragment.
6. The method of claim 1, wherein dividing the distributed block storage volume into a plurality of logical extents of the same size according to a preset block size comprises:
the distributed block storage volume is partitioned into logical intervals of 4KB size, wherein the payload tag is less than 4KB.
7. The method of claim 1, wherein determining a location of a storage unit in the distributed block storage volume corresponding to the difference amount, the step of rollback the storage unit with reference to the snapshot comprises:
Determining redundancy check codes of data fragments in the distributed block storage volume according to the difference positions of the numerical values of the payload labels in the distributed block storage volume;
Positioning at least one storage unit with data difference in the distributed block storage volume according to the redundancy check code;
And rolling back each storage unit of the distributed block storage volume one by one according to the snapshot, and rolling back the data of the storage units into the data of the snapshot.
8. The method of claim 7, wherein after determining the location of the storage unit in the distributed block storage volume corresponding to the difference amount, performing the rollback step on the storage unit with reference to the snapshot further comprises:
Acquiring data fragments corresponding to the bitmap units of the distributed block storage volumes and the snapshots thereof for data rollback;
Updating the data fragments, and recalculating the redundancy check codes of the head labels and the redundancy check codes of the corresponding data fragments stored in the updated data fragments.
9. The method according to claim 8, wherein the step of updating the data fragments and re-calculating redundancy check codes of the header tag and redundancy check codes of the corresponding data fragments stored in the updated data fragments comprises:
updating the data fragments;
recalculating redundancy check codes of the head labels in the updated data fragments;
And updating the redundancy check code of the data fragment stored in the foot tag.
10. The distributed block storage volume snapshot rollback method of claim 2, wherein said step of full caching said bitmap unit in said distributed block storage volume comprises:
monitoring whether the distributed block storage volume is read for the first time in response to reading the distributed block storage volume;
Responding to the distributed block storage volume for first reading, traversing the foot label in each data fragment of the distributed block storage volume, and acquiring the redundancy check code of the head label and the redundancy check code of the corresponding data fragment stored in the foot label;
The acquired redundancy check codes of the head labels and the redundancy check codes of all the data fragments are orderly arranged to form a redundancy check code list of the distributed block storage volume;
the redundancy check list is stored in the distributed block storage volume.
11. The distributed block storage volume snapshot rollback method of claim 10, wherein the step of full caching the bitmap unit in the distributed block storage volume further comprises:
and in response to the distributed block storage volume not being read for the first time, acquiring the redundancy check code list.
12. A distributed block storage volume snapshot rollback apparatus, the apparatus comprising:
The system comprises a slicing processing module, a data processing module and a data processing module, wherein the slicing processing module is used for dividing a distributed block storage volume into a plurality of data slices with the same size; the step of dividing the distributed block storage volume into a plurality of data fragments with the same size comprises the following steps: dividing the distributed block storage volume into a plurality of logic intervals with the same size according to the preset block size, wherein each logic interval is used as a data fragment; setting each data fragment to contain a head label, a payload label and a foot label which are continuously stored; setting the payload label for recording the state of the data fragment;
The bitmap unit management module is used for constructing a bitmap unit of the distributed block storage volume, wherein the bitmap unit and the data fragments form a mapping relation, and the state of each data fragment is recorded in the bitmap unit;
The loading bitmap unit control module is used for carrying out full-volume caching on the bitmap unit in the distributed block storage volume;
The snapshot management module is used for associating the distributed block storage volume with the snapshot after the distributed block storage volume builds the snapshot, and copying a current bitmap unit to be associated with the snapshot;
The difference amount calculating module is used for comparing bitmap unit data of the distributed block storage volume with bitmap unit data of the snapshot when the snapshot rolls back, and calculating the difference amount; wherein when the snapshot rolls back, comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference amount comprises: when the snapshot rolls back, bitmap unit data of the distributed block storage volume and bitmap unit data of the snapshot are obtained; comparing the bitmap unit data of the distributed block storage volume with the bitmap unit data of the snapshot, and calculating the difference quantity of the numerical value of the payload label in the distributed block storage volume;
and the rollback control module is used for determining the storage unit position corresponding to the difference quantity in the distributed block storage volume, and rolling back the storage unit by referring to the snapshot.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 11 when the computer program is executed by the processor.
14. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410473071.7A CN118069432B (en) | 2024-04-19 | 2024-04-19 | Method, device, equipment and storage medium for snapshot rollback of distributed block storage volume |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410473071.7A CN118069432B (en) | 2024-04-19 | 2024-04-19 | Method, device, equipment and storage medium for snapshot rollback of distributed block storage volume |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118069432A CN118069432A (en) | 2024-05-24 |
| CN118069432B true CN118069432B (en) | 2024-07-05 |
Family
ID=91108065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410473071.7A Active CN118069432B (en) | 2024-04-19 | 2024-04-19 | Method, device, equipment and storage medium for snapshot rollback of distributed block storage volume |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118069432B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110018996A (en) * | 2018-07-23 | 2019-07-16 | 郑州云海信息技术有限公司 | A kind of the snapshot rollback method and relevant apparatus of distributed memory system |
| CN111552437A (en) * | 2020-04-22 | 2020-08-18 | 上海天玑科技股份有限公司 | Snapshot method and snapshot device applied to distributed storage system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108271420B (en) * | 2016-11-02 | 2020-11-27 | 华为技术有限公司 | Method for managing files, file system and server system |
| CN108460045B (en) * | 2017-02-21 | 2022-11-18 | 中兴通讯股份有限公司 | Snapshot processing method and distributed block storage system |
| US10503427B2 (en) * | 2017-03-10 | 2019-12-10 | Pure Storage, Inc. | Synchronously replicating datasets and other managed objects to cloud-based storage systems |
| CN111901398B (en) * | 2020-07-10 | 2022-12-16 | 苏州浪潮智能科技有限公司 | Method and device for distributed block storage synchronization remote copy full-quantity synchronization |
| CN113010496B (en) * | 2021-03-19 | 2024-03-08 | 腾讯云计算（北京）有限责任公司 | Data migration method, device, equipment and storage medium |
| CN114741236A (en) * | 2022-04-12 | 2022-07-12 | 杭州海康威视系统技术有限公司 | Data storage method and data query method based on snapshot |
| CN115391341A (en) * | 2022-08-23 | 2022-11-25 | 抖音视界有限公司 | Distributed graph data processing system, method, device, equipment and storage medium |
| CN117093540A (en) * | 2023-08-09 | 2023-11-21 | 苏州浪潮智能科技有限公司 | Snapshot creation method and device of file system, electronic equipment and storage medium |
-
2024
- 2024-04-19 CN CN202410473071.7A patent/CN118069432B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110018996A (en) * | 2018-07-23 | 2019-07-16 | 郑州云海信息技术有限公司 | A kind of the snapshot rollback method and relevant apparatus of distributed memory system |
| CN111552437A (en) * | 2020-04-22 | 2020-08-18 | 上海天玑科技股份有限公司 | Snapshot method and snapshot device applied to distributed storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118069432A (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10303596B2 (en) | Read-write control method for memory, and corresponding memory and server | |
| CN108459826B (en) | Method and device for processing IO (input/output) request | |
| US10877898B2 (en) | Method and system for enhancing flash translation layer mapping flexibility for performance and lifespan improvements | |
| CN110568998B (en) | Trim command implementation method and device based on solid state disk and computer equipment | |
| US20180129429A1 (en) | Method and apparatus for writing data into cache | |
| EP3352071B1 (en) | Data check method and storage system | |
| US20120136842A1 (en) | Partitioning method of data blocks | |
| US10942811B2 (en) | Data processing method for solid state drive | |
| CN108255414B (en) | Solid state disk access method and device | |
| CN115639970B (en) | Storage virtualization method, system, device and equipment based on disk array card | |
| CN114253483B (en) | Command-based write cache management method and device, computer equipment and storage medium | |
| CN113608701A (en) | Data management method in storage system and solid state disk | |
| CN110780806B (en) | Method and system for facilitating atomicity guarantee for metadata and data bundled storage | |
| CN110569000A (en) | Host RAID (redundant array of independent disk) management method and device based on solid state disk array | |
| CN111026678B (en) | Cache design method and device based on solid state disk and computer equipment | |
| CN118069432B (en) | Method, device, equipment and storage medium for snapshot rollback of distributed block storage volume | |
| CN114115745B (en) | RAID optimization method and device for multi-Pass programming NAND and computer equipment | |
| WO2023245942A1 (en) | Ssd finite window data deduplication identification method and apparatus, and computer device | |
| CN114816228A (en) | Data processing method, device, server and storage medium | |
| US11579789B2 (en) | Techniques for managing context information for a storage device | |
| US12073081B2 (en) | Priority based raid (redundant array of independent disks) rebuild for a data storage system | |
| CN118747061A (en) | Data processing method, device, computer equipment and storage medium | |
| CN117215503B (en) | Method for reading flash memory data | |
| CN110888823B (en) | Page scanning efficiency improving method and device and computer equipment | |
| US11442929B2 (en) | Double header logging of slotted page structures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |