CN112148530A - Data timing volume level backup method and system thereof - Google Patents
Data timing volume level backup method and system thereof Download PDFInfo
- Publication number
- CN112148530A CN112148530A CN202011043723.1A CN202011043723A CN112148530A CN 112148530 A CN112148530 A CN 112148530A CN 202011043723 A CN202011043723 A CN 202011043723A CN 112148530 A CN112148530 A CN 112148530A
- Authority
- CN
- China
- Prior art keywords
- volume
- backup
- snapshot
- full
- bitmap
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000000977 initiatory effect Effects 0.000 claims abstract description 13
- 238000012216 screening Methods 0.000 claims abstract description 9
- 238000012544 monitoring process Methods 0.000 claims description 40
- 238000010586 diagram Methods 0.000 claims description 8
- 238000013500 data storage Methods 0.000 claims description 7
- 238000007726 management method Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 13
- 238000011084 recovery Methods 0.000 description 6
- 230000007547 defect Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Retry When Errors Occur (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method and a system for backing up a data timing volume level, wherein the method comprises the following steps: initiating a timed complete backup, and starting cdp to monitor the IO change sector of the volume to be protected; executing snapshot operation on the volume to be protected; generating a full backup time point; opening the complete snapshot volume, and screening to obtain an effective cluster; reading data from the corresponding sector of the full snapshot volume for backup according to the offset position of the effective cluster; initiating a timed incremental backup task; cdp bitmap information is exchanged after the complete backup task is detected to be successful, and cdp monitors IO changed sectors of the volume to be protected again; executing snapshot operation on the volume to be protected; generating an incremental backup time point; merging cdp bitmap information to obtain offset addresses of all IO changed sectors; and reading data from the corresponding sector of the incremental snapshot volume for backup. Compared with the prior art, the method and the device can ensure the consistency of the backup data and perform timed complete backup and incremental backup on the whole volume on the basis of ensuring the consistency of the file system by combining the snapshot technology and the cdp technology.
Description
Technical Field
The invention relates to the technical field of data timing backup, in particular to a data timing volume level backup method and a data timing volume level backup system.
Background
The data backup is to restore the data and system information needed by the computer system as quickly as possible, which not only plays a role of protection when the network system hardware fails or human error occurs, but also can be the basis for the backup to take over the key service normally after the failure switch in the cluster environment. From the perspective of backup mode, there are volume level backup and file level backup, which are also known as physical level and logical level backup. Volume level backup: copying data from a host to a standby machine by taking a disk block as a basic unit, and backing up the data by taking a sector (512B) as a unit each time, wherein the real-time backup is usually realized on the basis of the mode; file-level backup: the data is copied from the host computer to the standby computer by taking a file as a basic unit, and the data is also copied by taking a complete file as a backup unit, and the size of the data is determined by the file itself, so that the timed backup is usually realized based on the mode.
With the advancement of informatization, the data volume of a big data era is increased explosively, a protection scene of massive small file data also becomes a very typical application scene in the field of data protection, and in order to solve the problem of excessive massive small file data objects, many manufacturers adopt a volume-level backup mode to protect the massive small files.
The volume-level backup can well solve the problem of a backup long scene with excessive mass small file objects, excessive fragments and excessive handles, but for a volume-level file system which is continuously changed, how to ensure the consistency of the file system becomes a difficult problem of the volume-level backup, the Data in a volume to be protected is directly read by adopting cdp (Continuous Data Protection) technology to backup, and once other users change the Data in the volume to be protected in the backup process, the problem of inconsistent backup Data is easy to occur, so that the subsequently restored Data is unavailable, the Data is lost and even the restored operating system is damaged. In addition, how to extract incremental data for incremental backup on the premise of ensuring the consistency of the file system is also a difficult point which must be solved for data protection.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a method and a system for timed volume level backup of data, which combine a snapshot technology and an cdp technology to realize timed volume level backup on the basis of ensuring consistency of backup data.
The purpose of the invention can be realized by the following technical scheme: a data timing volume level backup method comprises the following steps:
s1, initiating a timing complete backup task, and simultaneously starting cdp to monitor the IO change sector of the volume to be protected;
s2, executing snapshot operation on the volume to be protected to obtain a complete snapshot volume;
s3, generating a complete backup time point;
s4, opening the full snapshot volume, acquiring cluster information of the full snapshot volume, and screening to obtain an effective cluster according to the cluster information of the full snapshot volume;
s5, reading data from the sector corresponding to the full snapshot volume according to the offset position of the effective cluster, backing up the read data and the corresponding data offset position, then deleting the full snapshot volume, and marking the full backup time point correspondingly so as to complete the timed full backup task;
s6, initiating a timed incremental backup task;
s7, detecting whether the complete backup task is successful, if so, executing the step S8, otherwise, returning to the step S1 to carry out complete backup again;
s8, through exchanging cdp bitmap information, cdp monitors the IO change sector of the volume to be protected again;
s9, executing snapshot operation on the volume to be protected to obtain an incremental snapshot volume;
s10, generating an incremental backup time point;
s11, combining cdp bitmap information to obtain all change IOs from the start of full backup to the end of snapshot of incremental backup;
and S12, acquiring offset addresses of all IO changed sectors according to the merged cdp bitmap information, reading data from the sectors corresponding to the incremental snapshot volume, backing up the read data and the corresponding data offset positions, deleting the incremental snapshot volume, and marking the incremental backup time points correspondingly so as to complete the incremental backup task.
Further, the cdp bitmap information includes a buffer bitmap table and a swap bitmap table.
Further, in the step S1, the step cdp of monitoring the IO change sector of the volume to be protected specifically records the IO change through the buffer bitmap.
Further, the step S4 specifically includes the following steps:
s41, opening the full snapshot volume, and acquiring the total number of clusters in the full snapshot volume and the attribute of each cluster;
and S42, traversing all the clusters in the complete snapshot volume according to the total number of the clusters and the attributes of the clusters, and sequentially judging whether the clusters are effective clusters or not so as to obtain the effective clusters through screening.
Further, in the step S42, it is specifically determined whether a valid flag exists in the attribute of the cluster, and if so, the cluster is a valid cluster, otherwise, the cluster is not a valid cluster.
Further, the step S7 specifically includes the following steps:
s71, detecting whether the full backup time exists, if yes, executing a step S72, otherwise, judging that the full backup task fails;
and S72, detecting whether the corresponding mark exists in the full backup time, if so, judging that the full backup task is successful, otherwise, judging that the full backup task fails.
Further, the step S8 specifically includes the following steps:
s81, exchanging data recorded by the buffer bitmap during the complete backup task to the swap bitmap, and enabling the swap bitmap to obtain change IO monitored during the complete backup task;
and S82, emptying the buffer bitmap table, and restarting recording the change IO during the incremental backup task by using the buffer bitmap table.
Further, in the step S11, the change IO recorded again in the buffer bitmap table is merged into the swap bitmap table, so that the swap bitmap table obtains all the change IOs from the full backup to the end of the incremental backup snapshot.
A data timing volume level backup system comprises a control unit, a monitoring unit and a snapshot unit, wherein the monitoring unit and the snapshot unit are respectively connected with the control unit, and the control unit is used for receiving a timing complete backup or timing incremental backup initiating instruction of a user and executing a timing volume level backup program;
the monitoring unit is used for monitoring IO change sectors of the volumes to be protected during the task of the timed complete backup or the timed incremental backup;
the snapshot unit is used for executing snapshot operation on the volume to be protected;
the timed volume-level backup program comprises a timed full backup subprogram and a positioning increment backup subprogram, wherein the timed full backup subprogram comprises the following steps:
a1, the control unit sends a control instruction to the monitoring unit, and the monitoring unit starts a buffer bitmap table to start recording the change IO of the volume to be protected;
a2, the control unit sends a control instruction to the snapshot unit, the snapshot unit executes snapshot operation on the volume to be protected to obtain a full snapshot volume, and then the full snapshot volume is transmitted to the control unit;
a3, the control unit generates a full backup time point, then opens the full snapshot volume to obtain the number of clusters in the full snapshot volume and the attributes of all clusters, and obtains effective clusters by screening in a mode of traversing clusters;
a4, the control unit reads data from the sector corresponding to the full snapshot volume according to the offset position of the effective cluster, backs up the read data and the corresponding data offset position, then deletes the full snapshot volume and correspondingly marks the full backup time point;
the timed incremental backup subroutine comprises the steps of:
b1, the control unit acquires the buffer bitmap table and the swap bitmap from the monitoring unit and switches the change IO recorded by the buffer bitmap table to the swap bitmap table;
b2, the control unit outputs a control instruction to the monitoring unit, and the monitoring unit starts a buffer bitmap table to restart recording the change IO of the volume to be protected;
b3, the control unit sends a control instruction to the snapshot unit, the snapshot unit executes snapshot operation on the volume to be protected to obtain an incremental snapshot volume, and then the incremental snapshot volume is transmitted to the control unit;
b4, the control unit generates an incremental backup time point;
b5, the control unit acquires the current buffer bitmap table and the swap bitmap table from the monitoring unit and merges the change IO recorded by the current buffer bitmap table into the swap bitmap table;
b6, the control unit acquires offset addresses of all IO change sectors according to the merged swap bit diagram, reads data from the sectors corresponding to the incremental snapshot volume, backs up the read data and the corresponding data offset positions, deletes the incremental snapshot volume, and correspondingly marks the incremental backup time points.
Further, the control unit includes a bitmap management module and a data storage module, the bitmap management module is configured to perform an exchange or merge operation on the buffer bitmap table and the swap bitmap table, and the data storage module is configured to store data read from the full snapshot volume or the incremental snapshot volume and a corresponding data offset position.
Compared with the prior art, the invention has the following advantages:
the invention combines snapshot technology and cdp technology, executes snapshot operation on a to-be-protected volume in advance by using snapshot technology to ensure consistency of a backup file system, monitors IO change sectors of the to-be-protected volume by using cdp technology, and can acquire accurate change IO, thereby ensuring consistency of backup data.
Based on cdp bitmap exchange and combination, the method and the device enable the change IO during the incremental backup to be reliably recorded, avoid the problem of losing the change IO and further ensure the consistency of backup data.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a schematic diagram of the system of the present invention;
FIG. 3 is a schematic diagram of a timed full backup process according to the present invention;
FIG. 4 is a schematic diagram of a process of a timed incremental backup in the present invention;
FIG. 5 is a schematic view of the life cycle monitored by cdp in accordance with the present invention;
FIG. 6 is a diagram illustrating the merging of cdp bitmap information;
the notation in the figure is: 1. the system comprises a control unit 2, a monitoring unit 3, a snapshot unit 4, a volume to be protected 101, a bitmap management module 102 and a data storage module.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments.
Examples
As shown in fig. 1, a method for timed volume level backup of data includes the following steps:
s1, initiating a timed complete backup task, and simultaneously starting cdp to monitor IO change sectors of a volume to be protected, wherein cdp monitoring specifically records change IO through a buffer bitmap, and bitmap information generated during cdp monitoring comprises a buffer bitmap and a swap bitmap;
s2, executing snapshot operation on the volume to be protected to obtain a complete snapshot volume;
s3, generating a complete backup time point;
s4, opening the full snapshot volume, acquiring cluster information of the full snapshot volume, and screening to obtain effective clusters according to the cluster information of the full snapshot volume, specifically, firstly, opening the full snapshot volume, and acquiring the total number of clusters in the full snapshot volume and the attribute of each cluster;
traversing all clusters in the complete snapshot volume according to the total number of the clusters and the attributes of each cluster, and sequentially judging whether an effective mark exists in the attributes of each cluster, wherein if the effective mark exists, the cluster is an effective cluster, otherwise, the cluster is not an effective cluster;
s5, reading data from the sector corresponding to the full snapshot volume according to the offset position of the effective cluster, backing up the read data and the corresponding data offset position, then deleting the full snapshot volume, and marking the full backup time point correspondingly so as to complete the timed full backup task;
s6, initiating a timed incremental backup task;
s7, detecting whether the complete backup task is successful, if so, executing the step S8, otherwise, returning to the step S1 to perform complete backup again, specifically, firstly detecting whether complete backup time exists, if so, continuously detecting whether a corresponding mark exists in the complete backup time, if so, judging that the complete backup task is successful, otherwise, judging that the complete backup task fails;
s8, through exchanging cdp bitmap information, cdp monitors IO change sectors of the volume to be protected again, specifically, firstly, data recorded by a buffer bitmap table during the complete backup task is exchanged to a swap bitmap table, and the swap bitmap table obtains change IO monitored during the complete backup task;
then emptying the buffer bitmap table, and restarting recording the changed IO during the incremental backup task by using the buffer bitmap table;
s9, executing snapshot operation on the volume to be protected to obtain an incremental snapshot volume;
s10, generating an incremental backup time point;
s11, combining cdp bitmap information to obtain all change IOs from the start of full backup to the end of snapshot making of incremental backup, and specifically combining the change IOs recorded again in a buffer bitmap into a swap bitmap when combining cdp bitmap information to enable the swap bitmap to obtain all the change IOs from the start of full backup to the end of snapshot making of incremental backup;
and S12, acquiring offset addresses of all IO changed sectors according to the merged cdp bitmap information, reading data from the sectors corresponding to the incremental snapshot volume, backing up the read data and the corresponding data offset positions, deleting the incremental snapshot volume, and marking the incremental backup time points correspondingly so as to complete the incremental backup task.
The invention also provides a data timing volume level backup system, as shown in fig. 2, comprising a control unit 1, a monitoring unit 2 and a snapshot unit 3, wherein the monitoring unit 2 and the snapshot unit 3 are respectively connected with the control unit 1, and the control unit 1 is used for receiving a timing complete backup or timing incremental backup initiating instruction of a user and executing a timing volume level backup program;
the monitoring unit 2 is used for monitoring IO change sectors of the volumes to be protected 4 during the task of timed complete backup or timed incremental backup;
the snapshot unit 3 is configured to perform a snapshot operation on the to-be-protected volume 4;
the timed volume-level backup program includes a timed full backup subroutine and a positioned incremental backup subroutine, as shown in fig. 3, the timed full backup subroutine includes the following steps:
a1, the control unit sends a control instruction to the monitoring unit, and the monitoring unit starts a buffer bitmap table to start recording the change IO of the volume to be protected;
a2, the control unit sends a control instruction to the snapshot unit, the snapshot unit executes snapshot operation on the volume to be protected to obtain a full snapshot volume, and then the full snapshot volume is transmitted to the control unit;
a3, the control unit generates a full backup time point, then opens the full snapshot volume to obtain the number of clusters in the full snapshot volume and the attributes of all clusters, and obtains effective clusters by screening in a mode of traversing clusters;
a4, the control unit reads data from the sector corresponding to the full snapshot volume according to the offset position of the effective cluster, backs up the read data and the corresponding data offset position, then deletes the full snapshot volume and correspondingly marks the full backup time point;
as shown in fig. 4, the timed incremental backup subroutine includes the steps of:
b1, the control unit acquires the buffer bitmap table and the swap bitmap from the monitoring unit and switches the change IO recorded by the buffer bitmap table to the swap bitmap table;
b2, the control unit outputs a control instruction to the monitoring unit, and the monitoring unit starts a buffer bitmap table to restart recording the change IO of the volume to be protected;
b3, the control unit sends a control instruction to the snapshot unit, the snapshot unit executes snapshot operation on the volume to be protected to obtain an incremental snapshot volume, and then the incremental snapshot volume is transmitted to the control unit;
b4, the control unit generates an incremental backup time point;
b5, the control unit acquires the current buffer bitmap table and the swap bitmap table from the monitoring unit and merges the change IO recorded by the current buffer bitmap table into the swap bitmap table;
b6, the control unit acquires offset addresses of all IO change sectors according to the merged swap bit diagram, reads data from the sectors corresponding to the incremental snapshot volume, backs up the read data and the corresponding data offset positions, deletes the incremental snapshot volume, and correspondingly marks the incremental backup time points.
In addition, the control unit 1 includes a bitmap management module 101 and a data storage module 102, where the bitmap management module 101 is configured to perform an exchange or merge operation on the buffer bitmap table and the swap bitmap table, and the data storage module 102 is configured to store data read from the full snapshot volume or the incremental snapshot volume and a corresponding data offset position.
In the whole timed volume-level backup process, the life cycle monitored by cdp is as shown in fig. 5, during the timed full backup, only the buffer bitmap table records the change IO, at this time, the swap bitmap table maintains the standby state, during the timed incremental backup, the change IO recorded by the buffer bitmap table is first switched to the swap bitmap table, then the buffer bitmap table records the change IO again, and finally the buffer bitmap table and the swap bitmap table are merged (as shown in fig. 6, data recorded by two bitmap tables are bitwise or operated), so as to obtain a merged swap bitmap table, and when the timed incremental backup is subsequently performed, the buffer bitmap table is also used to record the change IO, and the current swap bitmap table is emptied for subsequent switching and merging processing.
In summary, the timed full backup process proposed by the present invention is:
1. initiating a timing backup, starting cdp for monitoring, wherein the cdp technology is used for monitoring io change sectors of a to-be-protected volume, and a buffer bitmap table is used for recording the change sectors;
2. taking a snapshot of a volume to be protected, wherein the snapshot is used for ensuring the consistency of a complete backup file system;
3. generating time point information to be backed up for finding a recovery copy during recovery;
4. opening the snapshot volume, and acquiring cluster information of the snapshot volume, wherein the total cluster number is the total number of clusters;
5. whether the clusters traversing the volume are valid data or not, if the clusters are invalid data, continuing to execute the step 5, and if the clusters are valid data, executing the step 6;
6. backing up the data offset position of the effective data sector reading sector data record;
7. and (4) completing backup of all effective sectors, deleting the backup snapshot volume and completing complete backup.
The timed incremental backup process comprises the following steps:
1. initiating incremental backup, detecting whether the complete backup at the last time point is successful, if so, executing the step 2, otherwise, returning to the timed complete backup process, and carrying out the complete backup again;
2, cdp bitmap table exchange, wherein a buffer bitmap starts a new round cdp for monitoring, cdp is used for monitoring io change sectors of a volume to be protected, and monitored io changes before bitmap exchange are stored after swap bitmap exchange;
3. taking a snapshot of the volume to be protected, wherein the snapshot is used for ensuring the consistency of the backup file system;
4. generating time point information to be backed up for finding a recovery copy during recovery;
merging the Cdp double-bit graphs, wherein the swap bit graphs acquire bitmap changes during snapshot execution, and the double-bit graph merging can acquire io changes during snapshot execution to ensure data consistency;
6. obtaining from cdp a changed sector address from complete to incremental snapshot;
7. taking the changing sector offset address of the swap bitmap table, and reading the sector data of the offset address of the snapshot volume in the step 2 of the process for backup;
8. the backup is complete for all cdp provided offset address sector data and the incremental backup is complete.
To verify the data consistency of the timed volume level backup of the present invention, the first backup environment prepared in this embodiment is: preparing a 200GB lvm volume, and a file with the size of 10M-20M of valid data being 100 GB; the incremental backup environment prepared in this embodiment is: file 20GB is randomly added to the volume.
A first full backup step:
1. installing cdp on the machine where the volume to be backed up is located, establishing a volume level backup task, and selecting the data volume to be backed up on a software interface;
2. finding the backup task established in the step 1, and selecting a 'full backup' option to perform full backup;
3. starting a complete backup execution process, generating a backup time point, and selecting the time point for carrying out whole volume recovery when recovering;
4. obtaining metadata information of a volume to be backed up for backing up;
5. the execution process starts cdp monitoring for changed sectors;
6. snapshot the call command of the LVM backup volume;
7. the execution process reads the snapshot volume cluster information through the interface;
8. traversing the cluster information to judge whether the cluster is valid data, and executing step 11 after traversing;
a) if the cluster is not valid data, continuing to execute the step 8, and if the cluster is valid data, executing the step 9;
9. reading the data of the cluster and recording the position offset address;
10. backing up the data in the step 8;
11. deleting the snapshot volume;
12. the task is completely backed up successfully, the successful record is stored, and the complete backup is completed;
13. and detecting that the size of the backed-up data is 100GB, which indicates that the backup data consistency is realized by the complete backup.
Incremental backup step:
1. selecting the backup task established in the step 1 when the first full backup is carried out, and selecting an incremental backup option to carry out incremental backup;
2. starting an incremental backup execution process, generating a backup time point, and selecting the time point for incremental recovery when recovering;
3. whether the backup of the last time point of the task is successful is obtained from the storage end:
a) if the task backup at the last time point is successful, executing the step 4;
b) if the task backup fails at the last time point or the last time point does not exist, the task executes the complete backup and starts from the step 4 of the complete backup process;
4. obtaining metadata information of a volume to be backed up for backing up;
5. executing process exchange cdp bitmap information to monitor the changed sectors again;
6. snapshot the call command of the LVM backup volume;
7. merging the existing cdp bitmap information for obtaining incremental data sector offsets;
8. circularly acquiring the sector offset address of the step 7 through an cdp interface;
9. reading sector data corresponding to the offset address of the snapshot volume through the offset address in the step 8;
10. backing up the data in the step 9;
11. deleting the snapshot volume;
12. the task incremental backup is successful, the successful record is carried out in the storage, and the incremental backup is completed;
13. and detecting that the size of the backed-up data is 20GB, which shows that the incremental backup also realizes the consistency of the backed-up data.
Therefore, the method is suitable for backup scenes of massive small files, guarantees the consistency of backup data through a snapshot technology and an cdp technology, can completely backup and incrementally backup the whole file on the basis of guaranteeing the consistency of a file system, and belongs to the field of timed backup. The method can perform volume-level backup under the condition that client services are not interrupted, and even if IO changes during the incremental snapshot, the method can ensure that the changed IO is not lost through cdp bitmap switching and merging technology, thereby ensuring data consistency and avoiding the conditions that a file system is damaged or recovered data is unavailable or lost and the like due to the problem of data consistency.
Claims (10)
1. A method for backing up data in a timed volume level is characterized by comprising the following steps:
s1, initiating a timing complete backup task, and simultaneously starting cdp to monitor the IO change sector of the volume to be protected;
s2, executing snapshot operation on the volume to be protected to obtain a complete snapshot volume;
s3, generating a complete backup time point;
s4, opening the full snapshot volume, acquiring cluster information of the full snapshot volume, and screening to obtain an effective cluster according to the cluster information of the full snapshot volume;
s5, reading data from the sector corresponding to the full snapshot volume according to the offset position of the effective cluster, backing up the read data and the corresponding data offset position, then deleting the full snapshot volume, and marking the full backup time point correspondingly so as to complete the timed full backup task;
s6, initiating a timed incremental backup task;
s7, detecting whether the complete backup task is successful, if so, executing the step S8, otherwise, returning to the step S1 to carry out complete backup again;
s8, through exchanging cdp bitmap information, cdp monitors the IO change sector of the volume to be protected again;
s9, executing snapshot operation on the volume to be protected to obtain an incremental snapshot volume;
s10, generating an incremental backup time point;
s11, combining cdp bitmap information to obtain all change IOs from the start of full backup to the end of snapshot of incremental backup;
and S12, acquiring offset addresses of all IO changed sectors according to the merged cdp bitmap information, reading data from the sectors corresponding to the incremental snapshot volume, backing up the read data and the corresponding data offset positions, deleting the incremental snapshot volume, and marking the incremental backup time points correspondingly so as to complete the incremental backup task.
2. The method of claim 1, wherein the cdp bitmap information comprises a buffer bitmap and a swap bitmap.
3. The method according to claim 2, wherein in step S1, the step cdp of monitoring the IO change sector of the volume to be protected records the IO change through a buffer bitmap.
4. The method for backing up a timed volume level of data according to claim 1, wherein said step S4 specifically includes the following steps:
s41, opening the full snapshot volume, and acquiring the total number of clusters in the full snapshot volume and the attribute of each cluster;
and S42, traversing all the clusters in the complete snapshot volume according to the total number of the clusters and the attributes of the clusters, and sequentially judging whether the clusters are effective clusters or not so as to obtain the effective clusters through screening.
5. The method according to claim 4, wherein said step S42 is implemented by determining whether a valid flag exists in the attribute of a cluster, if so, the cluster is a valid cluster, otherwise, the cluster is not a valid cluster.
6. The method for backing up a timed volume level of data according to claim 1, wherein said step S7 specifically includes the following steps:
s71, detecting whether the full backup time exists, if yes, executing a step S72, otherwise, judging that the full backup task fails;
and S72, detecting whether the corresponding mark exists in the full backup time, if so, judging that the full backup task is successful, otherwise, judging that the full backup task fails.
7. The method for backing up a timed volume level of data according to claim 3, wherein said step S8 specifically includes the following steps:
s81, exchanging data recorded by the buffer bitmap during the complete backup task to the swap bitmap, and enabling the swap bitmap to obtain change IO monitored during the complete backup task;
and S82, emptying the buffer bitmap table, and restarting recording the change IO during the incremental backup task by using the buffer bitmap table.
8. The method according to claim 7, wherein step S11 is specifically to merge change IOs recorded again in the buffer bitmap into the swap bitmap, so that the swap bitmap obtains all change IOs from a full backup to an incremental backup.
9. A data timing volume level backup system based on the method of claim 1, characterized by comprising a control unit (1), a monitoring unit (2) and a snapshot unit (3), wherein the monitoring unit (2) and the snapshot unit (3) are respectively connected with the control unit (1), and the control unit (1) is configured to receive a timing full backup or timing incremental backup initiating instruction of a user and execute a timing volume level backup program;
the monitoring unit (2) is used for monitoring IO change sectors of the volumes to be protected (4) during the timed complete backup or timed incremental backup task;
the snapshot unit (3) is used for executing snapshot operation on the volume to be protected (4);
the timed volume-level backup program comprises a timed full backup subprogram and a positioning increment backup subprogram, wherein the timed full backup subprogram comprises the following steps:
a1, the control unit (1) sends a control instruction to the monitoring unit (2), and the monitoring unit (2) starts a buffer bitmap table to record the change IO of the volume (4) to be protected;
a2, the control unit (1) sends a control instruction to the snapshot unit (3), the snapshot unit (3) executes snapshot operation on the volume to be protected (4) to obtain a full snapshot volume, and then the full snapshot volume is transmitted to the control unit (1);
a3, a control unit (1) generates a full backup time point, then opens a full snapshot volume to obtain the number of clusters in the full snapshot volume and the attributes of all the clusters, and obtains effective clusters by screening in a mode of traversing the clusters;
a4, the control unit (1) reads data from the sector corresponding to the full snapshot volume according to the offset position of the effective cluster, backs up the read data and the corresponding data offset position, then deletes the full snapshot volume and correspondingly marks the full backup time point;
the timed incremental backup subroutine comprises the steps of:
b1, the control unit (1) acquires the buffer bitmap and the swap bitmap from the monitoring unit (2), and exchanges the change IO recorded by the buffer bitmap into the swap bitmap;
b2, the control unit (1) outputs a control instruction to the monitoring unit (2), and the monitoring unit (2) starts the buffer bit chart to restart recording the change IO of the to-be-protected volume (4);
b3, the control unit (1) sends a control instruction to the snapshot unit (3), the snapshot unit (3) executes snapshot operation on the volume to be protected (4) to obtain an incremental snapshot volume, and then the incremental snapshot volume is transmitted to the control unit (1);
b4, the control unit (1) generates an incremental backup time point;
b5, the control unit (1) acquires the current buffer bitmap and the swap bitmap from the monitoring unit (2), and merges the change IO recorded by the current buffer bitmap into the swap bitmap;
b6, the control unit (1) acquires offset addresses of all IO change sectors according to the merged swap bit diagram, reads data from the sectors corresponding to the incremental snapshot volume, backs up the read data and the corresponding data offset positions, deletes the incremental snapshot volume, and correspondingly marks the incremental backup time points.
10. The system according to claim 9, wherein the control unit (1) comprises a bitmap management module (101) and a data storage module (102), the bitmap management module (101) is configured to perform an exchange or merge operation on the buffer bitmap table and the swap bitmap table, and the data storage module (102) is configured to store data read from the full snapshot volume or the incremental snapshot volume and a corresponding data offset position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011043723.1A CN112148530A (en) | 2020-09-28 | 2020-09-28 | Data timing volume level backup method and system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011043723.1A CN112148530A (en) | 2020-09-28 | 2020-09-28 | Data timing volume level backup method and system thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112148530A true CN112148530A (en) | 2020-12-29 |
Family
ID=73895939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011043723.1A Pending CN112148530A (en) | 2020-09-28 | 2020-09-28 | Data timing volume level backup method and system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112148530A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297012A (en) * | 2021-07-27 | 2021-08-24 | 浙江豪联信息科技有限公司 | Backup method based on sector data |
| CN114461456A (en) * | 2022-04-11 | 2022-05-10 | 成都云祺科技有限公司 | CDP backup method, system, storage medium and recovery method based on continuous writing |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9558072B1 (en) * | 2013-09-27 | 2017-01-31 | EMC IP Holding Company LLC | Block-level incremental recovery of a storage volume |
| CN108255641A (en) * | 2017-12-25 | 2018-07-06 | 南京壹进制信息技术股份有限公司 | A kind of CDP disaster recovery methods based on cloud platform |
| CN110188068A (en) * | 2019-05-16 | 2019-08-30 | 上海爱数信息技术股份有限公司 | A kind of volume grade backup method and device guaranteeing file system data consistency |
| CN110851302A (en) * | 2019-10-31 | 2020-02-28 | 上海爱数信息技术股份有限公司 | Database information backup method and database information recovery method |
| CN111090549A (en) * | 2019-12-27 | 2020-05-01 | 航天壹进制(南京)数据科技有限公司 | Volume CBT drive-based method for incrementally migrating volume-level data back to original machine |
| CN111651238A (en) * | 2020-06-03 | 2020-09-11 | 上海英方软件股份有限公司 | System and method for acquiring effective data in virtual machine protection |
-
2020
- 2020-09-28 CN CN202011043723.1A patent/CN112148530A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9558072B1 (en) * | 2013-09-27 | 2017-01-31 | EMC IP Holding Company LLC | Block-level incremental recovery of a storage volume |
| CN108255641A (en) * | 2017-12-25 | 2018-07-06 | 南京壹进制信息技术股份有限公司 | A kind of CDP disaster recovery methods based on cloud platform |
| CN110188068A (en) * | 2019-05-16 | 2019-08-30 | 上海爱数信息技术股份有限公司 | A kind of volume grade backup method and device guaranteeing file system data consistency |
| CN110851302A (en) * | 2019-10-31 | 2020-02-28 | 上海爱数信息技术股份有限公司 | Database information backup method and database information recovery method |
| CN111090549A (en) * | 2019-12-27 | 2020-05-01 | 航天壹进制(南京)数据科技有限公司 | Volume CBT drive-based method for incrementally migrating volume-level data back to original machine |
| CN111651238A (en) * | 2020-06-03 | 2020-09-11 | 上海英方软件股份有限公司 | System and method for acquiring effective data in virtual machine protection |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297012A (en) * | 2021-07-27 | 2021-08-24 | 浙江豪联信息科技有限公司 | Backup method based on sector data |
| CN114461456A (en) * | 2022-04-11 | 2022-05-10 | 成都云祺科技有限公司 | CDP backup method, system, storage medium and recovery method based on continuous writing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7472139B2 (en) | Database recovery method applying update journal and database log | |
| US8898112B1 (en) | Write signature command | |
| US8543548B2 (en) | Hierarchical management storage system and storage system operating method | |
| US8600945B1 (en) | Continuous data replication | |
| US8732128B2 (en) | Shadow copy bookmark generation | |
| CN109542682B (en) | Data backup method, device, equipment and storage medium | |
| WO2014079028A1 (en) | Data processing method and storage device | |
| CN113626261A (en) | Data backup method, device and equipment and computer readable storage medium | |
| EP3862883A1 (en) | Data backup method and apparatus, and system | |
| CN114461456B (en) | CDP backup method, system, storage medium and recovery method based on continuous writing | |
| CN113886143B (en) | Virtual machine continuous data protection method and device and data recovery method and device | |
| CN112148530A (en) | Data timing volume level backup method and system thereof | |
| CN110188068B (en) | Volume-level backup method and device for guaranteeing consistency of file system data | |
| US20060018505A1 (en) | Method, system and software for enhanced data protection using raw device backup of copy-on-write snapshots | |
| CN112579363A (en) | MySQL data protection system and backup and recovery method thereof | |
| WO2013170702A1 (en) | File processing method based on snapshot and solid state disk with snapshot function | |
| KR101738965B1 (en) | Apparatus and method for journaling based on garbage collection | |
| CN112130760A (en) | Data writing method, device and medium based on True CDP | |
| WO2013107295A1 (en) | Method for recovering hard drive data, server and distributed storage system | |
| CN114356650A (en) | Data backup method, device, equipment, system and storage medium | |
| TW201516655A (en) | System and method for recovering distributed file system | |
| CN106909514B (en) | Method and device for positioning snapshot disk address | |
| CN111078359A (en) | Method and system for realizing virtual machine instant recovery through directory mapping | |
| US12007851B2 (en) | Differential checksum validation | |
| CN109325005A (en) | A kind of data processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201229 |
|
| RJ01 | Rejection of invention patent application after publication |