CN114780299B - BMR backup system and method based on disk CBT - Google Patents

BMR backup system and method based on disk CBT Download PDF

Info

Publication number
CN114780299B
CN114780299B CN202210682996.3A CN202210682996A CN114780299B CN 114780299 B CN114780299 B CN 114780299B CN 202210682996 A CN202210682996 A CN 202210682996A CN 114780299 B CN114780299 B CN 114780299B
Authority
CN
China
Prior art keywords
component
cbt
disk
snapshot
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210682996.3A
Other languages
Chinese (zh)
Other versions
CN114780299A (en
Inventor
郭勇
卢研平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Ecloud Technology Co ltd
Original Assignee
Nanjing Ecloud Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Ecloud Technology Co ltd filed Critical Nanjing Ecloud Technology Co ltd
Priority to CN202210682996.3A priority Critical patent/CN114780299B/en
Publication of CN114780299A publication Critical patent/CN114780299A/en
Application granted granted Critical
Publication of CN114780299B publication Critical patent/CN114780299B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1458Management of the backup or restore process

Abstract

The invention discloses a BMR backup system and method based on a disk CBT, and relates to the technical field of data backup. The system comprises: the user layer control component is used for scanning the target monitoring disk structure, extracting the effective data area of the target monitoring disk, and issuing a control command for creating a snapshot and starting synchronization to the kernel layer component; the kernel layer component is used for creating a snapshot for the target monitoring disk and copying snapshot data to a snapshot copy on the backup server in a copy-on-write and active synchronization mode; and the backup server is used for receiving and processing the data transmitted by the kernel layer component and storing and backing up the data. The invention adopts a unified mechanism and processing flow to establish consistency snapshots for disk block equipment under Windows and Linux; continuous tracking of disk change blocks is provided, the incremental data generated between two snapshots is effectively supported to be obtained, and the incremental backup of an operating system is supported.

Description

BMR backup system and method based on disk CBT
Technical Field
The invention relates to the technical field of data backup, in particular to a BMR backup system and method based on a disk CBT.
Background
Bare Metal Recovery (BMR) is a data protection technology specially for a computer operating system, and the technology realizes the rapid reconstruction of the operating system environment of a target computer by backing up key data (including a boot program, partition table information, system files and the like) of the operating system of the target computer and recovering the backed-up key data of the operating system on other machines when the target computer fails.
Change Block Tracing (CBT) is a technology for tracking disk Change data in a disk filter driver layer or a bottom storage layer, and the technology is widely applied to the backup field, and can realize snapshot of a disk Block device and acquisition of incremental data between two snapshots by combining Copy On Write (COW) technology.
The Windows operating system provides backup and restore functions, can create restore points for the system periodically, and can restore the system to the restore points at the specified time when software failure occurs in the system. The disadvantage of the Windows backup restore function is that the restore point data is stored locally in the target computer, and if the target computer has a hardware failure or physical damage, the restore point data cannot be restored.
Under the Linux operating system, important data of the operating system can be packaged through a tar command or the data of the whole disk or a partition can be cloned through a dd command, however, the two commands back up active data of the operating system, and the consistency of the back-up data cannot be guaranteed.
Some third-party backup software creates a consistency snapshot for an important system Volume by using a Volume shadow Copy Service (VSS) provided by Windows, and can realize bare computer recovery of an operating system by combining data transmission and backup data management capabilities provided by the backup software. Since VSS does not provide CBT capability, each snapshot created by VSS is a full data copy of the target backup volume, and therefore, using VSS technology, incremental data since the last backup cannot be obtained.
Various release versions of Linux only support block device snapshots based on logical volumes and file system snapshots such as ZFS and BTRFS, and do not provide an effective snapshot scheme for other standard partitions or mainstream file systems such as ext series file systems and XFS built on the standard partitions.
The present invention provides a system and a method for backing up BMR based on CBT of disk, which are all deficient in the prior art.
Disclosure of Invention
In order to overcome the above problems or at least partially solve the above problems, embodiments of the present invention provide a system and a method for backing up a BMR based on a CBT disk, which employ a unified mechanism and processing flow to create a consistent snapshot for disk block devices under Windows and Linux; continuous tracking of disk change blocks is provided, the increment data generated between two snapshots is effectively supported to be obtained, and the increment backup of an operating system is supported.
The embodiment of the invention is realized by the following steps:
in a first aspect, an embodiment of the present invention provides a disk CBT-based BMR backup system, including a user layer control component, a kernel layer component, and a backup server, where:
the user layer control component is used for scanning the target monitoring disk structure, extracting the effective data area of the target monitoring disk, and issuing a control command for creating a snapshot and starting synchronization to the kernel layer component;
the kernel layer component is used for creating a snapshot for the target monitoring disk and copying snapshot data to a snapshot copy on the backup server in a copy-on-write and active synchronization mode;
and the backup server is used for receiving and processing the data transmitted by the kernel layer component and storing and backing up the data.
In order to solve the problems in the prior art, the system establishes consistency snapshots for disk block equipment under Windows and Linux by mutually matching three plates, namely a user layer control assembly, a kernel layer assembly, a backup server and the like and adopting a unified mechanism and a processing flow; continuous tracking of disk change blocks is provided, the incremental data generated between two snapshots is effectively supported to be obtained, and the incremental backup of an operating system is supported. Scanning a disk structure through a user layer control assembly, extracting an effective data area of a disk, issuing control commands for creating a snapshot, starting synchronization and the like, and the like; and then, creating a snapshot for the target monitoring disk through the kernel layer component, copying snapshot data to a snapshot copy on a backup server through operations such as copy-on-write, active synchronization and the like, and further analyzing and storing and backing up the data packet through the backup server.
Based on the first aspect, in some embodiments of the present invention, the kernel layer component includes an IO monitoring component, a COW component, a synchronization component, and a transmission component, where:
the IO monitoring component is used for intercepting the IO write request in real time, generating a CBT record according to the IO write request, storing the CBT record into the increment CBT table and submitting the CBT record to the COW component;
the COW component is used for performing copy-on-write action based on the CBT record generated by the IO monitoring component and combining COW-Bitmap, and passively synchronizing old data of a disk area of a target monitoring disk corresponding to the CBT record to a snapshot copy on the backup server;
the synchronization component is used for actively synchronizing a disk area which is not written in since the snapshot is created on the target monitoring disk to the snapshot copy on the backup server;
and the transmission component is used for transmitting the data records of the COW component and the synchronization component to the backup server in real time.
In a second aspect, an embodiment of the present invention provides a disk CBT-based BMR backup method, including the following steps:
scanning a target monitoring disk structure through a user layer control component, extracting an effective data area of a target monitoring disk, and issuing a control command for creating a snapshot and starting synchronization to a kernel layer component;
creating a snapshot for a target monitoring disk through a kernel layer component and copying snapshot data to a snapshot copy on a backup server in a copy-on-write and active synchronization mode;
and receiving and processing the data transmitted by the kernel layer component based on the backup server, and storing and backing up the data.
The method is realized based on the system, and based on the mutual cooperation of three plates such as a user layer control assembly, a kernel layer assembly, a backup server and the like, a consistent snapshot is established on disk block equipment under Windows and Linux by adopting a unified mechanism and a unified processing flow; continuous tracking of disk change blocks is provided, the increment data generated between two snapshots is effectively supported to be obtained, and the increment backup of an operating system is supported. Scanning a disk structure through a user layer control assembly, extracting an effective data area of a disk, issuing control commands for creating a snapshot, starting synchronization and the like, and the like; and then, creating a snapshot for the target monitoring disk through the kernel layer component, copying snapshot data to a snapshot copy on a backup server through operations such as copy-on-write, active synchronization and the like, and further analyzing and storing and backing up the data packet through the backup server.
Based on the second aspect, in some embodiments of the present invention, the method for creating a snapshot for a target monitoring disk by a kernel layer component and copying snapshot data to a snapshot copy located on a backup server in a copy-on-write and active synchronization manner includes the following steps:
an IO write request is intercepted in real time through an IO monitoring component in the kernel layer component, a CBT record is generated according to the IO write request and stored in an increment CBT table, and the CBT record is submitted to the COW component;
performing copy-on-write action by combining a COW (communication per title) record generated by a COW (communication on-edge) component in a kernel layer component based on an IO (input/output) monitoring component with a COW-Bitmap, and passively synchronizing old data of a disk area of a target monitoring disk corresponding to the CBT record to a snapshot copy on a backup server;
the method comprises the steps that a synchronization component in a kernel layer component actively synchronizes an unwritten disk area on a target monitoring disk since a snapshot is created to a snapshot copy on a backup server;
and transmitting the data records of the COW component and the synchronization component to the backup server in real time through a transmission component in the kernel layer component.
Based on the second aspect, in some embodiments of the present invention, the method for CBT-based BMR backup on disk further includes the following steps:
the complete CBT record is obtained from the user layer control component through the synchronization component and a synchronization CBT table is generated.
Based on the second aspect, in some embodiments of the present invention, the method for CBT-based BMR backup on disk further includes the following steps:
converting the increment CBT table into a synchronous CBT table through an IO monitoring component and submitting the synchronous CBT table to a synchronous component;
and the synchronization component performs synchronization operation according to the synchronization CBT table and the COW-Bitmap, and synchronizes the disk blocks which are not written in the target monitoring disk since the snapshot is created to the snapshot copy on the backup service.
Based on the second aspect, in some embodiments of the present invention, the method for CBT-based BMR backup on disk further includes the following steps:
and combining the data records of the COW component and the synchronization component through the transmission component according to the preset data sending length, and then transmitting the combined data records to the backup server.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing one or more programs; a processor. The program or programs, when executed by a processor, implement the method of any of the second aspects as described above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method according to any one of the above second aspects.
The embodiment of the invention at least has the following advantages or beneficial effects:
the embodiment of the invention provides a BMR backup system and a method based on a disk CBT.A consistent snapshot is established for disk block equipment under Windows and Linux by adopting a unified mechanism and a processing flow through the mutual cooperation of three plates, namely a user layer control assembly, a kernel layer assembly, a backup server and the like; continuous tracking of disk change blocks is provided, the increment data generated between two snapshots is effectively supported to be obtained, and the increment backup of an operating system is supported. Scanning a disk structure through a user layer control assembly, extracting an effective data area of a disk, issuing control commands for creating a snapshot, starting synchronization and the like, and the like; and then, creating a snapshot for the target monitoring disk through the kernel layer component, copying snapshot data to a snapshot copy on a backup server through operations such as copy-on-write, active synchronization and the like, and further analyzing and storing and backing up the data packet through the backup server.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a block diagram of a BMR backup system based on a CBT disk according to an embodiment of the present invention;
fig. 2 is a flowchart of a disk CBT-based BMR backup method according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating tracking and copy-on-write of changed data blocks in an embodiment of the present invention;
FIG. 4 is a schematic diagram of an embodiment of an incremental backup based on CBT;
fig. 5 is a block diagram of an electronic device according to an embodiment of the present invention.
Description of the reference numerals: 101. a memory; 102. a processor; 103. a communication interface.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Example (b):
as shown in fig. 1, in a first aspect, an embodiment of the present invention provides a disk CBT-based BMR backup system, including a user-level control component, a kernel-level component, and a backup server, where:
the user layer control component is used for scanning the target monitoring disk structure, extracting the effective data area of the target monitoring disk, and issuing a control command for creating a snapshot and starting synchronization to the kernel layer component;
the kernel layer component is used for creating a snapshot for the target monitoring disk and copying snapshot data to a snapshot copy on the backup server in a copy-on-write and active synchronization mode;
further, the kernel layer component includes an IO monitoring component, a COW component, a synchronization component, and a transmission component, where:
the IO monitoring component is used for intercepting the IO write request in real time, generating a CBT record according to the IO write request, storing the CBT record into the increment CBT table and submitting the CBT record to the COW component; the COW component is used for performing copy-on-write action based on the CBT record generated by the IO monitoring component in combination with the COW-Bitmap, and passively synchronizing the old data of the disk area of the target monitoring disk corresponding to the CBT record to the snapshot copy on the backup server; the synchronization component is used for actively synchronizing a disk area which is not written in since the snapshot is created on the target monitoring disk to the snapshot copy on the backup server; and the transmission component is used for transmitting the data records of the COW component and the synchronization component to the backup server in real time.
In some embodiments of the present invention, the kernel layer component mainly includes an IO monitoring component, a COW component, a synchronization component, and a transmission component, and these four components cooperate with each other to create a snapshot for the target monitoring disk and copy snapshot data to a snapshot copy located on the backup server through operations such as copy-on-write and active synchronization.
An IO monitoring component is inserted into a disk filter driving layer to continuously track the change data block of a target monitoring disk, the change data block is constructed into a CBT record and stored in an Inc-CBT Table, and the Inc-CBT Table is used as an incremental data set between two snapshots. And a kernel COW component is added on the basis of the IO monitoring component, the COW component executes copy-on-write action based on the CBT record generated by the IO monitoring component in combination with the COW-Bitmap, and old data of a disk area corresponding to the CBT is passively synchronized to a snapshot copy on a backup server. A kernel synchronization component is added on the basis of the passive synchronization of the COW component, and under the condition of first full backup, the kernel synchronization component acquires a complete CBT record from a user layer control component to generate a Sync-CBT Table; under the condition of incremental backup, the IO monitoring component converts the Inc-CBT Table into the Sync-CBT Table and submits the Sync-CBT Table to the kernel synchronization component. And the kernel synchronization component executes synchronization operation according to the Sync-CBT Table in combination with the COW-Bitmap, and synchronizes the disk blocks which are not written in the target monitoring disk since the snapshot is created to the snapshot copy on the backup service.
The COW component maintains a copy-on-write Bitmap (COW-Bitmap) that logically divides the target supervisory disk into fixed-size disk blocks, and then marks one disk block by one bit. For each incremental CBT, the COW component converts it to one or more bits of the COW-Bitmap, and then checks and sets the corresponding bits. If the corresponding bit is not set before, the fact that the disk block corresponding to the bit is written for the first time since the snapshot is created is shown, and the COW component copies old data from the target monitoring disk to construct a data record and submits the data record to the transmission component. And after the COW component finishes processing, the IO monitoring component is informed to submit the IO request corresponding to the CBT to the lower-layer drive processing.
The synchronization component is responsible for actively synchronizing disk regions on the target monitoring disk that have not been written to since the snapshot was created to the snapshot copy on the backup server. The component executes synchronization operation based on a synchronization CBT Table (Sync-CBT Table), wherein each record of the Sync-CBT Table represents an effective data area or an area to be synchronized on a target monitoring disk. Under the condition of first full backup, the kernel synchronization component acquires complete CBT records from the user layer control component to generate Sync-CBT tables; under the condition of incremental backup, the IO monitoring component converts the Inc-CBT Table into the Sync-CBT Table and submits the Sync-CBT Table to the kernel synchronization component. And the kernel synchronization component executes synchronization operation according to the Sync-CBT Table in combination with the COW-Bitmap, and synchronizes the disk blocks which are not written in the target monitoring disk since the snapshot is created to the snapshot copy on the backup service.
The transport component is responsible for providing data transport services to the COW component and the sync component, which maintain a Cache queue (Cache). And after the COW component or the synchronization component submits the data record to the Cache, the transmission component transmits the data record to the backup server through a TCP/IP network in real time.
And the backup server is used for receiving and processing the data transmitted by the kernel layer component and storing and backing up the data.
In order to solve the problems in the prior art, the system establishes consistency snapshots for disk block devices under Windows and Linux by mutually matching three plates, namely a user layer control component, a kernel layer component, a backup server and the like, and adopting a unified mechanism and processing flow; continuous tracking of disk change blocks is provided, the increment data generated between two snapshots is effectively supported to be obtained, and the increment backup of an operating system is supported. Scanning a disk structure through a user layer control assembly, extracting an effective data area of a disk, issuing control commands for creating a snapshot, starting synchronization and the like, and the like; and then, creating a snapshot for the target monitoring disk through the kernel layer component, copying snapshot data to a snapshot copy on a backup server through operations such as copy-on-write, active synchronization and the like, and further analyzing and storing and backing up the data packet through the backup server.
As shown in fig. 2, in a second aspect, an embodiment of the present invention provides a disk CBT-based BMR backup method, including the following steps:
s1, scanning the target monitoring disk structure through the user layer control component, extracting the effective data area of the target monitoring disk, and issuing a control command for creating a snapshot and starting synchronization to the kernel layer component;
s2, creating a snapshot for the target monitoring disk through the kernel layer component and copying snapshot data to a snapshot copy on the backup server in a copy-on-write and active synchronization mode;
and S3, receiving and processing the data transmitted by the kernel layer component based on the backup server, and storing and backing up the data.
The method is realized based on the system, and based on the mutual cooperation of three plates such as a user layer control assembly, a kernel layer assembly, a backup server and the like, a consistent snapshot is established on disk block equipment under Windows and Linux by adopting a unified mechanism and a unified processing flow; continuous tracking of disk change blocks is provided, the increment data generated between two snapshots is effectively supported to be obtained, and the increment backup of an operating system is supported. Scanning a disk structure through a user layer control assembly, extracting an effective data area of a disk, issuing control commands for creating a snapshot, starting synchronization and the like, and the like; and then, creating a snapshot for the target monitoring disk through the kernel layer component, copying snapshot data to a snapshot copy on a backup server through operations such as copy-on-write, active synchronization and the like, and further analyzing and storing and backing up the data packet through the backup server.
As shown in fig. 3, according to the second aspect, in some embodiments of the present invention, the above method for creating a snapshot for a target monitoring disk by a kernel layer component and copying snapshot data to a snapshot copy located on a backup server by means of copy-on-write and active synchronization includes the following steps:
an IO write request is intercepted in real time through an IO monitoring component in the kernel layer component, a CBT record is generated according to the IO write request and stored in an increment CBT table, and the CBT record is submitted to the COW component;
performing copy-on-write action by combining a COW (communication per title) record generated by a COW (communication on-edge) component in a kernel layer component based on an IO (input/output) monitoring component with a COW-Bitmap, and passively synchronizing old data of a disk area of a target monitoring disk corresponding to the CBT record to a snapshot copy on a backup server;
the method comprises the steps that a synchronization component in a kernel layer component actively synchronizes an unwritten disk area on a target monitoring disk since a snapshot is created to a snapshot copy on a backup server;
and transmitting the data records of the COW component and the synchronization component to the backup server in real time through a transmission component in the kernel layer component.
The kernel layer component mainly comprises an IO monitoring component, a COW component, a synchronization component and a transmission component, and the four components cooperate with each other to establish a snapshot for a target monitoring disk and copy snapshot data to a snapshot copy located on a backup server through operations such as copy-on-write, active synchronization and the like.
The detailed work flow of the IO monitoring component is shown in fig. 3, and the IO monitoring component is located in a disk filter driver layer and is responsible for intercepting an IO write request submitted by an upper layer driver in real time, generating a CBT record < disk _ offset, length > according to IO request information, storing the CBT record in an incremental CBT Table (Inc-CBT Table), and submitting the CBT record to the COW component.
An IO monitoring component is inserted into a disk filter driving layer to continuously track the change data block of a target monitoring disk, the change data block is constructed into a CBT record and stored in an Inc-CBT Table, and the Inc-CBT Table is used as an incremental data set between two snapshots. And adding a kernel COW component on the basis of the IO monitoring component, wherein the COW component executes copy-on-write action based on a CBT record generated by the IO monitoring component in combination with the COW-Bitmap, and passively synchronizing the old data of the disk area corresponding to the CBT to the snapshot copy on the backup server. A kernel synchronization component is added on the basis of the passive synchronization of the COW component, and under the condition of first full backup, the kernel synchronization component acquires a complete CBT record from a user layer control component to generate a Sync-CBT Table; under the condition of incremental backup, the IO monitoring component converts the Inc-CBT Table into the Sync-CBT Table and submits the Sync-CBT Table to the kernel synchronization component. And the kernel synchronization component executes synchronization operation according to the Sync-CBT Table in combination with the COW-Bitmap, and synchronizes the disk blocks which are not written in the target monitoring disk since the snapshot is created to the snapshot copy on the backup service.
Based on the second aspect, in some embodiments of the present invention, the method for backing up a BMR based on CBT of a disk further includes the following steps:
the complete CBT record is obtained from the user layer control component through the synchronization component and a synchronization CBT table is generated.
The synchronization component is responsible for actively synchronizing disk regions on the target monitoring disk that have not been written to since the snapshot was created to the snapshot copy on the backup server. The component executes synchronization operation based on a synchronization CBT Table (Sync-CBT Table), wherein each record of the Sync-CBT Table represents an effective data area or an area to be synchronized on a target monitoring disk. Under the condition of primary full backup, the Sync-CBT Table is acquired by the user layer control component and is transmitted to the synchronization component through a user mode-kernel mode interactive interface.
As shown in fig. 4, according to the second aspect, in some embodiments of the present invention, the method for backing up a BMR based on a CBT of a disk further includes the following steps:
converting the increment CBT table into a synchronous CBT table through an IO monitoring component and submitting the synchronous CBT table to a synchronous component;
and the synchronization component performs synchronization operation according to the synchronization CBT table and the COW-Bitmap, and synchronizes the disk blocks which are not written in the target monitoring disk since the snapshot is created to the snapshot copy on the backup service.
Under the condition of incremental backup, the IO monitoring component converts the Inc-CBT Table into the Sync-CBT Table, as shown in FIG. 4. And the synchronous component reads a piece of continuous data from the target monitoring disk according to the CBT, and if the CBT exceeds the single maximum reading length of the synchronous component, the synchronous component divides the CBT into a plurality of smaller CBT records for data reading. After the data reading is completed, the synchronization component inquires the COW component whether the disk area corresponding to the CBT has been written with new data since the snapshot was created. Based on the returned results of the COW component, the synchronization component constructs the data of the unwritten area into a data record and submits the data record to the transmission component.
Based on the second aspect, in some embodiments of the present invention, the method for CBT-based BMR backup on disk further includes the following steps:
and combining the data records of the COW component and the synchronization component through a transmission component according to the preset data sending length, and transmitting the combined data records to a backup server.
The transmit component is responsible for providing data transmission services to the COW component and the sync component, which maintain a Cache queue (Cache). And after the COW component or the synchronization component submits the data record to the Cache, the transmission component transmits the data record to the backup server through a TCP/IP network in real time. Since the data records generated by the COW component are usually small, if the transmission component performs data transmission by taking the data records as a unit, the utilization rate of the transmission bandwidth is low, and a transmission bottleneck is generated. In order to solve the problem of transmission bandwidth utilization rate caused by small data records, a transmission assembly introduces a data record merging technology. The transmission merging technology has the general idea that a transmission assembly sets a single maximum sending length MSL, and when the Cache caches enough data records, the transmission assembly merges a plurality of data records to enable the total data length of a data packet to be close to the MSL to the maximum extent and not to exceed the MSL. Besides the data record merging technology, the transmission component also introduces an asynchronous confirmation mechanism to further improve the transmission efficiency. The general idea behind asynchronous acknowledgement mechanisms is that the transport component does not need to wait for the backup server to acknowledge whether the data packet has been successfully parsed and the data record replayed after sending the data packet to the backup server. If the backup server successfully processes the data packet, the processing result does not need to be confirmed to the transmission assembly, and if the backup server fails to process the data packet, the abnormal processing result is attached to the subsequent transmission confirmation packet.
In a third aspect, as shown in fig. 5, an embodiment of the present application provides an electronic device, which includes a memory 101 for storing one or more programs; a processor 102. The one or more programs, when executed by the processor 102, implement the method of any of the second aspects as described above.
Also included is a communication interface 103, and the memory 101, processor 102 and communication interface 103 are electrically connected to each other, directly or indirectly, to enable transfer or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 101 may be used to store software programs and modules, and the processor 102 executes the software programs and modules stored in the memory 101 to thereby execute various functional applications and data processing. The communication interface 103 may be used for communicating signaling or data with other node devices.
The Memory 101 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 102 may be an integrated circuit chip having signal processing capabilities. The Processor 102 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In the embodiments provided in the present application, it should be understood that the disclosed method and system and method can be implemented in other ways. The method and system embodiments described above are merely illustrative, for example, the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by the processor 102, implements the method according to any one of the second aspects described above. The functions may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The present invention has been described in terms of the preferred embodiment, and it is not intended to be limited to the embodiment. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.

Claims (7)

1. A disk CBT-based BMR backup system is characterized by comprising a user layer control component, a kernel layer component and a backup server, wherein:
the user layer control component is used for scanning the target monitoring disk structure, extracting the effective data area of the target monitoring disk, and issuing a control command for creating a snapshot and starting synchronization to the kernel layer component;
the kernel layer component is used for creating a snapshot for the target monitoring disk and copying snapshot data to a snapshot copy on the backup server in a copy-on-write and active synchronization mode; the kernel layer assembly comprises an IO monitoring assembly, a COW assembly, a synchronization assembly and a transmission assembly, wherein: the IO monitoring component is used for intercepting an IO write request in real time, generating a CBT record according to the IO write request, storing the CBT record into an incremental CBT table and submitting the CBT record to the COW component; the COW component is used for performing copy-on-write action based on the CBT record generated by the IO monitoring component and combining COW-Bitmap, and passively synchronizing old data of a disk area of a target monitoring disk corresponding to the CBT record to a snapshot copy on the backup server; the synchronization component is used for actively synchronizing a disk area which is not written in since the snapshot is created on the target monitoring disk to the snapshot copy on the backup server; the transmission component is used for transmitting the data records of the COW component and the synchronization component to the backup server in real time;
and the backup server is used for receiving and processing the data transmitted by the kernel layer component and storing and backing up the data.
2. A BMR backup method based on a disk CBT is characterized by comprising the following steps:
scanning a target monitoring disk structure through a user layer control component, extracting an effective data area of a target monitoring disk, and issuing a control command for creating a snapshot and starting synchronization to a kernel layer component;
creating a snapshot for a target monitoring disk through a kernel layer component and copying snapshot data to a snapshot copy located on a backup server in a copy-on-write and active synchronization mode, wherein the snapshot creating process comprises the following steps: an IO write request is intercepted in real time through an IO monitoring component in the kernel layer component, a CBT record is generated according to the IO write request and stored in an increment CBT table, and the CBT record is submitted to the COW component; performing copy-on-write action by combining a COW (communication per title) record generated by a COW (communication on-edge) component in a kernel layer component based on an IO (input/output) monitoring component with a COW-Bitmap, and passively synchronizing old data of a disk area of a target monitoring disk corresponding to the CBT record to a snapshot copy on a backup server; the method comprises the steps that a synchronization component in a kernel layer component actively synchronizes a disk area which is not written in since a snapshot is created on a target monitoring disk to a snapshot copy on a backup server; transmitting the data records of the COW component and the synchronization component to a backup server in real time through a transmission component in the kernel layer component;
and receiving and processing the data transmitted by the kernel layer component based on the backup server, and storing and backing up the data.
3. The method of claim 2, further comprising the steps of:
the complete CBT record is obtained from the user layer control component through the synchronization component and a synchronization CBT table is generated.
4. The method of claim 3, further comprising the steps of:
converting the incremental CBT table into a synchronous CBT table through an IO monitoring component and submitting the synchronous CBT table to a synchronous component;
and the synchronization component performs synchronization operation according to the synchronization CBT table and the COW-Bitmap, and synchronizes the disk blocks which are not written in the target monitoring disk since the snapshot is created to the snapshot copy on the backup service.
5. The disk CBT-based BMR backup method according to claim 2, further comprising the steps of:
and combining the data records of the COW component and the synchronization component through a transmission component according to the preset data sending length, and transmitting the combined data records to a backup server.
6. An electronic device, comprising:
a memory for storing one or more programs;
a processor;
the one or more programs, when executed by the processor, implement the method of any of claims 2-5.
7. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 2-5.
CN202210682996.3A 2022-06-17 2022-06-17 BMR backup system and method based on disk CBT Active CN114780299B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210682996.3A CN114780299B (en) 2022-06-17 2022-06-17 BMR backup system and method based on disk CBT

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210682996.3A CN114780299B (en) 2022-06-17 2022-06-17 BMR backup system and method based on disk CBT

Publications (2)

Publication Number Publication Date
CN114780299A CN114780299A (en) 2022-07-22
CN114780299B true CN114780299B (en) 2022-09-09

Family

ID=82421443

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210682996.3A Active CN114780299B (en) 2022-06-17 2022-06-17 BMR backup system and method based on disk CBT

Country Status (1)

Country Link
CN (1) CN114780299B (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7631158B2 (en) * 2007-01-31 2009-12-08 Inventec Corporation Disk snapshot method using a copy-on-write table in a user space
US7809908B2 (en) * 2007-02-23 2010-10-05 Inventec Corporation Disk snapshot acquisition method
CN110109777A (en) * 2019-07-02 2019-08-09 南京云信达科技有限公司 A kind of VM incremental backup method realized in the case of CBT corrupted data

Also Published As

Publication number Publication date
CN114780299A (en) 2022-07-22

Similar Documents

Publication Publication Date Title
US11429305B2 (en) Performing backup operations using replicas
US10831614B2 (en) Visualizing restoration operation granularity for a database
US10467109B2 (en) Replication based security
US9563517B1 (en) Cloud snapshots
US9740880B1 (en) Encrypted virtual machines in a cloud
US7844856B1 (en) Methods and apparatus for bottleneck processing in a continuous data protection system having journaling
US9722788B1 (en) Rekeying encrypted virtual machines in a cloud
US9367260B1 (en) Dynamic replication system
US7577867B2 (en) Cross tagging to data for consistent recovery
US9256605B1 (en) Reading and writing to an unexposed device
US9405481B1 (en) Replicating using volume multiplexing with consistency group file
US8898409B1 (en) Journal-based replication without journal loss
US9557925B1 (en) Thin replication
US8478955B1 (en) Virtualized consistency group using more than one data protection appliance
US7149858B1 (en) Synchronous replication for system and data security
US10101943B1 (en) Realigning data in replication system
US8954796B1 (en) Recovery of a logical unit in a consistency group while replicating other logical units in the consistency group
US9811430B1 (en) Method and system for incremental backup of data volumes
US9535801B1 (en) Xcopy in journal based replication
US8600945B1 (en) Continuous data replication
US8805786B1 (en) Replicating selected snapshots from one storage array to another, with minimal data transmission
US7934262B1 (en) Methods and apparatus for virus detection using journal data
US10223007B1 (en) Predicting IO
US8762342B1 (en) Method of inserting a validated time-image on the primary CDP subsystem in a continuous data protection and replication (CDP/R) subsystem
WO2000050999A1 (en) Method and system for mirroring and archiving mass storage

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant