CN112988473A - Backup data real-time recovery method and system - Google Patents
Backup data real-time recovery method and system Download PDFInfo
- Publication number
- CN112988473A CN112988473A CN202110503081.7A CN202110503081A CN112988473A CN 112988473 A CN112988473 A CN 112988473A CN 202110503081 A CN202110503081 A CN 202110503081A CN 112988473 A CN112988473 A CN 112988473A
- Authority
- CN
- China
- Prior art keywords
- data
- time
- block
- recovery
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
Abstract
The invention provides a method and a system for recovering backup data in real time, and relates to the technical field of data backup. The method for restoring the backup data in real time comprises the following steps: acquiring data to be backed up and a current timestamp in real time; comparing the data to be backed up acquired in real time with preset full data to obtain a change data block set; marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp; generating a data modification table according to the block bitmap and the change data block set of the current timestamp; updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table; and recovering the data according to the block bitmap, the data modification table and the time address table to generate real-time recovery data. Real-time block data change tracking and recording are realized through a block bitmap, a data modification table and a time address table, dynamic retrieval is carried out in the data recovery process, and online millisecond-level real-time data recovery is realized.
Description
Technical Field
The invention relates to the technical field of data backup, in particular to a method and a system for recovering backup data in real time.
Background
With the popularization of computer applications, data has become an important asset for data owners, and once data loss occurs, a huge loss is caused to computer users. Problems such as software platform faults, design bugs of an operating system, hardware faults of the system and the like can bring immeasurable loss to data of the system, influence normal operation of enterprise servers, and further cause serious loss to scientific research and production. Therefore, in the process of maintaining the server management software, a system administrator must back up data such as configuration information and database information in the current system at a certain node to prevent immeasurable loss caused by subsequent misoperation.
The conventional real-time data backup recovery technology is CDP (continuous data protection), and products for realizing backup recovery based on CDP in the field of data backup are all block-level real-time data protection recovery methods realized based on the realization or the design concept. However, such continuous data protection of CDP is performed by recovering the constructed data table and copy recovery, and cannot be performed in real time.
Disclosure of Invention
The invention aims to provide a method and a system for recovering backup data in real time, which are used for solving the problem that the recovery in the prior art is carried out by recovering a constructed data table and copying and recovering and can not realize the real-time recovery.
In a first aspect, an embodiment of the present application provides a method for restoring backup data in real time, including the following steps:
acquiring data to be backed up and a current timestamp in real time;
comparing the data to be backed up acquired in real time with preset full data to obtain a change data block set;
marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp;
generating a data modification table according to the block bitmap and the change data block set of the current timestamp;
updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table;
and recovering the data according to the block bitmap, the data modification table and the time address table to generate real-time recovery data.
In the implementation process, the data to be backed up and the current timestamp are acquired in real time, and then the data to be backed up acquired in real time is compared with the preset full data to obtain a change data block set; marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp; when the block bitmap of the changed data block, which is modified by the latest timestamp, determines whether the changed data block needs to be extracted or is directly read from the full data when the data is recovered, and when the block bitmap has no mark, the full data can be directly read, so that the data recovery rate is accelerated; then generating a data modification table according to the block bitmap and the change data block set of the current timestamp; the data modification table is realized on the basis of an array and open chain mode, is used for storing data areas of modified data blocks in the same timestamp, and is stored in corresponding positions according to block position number modulo, so that corresponding data can be quickly retrieved in data recovery, O (1) complexity recovery is realized, space-time complexity is reduced, and data recovery speed is improved; then, updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table; the time address table comprises a time stamp address table and a time pile data structure table, the latest modified hash time stamp address table is an array address table, which determines the hash array from which the data block should be read and the hash data modification table of the data change block, the array module is used for storing the change data block, list link and the same module number sequence are carried out, when data recovery is needed, data recovery is carried out on the data according to the block bitmap, the data modification table and the time address table, and real-time recovery data are generated. The time heap data structure table is a millisecond-based time heap data structure table, can realize real-time tracking and recording of block data change through a block bitmap, a data modification table and a time address table, and can be dynamically retrieved in the data recovery process. The online millisecond-level real-time data recovery is realized by mounting the initial full-amount backup data and the continuously changed data volume.
Based on the first aspect, in some embodiments of the present invention, the step of updating the preset time address table according to the current timestamp and the data modification table, and generating a new time address table includes the following steps:
updating a preset timestamp address table according to the data modification table and the current timestamp to generate a new timestamp address table;
and updating the preset time pile data table according to the new time stamp address table and the current time stamp to generate a new time pile data structure table.
Based on the first aspect, in some embodiments of the invention, the step of generating the data modification table from the block bitmap of the current timestamp and the set of changed data blocks comprises the steps of:
acquiring a block bit number marked in a block bitmap of a current timestamp;
and arranging the changed data blocks according to the marked block bit numbers to generate a data modification table.
Based on the first aspect, in some embodiments of the present invention, the step of arranging the changed data blocks according to the marked block bit numbers, and generating the data modification table includes the steps of:
performing modulo operation on the marked block bit numbers to generate a plurality of position numbers;
and arranging the changed data blocks according to the position numbers to generate a data modification table.
Based on the first aspect, in some embodiments of the present invention, the step of arranging the change data blocks by respective position numbers and generating the data modification table includes the steps of:
screening the change data block set to obtain change data blocks with the same position numbers;
storing the changed data blocks with the same position numbers in a linked list to generate a data modification table;
and marking the block bitmap according to the changed data blocks with the same position numbers, and generating a new block bitmap.
Based on the first aspect, in some embodiments of the present invention, the step of marking a preset block bitmap according to a change data block set, and generating a block bitmap of a current timestamp further comprises the steps of:
inquiring whether an old block bitmap exists or not, and if so, copying a latest block bitmap in the old block bitmap and taking the latest block bitmap as a preset block bitmap; if not, acquiring a new block bitmap as a preset block bitmap;
and marking the preset block bitmap according to the change data block set to generate the block bitmap of the current timestamp.
Based on the first aspect, in some embodiments of the present invention, the data recovery is performed according to a block bitmap, a data modification table, and a time address table, and the step of generating real-time recovery data includes the following steps:
a1: acquiring a recovery timestamp;
a2: extracting and judging whether the block bitmap of the recovery timestamp has a marked block bit, if not, directly reading data from a preset full data volume to obtain real-time recovery data; if so, go to step A3;
a3: inquiring in a time pile data structure table according to the recovery time stamp to obtain a time stamp address table corresponding to the recovery time stamp;
a4: inquiring a timestamp address table corresponding to the recovery timestamp according to the recovery timestamp to obtain a corresponding data modification table;
a5: extracting and modifying the block bitmap according to the change data block set in the corresponding data modification table to obtain a new block bitmap;
a6: and updating the preset full volume data according to the changed data block set to generate real-time recovery data.
In a second aspect, an embodiment of the present application provides a system for restoring backup data in real time, including:
the data acquisition module is used for acquiring data to be backed up and a current timestamp in real time;
the comparison module is used for comparing the data to be backed up acquired in real time with the preset full data to obtain a change data block set;
the marking module is used for marking a preset block bitmap according to the change data block set to generate a block bitmap of the current timestamp;
the storage module is used for generating a data modification table according to the block bitmap and the change data block set of the current timestamp;
the updating module is used for updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table;
and the data recovery module is used for recovering the data according to the block bitmap, the data modification table and the time address table to generate real-time recovery data.
In the implementation process, the data to be backed up and the current timestamp are acquired in real time through the data acquisition module, and then the data to be backed up acquired in real time is compared with the preset full data through the comparison module to obtain a change data block set; the marking module marks a preset block bitmap according to the change data block set to generate a block bitmap of the current timestamp; when the block bitmap of the changed data block, which is modified by the latest timestamp, determines whether the changed data block needs to be extracted or is directly read from the full data when the data is recovered, and when the block bitmap has no mark, the full data can be directly read, so that the data recovery rate is accelerated; then the storage module generates a data modification table according to the block bitmap and the change data block set of the current timestamp; the data modification table is realized on the basis of an array and open chain mode, is used for storing data areas of modified data blocks in the same timestamp, and is stored in corresponding positions according to block position number modulo, so that corresponding data can be quickly retrieved in data recovery, O (1) complexity recovery is realized, space-time complexity is reduced, and data recovery speed is improved; then the updating module updates the preset time address table according to the current timestamp and the data modification table to generate a new time address table; the time address table comprises a time stamp address table and a time pile data structure table, the latest modified hash time stamp address table is an array address table, which determines the hash array from which the data block should be read and the hash data modification table of the data change block, the array module is used for storing the change data block, list link and the same module number sequence are carried out, when data recovery is needed, the data recovery module carries out data recovery on the data according to the block bitmap, the data modification table and the time address table, and real-time recovery data are generated. The time heap data structure table is a millisecond-based time heap data structure table, can realize real-time tracking and recording of block data change through a block bitmap, a data modification table and a time address table, and can be dynamically retrieved in the data recovery process. The online millisecond-level real-time data recovery is realized by mounting the initial full-amount backup data and the continuously changed data volume.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing one or more programs; a processor. The program or programs, when executed by a processor, implement the method of any of the first aspects as described above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method according to any one of the first aspect described above.
The embodiment of the invention at least has the following advantages or beneficial effects:
the embodiment of the invention provides a method and a system for recovering backup data in real time, wherein data to be backed up and a current timestamp are acquired in real time, and then the data to be backed up acquired in real time is compared with preset full data to obtain a change data block set; marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp; when the block bitmap of the changed data block, which is modified by the latest timestamp, determines whether the changed data block needs to be extracted or is directly read from the full data when the data is recovered, and when the block bitmap has no mark, the full data can be directly read, so that the data recovery rate is accelerated; then generating a data modification table according to the block bitmap and the change data block set of the current timestamp; the data modification table is realized on the basis of an array and open chain mode, is used for storing data areas of modified data blocks in the same timestamp, and is stored in corresponding positions according to block position number modulo, so that corresponding data can be quickly retrieved in data recovery, O (1) complexity recovery is realized, space-time complexity is reduced, and data recovery speed is improved; then, updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table; the time address table comprises a time stamp address table and a time pile data structure table, the latest modified hash time stamp address table is an array address table, which determines the hash array from which the data block should be read and the hash data modification table of the data change block, the array module is used for storing the change data block, list link and the same module number sequence are carried out, when data recovery is needed, data recovery is carried out on the data according to the block bitmap, the data modification table and the time address table, and real-time recovery data are generated. The time heap data structure table is a millisecond-based time heap data structure table, can realize real-time tracking and recording of block data change through a block bitmap, a data modification table and a time address table, and can be dynamically retrieved in the data recovery process. The online millisecond-level real-time data recovery is realized by mounting the initial full-amount backup data and the continuously changed data volume.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a flowchart of a method for restoring backup data in real time according to an embodiment of the present invention;
fig. 2 is a block diagram of a system for real-time recovery of backup data according to an embodiment of the present invention;
fig. 3 is a block diagram of an electronic device according to an embodiment of the present invention.
Icon: 110-a data acquisition module; 120-a comparison module; 130-a labeling module; 140-a storage module; 150-an update module; 160-a data recovery module; 101-a memory; 102-a processor; 103-communication interface.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Examples
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for restoring backup data in real time according to an embodiment of the present invention. The method for recovering the backup data in real time comprises the following steps:
step S110: acquiring data to be backed up and a current timestamp in real time; in the process of full data backup, new data writing and data modification exist, the data to be backed up can be obtained in real time, and the current timestamp is used for authenticating the data generation time, so that whether the data is tampered after being generated is verified.
Step S120: comparing the data to be backed up acquired in real time with preset full data to obtain a change data block set; the preset full data is the primary full data backup realized by adopting the copy-on-write snapshot capability of the drive module, and the backup full data is placed on the mounted backup volume device. In this process, there will be new data writes and the drive module is initiating the backup process. The data to be backed up acquired in real time is compared with the preset full data, which data blocks are modified or newly written can be obtained, a plurality of data blocks can be modified or newly written at the same time, and the modified or newly written data blocks form a changed data block set.
Step S130: marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp; the block bitmap records the position of the modified data block in each timestamp and marks the data block in the block bitmap, so that the block bitmap of the current timestamp is generated, meanwhile, the bitmap has a progressive relation, namely the latest bitmap is updated on the basis of copying the last bitmap, and when the outdated timestamp bitmap is destroyed, the bitmap is required to perform redundancy elimination operation. The block bitmap includes the block bit number of each data block.
The step of marking a preset block bitmap according to the change data block set and generating the block bitmap of the current timestamp further comprises the following steps:
inquiring whether an old block bitmap exists or not, and if so, copying a latest block bitmap in the old block bitmap and taking the latest block bitmap as a preset block bitmap; if not, acquiring a new block bitmap as a preset block bitmap; if the block bitmap is the first block bitmap, the last block bitmap does not need to be copied for updating, otherwise, the last block bitmap is copied, and the block bitmap is updated on the basis of the last block bitmap.
And marking the preset block bitmap according to the change data block set to generate the block bitmap of the current timestamp. The change data blocks are collected with a plurality of change data blocks, and the change data blocks are marked on a preset block bitmap, so that a new block bitmap is obtained and used as a block bitmap of the current timestamp.
Step S140: generating a data modification table according to the block bitmap and the change data block set of the current timestamp; the generating of the data modification table includes the following steps:
firstly, acquiring a block bit number marked in a block bitmap of a current timestamp;
then, the changed data blocks are arranged according to the marked block bit numbers to generate a data modification table. The data modification table is a hash block data modification table, is realized based on an array and an open chain mode, is used for storing a data area of modified block data in the same timestamp, and is modulo according to a block number, and then is stored in a corresponding position according to a modulo value, and the step of arranging comprises the following steps:
firstly, carrying out modular operation on the marked block bit numbers to generate a plurality of position numbers; firstly, the block bit number of the changed data block is obtained, the estimated data capacity value is obtained, the block bit number of the changed data block and the estimated data capacity value are subjected to modular operation, and a value can be obtained and serves as the position number of the changed data block.
And secondly, arranging the changed data blocks according to the position numbers to generate a data modification table. The plurality of change data blocks have a plurality of position numbers, and the change data blocks are correspondingly stored according to the position numbers, so that a data modification table is obtained. For example, if the change data block has a change data block a and a change data block B, and the position numbers of the change data block a and the change data block B are 98 and 99, respectively, the change data block a is placed in the cell having the position number of 98, and the change data block B is placed in the cell having the position number of 99, thereby obtaining the data modification table. When the position numbers conflict, that is, the obtained position numbers are the same, the method may include the steps of:
firstly, screening a change data block set to obtain change data blocks with the same position numbers; and comparing the position numbers of all the changed data blocks in the changed data block set to obtain the changed data blocks with the same position numbers.
Then, storing the linked list of the changed data blocks with the same position number to generate a data modification table; and storing the changed data blocks with the same position numbers in a linked list to obtain a data modification table. The linked list storage belongs to the prior art and is not described in detail herein.
And finally, marking the block bitmap according to the changed data blocks with the same position numbers to obtain a new block bitmap. And marking the changed data blocks with the same position numbers in the block bitmap to obtain a new fast bitmap.
Step S150: updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table; the time address table is used for storing a data modification table and a time heap data structure, and comprises a time stamp address table and a time heap data structure table. The updating comprises the following steps:
firstly, updating a preset timestamp address table according to a data modification table and a current timestamp to generate a new timestamp address table; the timestamp address table is a hash timestamp-based address table, a plurality of hash block data modification tables in the system are subjected to primary storage mapping, and when specific time is specified for recovery, specific block data table addresses can be retrieved, so that corresponding block data can be retrieved from the hash table of the data table addresses, and the two retrievals are based on the time complexity O (1) level.
And then, updating the preset time pile data table according to the new time stamp address table and the current time stamp to generate a new time pile data structure table. The time-heap data structure table is a millisecond-based time-heap data structure table from which a set of time-stamp indexed time-stamp address tables can be found within a logarithmic level.
Step S160: and recovering the data according to the block bitmap, the data modification table and the time address table to generate real-time recovery data. The data recovery process is to mount the data volume of the first full backup of the clone and the data volume of the data change tracking. The process of performing read-write operation on the first full volume data volume mounted currently comprises the following steps:
a1: acquiring a recovery timestamp; the recovery time stamp is the point in time at which the data needs to be recovered. The recovery time stamp is acquired by inputting a time point at which recovery is required by the user.
A2: extracting and judging whether the block bitmap of the recovery timestamp has a marked block bit, if not, directly reading data from a preset full data volume to obtain real-time recovery data; retrieving in the fast bitmap according to the obtained recovery timestamp, thereby retrieving a block bitmap of the recovery timestamp; and further searching the block bitmap of the searched recovery time stamp, judging whether any data block is marked, and directly reading the data block from the full data volume if the data block is not marked. If so, step A3 is performed.
A3: inquiring in a time pile data structure table according to the recovery time stamp to obtain a time stamp address table corresponding to the recovery time stamp; the time heap data structure table may find a set of time stamp index time stamp address tables within a logarithmic level.
A4: inquiring a timestamp address table corresponding to the recovery timestamp according to the recovery timestamp to obtain a corresponding data modification table; and each timestamp is correspondingly stored with a data modification table corresponding to the timestamp, and the data modification table corresponding to the recovery timestamp can be obtained by inquiring the recovery timestamp in the timestamp address table.
A5: and extracting and modifying the block bitmap according to the change data block set in the corresponding data modification table to obtain a new block bitmap. And reading the hash block data modification table with the latest data change by module number according to the new-old relation between the blocks modified by the bitmap marks and the system backup dependency time stamps, and returning the hash block data modification table to an upper layer reading interface.
A6: and updating the preset full volume data according to the changed data block set to generate real-time recovery data. And updating the data blocks in the current preset full data volume according to the changed data blocks in the changed data block set, modifying the block bitmap, repeating the steps until all blocks of the bitmap are marked, and if all data in the current timestamp are dynamically restored, directly submitting the subsequent data block request to the backup volume.
In the implementation process, the data to be backed up and the current timestamp are acquired in real time, and then the data to be backed up acquired in real time is compared with the preset full data to obtain a change data block set; marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp; when the block bitmap of the changed data block, which is modified by the latest timestamp, determines whether the changed data block needs to be extracted or is directly read from the full data when the data is recovered, and when the block bitmap has no mark, the full data can be directly read, so that the data recovery rate is accelerated; then generating a data modification table according to the block bitmap and the change data block set of the current timestamp; the data modification table is realized on the basis of an array and open chain mode, is used for storing data areas of modified data blocks in the same timestamp, and is stored in corresponding positions according to block position number modulo, so that corresponding data can be quickly retrieved in data recovery, O (1) complexity recovery is realized, space-time complexity is reduced, and data recovery speed is improved; then, updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table; the time address table comprises a time stamp address table and a time pile data structure table, the latest modified hash time stamp address table is an array address table, which determines the hash array from which the data block should be read and the hash data modification table of the data change block, the array module is used for storing the change data block, list link and the same module number sequence are carried out, when data recovery is needed, data recovery is carried out on the data according to the block bitmap, the data modification table and the time address table, and real-time recovery data are generated. The time heap data structure table is a millisecond-based time heap data structure table, can realize real-time tracking and recording of block data change through a block bitmap, a data modification table and a time address table, and can be dynamically retrieved in the data recovery process. The online millisecond-level real-time data recovery is realized by mounting the initial full-amount backup data and the continuously changed data volume.
Based on the same inventive concept, the present invention further provides a real-time backup data recovery system, please refer to fig. 2, and fig. 2 is a structural block diagram of the real-time backup data recovery system according to an embodiment of the present invention. The backup data real-time recovery system comprises:
a data obtaining module 110, configured to obtain data to be backed up and a current timestamp in real time;
the comparison module 120 is configured to compare the data to be backed up obtained in real time with preset full data to obtain a changed data block set;
a marking module 130, configured to mark a preset block bitmap according to the change data block set, and generate a block bitmap of a current timestamp;
a storage module 140, configured to generate a data modification table according to the block bitmap and the change data block set of the current timestamp;
the updating module 150 is configured to update the preset time address table according to the current timestamp and the data modification table, and generate a new time address table;
and the data recovery module 160 is configured to perform data recovery on the data according to the block bitmap, the data modification table, and the time address table, and generate real-time recovery data.
In the implementation process, the data to be backed up and the current timestamp are acquired in real time through the data acquisition module 110, and then the comparison module 120 compares the data to be backed up acquired in real time with the preset full data to obtain a change data block set; the marking module 130 marks a preset block bitmap according to the change data block set to generate a block bitmap of the current timestamp; when the block bitmap of the changed data block, which is modified by the latest timestamp, determines whether the changed data block needs to be extracted or is directly read from the full data when the data is recovered, and when the block bitmap has no mark, the full data can be directly read, so that the data recovery rate is accelerated; then the storage module 140 generates a data modification table according to the block bitmap and the change data block set of the current timestamp; the data modification table is realized on the basis of an array and open chain mode, is used for storing data areas of modified data blocks in the same timestamp, and is stored in corresponding positions according to block position number modulo, so that corresponding data can be quickly retrieved in data recovery, O (1) complexity recovery is realized, space-time complexity is reduced, and data recovery speed is improved; then, the updating module 150 updates the preset time address table according to the current timestamp and the data modification table to generate a new time address table; the time address table comprises a time stamp address table and a time heap data structure table, the latest modified hash time stamp address table is an array address table, which determines the hash array from which the data block should be read and the hash data modification table of the data change block, the array module is used for storing the change data block, list link and the same module number sequence are adopted, when data recovery is needed, the data recovery module 160 recovers data according to the block bitmap, the data modification table and the time address table to generate real-time recovery data. The time heap data structure table is a millisecond-based time heap data structure table, can realize real-time tracking and recording of block data change through a block bitmap, a data modification table and a time address table, and can be dynamically retrieved in the data recovery process. The online millisecond-level real-time data recovery is realized by mounting the initial full-amount backup data and the continuously changed data volume.
Referring to fig. 3, fig. 3 is a schematic structural block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device comprises a memory 101, a processor 102 and a communication interface 103, wherein the memory 101, the processor 102 and the communication interface 103 are electrically connected with each other directly or indirectly to realize the transmission or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 101 may be used to store software programs and modules, such as program instructions/modules corresponding to the real-time backup data recovery system provided in the embodiments of the present application, and the processor 102 executes the software programs and modules stored in the memory 101, so as to execute various functional applications and data processing. The communication interface 103 may be used for communicating signaling or data with other node devices.
The Memory 101 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 102 may be an integrated circuit chip having signal processing capabilities. The Processor 102 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
It will be appreciated that the configuration shown in fig. 3 is merely illustrative and that the electronic device may include more or fewer components than shown in fig. 3 or have a different configuration than shown in fig. 3. The components shown in fig. 3 may be implemented in hardware, software, or a combination thereof.
In the embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. The above-described system embodiments are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
To sum up, in the data backup method and system provided in the embodiments of the present application, the data to be backed up and the current timestamp are obtained in real time, and then the data to be backed up obtained in real time is compared with the preset full data, so as to obtain the changed data block set; marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp; when the block bitmap of the changed data block, which is modified by the latest timestamp, determines whether the changed data block needs to be extracted or is directly read from the full data when the data is recovered, and when the block bitmap has no mark, the full data can be directly read, so that the data recovery rate is accelerated; then generating a data modification table according to the block bitmap and the change data block set of the current timestamp; the data modification table is realized on the basis of an array and open chain mode, is used for storing data areas of modified data blocks in the same timestamp, and is stored in corresponding positions according to block position number modulo, so that corresponding data can be quickly retrieved in data recovery, O (1) complexity recovery is realized, space-time complexity is reduced, and data recovery speed is improved; then, updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table; the time address table comprises a time stamp address table and a time pile data structure table, the latest modified hash time stamp address table is an array address table, which determines the hash array from which the data block should be read and the hash data modification table of the data change block, the array module is used for storing the change data block, list link and the same module number sequence are carried out, when data recovery is needed, data recovery is carried out on the data according to the block bitmap, the data modification table and the time address table, and real-time recovery data are generated. The time heap data structure table is a millisecond-based time heap data structure table, can realize real-time tracking and recording of block data change through a block bitmap, a data modification table and a time address table, and can be dynamically retrieved in the data recovery process. The online millisecond-level real-time data recovery is realized by mounting the initial full-amount backup data and the continuously changed data volume.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. A real-time recovery method for backup data is characterized by comprising the following steps:
acquiring data to be backed up and a current timestamp in real time;
comparing the data to be backed up acquired in real time with preset full data to obtain a change data block set;
marking a preset block bitmap according to the change data block set to generate a block bitmap of a current timestamp;
generating a data modification table according to the block bitmap and the change data block set of the current timestamp;
updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table;
and recovering the data according to the block bitmap, the data modification table and the time address table to generate real-time recovery data.
2. The method for real-time recovery of backup data according to claim 1, wherein the step of updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table comprises the following steps:
updating a preset timestamp address table according to the data modification table and the current timestamp to generate a new timestamp address table;
and updating the preset time pile data table according to the new time stamp address table and the current time stamp to generate a new time pile data structure table.
3. The method for real-time recovery of backup data according to claim 1, wherein the step of generating a data modification table according to the block bitmap of the current timestamp and the change data block set comprises the steps of:
acquiring a block bit number marked in a block bitmap of a current timestamp;
and arranging the changed data blocks according to the marked block bit numbers to generate a data modification table.
4. The method for real-time recovery of backup data according to claim 3, wherein the step of arranging the changed data blocks according to the marked block bit numbers and generating the data modification table comprises the steps of:
performing modulo operation on the marked block bit numbers to generate a plurality of position numbers;
and arranging the changed data blocks according to the position numbers to generate a data modification table.
5. The method for real-time recovery of backup data according to claim 4, wherein the step of arranging the changed data blocks according to the respective location numbers and generating the data modification table comprises the steps of:
screening the change data block set to obtain change data blocks with the same position numbers;
storing the changed data blocks with the same position numbers in a linked list to generate a data modification table;
and marking the block bitmap according to the changed data blocks with the same position numbers, and generating a new block bitmap.
6. The method for real-time recovery of backup data according to claim 1, wherein said step of marking a preset block bitmap according to a change data block set, and generating a block bitmap of a current time stamp further comprises the steps of:
inquiring whether an old block bitmap exists or not, and if so, copying a latest block bitmap in the old block bitmap and taking the latest block bitmap as a preset block bitmap; if not, acquiring a new block bitmap as a preset block bitmap;
and marking the preset block bitmap according to the change data block set to generate the block bitmap of the current timestamp.
7. The method for real-time recovery of backup data according to claim 1, wherein the step of performing data recovery on the data according to the block bitmap, the data modification table, and the time address table to generate real-time recovery data comprises the following steps:
a1: acquiring a recovery timestamp;
a2: extracting and judging whether the block bitmap of the recovery timestamp has a marked block bit, if not, directly reading data from a preset full data volume to obtain real-time recovery data; if so, go to step A3;
a3: inquiring in a time pile data structure table according to the recovery time stamp to obtain a time stamp address table corresponding to the recovery time stamp;
a4: inquiring a timestamp address table corresponding to the recovery timestamp according to the recovery timestamp to obtain a corresponding data modification table;
a5: extracting and modifying the block bitmap according to the change data block set in the corresponding data modification table to obtain a new block bitmap;
a6: and updating the preset full volume data according to the changed data block set to generate real-time recovery data.
8. A system for real-time recovery of backup data, comprising:
the data acquisition module is used for acquiring data to be backed up and a current timestamp in real time;
the comparison module is used for comparing the data to be backed up acquired in real time with the preset full data to obtain a change data block set;
the marking module is used for marking a preset block bitmap according to the change data block set to generate a block bitmap of the current timestamp;
the storage module is used for generating a data modification table according to the block bitmap and the change data block set of the current timestamp;
the updating module is used for updating the preset time address table according to the current timestamp and the data modification table to generate a new time address table;
and the data recovery module is used for recovering the data according to the block bitmap, the data modification table and the time address table to generate real-time recovery data.
9. An electronic device, comprising:
a memory for storing one or more programs;
a processor;
the one or more programs, when executed by the processor, implement the method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110503081.7A CN112988473B (en) | 2021-05-10 | 2021-05-10 | Backup data real-time recovery method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110503081.7A CN112988473B (en) | 2021-05-10 | 2021-05-10 | Backup data real-time recovery method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112988473A true CN112988473A (en) | 2021-06-18 |
| CN112988473B CN112988473B (en) | 2021-11-23 |
Family
ID=76337371
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110503081.7A Active CN112988473B (en) | 2021-05-10 | 2021-05-10 | Backup data real-time recovery method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988473B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115292094A (en) * | 2022-08-10 | 2022-11-04 | 广州鼎甲计算机科技有限公司 | Data recovery processing method, apparatus, device, storage medium, and program product |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5907672A (en) * | 1995-10-04 | 1999-05-25 | Stac, Inc. | System for backing up computer disk volumes with error remapping of flawed memory addresses |
| CN101183387A (en) * | 2007-12-14 | 2008-05-21 | 沈阳东软软件股份有限公司 | Increment data capturing method and system |
| CN101604269A (en) * | 2009-07-22 | 2009-12-16 | 中国科学院计算技术研究所 | A kind of management system of Backup Data and method |
| CN102479118A (en) * | 2010-11-30 | 2012-05-30 | 英业达集团(天津)电子技术有限公司 | Incremental backup method for data blocks |
| CN103049353A (en) * | 2012-12-21 | 2013-04-17 | 华为技术有限公司 | Data backup method and related device |
| CN106681862A (en) * | 2016-12-15 | 2017-05-17 | 华为技术有限公司 | Method and apparatus for incremental recovery of data |
| CN112114999A (en) * | 2020-09-01 | 2020-12-22 | 阿里云计算有限公司 | Data backup method, data recovery method, data backup device, data recovery device and electronic equipment |
| CN112328435A (en) * | 2020-12-07 | 2021-02-05 | 武汉绿色网络信息服务有限责任公司 | Method, device, equipment and storage medium for backing up and recovering target data |
-
2021
- 2021-05-10 CN CN202110503081.7A patent/CN112988473B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5907672A (en) * | 1995-10-04 | 1999-05-25 | Stac, Inc. | System for backing up computer disk volumes with error remapping of flawed memory addresses |
| CN101183387A (en) * | 2007-12-14 | 2008-05-21 | 沈阳东软软件股份有限公司 | Increment data capturing method and system |
| CN101604269A (en) * | 2009-07-22 | 2009-12-16 | 中国科学院计算技术研究所 | A kind of management system of Backup Data and method |
| CN102479118A (en) * | 2010-11-30 | 2012-05-30 | 英业达集团(天津)电子技术有限公司 | Incremental backup method for data blocks |
| CN103049353A (en) * | 2012-12-21 | 2013-04-17 | 华为技术有限公司 | Data backup method and related device |
| CN106681862A (en) * | 2016-12-15 | 2017-05-17 | 华为技术有限公司 | Method and apparatus for incremental recovery of data |
| CN112114999A (en) * | 2020-09-01 | 2020-12-22 | 阿里云计算有限公司 | Data backup method, data recovery method, data backup device, data recovery device and electronic equipment |
| CN112328435A (en) * | 2020-12-07 | 2021-02-05 | 武汉绿色网络信息服务有限责任公司 | Method, device, equipment and storage medium for backing up and recovering target data |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115292094A (en) * | 2022-08-10 | 2022-11-04 | 广州鼎甲计算机科技有限公司 | Data recovery processing method, apparatus, device, storage medium, and program product |
| CN115292094B (en) * | 2022-08-10 | 2023-11-14 | 广州鼎甲计算机科技有限公司 | Data recovery processing method, device, equipment, storage medium and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112988473B (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20150227538A1 (en) | Method and apparatus for eventually consistent delete in a distributed data store | |
| US11321291B2 (en) | Persistent version control for data transfer between heterogeneous data stores | |
| KR20040053142A (en) | Efficient management of large files | |
| US7487385B2 (en) | Apparatus and method for recovering destroyed data volumes | |
| CN112306743B (en) | Data processing method, device, electronic equipment and computer storage medium | |
| US11429498B2 (en) | System and methods of efficiently resyncing failed components without bitmap in an erasure-coded distributed object with log-structured disk layout | |
| CN112988473B (en) | Backup data real-time recovery method and system | |
| CN116467277A (en) | Metadata processing method, device, equipment, storage medium and product | |
| US11567839B2 (en) | Snapshot-based data corruption detection | |
| US20220121523A1 (en) | Identifying database backup copy chaining | |
| US20230124036A1 (en) | In-place garbage collection for state machine replication | |
| CN111770158B (en) | Cloud platform recovery method and device, electronic equipment and computer readable storage medium | |
| US11645333B1 (en) | Garbage collection integrated with physical file verification | |
| CN108694209B (en) | Distributed index method based on object and client | |
| US20050283665A1 (en) | Storing object recovery information within the object | |
| CN112988474B (en) | Method, system, equipment and medium for backing up hot data by mass small files | |
| CN110399098A (en) | A kind of data processing method and electronic equipment | |
| CN116257531B (en) | Database space recovery method | |
| Long et al. | Voting without version numbers | |
| CN114020764B (en) | Processing method and system for high-performance data clone copy | |
| US11886277B2 (en) | Systems, apparatuses, and methods for assessing recovery viability of backup databases | |
| US20230385239A1 (en) | Distributed management of file modification-time field | |
| KR20210104525A (en) | System for reporting of digital evidence by sorting data collection from object disk | |
| CN112988472A (en) | Method and system for retrieving incremental data backup | |
| CN115982782A (en) | Website file tamper-proof method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |