CN111159313B - Method, system, device and storage medium for database rapid synthesis backup - Google Patents
Method, system, device and storage medium for database rapid synthesis backup Download PDFInfo
- Publication number
- CN111159313B CN111159313B CN201911404692.5A CN201911404692A CN111159313B CN 111159313 B CN111159313 B CN 111159313B CN 201911404692 A CN201911404692 A CN 201911404692A CN 111159313 B CN111159313 B CN 111159313B
- Authority
- CN
- China
- Prior art keywords
- file
- incremental
- image copy
- backup
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a method, a system, a device and a storage medium for database rapid synthesis backup, wherein the method comprises the following steps: detecting the image copy, and comparing the database file with the image copy to obtain incremental data; backing up the incremental data and outputting the file name of the backup in real time; and acquiring a corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the increment is synthesized. The system comprises: the device comprises a comparison module, a backup module and a synthesis module. The device comprises a memory and a processor for executing the database rapid synthesis backup method. By using the method and the device, the speed of synthesizing the backup data by the database can be improved. The method, the system, the device and the storage medium for database rapid synthetic backup can be widely applied to the field of data backup synthesis.
Description
Technical Field
The present invention relates to the field of data backup and synthesis, and in particular, to a method, a system, an apparatus, and a storage medium for database fast synthesis and backup.
Background
The method comprises the steps that full backup is needed for the first time of data synthesis backup, incremental backup and incremental synthesis are conducted on incremental files afterwards, Oracle is composed of a plurality of data files, the incremental backup and the incremental synthesis of each data file are independent, and concurrent incremental backup and incremental synthesis of the data files are achieved by setting parallelism or distributing multiple channels for some users so as to save synthesis backup time.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a method, a system, a device and a storage medium for database fast synthetic backup, which can reduce the time required for data synthetic backup.
The first technical scheme adopted by the invention is as follows: a method for quickly synthesizing and backing up a database comprises the following steps:
detecting the image copy, and comparing the database file with the image copy to obtain incremental data;
backing up the incremental data and outputting the file name of the backup in real time;
and acquiring a corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the increment is synthesized.
Further, the method also comprises the following steps:
and storing and updating the image copy after the incremental synthesis into the database.
Further, the step of detecting the image copy, comparing the database file with the image copy to obtain incremental data, specifically includes:
detecting that no database image copy exists, and creating an image copy;
and acquiring the current database file and comparing the current database file with the image copy to obtain incremental data.
Further, the step of backing up the incremental data and outputting the file name of the backed up in real time specifically includes:
acquiring an increment file according to the increment data and creating a processing thread;
backing up each incremental file based on the processing thread, and outputting the file name of the incremental file which is backed up in real time;
further, the step of obtaining a corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the incremental file is synthesized specifically further includes:
acquiring a file name of the finished backup, and searching for a corresponding incremental file based on the file name;
and analyzing the incremental file to obtain the data block of the incremental file and synthesizing the data block into the corresponding image copy to obtain the image copy after the increment is synthesized.
Further, the creating processing threads are processing threads created by the RMAN process according to a preset rule in a corresponding number.
Further, the synthesizing the data block into the corresponding image copy further includes analyzing the image copy format and the incremental file format, and determining whether the image copy and the incremental file format are consistent.
The second technical scheme adopted by the invention is as follows: a database rapid synthetic backup system comprising:
the comparison module is used for acquiring the database file and the image copy and comparing the database file with the image copy to obtain incremental data;
the backup module is used for backing up the incremental data and outputting the file name of the backup in real time;
and the synthesis module is used for acquiring the corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the increment is synthesized.
Further, still include:
and the updating module is used for storing the image copy after the increment is synthesized and updating the image copy into the database.
The third technical scheme adopted by the invention is as follows: a database rapid synthetic backup apparatus, comprising:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement a database rapid synthetic backup method as described above.
The fourth technical scheme adopted by the invention is as follows: a storage medium having stored therein instructions executable by a processor, the storage medium comprising: the processor-executable instructions, when executed by the processor, are for implementing a database rapid synthetic backup method as described above.
The method, the system, the device and the storage medium have the advantages that: the invention enables the system to start the incremental synthesis of the mirror image copy of each data file after the incremental backup of the data file is finished by outputting the file name of the finished backup in real time, and does not need to start the incremental backup of all the data files after the incremental backup is finished, thereby greatly improving the speed of the data synthesis backup.
Drawings
FIG. 1 is a flow chart illustrating the steps of a method for fast synthetic backup of a database according to the present invention;
FIG. 2 is a block diagram of a database rapid synthetic backup system according to the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
As shown in fig. 1, the present invention provides a method for fast synthesizing and backing up a database, which comprises the following steps:
s101, detecting the image copy, and comparing the database file with the image copy to obtain incremental data.
Specifically, the RMAN process is created to compare the existing database with the image copy generated by the previous backup, and a new incremental backup is generated for the changed data block, i.e. the incremental data, if there is no image copy generated by the previous backup, which indicates that the backup is the first backup, the file of the database is fully backed up and the image copy is created.
S102, backing up the incremental data and outputting the file name of the backed-up incremental data in real time;
s103, acquiring a corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the increment is synthesized.
Specifically, incremental data are backed up based on the RMAN process, the file names of the incremental files which are backed up are output in real time in the backup process of all the incremental data, the file names output by the RMAN are captured by the system, the corresponding incremental files are analyzed to obtain data blocks in the incremental files, the incremental data blocks are synthesized to the image copy, and the steps are repeated until all the incremental data are synthesized to the image copy.
S104, storing the image copy after the increment is synthesized, and updating the image copy into a database;
specifically, the last step of completing the data synthesis backup is to update the synthesized image copy to the database and replace the old image copy, and there are two options, one is to automatically update the image copy information in the data based on RMAN.
As a preferred embodiment of the method, the step of detecting the image copy and comparing the database file with the image copy to obtain the incremental data specifically includes:
detecting that no database image copy exists, and creating an image copy;
and acquiring the current database file and comparing the current database file with the image copy to obtain incremental data.
Specifically, when no image copy is detected, a new image copy needs to be created to perform full backup on all files in the database, the incremental data is difference data obtained by comparing the database file with the image copy backed up at the previous time, and the image copy changes after each synthetic backup.
As a preferred embodiment of the method, the step of backing up the incremental data and outputting the filename of the backed up file in real time specifically includes:
acquiring an increment file according to the increment data and creating a processing thread;
backing up each incremental file based on the processing thread, and outputting the file name of the incremental file which is backed up in real time;
specifically, the RMAN process sets a processing thread for creating a processing task based on data of the incremental file, and may specifically be determined according to the number of CPUs, the number of main storage disks, the number of disks for backup storage, and the number of files of the incremental file, and in the process of backing up the incremental file, a file name for completing backup is output in real time so that the system can capture the file name.
Further as a preferred embodiment of the method, the step of obtaining the corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the incremental file is synthesized further specifically includes:
acquiring the file name of the finished backup, and searching a corresponding incremental file based on the file name;
and analyzing the incremental file to obtain the data block of the incremental file and synthesizing the data block into the corresponding image copy to obtain the image copy after the increment is synthesized.
Specifically, in cooperation with real-time output of the RMAN process, an incremental file is captured and analyzed to obtain an incremental data block therein for synthesizing an image copy, so that two cyclic processes of incremental backup and incremental synthesis are performed synchronously until the RMAN process finishes processing all data files and an incremental synthesis thread completes the whole synthetic backup after all incremental backup sets are synthesized.
Further as a preferred embodiment of the method, the creating processing threads are processing threads created by the RMAN process according to a preset rule in a corresponding number.
Further as a preferred embodiment of the method, the synthesizing the data block into the image copy corresponding thereto further includes analyzing the image copy format and the incremental file format, and confirming whether the image copy and the incremental file format are consistent.
Specifically, the process number of the RMAN process may be set according to the number of CPUs, the number of main storage disks, the number of backup disks, and the number of files of the incremental file, and the incremental data may be backed up by using a plurality of processing threads of the RMAN process, so that the speed of data synthesis backup may be further increased, and whether the process of data backup and data synthesis is faulty or not may be determined by detecting whether the image copy format is consistent with the incremental file format.
The specific embodiment of the invention is as follows:
the method is particularly applied to a scene of rapidly synthesizing incremental backups in Oracle, Oracle checks that no database image copy exists, creates an image copy to perform full backup on data of a database, creates an RMAN process to compare the existing data with the image copy generated by the previous backup, creates a processing thread based on the RMAN process, generates incremental backups for changed data blocks, namely the incremental data, outputs the incremental file names completing the backup in the data blocks in real time in the process of performing the backup on the incremental data, captures the incremental file names completing the backup, obtains the data blocks corresponding to each incremental file by analyzing the image copy format and the incremental backup format, synthesizes the data blocks corresponding to the incremental files into the image copy, circulates the steps until all the incremental data complete the incremental backups and the incremental backups, stores the synthesized image copy into a storage module, and updates a new image copy to the database after the storage is completed, and finally, creating a snapshot for the file system for storing the image copy, wherein the snapshot is used for restoring the database to the backup time point in the future.
As shown in fig. 2, a system for rapid synthetic backup of a database includes:
the comparison module is used for acquiring the database file and the image copy and comparing the database file with the image copy to obtain incremental data;
the backup module is used for backing up the incremental data and outputting the file name of the backup in real time;
the synthesis module is used for acquiring a corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the increment is synthesized;
and the updating module is used for storing the image copy after the increment is synthesized and updating the image copy into the database.
As a further preferred embodiment of the present system, the comparison module further comprises:
the detection submodule is used for detecting that no database image copy exists and creating an image copy;
and the obtaining submodule is used for obtaining the current database file and comparing the current database file with the image copy to obtain incremental data.
As a further preferred embodiment of the present system, the backup module further includes:
the creating submodule is used for acquiring the increment file according to the increment data and creating a processing thread;
the output submodule is used for backing up each incremental file based on the processing thread and outputting the file name of the incremental file which is backed up in real time;
as a further preferred embodiment of the present system, the synthesis module further comprises:
the searching submodule is used for acquiring the file name of the finished backup and searching the corresponding incremental file based on the file name;
and the analysis submodule is used for analyzing the incremental file to obtain the data block of the incremental file and synthesizing the data block into the corresponding image copy to obtain the image copy after the increment is synthesized.
The contents in the above method embodiments are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above method embodiment, and the beneficial effects achieved by the present system embodiment are also the same as those achieved by the above method embodiment.
A database rapid synthesis backup device:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement a database rapid synthetic backup method as described above.
The contents in the above method embodiments are all applicable to the present apparatus embodiment, the functions specifically implemented by the present apparatus embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present apparatus embodiment are also the same as those achieved by the above method embodiments.
A storage medium having stored therein instructions executable by a processor, the storage medium comprising: the processor-executable instructions, when executed by the processor, are for implementing a database rapid synthetic backup method as described above.
The contents in the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present storage medium embodiment are also the same as those achieved by the above method embodiments.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. A method for quickly synthesizing and backing up a database is characterized in that:
detecting the image copy, and comparing the database file with the image copy to obtain incremental data;
backing up the incremental data and outputting the file name of the backup in real time;
when a new file name is captured, synchronously acquiring a corresponding increment file based on the file name and synthesizing the increment file into an image copy to obtain the image copy after the increment is synthesized;
the step of backing up the incremental data and outputting the filename of the backed-up file in real time specifically includes:
acquiring an increment file according to the increment data and creating a processing thread;
backing up each incremental file based on the processing thread, and outputting the file name of the incremental file which is backed up in real time;
wherein, when capturing a new file name, synchronously acquiring a corresponding incremental file based on the file name and synthesizing the incremental file into the image copy to obtain the image copy after the incremental file is synthesized, which specifically comprises the following steps:
acquiring a file name of a newly finished backup, and searching for a corresponding incremental file based on the file name; acquiring a new file name and backing up the incremental data simultaneously and parallelly;
and analyzing the incremental file to obtain the data block of the incremental file and synthesizing the data block into the corresponding image copy to obtain the image copy after the increment is synthesized.
2. The method for rapid synthetic backup of database according to claim 1, further comprising:
and storing the image copy after the incremental synthesis, and updating the image copy into a database.
3. The method for fast synthetic backup of a database according to claim 1, wherein the step of detecting the image copy and comparing the database file with the image copy to obtain the incremental data specifically comprises:
detecting that no database image copy exists, and creating an image copy;
and acquiring the current database file and comparing the current database file with the image copy to obtain incremental data.
4. The method for rapid synthetic backup of database according to claim 1, wherein: and the creation processing threads are processing threads of which the RMAN process creates a corresponding number according to a preset rule.
5. The method for rapid synthetic backup of database according to claim 1, wherein: the compositing of the data blocks into their corresponding image copies further includes parsing the format of the image copies and the delta file.
6. A database rapid synthesis backup system is characterized in that:
the comparison module is used for acquiring the database file and the image copy and comparing the database file with the image copy to obtain incremental data;
the backup module is used for backing up the incremental data and outputting the file name of the backup in real time;
the synthesis module is used for synchronously acquiring corresponding increment files based on the file names and synthesizing the increment files into the image copies to obtain the image copies after the increment is synthesized when new file names are captured;
the backup module specifically comprises the following steps: acquiring an increment file according to the increment data and creating a processing thread; backing up each incremental file based on the processing thread, and outputting the file name of the incremental file which is backed up in real time;
the synthesis module specifically comprises the following steps: acquiring a file name of a newly finished backup, and searching for a corresponding incremental file based on the file name; acquiring a new file name and backing up the incremental data simultaneously and parallelly; and analyzing the incremental file to obtain the data block of the incremental file and synthesizing the data block into the corresponding image copy to obtain the image copy after the increment is synthesized.
7. A database rapid synthesis backup device is characterized in that: further comprising:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement a method for rapid synthetic backup of a database according to any of claims 1-5.
8. A storage medium having stored therein instructions executable by a processor, the storage medium comprising: the processor-executable instructions, when executed by a processor, are for implementing a method for fast synthetic backup of a database as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911404692.5A CN111159313B (en) | 2019-12-31 | 2019-12-31 | Method, system, device and storage medium for database rapid synthesis backup |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911404692.5A CN111159313B (en) | 2019-12-31 | 2019-12-31 | Method, system, device and storage medium for database rapid synthesis backup |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111159313A CN111159313A (en) | 2020-05-15 |
| CN111159313B true CN111159313B (en) | 2020-11-13 |
Family
ID=70559544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911404692.5A Active CN111159313B (en) | 2019-12-31 | 2019-12-31 | Method, system, device and storage medium for database rapid synthesis backup |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111159313B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113064760B (en) * | 2021-04-06 | 2022-02-15 | 广州鼎甲计算机科技有限公司 | Database synthesis backup method and device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104714859A (en) * | 2013-12-17 | 2015-06-17 | 南京壹进制信息技术有限公司 | Rapid backup and recovery method of mass files |
| CN104850598A (en) * | 2015-04-28 | 2015-08-19 | 江苏瑞中数据股份有限公司 | Method for recovering backup of real-time database |
| CN107066357A (en) * | 2017-05-31 | 2017-08-18 | 广州鼎甲计算机科技有限公司 | A kind of database synthetic backup and carry restoration methods |
| US10185504B1 (en) * | 2014-11-26 | 2019-01-22 | Acronis International Gmbh | Reducing data transmitted during backup |
| CN109426585A (en) * | 2017-08-22 | 2019-03-05 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus for backing up, restoring database data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981933B (en) * | 2012-12-18 | 2015-05-13 | 武汉邮电科学研究院 | Method and system for data increment backup of sensing layer of Internet of Things |
| CN108804253B (en) * | 2017-05-02 | 2021-08-06 | 中国科学院高能物理研究所 | Parallel operation backup method for mass data backup |
-
2019
- 2019-12-31 CN CN201911404692.5A patent/CN111159313B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104714859A (en) * | 2013-12-17 | 2015-06-17 | 南京壹进制信息技术有限公司 | Rapid backup and recovery method of mass files |
| US10185504B1 (en) * | 2014-11-26 | 2019-01-22 | Acronis International Gmbh | Reducing data transmitted during backup |
| CN104850598A (en) * | 2015-04-28 | 2015-08-19 | 江苏瑞中数据股份有限公司 | Method for recovering backup of real-time database |
| CN107066357A (en) * | 2017-05-31 | 2017-08-18 | 广州鼎甲计算机科技有限公司 | A kind of database synthetic backup and carry restoration methods |
| CN109426585A (en) * | 2017-08-22 | 2019-03-05 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus for backing up, restoring database data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111159313A (en) | 2020-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11221922B2 (en) | System and method for selective compression in a database backup operation | |
| EP2597579B1 (en) | Database backing-up and recovering method and device | |
| CN108376109A (en) | Volume and snapshot | |
| CN108536752B (en) | Data synchronization method, device and equipment | |
| CN107479881B (en) | Method for synchronizing difference codes, storage medium, electronic device and system | |
| CN110287101A (en) | User interface automated testing method, device, computer equipment and storage medium | |
| CN109840194B (en) | Method and system for detecting configuration file | |
| CN111158958B (en) | Backup method, system, device and storage medium capable of quickly recovering data | |
| CN109358898B (en) | Information processing method and device, electronic equipment and storage medium | |
| CN110309227B (en) | Distributed data rollback method, device and computer readable storage medium | |
| CN111159313B (en) | Method, system, device and storage medium for database rapid synthesis backup | |
| CN113360322A (en) | Method and equipment for recovering data based on backup system | |
| CN108241543B (en) | Method, service server and system for executing service operation breakpoint | |
| CN114637870A (en) | Image data processing method, device, equipment and storage medium | |
| US11809279B2 (en) | Automatic IO stream timing determination in live images | |
| CN110991983B (en) | Task processing method, device, medium and equipment | |
| CN111625397B (en) | Service log backup method, cluster, device, electronic equipment and storage medium | |
| CN111984624A (en) | Method and system for data migration through correction migration model | |
| US10922301B1 (en) | Apparatus, computer program, and method for trigger-based tracking of database modifications | |
| CN111522737B (en) | Automatic test verification method and device for front-end interface and storage medium | |
| CN115357429B (en) | Method, device and client for recovering data file | |
| CN114741401B (en) | Data synchronization method, device and medium based on update time and primary key | |
| CN112860492B (en) | Automatic regression testing method and system suitable for core system | |
| CN113626391B (en) | Processing method for cache file of operating system | |
| CN112711641A (en) | Data synchronization method and device for distributed system and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |