CN112433887A - Database data synchronization method, system, device and storage medium - Google Patents
Database data synchronization method, system, device and storage medium Download PDFInfo
- Publication number
- CN112433887A CN112433887A CN202011337993.3A CN202011337993A CN112433887A CN 112433887 A CN112433887 A CN 112433887A CN 202011337993 A CN202011337993 A CN 202011337993A CN 112433887 A CN112433887 A CN 112433887A
- Authority
- CN
- China
- Prior art keywords
- site
- disk
- database
- data
- log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000011084 recovery Methods 0.000 claims abstract description 93
- 230000010076 replication Effects 0.000 claims description 44
- 238000005516 engineering process Methods 0.000 claims description 42
- 230000001360 synchronised effect Effects 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 15
- 230000002093 peripheral effect Effects 0.000 claims description 12
- 230000003362 replicative effect Effects 0.000 claims description 7
- 238000007405 data analysis Methods 0.000 claims description 3
- 230000000295 complement effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 13
- 238000004519 manufacturing process Methods 0.000 description 11
- 238000002360 preparation method Methods 0.000 description 4
- 238000005553 drilling Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1456—Hardware arrangements for backup
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/80—Database-specific techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Hardware Redundancy (AREA)
Abstract
The invention discloses a database data synchronization method, a system, equipment and a storage medium, wherein the method comprises the following steps: synchronously copying the database logs written into the main disk of the first site and the key files to the secondary disk of the first site, synchronously copying the database logs written into the main disk to the database log disk of the second site, and synchronously copying the key files written into the main disk to the key file disk of the second site; when a database log disk of a second site receives a database log copied from a main disk, asynchronously copying the database log copied to the database log disk of the second site to a third site; and when the key file disk of the second site receives the key file copied from the main disk, asynchronously copying the key file copied to the key file disk of the second site to the third site. The invention can realize the recovery of the disaster recovery system at the minute level to the external service without using the application to participate in the complement.
Description
Technical Field
The invention relates to the technical field of data disaster recovery backup, in particular to a database data synchronization method, a system, equipment and a storage medium.
Background
At present, the disaster recovery construction of the domestic financial system mostly adopts a remote disaster recovery scheme, namely, a main center and a disaster recovery center are respectively deployed at two places, so that when a disaster occurs in the main center, the production system can be recovered at the remote disaster recovery center to continuously provide services to the outside. The technical indicators of the conventional disaster Recovery are Recovery Time Object (RTO) and Recovery Point Object (RPO), and ideally, the RPO is 0, and the shorter the RTO, the better.
The disaster recovery and backup replication scheme of the IBM mainframe system is mainly based on a disk replication technology or a database replication technology. (1) Regarding the time RTO required for disaster recovery: because the target disk copied as the disk in the current large-scale host system cannot be used online, when the adopted disaster recovery backup technology is the disk copy technology, the disaster recovery system needs to be restarted, and the time RTO required by the disaster recovery is long. When the adopted disaster recovery backup replication technology is the database replication technology, the system does not need to be restarted, and the time RTO required by disaster recovery backup recovery is short. (2) Recovery data point for disaster recovery RPO: in order to avoid disaster in two places, the disaster recovery center is usually far away from the main center, and the long distance can also cause the technical scheme of the remote disaster recovery data replication to be difficult to realize that the RPO is 0; if the data is copied by the disk technology, the copying distance is long, the time delay is large, and synchronous copying cannot be realized for a financial system with high service timeliness, so that data loss can be caused when recovery in different places is recovered; if asynchronously replicated through the database, asynchronous schemes are inherently unable to avoid data loss when restoring from a remote location.
Fig. 1 is a schematic diagram of a disaster recovery solution provided in the prior art, and please refer to fig. 1, where a technical solution provided in the prior art is based on an XRC asynchronous copy technology, and data of a production system is read by an SDM host and written into a disaster recovery data disk, so that data copy in a long distance and in a different place is implemented. The disaster recovery system provides service to the outside by using disaster recovery data obtained by data replication, and the current realization effect RPO is less than 2 minutes, and RTO is less than 2 hours. The technical scheme has the following defects: 1. a System Data Mover (SDM) host needs to be deployed independently at a disaster recovery side for XRC Data replication; 2, the XRC target disk is unreadable in the normal copying process, so that when the disaster recovery system is recovered, the system needs to be restarted, and the disaster recovery time is longer; and 3, the technical principle of XRC asynchronous replication determines that data loss cannot be avoided, the RPO is not zero, and the number needs to be complemented by other modes after the disaster recovery system is recovered.
Fig. 2 is a schematic diagram of another disaster recovery solution provided in the prior art, please refer to fig. 2, where a technical solution provided in the prior art is based on a multi-target disk synchronous replication technology and a database asynchronous replication technology, a database log is synchronously replicated from a site a to a site B, the log is read at the site B and asynchronously appended to a database of the site B, zero-loss replication of data of two sites in the same city is realized, and since a site B system is available online, a system does not need to be restarted, thereby realizing rapid switching of A, B sites. The current implementation effect RPO is 0 and RTO is in the order of minutes. However, due to the distance limitation of the synchronous disk replication technology, the technical scheme can only be used in the same city, and is not suitable for remote disaster recovery scenes.
Fig. 3 is a schematic diagram of another disaster recovery solution provided in the prior art, and please refer to fig. 3, where a technical solution provided in the prior art is based on a database asynchronous replication technology, and after a database log of a production system is read from a production center, the database log is transmitted to a disaster recovery center in a different place through a message queue, and the database log is asynchronously added to a recovery system, so that remote data replication between the production center and the disaster recovery center is realized. Because the disaster recovery system is always available online without restarting the system, fast disaster recovery switching can be realized. The effect of the current implementation is RPO >0 and RTO on the order of minutes. The method has the defects that the database asynchronous replication technology determines that the technical scheme cannot avoid data loss, and the data needs to be complemented by other schemes.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. To this end, a first aspect of the present invention provides a database data synchronization method, including:
synchronously copying a database log written into a main disk of a first site and a key file to a secondary disk of the first site, synchronously copying the database log written into the main disk to a database log disk of a second site, and synchronously copying the key file written into the main disk to a key file disk of the second site;
when the database log disk of the second site receives the database log copied from the main disk, asynchronously copying the database log copied to the database log disk of the second site to a third site;
when the key file disk of the second site receives the key file copied from the main disk, asynchronously copying the key file copied to the key file disk of the second site to a third site;
and the geographic distance between the first station and the second station is smaller than the geographic distance between the first station and the third station.
Further, still include:
when the secondary disk receives the database logs and the key files copied from the primary disk, copying the database logs and the key files copied to the secondary disk to a recovery system disk of a third site by adopting a disk asynchronous copying technology;
and when the recovery system disk receives the database log and the key file copied from the slave disk, initializing the content of the recovery system database of the third site according to the database log and the key file copied from the slave disk to the recovery system disk.
Further, the asynchronously copying the database logs of the database log disk copied to the second site to a third site includes:
copying the database log copied to the database log disk of the second site to a database log disk of a third site by adopting a disk asynchronous copying technology;
and when the database log disk of the third site receives the database log copied from the database log disk of the second site, copying the database log copied to the database log disk of the third site to a recovery system disk by adopting a database asynchronous copying technology.
Further, the copying the database log copied to the database log disk of the third site to the recovery system disk by using a database asynchronous copy technology includes:
reading the database log of the database log disk copied to the third site by a log reading process, and transmitting the read database log to an additional log process through a message queue;
and when receiving the read database log, the additional log process performs data analysis on the received read database log and then writes the data into a recovery system database of a third site again.
Further, after asynchronously copying the critical file of the critical file disk copied to the second site to the third site, the method further includes:
when the first station is detected to stop the external service and the connection between the first station and the peripheral system is disconnected, detecting whether the data of the second station and the data of the third station are completely synchronous;
if the third site is completely synchronous, judging whether the third site finishes reading and adding all database logs;
and if the recovery is finished, connecting the peripheral system to the third site recovery system, and opening the external service of the disaster recovery system.
Further, after determining whether the third site completes reading and appending all the database logs, the method further includes:
if the data is consistent with the data of the first station, detecting whether the data of the third station is consistent with the data of the first station;
when the data of the third station is consistent with the data of the first station, the step of connecting the peripheral system to the third station recovery system and opening the external service of the disaster recovery system is executed;
and when the data of the third site is inconsistent with the data of the first site, analyzing the reason for the inconsistency of the data of the third site with the data of the first site.
Further, after detecting whether the data of the second site is completely synchronized with the data of the third site, the method further includes: if not, the step of detecting whether the data of the second site and the third site are completely synchronous is executed again;
after the step of judging whether the third site finishes reading and appending all the database logs, the method further comprises the following steps: if not, the step of judging whether the third site finishes reading and adding all database logs is executed again.
The second aspect of the present invention provides a database data synchronization system, which includes a synchronous replication module, a first asynchronous replication module, and a second asynchronous replication module;
the synchronous replication module is used for synchronously replicating the database logs written into the main disk of the first site and the key files to the secondary disk of the first site, synchronously replicating the database logs written into the main disk to the database log disk of the second site, and synchronously replicating the key files written into the main disk to the key file disk of the second site;
the first asynchronous replication module is configured to, when the database log disk of the second site receives the database log replicated from the primary disk, asynchronously replicate the database log replicated to the database log disk of the second site to a third site;
the second asynchronous replication module is configured to, when the key file disk of the second site receives the key file replicated from the primary disk, asynchronously replicate the key file replicated to the key file disk of the second site to a third site;
and the geographic distance between the first station and the second station is smaller than the geographic distance between the first station and the third station.
A third aspect of the present invention provides an apparatus, which includes a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the database data synchronization method as set forth in the first aspect of the present invention.
A fourth aspect of the present invention provides a computer-readable storage medium having at least one instruction, at least one program, a set of codes, or a set of instructions stored therein, which is loaded and executed by a processor to implement the database data synchronization method as set forth in the first aspect of the present invention.
The database data synchronization method, the system, the equipment and the storage medium provided by the embodiment of the invention utilize the disk replication technology on the basis of keeping the characteristics of hot backup and short recovery time of the current database replication technology, the synchronously replicated data are firstly landed in the same city center and then are asynchronously transmitted to the disaster recovery center, so that the large host disaster recovery scheme with zero data loss at a system level is realized, the participation of complementation by application is not required, and meanwhile, a host does not need to be deployed in the same city center, and the recovery of external services of the disaster recovery system at a minute level can be realized.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic diagram of a disaster preparation scheme provided by the prior art;
FIG. 2 is a schematic diagram of another disaster preparation scheme provided by the prior art;
FIG. 3 is a schematic diagram of yet another disaster preparation scheme provided by the prior art;
FIG. 4 is a flowchart of a database data synchronization method according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a database data synchronization method according to an embodiment of the present invention;
fig. 6 is a flow of step S102;
fig. 7 is a schematic diagram of a disaster recovery process according to an embodiment of the present invention;
FIG. 8 is a schematic flow chart of an intra-planning disaster recovery drilling disclosed in the embodiments of the present invention;
fig. 9 is a block diagram of a database data synchronization system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. Examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout.
The following is an explanation of the noun terms that may be involved in embodiments of the present invention:
the Recovery Point Object (RPO) is a number that the disaster Recovery system can recover data to a Point in time before the disaster occurs after the disaster occurs.
The Recovery Time Object (RTO) refers to a Time period between a moment when a service is stopped due to a downtime of an IT system and a moment when the IT system is recovered to a state that the service can support operations of various departments after a disaster occurs, and is called RTO when the service is recovered to be operated.
The synchronous Copy technology (PPRC) is a disk Copy technology based on disk microcode, and the main technical principle is that a complete write operation includes the following steps:
1. writing the main magnetic disk;
2. writing into a target disk;
3. returning to the operating system;
and the data of the main disk and the target disk are completely consistent.
A Multi-target PPRC (MT-PPRC) refers to a disk copy technology in which one copy of disk data can be simultaneously and synchronously copied to two or more target disks.
The main technical principle is that a complete write operation does not include writing into a target disk part, and data synchronization between a main disk and a target disk is completed asynchronously.
An Extended Remote Asynchronous Copy technology (XRC) is a data Asynchronous Copy technology combining hardware microcode and software, and the main technical principle is that a Copy request is initiated by host software of a disaster recovery end, disk data of a production end is captured by the disk microcode, and sequencing is performed at the disaster recovery end, and finally data with a consistent time point is provided for disaster recovery.
System Data Mover (SDM) refers to a product component in XRC replication technology that interacts with the storage subsystem to replicate Data.
Examples
Fig. 4 is a flowchart of a database data synchronization method provided by an embodiment of the present invention, and fig. 5 is a schematic diagram of a database data synchronization method provided by an embodiment of the present invention, where the present specification provides the method operation steps as an embodiment or a flowchart, but may include more or less operation steps based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In practice, the system or server product may be implemented in a sequential or parallel manner (e.g., parallel processor or multi-threaded environment) according to the embodiments or methods shown in the figures. Specifically, as shown in fig. 4 and 5, the method may include the following steps:
s101: synchronously copying the database logs written into the main disk of the first site and the key files to the secondary disk of the first site, synchronously copying the database logs written into the main disk to the database log disk of the second site, and synchronously copying the key files written into the main disk to the key file disk of the second site;
the first site is the production center shown in fig. 5, the master disk of the first site is the production master disk shown in fig. 5, the slave disk of the first site, and the production slave disk shown in fig. 5.
A second site, namely the city-sharing center shown in fig. 5, a database log disk of the second site, namely a disk marked with a word of "database log" in fig. 5, and a key file disk of the second site, namely a disk marked with a word of "key file" in fig. 5;
specifically, the database log written into the primary disk of the first site is copied to the secondary disk of the first site and the database log disk of the second site by adopting a multi-target disk synchronous copying technology.
Specifically, the key files written into the main disk are copied to the secondary disk and the key file disk of the second site by adopting a multi-target disk synchronous copying technology.
S102: when a database log disk of a second site receives a database log copied from a main disk, asynchronously copying the database log copied to the database log disk of the second site to a third site;
fig. 6 is a flow of step S102, and specifically as shown in fig. 6, step S102 includes the following sub-steps:
s1021: when a database log disk of a second site receives a database log copied from a main disk, copying the database log copied to the database log disk of the second site to a database log disk of a third site by adopting a disk asynchronous copying technology;
the database log disk of the third site is a disk marked with the word "asynchronously copy to database log" in fig. 5.
S1022: and when the database log disk of the third site receives the database log copied from the database log disk of the second site, copying the database log copied to the database log disk of the third site to a recovery system disk by adopting a database asynchronous copying technology.
The database asynchronous Replication technology is a Q Replication technology based on an IBM DB2 system, and the main technical principle is that a database log of a main production center is read, transmitted to a disaster backup terminal through a Websphere MQ system, and re-executed in a database of the disaster backup terminal, so that the Replication of database contents of the main center and the disaster backup center is realized.
Specifically, step S1022 includes the following sub-steps:
the read log process reads the database log copied to the database log disk of the third site, and transmits the read database log to the additional log process through the message queue;
and when receiving the read database log, the additional log process performs data analysis on the received read database log and then rewrites the received read database log into the recovery system database of the third site.
S103: when the key file disk of the second site receives the key file copied from the main disk, asynchronously copying the key file copied to the key file disk of the second site to the third site; specifically, step S103 includes the following substeps:
and copying the key files copied to the key file disk of the second site to a recovery system disk of the third site by adopting a disk asynchronous copying technology.
And the geographic distance between the first station and the second station is smaller than the geographic distance between the first station and the third station. In one specific example, the first site (production center) is about 1500 kilometers from the third site (disaster recovery center) and the first site (production center) is about 60 kilometers from the second site (same city center).
Optionally, the geographic distance between the first station and the second station is smaller than a first preset distance, the geographic distance between the first station and the third station is smaller than a second preset distance, and the first preset distance is far smaller than the second preset distance.
Referring to fig. 5, the database data synchronization method provided in the embodiment of the present invention further includes the following steps:
when receiving the database logs and the key files copied from the main disk from the disk, copying the database logs and the key files copied to the secondary disk to a recovery system disk of a third site by adopting a disk asynchronous copying technology;
and when the recovery system disk receives the database log and the key file copied from the disk, initializing the content of the recovery system database of the third site according to the database log and the key file copied from the disk to the recovery system disk.
That is, the production database data is copied to the third site by using the disk as a data source through the disk asynchronous copying technology, and the database content of the recovery system is initialized.
Before the database data synchronization method provided by the embodiment of the invention is executed, the method further comprises the following steps:
1. build a second site (i.e. city center)
And deploying the disk and the peripheral equipment related to the disk (without deploying a host) and the equipment related to data transmission at the second site. Compared with the disaster preparation scheme provided by the prior art disclosed in fig. 1, the embodiment of the invention does not need to deploy a host computer in the same city center.
2. Build the third site (namely disaster recovery center)
And building a recovery system at the third site, and deploying a read log process, an additional log process, a message transmission queue and the like.
3. Establishing disk copy relationships
Establishing a multi-target synchronous replication relationship among a production master disk, a slave disk, database logs of a second site and a key file disk, wherein the production master disk and the slave disk are copied in full, and the second site is used for copying partial data;
establishing an asynchronous replication relation between a database log of a second site and a key file disk and a third site, wherein the database log is asynchronously replicated to the third site to be used as a data source for reading the log, and the key file is asynchronously replicated to the third site to be used as a part of recovery system data;
data consistency and copy management of multi-target synchronous copy and file asynchronous copy are controlled by an IBM large host GDPS (geophysics distributed Parallel system, disaster recovery product based on IBM z platform).
Fig. 7 is a schematic diagram of a disaster recovery procedure disclosed in an embodiment of the present invention, and specifically, as shown in fig. 7, the disaster recovery procedure includes the following steps:
s201: when a first site (namely a production center) is detected to stop external service and the first site is disconnected from a peripheral system, whether data of a second site (namely a city-level center) and data of a third site (namely a disaster recovery center) are completely synchronized is detected;
when an unplanned disaster occurs, the first site and the external connection thereof are passively stopped, and the database logs and the file data of the second site are not updated any more due to the disaster of the first site.
S202: if the third site is completely synchronous, judging whether the third site finishes reading and adding all database logs; if not, the step of detecting whether the data of the second site and the third site are completely synchronous is executed again;
s203: if the recovery is finished, connecting the peripheral system to a third site recovery system, and opening the external service of the disaster recovery system; if not, the step of judging whether the third site finishes reading and adding all the database logs is executed again.
Specifically, the data consistency check usually takes a long time to perform in the disaster recovery training, and the actual handover usually does not require the step of performing the data consistency check again because the handover is completed in the shortest time.
Fig. 8 is a schematic diagram of an intra-plan disaster recovery drilling process disclosed in the embodiment of the present invention, and specifically as shown in fig. 8, the disaster recovery drilling process includes the following steps:
s300: the first site (i.e., the production center) is actively stopped from out-of-service while the peripheral systems are stopped from continuing to access the first site's hosts.
S301: when the first station is detected to stop the external service and the connection between the first station and the peripheral system is disconnected, detecting whether the data of the second station (namely the same city center) and the data of the third station (namely the disaster recovery center) are completely synchronous or not;
s302: if the third site is completely synchronous, judging whether the third site finishes reading and adding all database logs; if not, executing step S301 again;
s303: if the data is consistent with the data of the first station, detecting whether the data of the third station is consistent with the data of the first station; if not, the step of judging whether the third site finishes reading and adding all the database logs is executed again.
S304: when the data of the third station is consistent with the data of the first station, connecting the peripheral system to the third station recovery system, and opening the external service of the disaster recovery system;
s305: and when the data of the third site is inconsistent with the data of the first site, analyzing the reason for the inconsistency of the data of the third site with the data of the first site.
Fig. 9 is a block diagram of a database data synchronization system according to an embodiment of the present invention, and specifically, as shown in fig. 9, the database data synchronization system according to the embodiment of the present invention includes a synchronous replication module 401, a first asynchronous replication module 402, and a second asynchronous replication module 403;
the synchronous replication module 401 is configured to synchronously replicate the database log and the key file written to the primary disk of the first site to the secondary disk of the first site, synchronously replicate the database log written to the primary disk to the database log disk of the second site, and synchronously replicate the key file written to the primary disk to the key file disk of the second site;
a first asynchronous replication module 402, configured to, when a database log disk of a second site receives a database log replicated from a primary disk, asynchronously replicate the database log replicated to the database log disk of the second site to a third site;
a second asynchronous replication module 403, configured to, when the key file disk of the second site receives the key file replicated from the primary disk, asynchronously replicate the key file replicated to the key file disk of the second site to the third site;
and the geographic distance between the first station and the second station is smaller than the geographic distance between the first station and the third station.
An embodiment of the present invention also provides an apparatus, which includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the database data synchronization method as in the method embodiment.
Embodiments of the present invention also provide a storage medium, which may be disposed in a server to store at least one instruction, at least one program, a code set, or a set of instructions related to implementing the database data synchronization method in the method embodiment, where the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the database data synchronization method provided in the method embodiment.
Alternatively, in this embodiment, the storage medium may be located in at least one network server of a plurality of network servers of a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
As can be seen from the embodiments of the database data synchronization method, system, device or storage medium provided by the present invention, the database data synchronization method, system, device and storage medium provided by the embodiments of the present invention construct a two-place-three-center disaster-tolerant solution with zero data loss by using a database asynchronous replication technology, a multi-target disk synchronization technology and an asynchronous replication technology. The main technical principle is that a database log of a production system (a first site) is synchronously backed up to a city-sharing center (a second site) through a multi-target synchronous copying technology, data of the city-sharing center (the second site) is used as a data source, and the production data is asynchronously copied to a disaster recovery system in a different place through a database asynchronous copying technology. According to the scheme, zero data loss is completely realized by a system-level disaster recovery scheme, and application participation in number complementing is not needed.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the device and server embodiments, since they are substantially similar to the method embodiments, the description is simple, and the relevant points can be referred to the partial description of the method embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A database data synchronization method, comprising:
synchronously copying a database log written into a main disk of a first site and a key file to a secondary disk of the first site, synchronously copying the database log written into the main disk to a database log disk of a second site, and synchronously copying the key file written into the main disk to a key file disk of the second site;
when the database log disk of the second site receives the database log copied from the main disk, asynchronously copying the database log copied to the database log disk of the second site to a third site;
when the key file disk of the second site receives the key file copied from the main disk, asynchronously copying the key file copied to the key file disk of the second site to a third site;
and the geographic distance between the first station and the second station is smaller than the geographic distance between the first station and the third station.
2. The method of claim 1, further comprising:
when the secondary disk receives the database logs and the key files copied from the primary disk, copying the database logs and the key files copied to the secondary disk to a recovery system disk of a third site by adopting a disk asynchronous copying technology;
and when the recovery system disk receives the database log and the key file copied from the slave disk, initializing the content of the recovery system database of the third site according to the database log and the key file copied from the slave disk to the recovery system disk.
3. The method of claim 1, wherein asynchronously replicating the database logs of the database log disk replicated to the second site to a third site comprises:
copying the database log copied to the database log disk of the second site to a database log disk of a third site by adopting a disk asynchronous copying technology;
and when the database log disk of the third site receives the database log copied from the database log disk of the second site, copying the database log copied to the database log disk of the third site to a recovery system disk by adopting a database asynchronous copying technology.
4. The method of claim 3, wherein the copying the database logs copied to the database log disk at the third site to the recovery system disk using a database asynchronous copy technique comprises:
reading the database log of the database log disk copied to the third site by a log reading process, and transmitting the read database log to an additional log process through a message queue;
and when receiving the read database log, the additional log process performs data analysis on the received read database log and then writes the data into a recovery system database of a third site again.
5. The method of claim 1, wherein after asynchronously copying critical files of the critical file disks copied to the second site to a third site, further comprising:
when the first station is detected to stop the external service and the connection between the first station and the peripheral system is disconnected, detecting whether the data of the second station and the data of the third station are completely synchronous;
if the third site is completely synchronous, judging whether the third site finishes reading and adding all database logs;
and if the recovery is finished, connecting the peripheral system to the third site recovery system, and opening the external service of the disaster recovery system.
6. The method of claim 5, wherein after determining whether the third site has completed reading and appending all database logs, the method further comprises:
if the data is consistent with the data of the first station, detecting whether the data of the third station is consistent with the data of the first station;
when the data of the third station is consistent with the data of the first station, the step of connecting the peripheral system to the third station recovery system and opening the external service of the disaster recovery system is executed;
and when the data of the third site is inconsistent with the data of the first site, analyzing the reason for the inconsistency of the data of the third site with the data of the first site.
7. The method of claim 6,
after detecting whether the data of the second site and the data of the third site are completely synchronized, the method further includes: if not, the step of detecting whether the data of the second site and the third site are completely synchronous is executed again;
after the step of judging whether the third site finishes reading and appending all the database logs, the method further comprises the following steps: if not, the step of judging whether the third site finishes reading and adding all database logs is executed again.
8. A database data synchronization system is characterized by comprising a synchronous replication module, a first asynchronous replication module and a second asynchronous replication module;
the synchronous replication module is used for synchronously replicating the database logs written into the main disk of the first site and the key files to the secondary disk of the first site, synchronously replicating the database logs written into the main disk to the database log disk of the second site, and synchronously replicating the key files written into the main disk to the key file disk of the second site;
the first asynchronous replication module is configured to, when the database log disk of the second site receives the database log replicated from the primary disk, asynchronously replicate the database log replicated to the database log disk of the second site to a third site;
the second asynchronous replication module is configured to, when the key file disk of the second site receives the key file replicated from the primary disk, asynchronously replicate the key file replicated to the key file disk of the second site to a third site;
and the geographic distance between the first station and the second station is smaller than the geographic distance between the first station and the third station.
9. An apparatus comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the database data synchronization method of any of claims 1-7.
10. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the database data synchronization method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011337993.3A CN112433887A (en) | 2020-11-25 | 2020-11-25 | Database data synchronization method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011337993.3A CN112433887A (en) | 2020-11-25 | 2020-11-25 | Database data synchronization method, system, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112433887A true CN112433887A (en) | 2021-03-02 |
Family
ID=74698897
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011337993.3A Pending CN112433887A (en) | 2020-11-25 | 2020-11-25 | Database data synchronization method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112433887A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060271815A1 (en) * | 2005-05-31 | 2006-11-30 | Kazuhiko Mizuno | System and method for disaster recovery of data |
| CN103226502A (en) * | 2013-05-21 | 2013-07-31 | 中国工商银行股份有限公司 | Data disaster backup control system and data recovery method |
| CN103488668A (en) * | 2012-06-07 | 2014-01-01 | 国际商业机器公司 | Reducing data transfers while eliminating data loss for asynchronous replication of databases |
| CN104850628A (en) * | 2015-05-21 | 2015-08-19 | 中国工商银行股份有限公司 | Data synchronization method and apparatus in database |
| CN106776121A (en) * | 2016-11-23 | 2017-05-31 | 中国工商银行股份有限公司 | A kind of data calamity is for device, system and method |
| CN106815097A (en) * | 2017-01-18 | 2017-06-09 | 北京许继电气有限公司 | Database disaster tolerance system and method |
| CN109901949A (en) * | 2019-02-25 | 2019-06-18 | 中国工商银行股份有限公司 | The application disaster recovery and backup systems and method of dual-active data center |
-
2020
- 2020-11-25 CN CN202011337993.3A patent/CN112433887A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060271815A1 (en) * | 2005-05-31 | 2006-11-30 | Kazuhiko Mizuno | System and method for disaster recovery of data |
| CN103488668A (en) * | 2012-06-07 | 2014-01-01 | 国际商业机器公司 | Reducing data transfers while eliminating data loss for asynchronous replication of databases |
| CN103226502A (en) * | 2013-05-21 | 2013-07-31 | 中国工商银行股份有限公司 | Data disaster backup control system and data recovery method |
| CN104850628A (en) * | 2015-05-21 | 2015-08-19 | 中国工商银行股份有限公司 | Data synchronization method and apparatus in database |
| CN106776121A (en) * | 2016-11-23 | 2017-05-31 | 中国工商银行股份有限公司 | A kind of data calamity is for device, system and method |
| CN106815097A (en) * | 2017-01-18 | 2017-06-09 | 北京许继电气有限公司 | Database disaster tolerance system and method |
| CN109901949A (en) * | 2019-02-25 | 2019-06-18 | 中国工商银行股份有限公司 | The application disaster recovery and backup systems and method of dual-active data center |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3474516B1 (en) | Data processing method and device | |
| US20150213100A1 (en) | Data synchronization method and system | |
| WO2018000812A1 (en) | Data storage method and apparatus | |
| US9189348B2 (en) | High availability database management system and database management method using same | |
| US7571268B2 (en) | Consistent updates across storage subsystems coupled to a plurality of primary and secondary units at selected times | |
| CN106776130B (en) | Log recovery method, storage device and storage node | |
| US7111004B2 (en) | Method, system, and program for mirroring data between sites | |
| CN103548011A (en) | Asynchronous replication in a distributed storage environment | |
| CN109189608A (en) | A kind of method guaranteeing duplication transaction consistency and corresponding reproducing unit | |
| CN102306115A (en) | Asynchronous remote copying method, system and equipment | |
| CN115098299B (en) | Backup method, disaster recovery method, device and equipment of virtual machine | |
| US9330153B2 (en) | System, method, and computer readable medium that coordinates between devices using exchange of log files | |
| CN106815094B (en) | Method and equipment for realizing transaction submission in master-slave synchronization mode | |
| CN114610532A (en) | Database processing method and device | |
| CN113467993A (en) | Method, system, equipment and storage medium for restoring cloud platform resources | |
| EP4060514A1 (en) | Distributed database system and data disaster backup drilling method | |
| US8312237B2 (en) | Automated relocation of in-use multi-site protected data storage | |
| WO2023151443A1 (en) | Synchronizing main database and standby database | |
| CN112433887A (en) | Database data synchronization method, system, device and storage medium | |
| CN106502831B (en) | A kind of method and device of image file duplication | |
| CN115454958A (en) | Data processing method, device, equipment, system and medium based on artificial intelligence | |
| CN101242201B (en) | A master-slave system maintenance method, system and device | |
| CN111400098A (en) | Copy management method and device, electronic equipment and storage medium | |
| JP2010182154A (en) | Information system and disaster recovery method | |
| CN113467717B (en) | Dual-machine volume mirror image management method, device and equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |